Methods, apparatus, and computer programs for video encoding and decoding.

Abstract

To provide video encoding / decoding method and device using an in-frame block copy mode.SOLUTION: A device for video decoding includes a reception circuit and a processing circuit, wherein the processing circuit encodes prediction information on a current block which is being encoded from an encoded video bitstream. The prediction information indicates an in-frame block copy mode, and determines a block vector indicating a reference block in the same picture as the current block. The reference block is restricted to an encoded region, and a reconstructed sample is buffered in a reference sample memory. The encoded region is one of a plurality of previous definition regions of an encoding tree unit (CTU). The processing circuit reconstructs at least a sample of the current block on the basis of the reconstructed sample of the reference block retrieved from the reference sample memory.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] This application claims priority to U.S. Patent Application No. 16 / 699,235, entitled "METHOD AND APPARATUS FOR VIDEO CODING", filed on Nov. 29, 2019, which claims priority to U.S. Provisional Application No. 62 / 774,148, entitled "CONSTRAINTS ON REFERENCE BLOCK LOCATIONS FOR INTRA BLOCK COPY", filed on Nov. 30, 2018, and U.S. Provisional Application No. 62 / 790,454, entitled "INTRA PICTURE BLOCK COMPENSATION BUFFER REUSE WITH NON SQUARE BLOCK PARTITIONS", filed on Jan. 9, 2019, the entire contents of which are incorporated herein by reference.

[0002] The content of the present disclosure generally describes embodiments related to video coding.

Background Art

[0003] The background description provided herein is for the purpose of generally presenting the background of the content of the present disclosure. From the work described in this background art section, the work of the currently named inventors and aspects that are not otherwise limited as the description of the prior art at the time of filing are not expressly or implicitly admitted as the prior art to the content of the present disclosure.

[0004] Video encoding and decoding can be performed using inter-frame prediction with motion compensation. Uncompressed digital video may consist of a series of pictures, each picture having spatial dimensions, e.g., 1920×1080 luminance samples and associated chrominance samples. The series of pictures may have, for example, 60 pictures per second or a fixed or variable picture rate (also informally called frame rate) of 60Hz. Uncompressed video has high bitrate requirements. For example, 8-bit per sample 1080p60 4:2:0 video (1920×1080 luminance sample resolution at a 60Hz frame rate) requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 GB of storage space.

[0005] Video encoding and decoding aim to reduce redundancy in the video signal input through compression. Compression contributes to reducing the bandwidth or memory space requirements mentioned above, reducing them by two or more orders of magnitude in some situations. Lossless compression, lossy compression, and combinations thereof are available. Lossless compression refers to techniques that allow for the reconstruction of an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for expected applications. In the case of video, lossy compression is widely used. The acceptable amount of distortion depends on the application; for example, users of consumer streaming media applications may tolerate higher distortion than users of television distribution applications. The compression ratio achieved reflects that the higher the allowed / acceptable distortion, the higher the compression ratio.

[0006] Video encoders and decoders can utilize several broad categories of techniques, including, for example, motion compensation, transformation, quantization, and entropy coding.

[0007] Video codec technology includes a technique called intra-frame coding. In intra-frame coding, sample values ​​are represented without referencing samples or other data from a previously reconstructed reference picture. In some video codecs, the picture is spatially subdivided into sample blocks. If all sample blocks are coded in intra-frame mode, the picture may be an intra-frame picture. An intra-frame picture and its derivatives (e.g., an independent decoder refresh picture) can reset the decoder state and can therefore be used as the first picture or still picture in the encoded video bitstream and video session. Samples in intra-frame blocks may be subjected to a transformation, and the transformation coefficients may be quantized before entropy coding. Intra-frame prediction may be a technique to minimize the sample values ​​in the pre-transformation domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0008] For example, traditional in-frame coding, known from MPEG-2 encoding techniques, does not use in-frame prediction. However, newer video compression techniques include, for example, techniques that utilize metadata and / or surrounding sample data obtained during the encoding / decoding of spatially adjacent and preceding data blocks in the decoding order. Such techniques are called “in-frame prediction” techniques. In at least some cases, in-frame prediction uses only reference data from the current picture being reconstructed, and not reference data from the reference picture.

[0009] In-frame prediction can take various forms. If more than one such technique can be used with a given video coding technique, the technique used can be coded in in-frame prediction mode. In some cases, a mode may have submodes and / or parameters, which may be coded individually or included in a mode codeword. The choice of codeword for a given mode / submode / parameter combination may affect the gain in coding efficiency due to in-frame prediction, and the entropy coding technique used to convert the codeword to a bitstream may also affect the gain in coding efficiency.

[0010] Modes with in-frame prediction were introduced in H.264, refined in H.265, and further refined with newer coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Prediction blocks may be formed using adjacent sample values ​​belonging to already available samples. The sample values ​​of adjacent samples are copied into the prediction block according to direction. The reference to the direction of use may be encoded in the bitstream or predicted itself. [Overview of the project] [Means for solving the problem]

[0011] Each aspect of this disclosure provides methods and apparatus for video encoding / decoding. In some examples, the apparatus for video decoding includes a receiving circuit and a processing circuit. In some embodiments, the processing circuit decodes prediction information for the current block currently being encoded from the encoded video bitstream. The prediction information indicates an in-frame block copy mode. The processing circuit then determines a block vector that points to a reference block in the same picture as the current block. The reference block is confined to an encoding region, and the reconstructed samples are buffered in a reference sample memory. The encoding region is one of several predefined regions of an encoding tree unit (CTU). The processing circuit then reconstructs at least samples of the current block based on the reconstructed samples of the reference block retrieved from the reference sample memory.

[0012] In some examples, the encoded region has a square shape. In the example, the CTU is arranged to have 128 × 128 samples of luminance components.

[0013] In one embodiment, the reference sample memory is configured to update the samples buffered in the reference sample memory on a coding region basis.

[0014] In some embodiments, the processing circuit checks whether multiple corners of the reference block are within the encoding region.

[0015] In one example, if the CTU contains 128 × 128 luminance samples, the size of the encoded region is the size of 64 × 64 luminance samples. In another example, if the CTU contains fewer than 128 × 128 luminance samples, the size of the encoded region is the same as the size of the CTU.

[0016] In some embodiments, the processing circuit moves a reference block to the coding region if the reference block is partially in the coding region before being moved. In one example, the processing circuit moves the reference block in a direction that decreases the distance between the reference block and the current block. In another example, the processing circuit moves the reference block in a direction that increases the percentage of the reference sample in the coding region.

[0017] In some embodiments, when a reference block is partially in the encoded region, the processing circuit uses information from a first reference sample in the encoded region as information from a second reference sample outside the encoded region. In some examples, flexible block partitioning is prohibited in the partitioning of the CTU.

[0018] Each aspect of the present disclosure further provides a non-temporary computer-readable medium on which instructions are stored, and when such instructions are executed by a computer, causes the computer to perform a method for video encoding. [Brief explanation of the drawing]

[0019] Based on the following detailed description and drawings, other features, properties, and various advantages of the disclosed topic will become clearer, as shown in the drawings.

[0020] [Figure 1] This is a schematic diagram of a simplified block diagram of a communication system (100) according to an embodiment.

[0021] [Figure 2] This is a schematic diagram of a simplified block diagram of a communication system (200) according to an embodiment.

[0022] [Figure 3] This is a schematic diagram of a simplified block diagram of a decoder according to an embodiment.

[0023] [Figure 4] This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0024] [Figure 5] Shows a block diagram of an encoder according to another embodiment.

[0025] [Figure 6] Shows a block diagram of a decoder according to another embodiment.

[0026] [Figure 7] Shows an example of in-frame block copy according to an embodiment of the content of the present disclosure.

[0027] [Figure 8] A to D show examples of the effective search range of reference sample memory update and in-frame block copy mode according to an embodiment of the content of the present disclosure.

[0028] [Figure 9] A to H show some examples that may result in the division of non-square blocks.

[0029] [Figure 10A] It is one of the figures showing two examples of encoding order and use of reference sample memory. [Figure 10B] It is one of the figures showing two examples of encoding order and use of reference sample memory.

[0030] [Figure 11] A and B show examples of samples from two different encoding regions in a reference block.

[0031] [Figure 12] A to F show examples of a reuse strategy for reference sample memory.

[0032] [Figure 13] A to F show examples of update processing according to some embodiments of the content of the present disclosure.

[0033] [Figure 14] A flowchart outlining examples of processing according to several embodiments of the contents of this disclosure is provided.

[0034] [Figure 15] This is a schematic diagram of a computer system according to an embodiment. [Modes for carrying out the invention]

[0035] Figure 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the contents of this disclosure. The communication system (100) includes a plurality of terminal devices that can communicate with each other, for example, via a network (150). For example, the communication system (100) has a first pair of terminal devices (110), (120) connected to each other via a network (150). In the example of Figure 1, the first pair of terminal devices (110), (120) perform one-way data transmission. For example, terminal device (110) encodes video data (for example, a video picture stream captured by terminal device (110)) and transmits it to other terminal device (120) via the network (150). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. Terminal device (120) receives the encoded video data from the network (150), decodes the encoded video data to recover the video picture, and displays the video picture according to the recovered video data. One-way data transmission is commonly seen in media service applications and other similar applications.

[0036] In another example, the communication system (100) includes a second pair of terminal devices (130), (140) for performing bidirectional transmission of encoded video data, which may occur, for example, during a video conference. With respect to bidirectional data transmission, in the example, each terminal device in terminal devices (130), (140) can encode video data (e.g., a video picture stream captured by the terminal device) and transmit it to the other terminal device in terminal devices (130), (140) via the network (150). Each terminal device in terminal devices (130), (140) can further receive encoded video data transmitted from the other terminal device in terminal devices (130), (140), decode the encoded video data to recover the video picture, and display the video picture on an accessible display device according to the recovered video data.

[0037] In the example shown in Figure 1, terminal devices (110), (120), (130), and (140) are shown as a server, a personal computer, and a smartphone, respectively, but the principles of the Disclosure are not limited thereto. Embodiments of the Disclosure apply to laptop computers, tablets, media players, and / or professional video conferencing equipment. Network (150) refers to any number of networks, including, for example, wired and / or wireless networks, for transmitting encoded video data between terminal devices (110), terminal device (120), terminal device (130), and terminal device (140). Communication network (150) can exchange data over circuit-switched and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, unless otherwise described herein, the architecture and topology of network (150) are not important to the operation of the Disclosure.

[0038] As an example of the application of the disclosed theme, Figure 2 shows the arrangement of a video encoder and a video decoder in a streaming transmission environment. The disclosed theme is equivalent to other applications that support video, such as video conferencing and digital television, where compressed video is stored on digital media such as CDs, DVDs, and memory sticks.

[0039] The streaming transmission system includes a capture subsystem (213), which includes a video source (201), such as a digital camera, for constructing an uncompressed video picture stream (202). In this example, the video picture stream (202) includes a sample captured by the digital camera. The video picture stream (202), which is drawn as a thick line to emphasize its larger data volume compared to encoded video data (204) (or encoded video bitstream), is processed by electronic equipment (220) including a video encoder (203) coupled to the video source (201). The video encoder (203) includes hardware, software, or a combination thereof to implement or carry out each aspect of the themes of the disclosure described in more detail below. The encoded video data (204) (or encoded video bitstream (204)), which is drawn as a thin line to emphasize its smaller data volume compared to the video picture stream (202), is stored in a streaming server (205) for later use. One or more streaming client subsystems, such as client subsystems (206) and (208) in Figure 2, can access a streaming server (205) to retrieve copies (207) and (209) of encoded video data (204). Client subsystem (206) includes, for example, a video decoder (210) in an electronic device (230). The video decoder (210) decodes the input copy (207) of the encoded video data and constructs a derived video picture stream (211) that is displayed on a display (212) (e.g., a screen) or other display device (not shown). In a streaming transmission system, the encoded video data (204), (207), and (209) (e.g., video bitstreams) can be encoded based on a video encoding / compression standard. Examples of these standards include the ITU-T H.265 proposal. In the examples, video encoding standards under development are informally referred to as multipurpose video coding or VVC. The disclosed topics apply in the context of VVC.

[0040] The electronic devices (220) and (230) may include other components (not shown). For example, the electronic device (220) may include a video decoder (not shown), and the electronic device (230) may include a video encoder (not shown).

[0041] Figure 3 is a block diagram of a video decoder (310) according to an embodiment of the present disclosure. The video decoder (310) may be included in an electronic device (330). The electronic device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used as a substitute for the video decoder (210) in the example of Figure 2.

[0042] The receiver (331) can receive one or more encoded video sequences to be decoded by the video decoder (310), and in the same embodiment or other embodiments, it may receive one encoded video sequence at a time, with the decoding of each encoded video sequence being independent of other encoded video sequences. It may also receive encoded video sequences from a channel (301), which may be a hardware / software link to a storage device for storing encoded video data. The receiver (331) can receive encoded video data and other data, for example, encoded audio data and / or auxiliary data streams that can be transferred to their respective user entities (not shown). The receiver (331) can separate the encoded video sequences from the other data. To prevent network jitter, a buffer memory (315) is connected between the receiver (331) and the entropy decoder / parser (320) (hereinafter referred to as "parser (320)"). In some applications, the buffer memory (315) is part of the video decoder (310). In other applications, the buffer memory (315) may be located outside the video decoder (310) (not shown). Furthermore, in other applications, a buffer memory (not shown) outside the video decoder (310) may be provided to prevent network jitter, for example, and another buffer memory (315) inside the video decoder (310) may be provided to process playback timing, for example. When the receiver (331) receives data from a storage / transfer device or an isochronous network with sufficient bandwidth and controllability, the buffer memory (315) may not be required, or the buffer memory (315) may be small. For example, a buffer memory (315) may be required for use in a best-effort packet network such as the Internet, and the buffer memory (315) may be relatively large and advantageously self-adaptive in size, and may be implemented in an operator system or a similar element (not shown) located at least in part outside the video decoder (310).

[0043] The video decoder (310) has a parser (320) that reconstructs codes (321) based on the encoded video sequence. These codes include information for managing the operation of the video decoder (310) and latent information for controlling a display device (312) (e.g., a screen), which, as shown in Figure 3, is not an integral part of the electronic device (330) but is linked to the electronic device (330). The control information used for (one or more) display devices may be in the form of supplementary enhancement information (SEI messages) or video usability information (VUI) parameter set segments (not shown). The parser (320) performs parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence is based on video coding techniques or standards and follows various principles such as variable-length coding, Huffman coding, and context-sensitive or non-context-sensitive arithmetic coding. The parser(320) extracts a set of subgroup parameters from the encoded video sequence for at least one subgroup of pixels in the video decoder, based on at least one parameter corresponding to a group. Subgroups include picture groups (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), and predictive units (PU). The parser(320) can further extract information from the encoded video sequence such as transform coefficients, quantizer parameter values, and motion vectors.

[0044] The parser (320) can construct a code (321) by performing an entropy decoding / analysis operation on the video sequence received from the buffer memory (315).

[0045] Depending on the type of encoded video picture or part thereof (e.g., interframe and intraframe pictures, interframe and intraframe blocks) and other factors, the reconstruction of the code (321) involves several different units. Which units are involved and how they are involved can be controlled by subgroup control information analyzed by the parser (320) from the encoded video sequence. For brevity, the flow of such subgroup control information between the parser (320) and the following multiple units is not described.

[0046] Aside from the functional blocks already mentioned, the video decoder (310) can be conceptually subdivided into several functional units described below. In actual implementations performed under commercial constraints, multiple units in these units may interact closely with each other and be at least partially integrated with each other. However, for the purpose of illustrating the theme of the disclosure, it is appropriate to subdivide it conceptually into the following functional units.

[0047] The first unit is the scaler / inverse unit (351). The scaler / inverse unit (351) receives quantization transformation coefficients as (one or more) codes (321) and control information from the parser (320). The control information includes the transformation scheme to be used, the block size, the quantization factor, the quantization scaling matrix, etc. The scaler / inverse unit (351) can output a block containing sample values ​​that is input to the aggregator (355).

[0048] In some cases, the output samples of the scaler / inverse unit (351) may belong to an in-frame coding block, that is, it does not utilize predictive information from a previously reconstructed picture, but it can utilize a block of predictive information from a portion of the current picture that was reconstructed before it. Such predictive information is provided by the in-frame picture prediction unit (352). In some situations, the in-frame picture prediction unit (352) generates a block that is the same size and shape as the block being reconstructed, using the surrounding reconstructed information extracted from the current picture buffer (358). For example, the current picture buffer (358) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some situations, the aggregator (355) adds the prediction information generated by the in-frame prediction unit (352) to the output sample information provided by the scaler / inverse unit (351), based on each sample.

[0049] In other cases, the output samples of the scaler / inverse unit (351) may belong to an interframe coding block and a latent motion compensation block. In such a situation, the motion compensation prediction unit (353) can extract samples for prediction by accessing the reference picture memory (357). After motion compensation is performed on the extracted samples based on the codes (321) belonging to the block, these samples are added from the aggregator (355) to the output of the scaler / inverse unit (351) (in this case, called residual samples or residual signals) to generate output sample information. The address in the reference picture memory (357) from which the motion compensation prediction unit (353) extracts prediction samples can be controlled by a motion vector, which is used by the motion compensation prediction unit (353) in the form of a code (321), which may have, for example, X, Y, and reference picture components. Motion compensation may further include interpolation of sample values ​​extracted from reference picture memory (357), motion vector prediction mechanisms, etc., when using the precise motion vectors of subsamples.

[0050] The output samples from the aggregator (355) can be used in the loop filter unit (356) to utilize various loop filtering techniques. The video compression technique includes an in-loop filtering technique, which is included in the encoded video sequence (also called the encoded video bitstream) and is used as a code (321) from the parser (320) to control the parameters of the loop filter unit (356). The video compression technique can respond to metadata obtained during the decoding period of the earlier part of the encoded picture or encoded video sequence (in the order of decoding), and to sample values ​​that have been previously reconstructed and loop filtered.

[0051] The output of the loop filter unit (356) may be a sample stream, which is output to the display device (312) and stored in the reference picture memory (357) for later use in inter-frame picture prediction.

[0052] Once fully reconstructed, an encoded picture can be used as a reference picture for later predictions. For example, when an encoded picture corresponding to the current picture is fully reconstructed and the encoded picture (e.g., via the parser (320)) is recognized as a reference picture, the current picture buffer (358) becomes part of the reference picture memory (357), and a new current picture buffer is reallocated before any subsequent encoded pictures are reconstructed.

[0053] The video decoder (310) can perform decoding operations based on a specified video compression technique in a standard, such as the ITU-T H.265 proposal. The encoded video sequence conforms to the grammar specified by the video compression technique or standard in use, in the sense that it follows both the grammar of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile can select several tools from all the tools available in the video compression technique or standard as tools available under that profile. For compliance, the complexity of the encoded video sequence may be required to be within the range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (e.g., measured in units of millions of samples / second), and maximum reference picture size. In some cases, the limits set by the level are further restricted through the specifications of the virtual reference decoder (HRD) and metadata for managing the signaled HRD buffer in the encoded video sequence.

[0054] In the embodiment, the receiver (331) can receive additional (redundant) data and encoded video. The additional data is included as part of one or more encoded video sequences. The additional data is utilized by the video decoder (310) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, time, space, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0055] Figure 4 shows a block diagram of a video encoder (403) according to an embodiment of the contents of this disclosure. The video encoder (403) is contained within an electronic device (420). The electronic device (420) has a transmitter (440) (e.g., a transmission circuit). The video encoder (403) can replace the video encoder (203) in the example of Figure 2.

[0056] The video encoder (403) can receive video samples from a video source (401) (not part of the electronic device (420) in the example in Figure 4) that can capture video images encoded by the video encoder (403). In other examples, the video source (401) is part of the electronic device (420).

[0057] A video source (401) can be encoded by a video encoder (403) to provide a source video sequence in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601 YCrCB, RGB…), and any suitable sampling configuration (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (401) may be a storage device for storing previously prepared video. In a video conferencing system, the video source (401) may be a camera for capturing local image information as a video sequence. The video data may be provided as a series of individual pictures with motion when viewed in order. The pictures themselves may be organized as spatial pixel arrays, and depending on the sampling configuration, color space, etc. used, each pixel may contain one or more samples. The relationship between pixels and samples will be easily understood by those skilled in the art. The following description focuses on samples.

[0058] According to the embodiment, the video encoder (403) encodes the pictures of the source video sequence in real time or under any other time constraints required by the application, and compresses them as an encoded video sequence (443). Performing this at an appropriate encoding speed is one function of the controller (450). In some embodiments, the controller (450) controls and is functionally coupled to other functional units described below. For simplicity, such coupling is not illustrated. Parameters set by the controller (450) may include parameters related to rate control (picture skip, quantizer, λ value of rate distortion optimization technique, etc.), picture size, picture group (GOP) arrangement, maximum motion vector search range, etc. The controller (450) may be configured to have other appropriate functions, which belong to the video encoder (403) optimized for a given system design.

[0059] In some embodiments, the video encoder (403) is arranged to operate in an encoding loop. In a very brief description, in one example, the encoding loop includes a source encoder (430) (for example, responsible for constructing a code, such as a code stream, based on the input picture to be encoded and one or more reference pictures) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the code and constructs sample data in the same manner as a (remote) decoder constructs sample data (because in the video compression techniques considered in the theme of disclosure, both compression between the code and the encoded video bitstream are reversible). The reconstructed sample stream (sample data) is input to a reference picture memory (434). Since decoding the code stream produces a bit-accurate result independent of the decoder location (local or remote), the contents in the reference picture memory (434) are bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the predictive portion of the encoder are exactly the same as the sample values ​​"seen" when the decoder attempts to utilize the prediction during the decoding period. The fundamental principles of the synchronization of the reference picture (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) also apply to related fields.

[0060] The operation of the “local” decoder (433) may be the same as the operation of a “remote” decoder, such as the video decoder (310) described in detail by combining Figure 3 above. However, referring briefly to Figure 3, if the codes are available and the entropy encoder (445) and parser (320) can losslessly encode / decode the codes into an encoded video sequence, the entropy decoding portion of the video decoder (310), including the buffer memory (315) and parser (320), may not be fully realized in the local decoder (433).

[0061] In this case, any decoder techniques other than analysis / entropy decoding present in the decoder also inevitably exist in the corresponding encoder in the form of essentially the same function. Therefore, the theme of the disclosure focuses on the operation of the decoder. This is in contrast to the decoder techniques described in full with the encoder techniques, and thus the description of the encoder techniques can be simplified. Only specific areas require a more detailed description, which is provided below.

[0062] During the operation period, in some examples, the source encoder (430) can perform motion-compensated predictive coding, which references one or more previously coded pictures from a video sequence designated as “reference pictures” and performs predictive coding on the input picture. In this scheme, the coding engine (432) codes on the difference between the pixel blocks of the input picture and the pixel blocks selected as the (one or more) reference pictures of the (one or more) predictive references of the input picture.

[0063] The local video decoder (433) can decode the encoded video data of a picture that can be designated as a reference picture based on the code constructed by the source encoder (430). The operation of the encoding engine (432) is preferably lossy. Once the encoded video data can be decoded by the video decoder (not shown in Figure 4), the reconstructed video sequence may generally be a copy of the source video sequence with some degree of error. The local video decoder (433) copies the decoding process that the video decoder performs on the reference picture and stores the reconstructed reference picture in the reference picture cache (434). In this scheme, the video encoder (403) locally stores a copy of the reconstructed reference picture, and this copy has common content (no transmission error) with the reconstructed reference picture obtained by the remote video decoder.

[0064] The predictor (435) can perform a predictive search on the encoding engine (432). That is, for a new picture to be encoded, the predictor (435) searches the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as a reference picture motion vector or block shape, of a suitable predictive reference for the new picture. Based on the sample blocks, the predictor (435) can find a suitable predictive reference by operating on each pixel block. In some situations, the input picture may have predictive references obtained from multiple reference pictures stored in the reference picture memory (434), as determined based on the search results obtained by the predictor (435).

[0065] The controller (450) can manage the encoding operations of the source encoder (430), for example, by setting parameters and subgroup parameters for encoding video data.

[0066] The entropy encoder (445) can perform entropy coding on the outputs of all the functional units mentioned above. The entropy encoder (445) converts the codes into an encoded video sequence by performing lossless compression on the codes generated from the various functional units, based on techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0067] The transmitter (440) buffers (one or more) encoded video sequences constructed by the entropy encoder (445) to prepare them for transmission over the communication channel (460), which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter (440) merges the encoded video data from the video encoder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0068] The controller (450) can manage the operation of the video encoder (403). During the encoding period, the controller (450) assigns a specific encoding picture type to each encoded picture, which can affect the encoding technique that may be applied to each image. For example, generally, a picture is assigned one of the following picture types:

[0069] An in-frame picture (I-picture) may be a picture that is encoded and decoded without using any other picture in the sequence as a source for prediction. Some video codecs allow different types of in-frame pictures, including, for example, independent decoder refresh ("IDR") pictures. Those skilled in the art will be able to understand the variants of I-pictures and their corresponding applications and characteristics.

[0070] A predictive picture (P-picture) may be a picture that is encoded and decoded using intra-frame prediction or inter-frame prediction, the intra-frame or inter-frame prediction using at most one motion vector and reference index to predict the sample value of each block.

[0071] A bidirectional predictive picture (B-picture) may be a picture that is encoded and decoded using intra-frame or inter-frame prediction, which predicts the sample values ​​of each block using at most two motion vectors and reference indices. Similarly, a multi-predictive picture can use more than two reference pictures and associated metadata to reconstruct a single block.

[0072] A source picture can generally be subdivided in space into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples) and each block can be encoded. These blocks are predictively encoded by referencing other (already encoded) blocks, which are determined by the encoding assignment applied to the corresponding picture of the block. For example, a block of picture I may be non-predictively encoded, or it may be predictively encoded (spatial or intra-frame) by referencing an already encoded block of the same picture. A pixel block of picture P may be predictively encoded via spatial or temporal prediction by referencing one previously encoded reference picture. A block of picture B may be predictively encoded via spatial or temporal prediction by referencing one or two previously encoded reference pictures.

[0073] The video encoder (403) can perform encoding operations based on, for example, the specified video encoding techniques or standards in the ITU-T H.265 proposal. In its operations, the video encoder (403) can perform various compression operations, including predictive encoding operations that utilize temporal and spatial redundancy in the input video sequence. Thus, the encoded video data conforms to the grammar specified by the video encoding technique or standard used.

[0074] In the embodiment, the transmitter (440) can transmit additional data and encoded video. The source encoder (430) may include such data as part of the encoded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, supplemental enhancement information (SEI) messages, video usability information (VUI) parameter set segments, and the like.

[0075] Video can be captured as multiple source pictures (video pictures) in chronological order. In-frame picture prediction (generally simplified to intra-frame prediction) utilizes spatial relationships in a particular picture, while inter-frame picture prediction utilizes (temporal or other) relationships between pictures. In the example, a particular picture in encoding / decoding, called the current picture, is divided into blocks. If the blocks in the current picture are analogous to reference blocks in a previously encoded and still buffered reference picture in the video, then the blocks in the current picture can be encoded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension for recognizing the reference picture if multiple reference pictures are used.

[0076] In some embodiments, bidirectional prediction techniques are used for inter-frame picture prediction. Based on bidirectional prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used, both of which, in decoding order, precede the current picture in the video (however, in display order, they may be past and future, respectively). A block in the current picture can be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The combination of the first and second reference blocks allows for block prediction.

[0077] Furthermore, merge mode technology improves encoding efficiency by being used for inter-frame picture prediction.

[0078] According to some embodiments of the present disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed block by block. For example, according to the HEVC standard, a picture in a video picture sequence is divided into coded tree units (CTUs) used for compression, and the CTUs in a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU contains three coded tree blocks (CTBs) that include one luminance CTB and two chrominance CTBs. Each CTU is recursively divided into one or more coded units (CUs) in a quad tree. For example, a CTU that is 64x64 pixels is divided into one CU that is 64x64 pixels, or four CUs that are 32x32 pixels, or sixteen CUs that are 16x16 pixels. In the example, by analyzing each CU, the prediction type used for the CU, such as inter-frame prediction type or intra-frame prediction type, is determined. Depending on the temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Generally, each PU includes a luminance prediction block (PB) and two chrominance PBs. In embodiments, the prediction operation in encoding (encoding / decoding) is performed for each prediction block. An example of a prediction block is a luminance prediction block, which includes a matrix of pixel values ​​(e.g., luminance values) such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0079] Figure 5 shows a diagram of a video encoder (503) according to another embodiment of the present disclosure. The video encoder (503) is configured to receive a processing block (e.g., a prediction block) of sample values ​​in the current video picture in a video picture sequence and to encode the processing block into an encoded picture as part of an encoded video sequence. In the example, the video encoder (503) is used in place of the video encoder (203) in the example of Figure 2.

[0080] In the HEVC example, the video encoder (503) receives a matrix of sample values, such as a processing block, for example, an 8x8 sample prediction block. The video encoder (503) determines, for example by rate distortion optimization, whether to best encode the processing block using an intra-frame mode, an inter-frame mode, or a bidirectional prediction mode. If the processing block is to be encoded in intra-frame mode, the video encoder (503) encodes the processing block into an encoded picture using an intra-frame prediction technique. If the processing block is to be encoded in inter-frame mode or a bidirectional prediction mode, the video encoder (503) can encode the processing block into an encoded picture using an inter-frame prediction technique or a bidirectional prediction technique, respectively. In some video encoding techniques, the merge mode may be an inter-frame picture prediction submode, and if no encoded motion vector component outside the prediction values ​​is borrowed, the motion vector is derived from one or more motion vector predictors. In some other video encoding techniques, there may be a motion vector component applied to the theme block. In the example, the video encoder (503) includes other components, such as a mode determination module (not shown) for determining the mode of a processing block.

[0081] In the example shown in Figure 5, the video encoder (503) includes a concatenated interframe encoder (530), an intraframe encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general-purpose controller (521), and an entropy encoder (525), as shown in Figure 5.

[0082] The interframe encoder (530) is configured to receive a sample of the current block (e.g., a processing block), compare the block with one or more reference blocks in the reference picture (e.g., blocks in the previous and subsequent pictures), generate interframe prediction information (e.g., motion vectors, merge mode information, descriptions of redundant information by interframe coding techniques), and compute an interframe prediction result (e.g., a block of predictions) using any appropriate technique based on the interframe prediction information. In some examples, the reference picture is a decoded reference picture that has been decoded based on encoded video information.

[0083] The in-frame encoder (522) is configured to receive a sample of the current block (e.g., a processing block), compare it in some cases with an encoded block in the same picture as the block, transform it, generate quantization coefficients, and in some situations generate in-frame prediction information (e.g., in-frame prediction direction information by one or more in-frame coding techniques). In one example, the in-frame encoder (522) further calculates an in-frame prediction result (e.g., a prediction block) based on the in-frame prediction information and a reference block in the same picture.

[0084] The general-purpose controller (521) is configured to determine general-purpose control data and to control other components of the video encoder (503) based on the general-purpose control data. In one example, the general-purpose controller (521) determines the mode of a block and provides control signals to the switch (526) based on that mode. For example, if the mode is an in-frame mode, the general-purpose controller (521) controls the switch (526) to select the in-frame mode result using the residual calculator (523), controls the entropy encoder (525) to select the in-frame prediction information, and includes the in-frame prediction information in the bitstream. If the mode is an inter-frame mode, the general-purpose controller (521) controls the switch (526) to select the inter-frame prediction result using the residual calculator (523), controls the entropy encoder (525) to select the inter-frame prediction information, and includes the inter-frame prediction information in the bitstream.

[0085] A residual calculator (523) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the in-frame encoder (522) or inter-frame encoder (530). A residual encoder (524) is configured to operate on the residual data and generate transformation coefficients by encoding the residual data. In one example, the residual encoder (524) is configured to transform the residual data from the spatial domain to the frequency domain and generate transformation coefficients. The coefficients are transformed and then, through a quantization process, the quantized transformation coefficients are obtained. In each embodiment, the video encoder (503) further includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data is appropriately available to the in-frame encoder (522) and inter-frame encoder (530). For example, an interframe encoder (530) can generate a decoded block based on the decoded residual data and interframe prediction information, and an intraframe encoder (522) can generate a decoded block based on the decoded residual data and intraframe prediction information. In some examples, the decoded block is processed appropriately to generate a decoded picture, which is buffered in a memory circuit (not shown) and can be used as a reference picture.

[0086] The entropy encoder (525) is configured to format the bitstream to include the encoded blocks. The entropy encoder (525) is configured to include various information based on an appropriate standard such as the HEVC standard. In the example, the entropy encoder (525) is configured to include general control data, selection prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other appropriate information in the bitstream. Note that, based on the theme of disclosure, if blocks are encoded in inter-frame mode or bidirectional prediction mode merge submode, residual information is not present.

[0087] Figure 6 shows a diagram of a video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive an encoded picture as part of an encoded video sequence and to decode the encoded picture to produce a reconstructed picture. In this example, the video decoder (610) is used in place of the video decoder (210) in the example of Figure 2.

[0088] In the example shown in Figure 6, the video decoder (610) includes a coupled entropy decoder (671), interframe decoder (680), residual decoder (673), reconstruction module (674), and intraframe decoder (672), as shown in Figure 6.

[0089] The entropy decoder (671) is configured to reconstruct several codes representing the grammatical elements that make up the encoded picture, based on the encoded picture. Such codes include, for example, a mode for encoding a block (e.g., an in-frame mode, an inter-frame mode, a bidirectional prediction mode, a later merge submode of both, or other submodes), prediction information (e.g., in-frame prediction information or inter-frame prediction information) that can recognize several samples or metadata for prediction, which are used by the in-frame decoder (672) or inter-frame decoder (680), respectively, and residual information, for example, in the form of quantization conversion coefficients. In the example, if the prediction mode is an inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (680), and if the prediction type is an in-frame prediction type, in-frame prediction information is provided to the in-frame decoder (672). Residual information is provided to the residual decoder (673) via inverse quantization.

[0090] The interframe decoder (680) is configured to receive interframe prediction information and generate interframe prediction results based on the interframe prediction information.

[0091] The in-frame decoder (672) is configured to receive in-frame prediction information and generate prediction results based on that information.

[0092] The residual decoder (673) is configured to perform inverse quantization, extract the inversely quantized transformation coefficients, and then convert the residual from the frequency domain to the spatial domain by processing the inversely quantized transformation coefficients. The residual decoder (673) may require some control information (including quantizer parameters (QP)), which is provided by the entropy decoder (671) (this is minimal control information, so the data path is not shown).

[0093] The reconstruction module (674) is positioned in the spatial domain to form a reconstruction block by combining the residual output from the residual decoder (673) with the prediction result (which may optionally be output from an inter-frame prediction module or an intra-frame prediction module), and the reconstruction block may be part of the picture to be reconstructed, and the picture to be reconstructed may be part of the video to be reconstructed. Visual quality may be improved by performing other appropriate operations, such as a deblocking operation.

[0094] Furthermore, the video encoder (203), video encoder (403), video encoder (503), and video decoder (210), video decoder (310), and video decoder (610) can be implemented using any suitable technology. In one embodiment, the video encoder (203), video encoder (403), video encoder (503), and video decoder (210), video decoder (310), and video decoder (610) can be implemented via one or more integrated circuits. In other embodiments, the video decoders (203), (403), (403), and video decoders (210), (310), and (610) are implemented by one or more processors for executing software instructions.

[0095] Each aspect of this disclosure provides techniques for restricting the reference block position in an in-frame block copy mode, and further provides techniques for reusing an in-frame picture block compensation buffer with a non-square block partition.

[0096] Block-based compensation may be used for inter-frame and intra-frame predictions. In the case of inter-frame predictions, block-based compensation from different pictures is called motion compensation. In the case of intra-frame predictions, block-based compensation may be performed on previously reconstructed regions within the same picture. Block-based compensation for reconstructed regions from the same picture is called intra-frame picture block compensation, current picture referencing (CPR), or intra-frame block copy (IBC). The displacement vector that indicates the offset between the current block and the reference block in the same picture is called the block vector (or abbreviated as BV). Unlike the motion vector in motion compensation, which can be any value (positive or negative in the x or y direction), the block vector is constrained to ensure that the reference block is available and has already been reconstructed. Also, in some examples, when considering parallel processing, some reference regions that are tile boundaries or wavefront ladder shape boundaries are excluded.

[0097] Block vectors may be coded explicitly or implicitly. In explicit mode (similar to the Advanced Motion Vector Prediction (AMVP) mode in interframe coding), the difference between the block vector and its predicted value is signaled, while in implicit mode, the block vector is reconstructed from the predicted value (called the block vector predicted value), similar to motion vectors in merge mode. In some implementations, the resolution of the block vector is limited to integer positions, while in other systems, the block vector is allowed to point to decimal positions.

[0098] In some embodiments, a block-level flag (IBC flag) may be used to signal the use of block-level intra-frame block copying. In one embodiment, the IBC flag is signaled when the current block is not encoded in merge mode. In another embodiment, the IBC flag is signaled using a reference index method. The current picture being decoded is then treated as a reference picture. In one example, such a reference picture is set to the last position in the list of reference pictures. This particular reference picture is further managed in a buffer, such as a decoded picture buffer (DPB), along with other time reference pictures.

[0099] There are also several variants of intraframe block copying, such as flipped intraframe block copying (where the reference block is flipped horizontally or vertically before being used to predict the current block), or row-based intraframe block copying (where each compensation unit in an M×N coded block is M×1 or 1×N rows).

[0100] Figure 7 shows an example of an in-frame block copy according to an embodiment of the present disclosure. The current picture (700) is being decoded. The current picture (700) includes the reconstructed region (710) (gray region) and the region to be decoded (720) (white region). The current block (730) is being reconstructed by the decoder. The current block (730) can be reconstructed according to the reference block (740) in the reconstructed region (710). The positional offset between the reference block (740) and the current block (730) is called the block vector (750) (or BV(750)).

[0101] In some implementation examples, reconstructed samples of the current picture may be buffered in dedicated memory. Considering the implementation cost, the reference area cannot be as large as the full frame and is limited to the storage size of the dedicated memory. In some embodiments, block vectors in an in-frame block copy are permitted to reference only a few adjacent areas, rather than the full picture. In one example, if the memory size of the dedicated memory is one CTU and the referenced block is in the same CTU as the current block, the in-frame block copy mode may be used. In another example, if the memory size is two CTUs and the referenced block is in the current CTU or in the CTU to the left of the current CTU, the in-frame block copy mode may be used. In one example, if the referenced block is outside a specified local area, the referenced sample cannot be used for in-frame picture block compensation, even if the referenced block has already been reconstructed.

[0102] In a constrained reference region, the efficiency of intra-frame block copying is limited. Further improvements are needed to improve the efficiency of intra-frame block copying in a constrained reference region.

[0103] In some versions of video encoding standards, for example, in the VVC version, a picture is divided into an array of non-overlapping CTUs. The size of a CTU is set at 128 x 128 luminance samples (and corresponding chroma samples depending on the color format). For example, a CTU may be divided into encoding units (CUs) using one or a combination of tree partitioning methods such as binary tree (BT) partitioning or ternary tree (TT) partitioning. BT partitioning divides the parent block in half horizontally or vertically. Compared to the parent block, the two smaller partitions obtained by BT partitioning are half the size. TT partitioning divides the parent block into three parts horizontally or vertically. The middle part of these three parts is twice the size of the other two parts. Compared to the parent block, the three smaller partitions obtained by TT partitioning are 1 / 4, 1 / 2, and 1 / 4 the size.

[0104] In some embodiments, the division of a 128x128 level block is constrained to allow several outcomes, for example, one 128x128 block, two 128x64 blocks, two 64x128 blocks, or four 64x64 blocks. In some examples, the division is further constrained so that TT divisions in either direction (horizontal or vertical) are not allowed at the 128x64 or 64x128 block level. Also, in the examples, if a 128x64 or 64x128 is further divided, it is only allowed to be divided into two 64x64 blocks. The constrained division type is called the normal division type.

[0105] In some examples, in-frame block copying is implemented with limited memory requirements. In the implementation examples, the designated memory that stores reference samples of previously encoded CUs and is used for future in-frame block copy references is called reference sample memory. For example, the maximum size of the reference sample memory (also called a buffer) for storing unfiltered samples in in-frame block copy mode is 1 CTU (128 x 128 luminance samples and corresponding chroma samples) size. To make full use of such constrained memory, in some examples, the buffer is updated every 64 x 64. In the example, if some of the reference sample memory has not yet been updated with reconstructed samples from the current CTU, then some of the data stored in memory is actually reconstructed samples from the left CTU. These samples are used as references to predict the current block in in-frame block copy mode, and the effective search range is extended from within the current CTU to a portion of the left CTU.

[0106] Therefore, in some embodiments, the reference sample memory may be reused to store previously encoded samples in the left CTU before updating the reference sample memory with reconstructed samples in the current CTU. In embodiments, the availability of reference sample memory is updated in 64x64 increments. In an example, if any sample of the reference sample in a given 64x64 block is updated, the entire 64x64 block is considered updated with samples reconstructed from the current CTU and is not considered to store the sample before the update. In some implementation examples, the addressing operations of allocating contiguous memory space in memory (e.g., buffer memory, on-chip memory, etc.) to store reconstructed samples in a 64x64 block, and storing and reading back reconstructed samples from contiguous memory space, can be performed with relatively simple calculations. In some examples, the contiguous memory space storing reconstructed samples of an encoded region (e.g., a 64x64 encoded region) is called a memory update unit. Access to samples in a memory update unit may be performed using reduced addressing calculations, and accessing samples in a different memory update unit requires an increase in the number of addressing calculations.

[0107] Figures 8A to 8D illustrate examples of 64x64 reference sample memory updates and effective search ranges in frame block copy mode according to embodiments of the present disclosure. In some examples, the encoder / decoder includes cache memory capable of storing samples from one CTU (e.g., 128x128 samples). Also, in the examples in Figures 8A to 8D, the CTU is divided into four 64x64 coding regions. The coding region with the current block for prediction (which may be equal to or smaller than the coding region) is called the current coding region (denoted "Curr"). Note that the examples may be appropriately modified for coding regions of other suitable sizes.

[0108] Figures 8A to 8D show the current CTU(820) and the left CTU(810), respectively. The left CTU(810) contains four coded regions (811) to (814), and each coded region has a sample size of 64 × 64 samples. The current CTU(820) contains four coded regions (821) to (824), and each coded region has a sample size of 64 × 64 samples. The current CTU(820) contains a CTU that includes the current coded region being reconstructed (for example, having a vertical stripe pattern as shown in the label "Curr"). The left CTU(810) is adjacent to the left of the current CTU(820). In Figures 8A to 8D, gray blocks are coded regions that have already been reconstructed, and white blocks are coded regions to be reconstructed.

[0109] In Figure 8A, the currently reconstructed coding region is coding region (821). The cache memory stores the reconstructed samples in coding regions (812), (813), and (814), and is also used to store the reconstructed samples in the current coding region (821). In the example in Figure 8A, the effective search range of the current coding region (821) includes coding regions (812), (813), and (814) in the left CTU (810), and the reconstructed samples in these regions are stored in the cache memory. In the example, if the cache memory is updated using one or more reconstructed samples from the current coding region (821), a portion of coding region (811) has already been updated with reconstructed samples from the current CTU, and therefore coding region (811) is considered unavailable for CPR prediction in the cache memory (marked "X"). In this embodiment, it should be noted that the reconstructed samples of the encoding region (811) are stored in main memory, which has a slower access speed than the cache memory (for example, they are copied from the cache memory to main memory before the encoding region (821) is reconstructed).

[0110] In Figure 8B, the currently reconstructed coding region is coding region (822). The cache memory stores the reconstructed samples in coding regions (813), (814), and (821), and is used to store the reconstructed samples in the current coding region (822). In the example in Figure 8B, the effective search range of the current coding region (822) includes coding regions (813) and (814) in the left CTU (810), and coding region (821) in the current CTU (820), and the reconstructed samples in these regions are stored in the cache memory. In the example, if the cache memory is updated using one or more reconstructed samples from the current coding region (822), a portion of coding region (812) has already been updated with reconstructed samples from the current CTU, and therefore coding region (812) is considered unavailable for CPR prediction in the cache memory (marked "X"). In this embodiment, it should be noted that the reconstructed samples of the encoding region (812) are stored in main memory, which has a slower access speed than the cache memory (for example, they are copied from the cache memory to the main memory before the encoding region (822) is reconstructed).

[0111] In Figure 8C, the currently reconstructed coding region is coding region (823). The cache memory stores the reconstructed samples in coding regions (814), (821), and (822), and is used to store the reconstructed samples in the current coding region (823). In the example in Figure 8C, the effective search range of the current coding region (823) includes coding region (814) in the left CTU (810), and coding regions (821) and (822) in the current CTU (820), and the reconstructed samples in these regions are stored in the cache memory. In the example, if the cache memory is updated using one or more reconstructed samples from the current coding region (823), a portion of coding region (813) has already been updated with reconstructed samples from the current CTU, and therefore coding region (813) is considered unavailable for CPR prediction in the cache memory (marked "X"). It should be noted that in this embodiment, the reconstructed sample of the encoding region (813) is stored in main memory, which has a slower access speed than the cache memory (for example, it is copied from the cache memory to main memory before the encoding region (823) is reconstructed).

[0112] In Figure 8D, the current coding region being reconstructed is coding region (824). The cache memory stores the reconstructed samples in coding regions (821), (822), and (823), and is used to store the reconstructed samples in the current coding region (824). In the example in Figure 8D, the effective search range of the current coding region (824) includes coding regions (821), (822), and (823) in the current CTU (820), and the reconstructed samples in these regions are stored in the cache memory. In the example, if the cache memory is updated using one or more reconstructed samples from the current coding region (824), a portion of coding region (814) has already been updated with reconstructed samples from the current CTU, and therefore coding region (814) is considered unavailable for CPR prediction in the cache memory (marked "X"). In this embodiment, it should be noted that the reconstructed samples of the encoding region (814) are stored in main memory, which has a slower access speed than the cache memory (for example, they are copied from the cache memory to main memory before the encoding region (824) is reconstructed).

[0113] In the examples in Figures 8A to 8D, the total memory of 4 × 64 × 64 luminance samples (along with the corresponding chroma samples) is used for in-frame block copy compensation. Note that the technique used in the examples in Figures 8A to 8D can be appropriately modified for other memory sizes (e.g., total memory of 3 × 64 × 64, 2 × 64 × 64 luminance samples).

[0114] Note that in some embodiments, ternary tree partitioning at 128x128 levels and lower levels is permitted, and the resulting partitions may be non-square blocks.

[0115] In some examples, the CTU may be partitioned using ternary tree partitioning at the 128x128 level. In the case of horizontal TT partitioning, the partitioned blocks are 128x32, 128x64, and 128x32. In the case of vertical partitioning, the partitioned blocks are 32x128, 64x128, and 32x128.

[0116] Furthermore, in some examples, a ternary tree partition may be used at the 128×64 / 64×128 level, and the resulting blocks may be non-square blocks.

[0117] Figures 9A to 9H show examples of non-square block partitioning.

[0118] According to certain aspects of the content of this disclosure, certain partitioning types, such as 128x128 level TT partitioning, may result in coded blocks that are not included in a 64x64 coded area or that cannot include one or more 64x64 coded areas.

[0119] If these partition types are allowed on top of the usual partition types, the reference sample update process may no longer be one-to-one aligned with the 64x64 encoded area. A method needs to be developed to implement the reference sample memory cache reuse strategy mentioned above.

[0120] According to one aspect of this disclosure, the performance of intra-frame block copying can be improved by using several reference area constraints. More specifically, in the example, the location of a reference sample block in the reference sample memory is constrained to the same 64x64 encoded area. In some embodiments, the size of the reference sample memory is fixed to two 64x64 luminance samples (along with the corresponding chroma samples). It should be noted that the techniques used in this disclosure can be appropriately modified for use with different reference sample memory sizes, such as memory sizes of three 64x64 luminance samples and corresponding chroma samples, or four 64x64 luminance samples and corresponding chroma samples (i.e., 1 CTU size).

[0121] In general, the size of an IBC coded block may be the same as any ordinary interframe coded block. In one aspect of this disclosure, the size of the IBC coded block is limited to make more efficient use of the reference sample memory. In an example, the size of the IBC coded block is limited to an upper limit, e.g., width or height, not exceeding 64 luminance samples (the corresponding size constraint applies to chroma samples, depending on the color format, e.g., 4:2:0 format, where the size of the chroma block in IBC mode must not exceed 32 samples on each side). Also, in some examples, the size of the IBC coded block is limited to a lower limit, e.g., width or height, not exceeding 32 luminance samples. While the maximum IBC size of 64 × 64 luminance samples is used in this description, it should be noted that the technique can be appropriately modified for other maximum IBC sizes.

[0122] In the following explanation, the maximum size of the reference sample memory for storing in-frame block copy reference samples corresponds to the memory size for storing two 64x64 luminance samples and their corresponding chroma samples.

[0123] According to some aspects of the present disclosure, a reference sample memory has a size for storing sample information for two encoded regions, each encoded region having a size of 64 × 64 luminance samples, and the sample information includes luminance sample information for 64 × 64 luminance samples and chroma sample information for corresponding chroma samples. The reference sample memory then stores sample information for the current encoded region in the current CTU, and sample information for another encoded region which is one of several previously encoded 64 × 64 encoded regions.

[0124] Figures 10A and 10B show two examples of coding order and the use of reference sample memory. In Figures 10A and 10B, the gray areas are coding regions that have already been coded (e.g., 64 × 64 luminance samples), the white areas are regions that have not yet been coded, and the (one or more) shaded areas are coding regions where the current coding block is located. Also in Figures 10A and 10B, the two coding regions in the reference sample memory are indicated by (one or more) dotted boxes.

[0125] In Figure 10A, in the example, the CTU performs horizontal binary partitioning or quadtree partitioning at the 128x128 level. The coding order of the 64x64 coded region follows (1001) to (1004).

[0126] In Figure 10B, in the example, the CTU performs vertical binary partitioning at a 128x128 level. The coding order of the 64x64 coded region follows (1011) to (1014).

[0127] In the examples in Figures 10A and 10B, if the current encoded block is located in one of the four 64x64 encoded regions in the current CTU, the reference sample memory may store another 64x64 encoded region that is allocated as a reference for the intraframe block copy mode.

[0128] Figures 10A and 10B are examples, and it should be noted that other possible reference area assignments are also possible. In one embodiment, the upper right 64x64 encoded area is used as the reference area for the lower right 64x64 encoded area, as shown in Figures 10A and 10B. In another embodiment, the upper left 64x64 encoded area is used as the reference area for the lower right 64x64 encoded area.

[0129] Ideally, the reconstructed samples in the reference sample memory may be used to predict the current block in an in-frame block copy mode. However, the samples in the reference block may come from two different 64x64 encoded regions.

[0130] Figures 11A and 11B show examples of samples in a reference block from two different 64x64 coding regions. In Figure 11A, part of the reference block is from one 64x64 coding region, and another part belongs to another 64x64 previously coding region that has already been reconstructed. In Figure 11B, part of the reference block is from one 64x64 coding region, and another part belongs to the current 64x64 coding region that has already been reconstructed. Since each of the 64x64 coding regions in the reference sample memory is operated separately, the two 64x64 coding regions in memory cannot be physically connected. These locations of the reference block in different coding regions, as shown in Figures 11A and 11B, can result in undesirable extra memory access or operation.

[0131] In some aspects of this disclosure, additional constraints are used in conjunction with other appropriate block vector constraints to enable in-frame block copying to function in a particular video / image encoding system. By imposing additional constraints on the block vectors for the in-frame block copy mode, it is ensured that the entire reference block for the in-frame block copy mode resides within the same 64x64 encoded region. A 64x64 encoded region means that each CTU with 128x128 luminance samples (plus the corresponding chroma samples) can be divided into four non-overlapping 64x64 regions. In some examples, each of these 64x64 regions is considered a 64x64 encoded region under the constraints.

[0132] In this embodiment, for example, assuming the top-most corner of the CTU is (0,0), the upper-left corner of the reference block is indicated as (Ref_TLx,Ref_TLy) and the lower-right corner of the reference block is indicated as (Ref_BRx,Ref_BRy). Additional constraints include adopting a format that must satisfy the bitstream consistency requirements of the following conditions (Equation 1) and (Equation 2). Ref_BRx / 64=Ref_TLx / 64 (Formula 1) Ref_BRy / 64 = Ref_TLy / 64 (Equation 2)

[0133] If (Equation 1) is satisfied, the integer part of Ref_BRx / 64 is equal to the integer part of Ref_TLx / 64. If (Equation 2) is satisfied, the integer part of Ref_BRy / 64 is equal to the integer part of Ref_TLy / 64. Note that the division operation may also be performed by shifting 6 bits to the right.

[0134] In some embodiments, if a block vector points to a reference block location where not all samples within the reference block belong to the same 64x64 encoded region, the block vector may be clipped to a nearby reference block location that satisfies the same 64x64 encoded region requirement. In embodiments, clipping refers to moving the original reference block location (which covers at least two 64x64 regions) vertically or horizontally in one direction. In the example, the movement is in a direction that decreases the distance between the reference block and the current block. Alternatively, the movement is in a direction that increases the percentage of reference samples belonging to the same 64x64 sample region, ultimately causing the entire reference block to belong to the same 64x64 region.

[0135] In the example shown in Figure 11A, the reference block can be moved vertically upward until all samples belong to the upper 64x64 coding region. Note that in the example shown in Figure 11A, moving the reference block downward may increase the distance between the two blocks, while moving the reference block horizontally does not change the percentage of samples in each 64x64 coding region.

[0136] In the example shown in Figure 11B, the reference block can be moved horizontally to the right until all samples belong to the current 64x64 coding area. Note that in the example shown in Figure 11B, moving the reference block to the left may increase the distance between the two blocks, while moving the reference block vertically does not change the percentage of samples in each 64x64 coding area.

[0137] In some embodiments, if the block vector points to a reference block location where not all samples within the reference block belong to the same 64x64 coding region, sample padding (copying samples vertically and / or horizontally at the boundaries of the 64x64 coding region) may generate samples outside the 64x64 coding region. The padding process can be performed in a manner similar to the boundary extension of a reference picture in motion compensation.

[0138] In the example, the upper-left corner of the reference block is located within a 64x64 coding region, and this 64x64 coding region is considered the coding region from which the reference block should originate. If any part of the reference block lies outside this coding region, padding can be used to determine the sample for that part.

[0139] In another example, the lower right corner of the reference block is located within a 64x64 coding region, and this 64x64 coding region is considered the coding region from which the reference block should originate. If any part of the reference block lies outside this coding region, padding can be used to determine the sample of that part.

[0140] In another example, the upper-left corner of the reference block is located in a 64x64 encoding region, and first, the encoding region from which the reference block originates is determined using this 64x64 encoding region. If the upper-left corner is not a valid reference location, the lower-right corner of the reference block is considered to determine the encoding region from which the reference block originates.

[0141] In another example, the lower right corner of the reference block is located in a 64x64 encoding region, and first, the encoding region from which the reference block originates is determined using this 64x64 encoding region. If the lower right corner is not a valid reference location, the upper left corner of the reference block is considered to determine the encoding region from which the reference block originates.

[0142] In another embodiment, if the block vector points to a reference block location where all samples within the reference block are outside the valid reference region (the current 64x64 encoded region or one or more previously encoded 64x64 encoded regions), sample padding (vertically and / or horizontally copying samples at the nearest valid reference region boundary) may generate samples outside the 64x64 encoded region. The padding process may be performed similarly to the boundary extension of a reference picture in motion compensation, where the boundary can be considered the boundary of one or more permitted reference regions.

[0143] Each aspect of this disclosure further provides a flexible block partitioning strategy and a buffer reuse mechanism for in-frame block copying.

[0144] In some embodiments, when using in-frame block copying for a given encoding level (e.g., sequence, picture, or slice / tile / tile group), flexible block division types such as 128×32 / 32×128 cannot be used.

[0145] In embodiments where a flexible block partitioning type is not used for a given encoding level (e.g., sequence, picture, or slice / tile / tile group), the in-frame block copy mode may use a reference sample memory reuse strategy. For example, an update and buffering reuse strategy based on a 64x64 encoding area may be used.

[0146] In embodiments, when using a flexible block partitioning type for a given encoding level (e.g., sequence, picture, or slice / tile / tile group), the in-frame block copy mode cannot use the reference sample memory reuse strategies disclosed above. However, in the example, the block vector is restricted to referencing the reconstructed portion of the current CTU.

[0147] In embodiments, the reference sample memory reuse strategy described above may be modified when using the flexible block partitioning type described above for a given encoding level (e.g., sequence, picture, or slice / tile / tile group).

[0148] In some cases, when the encoded block is in a first TT partition (e.g., left 32x128 or top 128x32) at the CTU (128x128) level, or a BT partition (top 128x64 or left 64x128), or a QT partition (top left 64x64), a portion of the reference samples in the left CTU can be used.

[0149] Figures 12A to 12F show examples of reference sample memory reuse strategies. In each example shown in Figures 12A to 12F, the current encoded block is indicated by diagonal lines, and the available reference area is indicated by gray.

[0150] In the example, the 64x64 (sample) area in the upper right corner of the left CTU is the available reference area, which is shown in gray in the examples in Figures 12A and 12C.

[0151] In another example, the rightmost 32 × 128 (sample) area in the left CTU is the available reference area, shown in gray in the examples in Figures 12D to 12F.

[0152] In another example, the top 128x32 area within the left CTU is the available reference area.

[0153] In another example, the selection of a region is determined by the partition of the left CTU. For example, if the left CTU uses a 128x128 level quadtree partition, the upper right 64x64 (sample) region of the left CTU may be used to predict the encoded block in the first partition of the current CTU.

[0154] In another example, if a total of 3x64x64 size memory is used for the encoded blocks in the first partition of the current CTU, the size of the memory for storing the reference samples from the left CTU is 64x64 (not necessarily square). Memory space does not necessarily store square-shaped samples.

[0155] In another embodiment, if a total of 4x64x64 size memory is used for the encoded blocks in the first division of the current CTU, the size of the memory for storing the reference samples from the left CTU is 2x64x64 (not necessarily two squares). The memory space does not necessarily store two square samples.

[0156] In some embodiments, using both flexible block partitioning type and in-frame block copy mode at a given encoding level (e.g., for sequences, pictures, or slices / tiles / tile groups) allows the technique to reuse reference sample memory, effectively increasing the search range of in-frame block copies while maintaining or reducing storage requirements.

[0157] In some examples, instead of using the previous 64x64-based update process, we propose a smaller or same-granularity update process using a larger block shape (e.g., 32x32 or 32x64 or 64x32 or 16x128 shapes). In the examples, the update process may use 32x32 (sample) units, or it may divide the current CTU using a 128x128 level TT.

[0158] Figures 13A to 13C show an example of an update process using 32×32 (sample) units, and Figures 13D to 13F show another example of an update process using 32×32 (samples). Each 32×32 (sample) unit is shown as a small square. In the examples in Figures 13A to 13F, the total reference sample memory size is 128×128. Then, a first memory space of 3×64×64 samples is allocated to store the reference samples intended for in-frame block copy reference, and a second memory space of 64×64 samples is allocated to units of currently 64×64 size (not necessarily squares, but the total number of luminance samples is equal to 64×64 samples). For each 64x64 or 64x128 size division and its subdivisions (not necessarily square, with luminance samples equal to 64x64 samples), the reconstructed samples within the current CTU and several other previously encoded regions (e.g., shown as gray areas in Figures 13A to 13F) may be stored in reference sample memory and can be referenced for predicting in-frame block copies.

[0159] Figure 14 shows a flowchart outlining the process (1400) according to an embodiment of the contents of this disclosure. Process (1400) is used to reconstruct blocks encoded in in-frame mode and generates predicted blocks for the blocks being reconstructed. In various embodiments, process (1400) is executed by processing circuits such as processing circuits in terminal devices (110), (120), (130) and (140), processing circuits that perform the functions of a video encoder (203), processing circuits that perform the functions of a video decoder (210), processing circuits that perform the functions of a video decoder (310), and processing circuits that perform the functions of a video encoder (403). In some embodiments, since process (1400) is implemented by software instructions, when a processing circuit executes a software instruction, the processing circuit executes process (1400). The process starts at (S1401) and is executed up to (S1410).

[0160] In (S1410), the prediction information for the current block is decoded from the encoded video bitstream. The prediction information indicates the intra-frame block copy mode.

[0161] In (S1420), a block vector is determined that points to a reference block in the same picture as the current block.

[0162] In (S1430), the reference block is checked to determine whether the reference block is entirely within the coding region. In some embodiments, a reference sample memory with limited storage space is reused to store previously coded samples. For example, before updating the reference sample memory with reconstructed samples in the current CTU, the reference sample memory stores the samples in the CTU to the left of the current CTU. The reference sample memory is updated with reconstructed samples in units of the coding region in the current CTU. In embodiments, the availability of the reference sample memory is updated in 64x64 increments. In the example, if any of the samples in a 64x64 block of reference samples has already been updated, the entire 64x64 block is considered to be updated with reconstructed samples from the current CTU and is not considered a stored sample before the update. In embodiments, a 128x128 sample CTU includes four 64x64 regions which may be called coding regions.

[0163] In this embodiment, multiple corners of the reference block are checked to determine whether the reference block is entirely within the encoding area. If the reference block is entirely within the encoding area, the process proceeds to (S1450); otherwise, the process proceeds to (S1440).

[0164] In (S1440), the reference block is appropriately restricted to the encoding area. In some embodiments, the reference block is appropriately moved to the encoding area. In some embodiments, some reference samples that are not in the encoding area may be padded with other reference samples that are not in the encoding area. And, in the example, memory access to the reconstructed reference samples of the current block can be achieved by accessing only one memory update unit.

[0165] In (S1450), the current block is reconstructed based on the reconstructed sample of the reference block retrieved from the reference sample memory. For example, the reference sample memory is accessed to retrieve the reconstructed sample of the reference block, and then the sample of the current block is reconstructed based on the reconstructed sample retrieved from the reference sample memory. Since the reference block is restricted to having all of its reference samples in one encoding region or being determined based on one encoding region, the addressing operation to retrieve the reference samples can be performed easily and efficiently. The process then proceeds to (S1499) and terminates.

[0166] The aforementioned technology may be implemented as computer software by computer-readable instructions and may be stored physically on one or more computer-readable media. For example, Figure 15 shows a computer system (1500) for implementing some embodiments of the themes of the disclosure.

[0167] Computer software is encoded in any appropriate machine code or computer language, and the machine code or computer language constructs code containing instructions through mechanisms such as editing, compiling, and linking, and these instructions are executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or by interpretation, microcode execution, etc.

[0168] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, and Internet of Things devices.

[0169] The components of the computer system (1500) shown in Figure 15 are essentially illustrative and not limit the scope or functionality of the computer software used in embodiments to realize the contents of this disclosure. The arrangement of the components should not be interpreted as having any dependence or requirement on any of the components or combinations thereof shown in the exemplary implementation of the computer system (1500).

[0170] The computer system (1500) may include several human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users, for example, tactile input (e.g., keystrokes, slides, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., posture), and olfactory input (not shown). The human-machine interface devices can further capture certain media, such as audio (e.g., voices, music, ambient sounds), images (e.g., scanned images, photographic images acquired from a static image capture device), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0171] The input human-machine interface device may have one or more of the following: keyboard (1501), mouse (1502), touchpad (1503), touch panel (1510), data glove (not shown), joystick (1505), microphone (1506), scanner (1507), and imaging device (1508) (only one of each of the listed).

[0172] The computer system (1500) may further have human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users, for example, through tactile output, sound, light and smell / taste. Such human-machine interface output devices include tactile output devices (e.g., tactile feedback via touch panels (1510), data gloves (not shown) or joysticks (1505), although tactile feedback devices may also exist that are not used as input devices), audio output devices (e.g., speakers (1509), headphones (not shown)), visual output devices (e.g., screens (1510), CRT screens, LCD screens, plasma screens, OLED screens, each screen may or may not have touch panel input capability and tactile feedback capability, some of which output two-dimensional vision or three-dimensional or more-dimensional vision by means such as stereoscopic image output, and include virtual reality glasses (not shown), holographic displays and smoke tanks (not shown)), and printers (not shown).

[0173] The computer system (1500) may further have human-accessible memory devices and related media, including, for example, optical media including CD / DVD ROM / RW (1520) having media such as CD / DVD (1521), thumb drives (1522), removable hard drives or solid-state drives (1523), traditional magnetic media such as magnetic tape and floppy disks (not shown), and devices based on dedicated ROM / ASIC / PLD, such as dongles (not shown).

[0174] Those skilled in the art will understand, by combining the themes of the present disclosure, that the term “computer-readable medium” as used does not include a transmission medium, carrier wave, or other instantaneous signal.

[0175] The computer system (1500) may further have interfaces to one or more communication networks. These networks may be, for example, wireless, wired, or optical. The networks may also be local, wide-area, urban, vehicle, industrial, real-time, or latency-tolerant networks. Examples of networks include local area networks such as Ethernet, cellular networks including wireless LAN, GSM, 3G, 4G, 5G, LTE, etc., wired or wireless wide-area digital television networks including wired television, satellite television, and terrestrial television, and vehicle and industrial networks including CANBus. Some networks generally require an external network interface adapter connected to a general-purpose data port or peripheral bus (1549) (e.g., a USB port on the computer system (1500)), while others are generally integrated into the core of the computer system (1500) by being connected to system buses described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Through any of these networks, the computer system (1500) can communicate with other entities. Such communications can be one-way and receive only (e.g., broadcast television), one-way and transmit only (e.g., CANbus to a CANbus device), or two-way (e.g., to another computer system via a local area or wide area digital network). Specific protocols and protocol stacks can be used for each of these networks and network interfaces described above.

[0176] The human-machine interface devices, human-accessible memory devices, and network interfaces mentioned above can be connected to the core (1540) of the computer system (1500).

[0177] The core (1540) includes one or more central processing units (CPUs) (1541), graphics processing units (GPUs) (1542), specialized programmable processing units in the form of field-programmable gate arrays (FPGAs) (1543), and hardware accelerators (1544) for specific tasks. These devices, along with read-only memory (ROM) (1545), random access memory (1546), and internal mass storage devices such as hard disk drives and SSDs (1547) that are inaccessible to the internal user, are connected via a system bus (1548). In a computer system, it can be expanded by other CPUs, GPUs, etc., by accessing the system bus (1548) in the form of one or more physical plugs. Peripheral devices are connected to the core's system bus (1548) directly or via peripheral buses (1549). Peripheral bus architectures include PCI, USB, etc.

[0178] The CPU (1541), GPU (1542), FPGA (1543), and accelerator (1544) can execute several instructions, and these instructions are combined to form the computer code mentioned above. This computer code is stored in ROM (1545) or RAM (1546). Transition data is stored in RAM (1546), and permanent data may be stored in, for example, an internal mass storage device (1547). Cache memory enables fast storage and retrieval of any of the memory devices, and the cache memory can be closely associated with one or more CPUs (1541), GPUs (1542), mass storage devices (1547), ROMs (1545), RAM (1546), etc.

[0179] A computer-readable medium may contain computer code for performing various operations that a computer can perform. The medium and computer code may be specially designed and constructed for the purposes of the subject matter of this disclosure, or they may be of a type known and available to those skilled in the art of computer software.

[0180] To the present and not to the extent of an example, a computer system having architecture (1500), particularly a core (1540), can provide functionality by having (one or more) processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in one or more tangible computer-readable media. Such computer-readable media may be media relating to user-accessible mass storage devices as described above, and storage devices such as core-internal mass storage devices (1547) or ROM (1545) having a non-temporary core (1540). Software for various embodiments of the present disclosure is stored in such devices and executed by the core (1540). Depending on the specific requirements, the computer-readable media may include one or more storage devices or chips. The software includes causing the core (1540), particularly its processors (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific parts of specific processes as described herein, to limit data configurations stored in RAM (1546), and to modify such data configurations based on the processes limited by the software. Alternatively, a computer system may provide functionality by being embodied in a circuit (e.g., an accelerator (1544)) through a logical fixed connection or other means, and such circuit may, in place of or in conjunction with software, perform a particular process or a particular part of a particular process described herein. Where appropriate, the software referred to may include logic, and conversely, the logic referred to may include software. Where appropriate, the computer-readable medium referred to may include a circuit (e.g., an integrated circuit (IC)) on which software for execution is stored, a circuit embodying logic for execution, or both. The scope of this disclosure includes any appropriate combination of hardware and software. Appendix A: Acronyms JEM: Collaborative Search Model VVC: Multifunctional Video Coding BMS: Benchmark Set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Replenishment Enhancement Information VUI: Video Usability Information GOP: Picture Group TU: Conversion Unit PU: Prediction Unit CTU: Encoding Tree Unit CTB: Encoded Tree Block PB: Prediction Block HRD: Virtual Reference Decoder SNR: Signal-to-Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-only memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global Mobile System of Systems LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnection FPGA: Field-programmable gate array SSD: Solid State Drive IC: Integrated Circuit CU: Encoding Unit

[0181] While this disclosure includes some illustrative examples, there are many variations, substitutions, and alternative equivalents that fall within the scope of this disclosure. Therefore, many systems and methods that embody the principles of this disclosure and fall within the spirit and scope of this disclosure, although not explicitly described herein, can be conceived by those skilled in the art.

Claims

1. A method for encoding video, which is performed by an encoder, A step of determining predictive information for the current block of the current coding region and encoding the predictive information in the encoded video bitstream, wherein the predictive information indicates an intraframe block copy mode, and if the coding tree unit (CTU) to which the current block belongs contains fewer than 128 x 128 luminance samples, the size of the coding region is the same as the size of the CTU; otherwise, the CTU is divided into multiple coding regions, and the current coding region is the coding region to which the current block belongs among the multiple coding regions. A step of determining a block vector that points to a reference block in the same picture as the current block, wherein the reference block is in a search region that includes an encoding region having reconstructed samples buffered in a reference sample memory; The step includes: reconstructing at least one sample of the current block based on the reconstructed sample of the reference block obtained from the reference sample memory; The search area is updated in units of the coding area. method.

2. The method according to claim 1, wherein each encoding region has a square shape.

3. A method for encoding video, which is performed by an encoder, A step of determining predictive information for the current block of the current coding region and encoding the predictive information in the encoded video bitstream, wherein the predictive information indicates an intraframe block copy mode, and if the coding tree unit (CTU) to which the current block belongs contains fewer than 128 x 128 luminance samples, the size of the coding region is the same as the size of the CTU; otherwise, the CTU is divided into multiple coding regions, and the current coding region is the coding region to which the current block belongs among the multiple coding regions. A step of determining a block vector that points to a reference block in the same picture as the current block, wherein the reference block is in a search region that includes an encoding region having reconstructed samples buffered in a reference sample memory; The step includes: reconstructing at least one sample of the current block based on the reconstructed sample of the reference block obtained from the reference sample memory; The further step includes checking whether multiple corners of the aforementioned reference block fall within the same encoding region. method.

4. The method according to claim 1, further comprising the step of moving the reference block into the search area if the reference block is partially within the search area.

5. The method according to claim 4, further comprising the step of moving the reference block in a direction that decreases the distance between the reference block and the current block.

6. The method according to claim 4, further comprising the step of moving a reference block in a direction that increases the percentage of reference samples in the encoding region.

7. The method according to claim 1, further comprising the step of using information of a first reference sample located within the search area as information of a second reference sample located outside the search area, when the reference block is partially located within the search area.

8. The method according to claim 1, wherein flexible block division is prohibited in the division of the CTU.

9. A device for encoding video, Includes processing circuitry, The processing circuit is configured to perform the method described in any one of claims 1 to 8. Device.

10. A computer program for causing the processor of an encoder to perform the method described in any one of claims 1 to 8.

11. A method for encoding video, performed by an encoder, the method comprising generating and transmitting an encoded bitstream, The encoded video bitstream includes predictive information indicating an intraframe block copy mode, and if the coding tree unit (CTU) to which the current block of the current coding region belongs contains fewer than 128 x 128 luminance samples, the size of the coding region is the same as the size of the CTU; otherwise, the CTU is divided into multiple coding regions, and the current coding region is the coding region to which the current block belongs among the multiple coding regions. A step of determining a block vector that points to a reference block in the same picture as the current block, wherein the reference block is in a search region that includes an encoding region having reconstructed samples buffered in a reference sample memory; The step includes: reconstructing at least one sample of the current block based on the reconstructed sample of the reference block obtained from the reference sample memory; The search area is updated in units of the coding area. method.

Images