Methods of video coding, computer devices, apparatuses, and computer-readable media

Abstract

Aspects of the disclosure provide methods and apparatuses for video encoding / decoding. In some examples, a device for video decoding includes processing circuitry. The processing circuitry determines a string vector for a current string in a current coding tree unit (CTU). The string vector points to a reference string for the current string. The processing circuitry determines that the string vector satisfies a requirement that allows the current string to partially overlap with the reference string, and then reconstructs the current string based on the reference string.

Description

[0001] Incorporation

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 368,616, filed July 6, 2021, entitled “METHOD AND APPARATUS FOR VIDEO CODING,” which also claims the benefit of priority to U.S. Provisional Application No. 63 / 122,342, filed December 7, 2020, entitled “STRING MATCHING WITH REFERENCE LOCATION CONSTRAINTS.” The entire disclosure of the earlier applications is incorporated herein by reference. Technical Field

[0003] This disclosure describes implementations of video decoding in general, and more particularly relates to a method, computer apparatus, device, and non-transitory computer-readable medium for video decoding. Background Technology

[0004] The purpose of the background description provided herein is to present the general context of this disclosure. To the extent that the work of the currently identified inventors is described in this background section, neither the work of the currently identified inventors nor aspects of the description which at the time of submission may be otherwise considered prior art are explicitly or implicitly acknowledged as prior art to this disclosure.

[0005] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with a spatial size of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of pictures can have a fixed or variable picture rate (also informally referred to as the frame rate), such as 60 pictures per second or 60Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at a 60Hz frame rate) requires approximately 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 gigabytes (GByte) of storage space.

[0006] One objective of video encoding and decoding is to reduce the redundancy of the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements, in some cases by two or more orders of magnitude. Lossless compression and lossy compression, or combinations thereof, can be employed. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can be reflected in the fact that higher allowable / tolerable distortion can result in a higher compression ratio.

[0007] Video encoders and decoders can utilize techniques from several broad categories, including motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques can include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the picture can be an intra-frame picture. Intra-frame pictures and their derivatives (e.g., independent decoder refresh pictures) can be used to reset the decoder state and therefore can be used as the first picture in an encoded video bitstream and video session or as a still image. Samples of an intra-frame block can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0009] Traditional intra-frame coding—such as intra-frame coding techniques known from MPEG-2 generation coding techniques—does not use intra-frame prediction. However, some newer video compression techniques include attempts to use metadata obtained during the encoding and / or decoding of, for example, spatially adjacent data blocks that are first in the decoding order, along with / or surrounding sample data. Such techniques are hereby referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses only reference data from the current frame in the reconstruction, and not reference data from a reference frame. Summary of the Invention

[0010] According to an embodiment, a method for video decoding in a decoder is provided, characterized in that the method includes: determining, by a processor, a string vector of a current string in a current coding tree unit (CTU), the string vector pointing to a reference string of the current string; determining, by the processor, that the string vector satisfies a requirement allowing partial overlap between the current string and the reference string; and reconstructing, by the processor, the current string based on the reference string.

[0011] According to an embodiment, a computer device is provided, characterized in that the computer device comprises: one or more computer-readable non-transitory storage media configured to store computer program code; and one or more computer processors configured to access the computer program code and execute the above-described method for video decoding in a decoder as instructed by the computer program code.

[0012] According to an embodiment, an apparatus for video decoding is provided, comprising a processing circuit system, characterized in that the processing circuit system is configured to: determine a string vector of a current string in a current coding tree unit (CTU), the string vector pointing to a reference string of the current string; determine that the string vector satisfies a requirement allowing partial overlap between the current string and the reference string; and reconstruct the current string based on the reference string.

[0013] According to an embodiment, a non-transitory computer-readable medium is provided, which stores instructions, characterized in that, when executed by a computer for video decoding, the instructions cause the computer to perform the above-described method for video decoding in a decoder.

[0014] The video decoding method, computer device, apparatus, and non-transitory computer-readable medium of the present invention provide techniques for reference position constraints in string copy mode. Attached Figure Description

[0015] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, which are shown in the drawings:

[0016] Figure 1A is a schematic diagram of an exemplary subset of intra-prediction modes.

[0017] Figure 1B is an illustration of an exemplary intra-frame prediction direction.

[0018] Figure 2 is a schematic diagram of the current block and its surrounding space merge candidates in an example.

[0019] Figure 3 This is a simplified block diagram of a communication system according to an implementation method.

[0020] Figure 4This is a simplified block diagram of another communication system according to an embodiment.

[0021] Figure 5 This is a simplified block diagram of the decoder according to the implementation method.

[0022] Figure 6 This is a simplified block diagram of an encoder according to an implementation method.

[0023] Figure 7 A block diagram of an encoder according to another embodiment is shown.

[0024] Figure 8 A block diagram of a decoder according to another embodiment is shown.

[0025] Figure 9 An example of intra-block copying according to an embodiment of this disclosure is shown.

[0026] Figure 10 An example of intra-block copying according to an embodiment of this disclosure is shown.

[0027] Figure 11 An example of intra-block copying according to an embodiment of this disclosure is shown.

[0028] Figures 12A to 12D An example of intra-block copying according to an embodiment of this disclosure is shown.

[0029] Figure 13 An example of the spatial class for intra-block copy block vector prediction of the current block is shown according to an embodiment of this disclosure.

[0030] Figure 14 An example of a string copying mode according to an embodiment of this disclosure is shown.

[0031] Figure 15 The diagram shows images during decoding processing according to some embodiments of this disclosure.

[0032] Figure 16 An example of the reconstruction process of a string based on some examples is shown.

[0033] Figure 17 An example of a reference string that spans a region boundary is shown.

[0034] Figure 18 Another example of a reference string that spans a region boundary is shown.

[0035] Figure 19 An example of a reference string with a sample above the current CTU is shown.

[0036] Figure 20A flowchart outlining the process according to an embodiment of this disclosure is shown.

[0037] Figure 21 This is a schematic diagram of a computer system according to an implementation method. Detailed Implementation

[0038] Many different forms of intra-prediction can exist. When more than one such technique can be used in a given video coding technique, the technique used can be encoded in an intra-prediction mode. In some cases, a mode can have sub-modes and / or parameters, and these sub-modes and / or parameters can be encoded separately or included in the mode codeword. Which codeword is used for a given mode, sub-mode, and / or parameter combination can affect the coding efficiency gain through intra-prediction, and therefore the entropy coding technique used to convert the codeword into a bitstream can also affect it.

[0039] Some patterns of intra-frame prediction were introduced with H.264, refined in H.265, and further refined in newer coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Prediction blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the prediction block according to the direction. The reference to the direction used can be encoded in the bitstream, or it can be predicted itself.

[0040] Referring to Figure 1A, the lower right corner depicts a subset of nine prediction directions known from the 33 possible prediction directions of H.265 (corresponding to 33 angular modes out of 35 intra-frame modes). The point (101) where the arrows converge represents the sample being predicted. The arrows indicate the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted based on one or more samples at a 45-degree angle to the horizontal at the upper right. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more samples at a 22.5-degree angle to the horizontal at the lower left.

[0041] Referring again to Figure 1A, a 4×4 square block (104) of samples is depicted in the upper left (indicated by a bold dashed line). The square block (104) comprises 16 samples, each labeled with “S”, its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top) and the first sample in the X dimension (from left). Similarly, sample S44 is the fourth sample in block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is in the lower right. Also shown are reference samples following a similar numbering scheme. Reference samples are labeled with R, their Y position (e.g., row index) and X position (column index) relative to block (104). In both H.264 and H.265, predicted samples are adjacent to the blocks in the reconstruction; therefore, negative values ​​are not required.

[0042] Intra-frame image prediction works by appropriately copying reference sample values ​​from neighboring samples according to the prediction direction indicated by a signal. For example, suppose the encoded video bitstream includes signaling for that block, the signaling indicating a prediction direction consistent with arrow (102)—that is, predicting samples based on one or more prediction samples at a 45-degree angle to the horizontal from the upper right. In this case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Then sample S44 is predicted based on reference sample R08.

[0043] In some cases, the values ​​of multiple reference samples can be combined, for example, by interpolation, to calculate the reference sample; especially when the direction is not divisible by 45 degrees.

[0044] As video coding techniques have evolved, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS could support up to 65 directions at the time of publication. Experiments have been conducted to identify the most probable directions, and certain techniques in entropy coding have been used to represent these possible directions with a small number of bits, thus incurring some penalty for less likely directions. Furthermore, the direction itself can sometimes be predicted based on adjacent directions used in adjacent decoded blocks.

[0045] Figure 1B shows a schematic diagram (180) depicting 65 intra-frame prediction directions based on JEM to illustrate how the number of prediction directions increases over time.

[0046] The mapping of intra-predicted direction bits representing direction in an encoded video bitstream can vary depending on the video coding technique; and this mapping can range from, for example, a simple direct mapping of the predicted direction to the intra-predicted mode, to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, there may exist some directions that are statistically less likely to appear in the video content than some other directions. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less probable directions will be represented by a larger number of bits compared to the more probable directions.

[0047] Motion compensation can be a lossy compression technique and can involve using blocks of sample data from a previously reconstructed image or a portion thereof (the reference image), spatially shifted in a direction indicated by a motion vector (hereinafter referred to as MV), to predict a newly reconstructed image or a portion thereof. In some cases, the reference image may be the same as the image in the current reconstruction. The MV may have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference image used (the third dimension may indirectly be a temporal dimension).

[0048] In some video compression techniques, an MV applicable to a specific region of sample data can be predicted based on other MVs, such as an MV that is spatially adjacent to the region being reconstructed and precedes that MV in the decoding order. This significantly reduces the amount of data required to encode the MV, eliminating redundancy and increasing compression. MV prediction works effectively, for example, because when encoding an input video signal (called natural video) from a camera device, there is a statistical probability that a region larger than the area applicable to a single MV moves in similar directions, and therefore, in some cases, prediction can be made using similar motion vectors derived from MVs of adjacent regions. This makes the MV obtained for a given region similar or identical to the MV predicted based on surrounding MVs, and can be represented after entropy encoding with fewer bits than would be used if the MV were encoded directly. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating the predictor based on several surrounding MVs.

[0049] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016). This section describes one of the many MV prediction mechanisms provided by H.265, hereinafter referred to as “spatial combining”.

[0050] Referring to Figure 2, the current block (201) includes samples that the encoder obtained during the motion search process can predict based on previous blocks of the same size that have been spatially shifted. Instead of directly encoding this MV, the MV can be derived from metadata associated with one or more reference images, for example, from the most recent (in decoding order) reference image, using the MV associated with any of the five surrounding samples denoted as A0, A1 and B0, B1, B2 (corresponding to 202 to 206, respectively). In H.265, MV prediction can use a predictor from the same reference image that is also being used in adjacent blocks.

[0051] Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3 In the example, the first pair of terminal devices (310) and (320) perform unidirectional data transmission. For example, terminal device (310) may encode video data (e.g., a video image stream captured by terminal device (310)) for transmission via network (350) to another terminal device (320). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. Terminal device (320) may receive the encoded video data from network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. Unidirectional data transmission can be common in media service applications, etc.

[0052] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, in this example, each of the terminal devices (330) and (340) can encode video data (e.g., a stream of video images captured by the terminal device) for transmission via a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) can also receive encoded video data transmitted by the other terminal device (330) and (340), and can decode the encoded video data to recover the video images, and can display the video images at an accessible display device based on the recovered video data.

[0053] exist Figure 3 In the examples, terminal devices (310), (320), (330), and (340) may be shown as servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) refers to any number of networks that transmit encoded video data between terminal devices (310), (320), (330), and (340), including, for example, wired (connected) and / or wireless communication networks. Communication networks (350) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless otherwise stated below, the architecture and topology of the network (350) may be irrelevant to the operation of this disclosure.

[0054] As an example of the application of the disclosed topic Figure 4 The arrangement of a video encoder and video decoder in a streaming environment is illustrated. The disclosed subject matter can be equally applied to other video-enabled applications, including, for example, video conferencing; digital TV; and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0055] The streaming system may include a capture subsystem (413), which may include a video source (401), such as a digital camera device, that creates, for example, an uncompressed video picture stream (402). In the example, the video picture stream (402) includes samples captured by the digital camera device. The video picture stream (402) is depicted as a thick line to emphasize the high data volume when compared with encoded video data (404) (or encoded video bitstream), which may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or carry out aspects of the disclosed subject matter as described in more detail below. Encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize the lower data volume when compared to the video picture stream (402). The encoded video data (404) can be stored on a streaming server (405) for future use. One or more streaming client subsystems (e.g., Figure 4 The client subsystems (406) and (408) can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video picture stream (411) that can be displayed on a display (412) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In the example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0056] Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0057] Figure 5 A block diagram of a video decoder (510) according to an embodiment of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuitry system). The video decoder (510) may be used in place of... Figure 4 The example video decoder (410).

[0058] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (531) can receive the encoded video data as well as other data such as encoded audio data and / or auxiliary data streams, which can be forwarded to their respective user entities (not depicted). The receiver (531) can separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other applications, the buffer memory (515) can be external to the video decoder (510) (not depicted). In some other applications, a buffer memory (not depicted) may exist outside the video decoder (510) to prevent network jitter, for example, and another buffer memory (515) may exist inside the video decoder (510) to handle broadcast timing, for example. The buffer memory (515) may not be necessary, or the buffer memory (515) may be small, when the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability, or from an isosynchronous network. For use on best-effort packet networks such as the Internet, the buffer memory (515) may be required; the buffer memory (515) may be relatively large and advantageously have an adaptive size, and may be implemented at least partially in the operating system or in a similar element (not depicted) outside the video decoder (510).

[0059] The video decoder (510) may include a parser (520) to reconstruct symbols (521) based on the encoded video sequence. These symbols include information for managing the operation of the video decoder (510), and may include information for controlling a presentation device, such as a presentation device (512) (e.g., a display screen), which is not part of the electronic device (530) but may be coupled to it, such as... Figure 5As shown. Control information for one or more presentation devices may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (520) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may conform to video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set for at least one subgroup of pixel subgroups in the video decoder from the encoded video sequence based on at least one parameter corresponding to a group. Subgroups may include: Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The parser (520) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0060] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0061] The reconstruction of the symbol (521) may involve multiple different units depending on the type of the encoded video picture or a portion thereof (e.g., inter-frame picture and intra-frame picture, inter-frame block and intra-frame block) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For the sake of brevity, the flow of such subgroup control information between the parser (520) and the multiple units described below is not depicted.

[0062] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into multiple functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other at least partially. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0063] The first unit is the scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives from the parser (520) quantized transform coefficients as symbols (521) and control information, including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output a block containing sample values, which can be input into the aggregator (555).

[0064] In some cases, the output samples of the scaler / inverse transform (551) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape as the blocks in the reconstruction using surrounding reconstructed information obtained from the current picture buffer (558). For example, the current picture buffer (558) buffers partially reconstructed current images and / or fully reconstructed current images. In some cases, the aggregator (555) adds the predictive information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) based on each sample.

[0065] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to inter-frame coded and possibly motion-compensated blocks. In such cases, the motion-compensated prediction unit (553) can access the reference image memory (557) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the block, these samples can be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (referred to in this case as residual samples or residual signals) to generate output sample information. The address from which the motion-compensated prediction unit (553) obtains the predicted samples within the reference image memory (557) can be controlled by motion vectors, which can be obtained by the motion-compensated prediction unit (553) in the form of symbols (521), which may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (557) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0066] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). The video compression technique may include an in-loop filtering technique controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and which may be obtained by the loop filter unit (556) as symbols (521) from the parser (520). However, the in-loop filtering technique may also respond to metadata obtained during decoding of previous portions of the encoded picture or encoded video sequence (in the order of decoding) and to previously reconstructed and loop-filtered sample values.

[0067] The output of the loop filter unit (556) can be a sample stream, which can be output to the presentation device (512) and stored in the reference image memory (557) for future inter-frame image prediction.

[0068] Once fully reconstructed, certain encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image (by, for example, the parser (520)) is identified as the reference image, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before reconstructing subsequent encoded images begins.

[0069] The video decoder (510) can perform decoding operations according to a predetermined video compression technique as specified in a standard such as ITU-T Recommendation H.265. An encoded video sequence can conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile can select certain tools from all available tools in the video compression technique or standard as tools available only under that profile. For compliance, the complexity of the encoded video sequence also needs to be within the limits defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, in megapixels per second), maximum reference picture size, etc. In some cases, the limitations set by the level can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, indicated by signals in the encoded video sequence.

[0070] In this implementation, the receiver (531) may receive supplemental (redundant) data along with the encoded video. The supplemental data may be included as part of one or more encoded video sequences. The supplemental data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0071] Figure 6 A block diagram of a video encoder (603) according to an embodiment of the present disclosure is shown. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit system). The video encoder (603) can be used in place of Figure 4 The example video encoder (403).

[0072] The video encoder (603) can obtain data from the video source (601) (which is not...). Figure 6 In one example, the electronic device (620) receives video samples, and the video source (601) can capture one or more video images to be encoded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0073] The video source (601) can provide a sequence of source video samples in the form of a digital video sample stream to be encoded by the video encoder (603). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601YCrCb, RGB…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (601) can be a storage device storing previously prepared video. In a video conferencing system, the video source (601) can be a camera device capturing local image information as a video sequence. Video data can be provided as multiple individual pictures that are given motion when viewed sequentially. The pictures themselves can be organized as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0074] According to the implementation, the video encoder (603) can encode and compress images of the source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (650). In some implementations, the controller (650) controls and is functionally coupled to other functional units as described below. For simplicity, the coupling is not depicted. Parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions related to the video encoder (603) optimized for a specific system design.

[0075] In some implementations, the video encoder (603) is configured to operate within an encoding loop. As a highly simplified description, in this example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input picture to be encoded and one or more reference pictures) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (since any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local and remote encoders. In other words, the encoder's prediction portion "treats" the reference picture samples as sample values ​​that are exactly the same as the sample values ​​the decoder "sees" when using prediction during decoding. This fundamental principle of image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related techniques.

[0076] The operation of the "local" decoder (633) can be combined with that of the "remote" decoder, as already mentioned above. Figure 5 The operation of the video decoder (510) described in detail is the same. However, a brief additional reference is provided. Figure 5 Since symbols are available and the encoding of symbols into an encoded video sequence by the entropy encoder (645) and the decoding of symbols by the parser (520) can be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and the parser (520), can be completely implemented in the local decoder (633).

[0077] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the subject matter disclosed focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are the opposite of those comprehensively described decoder techniques. Only certain aspects require a more detailed description, which is provided below.

[0078] In some examples, during operation, the source encoder (630) may perform motion-compensated predictive coding, which predictively codes the input image with reference to one or more previously encoded images designated as "reference images" from the video sequence. In this way, the encoding engine (632) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images (one or more) that can be selected as predictive references to the input image.

[0079] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (630). The operation of the encoding engine (632) can be advantageously for lossy processing. When the encoded video data can be decoded by the video decoder (633), Figure 6 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding processing performed on the reference image by the video decoder and can store the reconstructed reference image in a reference image buffer (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0080] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search in the reference image memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. The predictor (635) can operate on a sample-by-sample block-by-pixel basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0081] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0082] The outputs of all the functional units mentioned above can undergo entropy encoding in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into an encoded video sequence by performing lossless compression on the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0083] The transmitter (640) can buffer one or more encoded video sequences created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0084] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques that can be applied to the corresponding image. For example, one of the following image types can typically be assigned to an image:

[0085] An intra-frame picture (I-picture) can be a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand those variations of I-pictures and their corresponding applications and characteristics.

[0086] Predictive images (P-images) can be images that can be encoded and decoded using inter-frame or intra-frame prediction—which uses at most one motion vector and a reference index to predict the sample values ​​of each block.

[0087] A bidirectional predictive picture (B-picture) can be an image that can be encoded and decoded using either inter-frame or intra-frame prediction—which uses up to two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0088] Source images are typically spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded block by block. These blocks can be predictively encoded with reference to other (already encoded) blocks, determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of image I can be non-predictively encoded, or blocks of image I can be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of image P can be predictively encoded with reference to a previously encoded reference image via spatial prediction or temporal prediction. Blocks of image B can be predictively encoded with reference to one or two previously encoded reference images via spatial prediction or temporal prediction.

[0089] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0090] In this implementation, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0091] Video can be captured temporally as multiple source images (video frames). Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, the specific image being encoded / decoded—referred to as the current image—is segmented into blocks. Where a block in the current image resembles a reference block in a previously encoded and buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.

[0092] In some implementations, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image that both precede the current image in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0093] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0094] According to some embodiments of this disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video picture sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively divided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be divided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type for that CU, such as inter-frame prediction or intra-frame prediction. Based on temporal and / or spatial predictability, a CU is divided into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In implementations, prediction operations in encoding / decoding are performed on a per-prediction-block basis. Using a luminance prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luminance values) such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0095] Figure 7 A diagram of a video encoder (703) according to another embodiment of the present disclosure is shown. The video encoder (703) is configured to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video frame in a video frame sequence, and to encode the processing blocks into an encoded frame that is part of an encoded video sequence. In the example, the video encoder (703) is used instead of Figure 4 The example video encoder (403).

[0096] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (703) uses, for example, rate-distortion optimization to determine whether to best encode the processing block using intra-frame mode, inter-frame mode, or bidirectional prediction mode. When encoding the processing block in intra-frame mode, the video encoder (703) can encode the processing block into an encoded picture using intra-frame prediction techniques; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (703) can encode the processing block into an encoded picture using inter-frame prediction or bidirectional prediction techniques, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector predictors without relying on encoded motion vector components outside of said predictors. In some other video coding techniques, motion vector components applicable to the object block may exist. In the example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0097] exist Figure 7 In the example, the video encoder (703) includes, for example, Figure 7 The inter-frame encoder (730), intra-frame encoder (722), residual calculator (723), switch (726), residual encoder (724), overall controller (721), and entropy encoder (725) are shown coupled together.

[0098] The inter-frame encoder (730) is configured to: receive samples of the current block (e.g., the processing block); compare the block with one or more reference blocks in a reference image (e.g., blocks in a previous image and a subsequent image); generate inter-frame prediction information (e.g., a description based on redundancy information, motion vectors, and merging mode information of the inter-frame coding technique); and compute inter-frame prediction results (e.g., prediction blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on encoded video information.

[0099] The intra encoder (722) is configured to: receive samples of the current block (e.g., the processing block); in some cases compare the block with already encoded blocks in the same image; generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). In the example, the intra encoder (722) also computes an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same image.

[0100] The overall controller (721) is configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In an example, the overall controller (721) determines the mode of the block and provides control signals to the switch (726) based on the mode. For example, when the mode is intra-frame mode, the overall controller (721) controls the switch (726) to select an intra-frame mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include the intra-frame prediction information in the bitstream; and when the mode is inter-frame mode, the overall controller (721) controls the switch (726) to select an inter-frame prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0101] A residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from an intra-encoder (722) or an inter-encoder (730). A residual encoder (724) is configured to operate based on the residual data to encode the residual data, thereby generating transform coefficients. In an example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra-encoder (722) and the inter-encoder (730). For example, an inter-frame encoder (730) can generate decoded blocks based on decoded residual data and inter-frame prediction information, and an intra-frame encoder (722) can generate decoded blocks based on decoded residual data and intra-frame prediction information. In some examples, the decoded blocks are appropriately processed to generate decoded images, and these decoded images can be buffered in memory circuitry (not shown) and used as reference images.

[0102] An entropy encoder (725) is configured to format the bitstream to include encoded blocks. The entropy encoder (725) is configured to include various information according to a suitable standard, such as the HEVC standard. In this example, the entropy encoder (725) is configured to include overall control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Note that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0103] Figure 8A diagram of a video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In the example, the video decoder (810) is used instead of Figure 4 The example video decoder (410).

[0104] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The entropy decoder (871), inter-frame decoder (880), residual decoder (873), reconstruction module (874), and intra-frame decoder (872) are shown coupled together.

[0105] The entropy decoder (871) can be configured to reconstruct certain symbols from the encoded picture, these symbols representing the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode encoding the block (e.g., intra-mode, inter-mode, bidirectional prediction mode, a combined sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can identify certain samples or metadata used by the intra-decoder (872) or inter-decoder (880) for prediction, and residual information in the form of quantized transform coefficients, etc. In the example, when the prediction mode is inter-mode or bidirectional prediction mode, the inter-prediction information is provided to the inter-decoder (880); and when the prediction type is intra-prediction type, the intra-prediction information is provided to the intra-decoder (872). The residual information may be inversely quantized and provided to the residual decoder (873).

[0106] The inter-frame decoder (880) is configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0107] The intra-frame decoder (872) is configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0108] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients, and to process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require some control information (including quantizer parameters (QP)), and this information may be provided by the entropy decoder (871) (the data path is not depicted because this may only be a small amount of control information).

[0109] The reconstruction module (874) is configured to combine the residual output by the residual decoder (873) with the prediction result (output by the inter-frame prediction module or intra-frame prediction module, depending on the situation) in the spatial domain to form a reconstructed block, which can be a part of a reconstructed image, which in turn can be a part of a reconstructed video. Note that other suitable operations, such as deblocking, can be performed to improve visual quality.

[0110] Note that any suitable technology can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In one implementation, one or more integrated circuits can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In another implementation, one or more processors that execute software instructions can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810).

[0111] Block-based compensation can be used for both inter-frame and intra-frame prediction. For inter-frame prediction, block-based compensation based on different pictures is called motion compensation. Block-based compensation can also be done, for example, in intra-frame prediction based on previously reconstructed regions within the same picture. Block-based compensation from reconstructed regions within the same picture is called intra-picture block compensation, current picture referencing (CPR), or intra-block copy (IBC). The displacement vector indicating the offset between the current block and the reference block (also called the prediction block) in the same picture is called the block vector (BV), where the current block can be encoded / decoded based on the reference block. Unlike the motion vector in motion compensation, which can be any value (positive or negative in the x or y direction), the BV has some constraints to ensure that the reference block is available and has been reconstructed. Furthermore, in some examples, some reference regions are excluded as tile boundaries, slice boundaries, or wavefront trapezoidal boundaries for parallel processing considerations.

[0112] The encoding of block vectors can be explicit or implicit. In explicit mode, a signal indicates the BV difference between the block vector and its predictor. In implicit mode, the block vector is recovered from the predictor (called the block vector predictor) in a manner similar to that of motion vectors in merging mode, without using the BV difference. Explicit mode can be referred to as non-merging BV prediction mode. Implicit mode can be referred to as merging BV prediction mode.

[0113] In some implementations, the resolution of the block vector is limited to integer positions. In other systems, the block vector is allowed to point to decimal positions.

[0114] In some examples, block-level flags, such as the IBC flag, can be used to signal the use of intra-block copying at the block level. In implementations, the block-level flag is signaled when the current block is explicitly encoded. In some examples, a reference indexing method can be used to signal the use of intra-block copying at the block level. The current picture being decoded is then treated as a reference picture or a special reference picture. In an example, such a reference picture is placed at the end of the list of reference pictures. This special reference picture, along with other time reference pictures, is also managed in a buffer, such as a decoded picture buffer (DPB).

[0115] IBC mode can have variations. In the example, IBC mode is considered a third mode, distinct from intra-prediction and inter-prediction modes. Therefore, BV prediction in implicit (or merge) mode and explicit mode is separate from the regular inter-prediction mode. A separate merge candidate list can be defined for IBC mode, where entries are BV. Similarly, in the example, the BV prediction candidate list in explicit IBC mode includes only BV. The general rule applied to these two lists (i.e., the separate merge candidate list and the BV prediction candidate list) is that, in terms of candidate derivation processing, these lists can follow the same logic as the merge candidate list used in regular merge mode or the AMVP predictor list used in regular AMVP mode. For example, accessing five spatially adjacent locations (e.g., A0, A1 and B0, B1, B2 in Figure 2) for IBC mode, such as in HEVC or VVC inter-merge mode, yields a separate merge candidate list for IBC mode.

[0116] As described above, the BV of the current block being reconstructed in the image can have certain constraints, and therefore, the reference block of the current block is within the search range. The search range refers to a portion of the image from which reference blocks can be selected. For example, the search range can be within certain portions of the reconstructed region in the image. The size, position, shape, etc., of the search range can be restricted. Alternatively, the BV can be constrained. In the example, the BV is a two-dimensional vector comprising x and y components, and at least one of the x and y components can be constrained. Constraints can be specified for the BV, the search range, or a combination of BV and the search range. In various examples, when certain constraints are specified for the BV, the search range is constrained accordingly. Similarly, when certain constraints are specified for the search range, the BV is constrained accordingly.

[0117] Figure 9An example of intra-frame block copying according to an embodiment of this disclosure is shown. The current image (900) will be reconstructed during decoding. The current image (900) includes a reconstruction region (910) (gray area) and a region to be decoded (920) (white area). The current block (930) is being reconstructed by the decoder. The current block (930) can be reconstructed based on a reference block (940) located in the reconstruction region (910). The positional offset between the reference block (940) and the current block (930) is called the block vector (950) (or BV (950)). Figure 9 In the example, the search range (960) is within the reconstructed region (910), the reference block (940) is within the search range (960), and the block vector (950) is constrained to point to the reference block (940) within the search range (960).

[0118] Various constraints can be applied to the BV and / or search range. In this implementation, the search range of the current block being reconstructed in the current CTB is limited to the current CTB.

[0119] In this implementation, the effective memory requirement for storing reference samples to be used in intra-block copying is one CTB size. In the example, the CTB size is 128×128 samples. The current CTB includes the current region in reconstruction. The current region is 64×64 samples in size. Since the reference memory can also store reconstructed samples in the current region, it can store three more 64×64 sample regions when the reference memory size is equal to the 128×128 sample CTB size. Therefore, the search range can include portions of the previously reconstructed CTB, while the total memory requirement for storing reference samples remains unchanged (e.g., one CTB size with 128×128 samples or a total of four 64×64 reference samples). In the example, as... Figure 10 As shown, the previously reconstructed CTB is the left neighbor of the current CTB.

[0120] Figure 10An example of intra-block copying according to an embodiment of the present disclosure is shown. The current frame (1001) includes a current CTB (1015) being reconstructed and a previously reconstructed CTB (1010) that is the left neighbor of the current CTB (1015). The CTB in the current frame (1001) has, for example, a CTB size of 128 × 128 and a CTB width of, for example, 128 samples. The current CTB (1015) includes four regions (1016) to (1019), wherein the current region (1016) is under reconstruction. The current region (1016) includes multiple coded blocks (1021) to (1029). Similarly, the previously reconstructed CTB (1010) includes four regions (1011) to (1014). Encoding blocks (1021) to (1025) have been reconstructed, the current block (1026) is being reconstructed, and encoding blocks (1026) to (1027) and regions (1017) to (1019) will be reconstructed.

[0121] The current region (1016) has a juxtaposed region (i.e., region (1011) in the previously reconstructed CTB (1010)). The relative position of the juxtaposed region (1011) with respect to the previously reconstructed CTB (1010) can be the same as the relative position of the current region (1016) with respect to the current CTB (1015). Figure 10 In the example shown, the current region (1016) is the top-left region in the current CTB (1015), therefore, the juxtaposed region (1011) is also the top-left region in the previously reconstructed CTB (1010). Since the position of the previously reconstructed CTB (1010) is offset by the CTB width from the position of the current CTB (1015), the position of the juxtaposed region (1011) is offset by the CTB width from the position of the current region (1016).

[0122] In this implementation, the juxtaposed region of the current region (1016) is within a previously reconstructed CTB, wherein the position of the previously reconstructed CTB is offset from the position of the current CTB (1015) by one or more CTB widths. Therefore, the position of the juxtaposed region is also offset from the position of the current region (1016) by a corresponding one or more CTB widths. The position of the juxtaposed region can be shifted to the left, up, etc., from the current region (1016).

[0123] As mentioned above, the search range of the current block (1026) is limited by the CTB size. Figure 10In the example, the search range may include regions (1012) to (1014) in the previously reconstructed CTB (1010), as well as reconstructed portions of the current region (1016), such as coded blocks (1021) to (1025). The search range further excludes juxtaposed regions (1011), ensuring that the size of the search range is within the CTB size. See [reference] Figure 10 The reference block (1091) is located in region (1014) of the previously reconstructed CTB (1010). The block vector (1020) indicates the offset between the current block (1026) and the corresponding reference block (1091). The reference block (1091) is within the search range.

[0124] Figure 10 The example shown can be appropriately applied to other scenarios where the current region is located at another position within the current CTB (1015). In the example, when the current block is in region (1017), the juxtaposed region of the current block is region (1012). Therefore, the search range can include regions (1013) to (1014), region (1016), and the reconstructed portion of region (1017). The search range further excludes region (1011) and the juxtaposed region (1012), such that the size of the search range is within the CTB size. In the example, when the current block is in region (1018), the juxtaposed region of the current block is region (1013). Therefore, the search range can include regions (1014), regions (1016) to (1017), and the reconstructed portion of region (1018). The search range further excludes regions (1011) to (1012) and the juxtaposed region (1013), such that the size of the search range is within the CTB size. In the example, when the current block is in region (1019), the juxtaposed region of the current block is region (1014). Therefore, the search range can include regions (1016) to (1018) as well as the reconstructed portions of region (1019). The search range further excludes the previously reconstructed CTB (1010), making the size of the search range within the size of the CTB.

[0125] In the above description, the reference block can be in the previously reconstructed CTB (1010) or the current CTB (1015).

[0126] In this implementation, the search range can be specified as follows. In the example, the current image is a luminance image and the current CTB is a luminance CTB that includes multiple luminance samples, and BV(mvL) satisfies the following constraints for bitstream consistency. In the example, BV(mvL) has a fractional resolution (e.g., 1 / 16 pixel resolution).

[0127] The constraint includes the first condition: the reference block of the current block has been reconstructed. When the reference block has a rectangular shape, a neighboring block availability check (or a reference block availability check) can be performed to check whether the top-left and bottom-right samples of the reference block have been reconstructed. If both the top-left and bottom-right samples of the reference block have been reconstructed, the reference block is determined to be reconstructed.

[0128] For example, when the derivation process for the availability of the reference block is invoked with the position (xCurr, yCurr) of the top left sample of the current block set to (xCb, yCb) and the position (xCb+(mvL[0]>>4), yCb+(mvL[1]>>4)) of the top left sample of the reference block as input, the output is true if the top left sample of the reference block is reconstructed, where the block vector mvL is a two-dimensional vector with x-component mvL[0] and y-component mvL[1]. In the case where BV(mvL) has a fractional resolution, such as 1 / 16 pixel resolution, the x-component mvL[0] and y-component mvL[1] are shifted to have an integer resolution, as indicated by mvL[0]>>4 and mvL[1]>>4, respectively.

[0129] Similarly, when the block availability derivation process is called with the position (xCurr, yCurr) of the top-left sample of the current block set to (xCb, yCb) and the position (xCb+(mvL[0]>>4)+cbWidth-1, yCb+(mvL[1]>>4)+cbHeight-1) of the bottom-right sample of the reference block as input, the output is TRUE if the bottom-right sample of the reference block is reconstructed. The parameters cbWidth and cbHeight represent the width and height of the reference block.

[0130] The constraint may also include at least one of the following second conditions: 1)(mvL[0]>>4)+cbWidth is less than or equal to 0, which indicates that the reference block is to the left of the current block and does not overlap with the current block; 2)(mvL[1]>>4)+cbHeight is less than or equal to 0, which indicates that the reference block is above the current block and does not overlap with the current block.

[0131] Constraints may also include: the block vector mvL satisfies the following third condition:

[0132] (yCb+(mvL[1]>>4))>>CtbLog2SizeY=yCb>>CtbLog2SizeY (1)

[0133] (yCb+(mvL[1]>>4+cbHeight-1)>>CtbLog2SizeY=yCb>>CtbLog2Size (2)

[0134] (xCb+(mvL[0]>>4))>>CtbLog2SizeY>>=(xCb>>CtbLog2SizeY)-1 (3)

[0135] (xCb+(mvL[0]>>4)+cbWidth-1)>>CtbLog2SizeY<=(xCb>>CtbLog2SizeY) (4)

[0136] Wherein, the parameter CtbLog2SizeY represents the CTB width in log2 form. For example, when the CTB width is 128 samples, CtbLog2SizeY is 7. Equations (1) to (2) specify that the CTB including the reference block is in the same CTB row as the current CTB (e.g., when the reference block is in the previously reconstructed CTB (1010), the previously reconstructed CTB (1010) and the current CTB (1015) are in the same row). Equations (3) to (4) specify that the CTB including the reference block is in the left CTB column of the current CTB or in the same CTB column as the current CTB. Similar to the reference... Figure 10 The description, as described in Equations (1) to (4), specifies the third condition: the CTB including the reference block is the current CTB, such as the current CTB (1015), or the left neighbor of the current CTB, such as the previously reconstructed CTB (1010).

[0137] The constraint can also include a fourth condition: when the reference block is among the left neighbors of the current CTB, the juxtaposed region of the reference block is not reconstructed (i.e., no samples are reconstructed in the juxtaposed region). Furthermore, the juxtaposed region of the reference block is within the current CTB. Figure 10 In the example, the juxtaposed region of the reference block (1091) is the region (1019) offset from the region (1014) where the reference block (1091) is located by the CTB width, and the region (1019) has not yet been reconstructed. Therefore, the block vector (1020) and the reference block (1091) satisfy the fourth condition mentioned above.

[0138] In the example, the fourth condition can be specified as follows: when (xCb+(mvL[0]>>4))>>CtbLog2SizeY equals (xCb>>CtbLog2SizeY)-1, the export processing for reference block availability is called with the current block's position (xCurr, yCurr) set to (xCb, yCb) and the position (((xCb+(mvL[0]>>4)+CtbSizeY)>>(CtbLog2SizeY-1)))<<(CtbLog2SizeY-1) and ((yCb+(mvL[1]>>4))>>(CtbLog2SizeY-1))<<(CtbLog2SizeY-1)) as input. The output is equal to FALSE, which indicates that the juxtaposed region has not been reconstructed, such as Figure 10 As shown.

[0139] Constraints on the search range and / or block vector can include appropriate combinations of the first, second, third, and fourth conditions described above. In the example, such as... Figure 10 As shown, the constraints include a first condition, a second condition, a third condition, and a fourth condition. In the example, the first, second, third, and / or fourth conditions can be modified, and the constraints include the modified first, second, third, and / or fourth conditions.

[0140] According to the fourth condition, if one of the coding blocks (1022) to (1029) is the current block, the reference block cannot be in region (1011), and therefore, the search range for said one coding block (1022) to (1029) excludes region (1011). The reason for excluding region (1011) is as follows: if the reference block is in region (1011), then the juxtaposition region of the reference block is region (1016); however, at least the samples in coding block (1021) have already been reconstructed, thus violating the fourth condition. On the other hand, for the coding block that must be reconstructed first in the current region, for example... Figure 11 The fourth condition does not prevent the reference block in region (1111) from being in region (1121) since the juxtaposed region (1116) of the reference block has not yet been reconstructed.

[0141] Figure 11An example of intra-block copying according to an embodiment of the present disclosure is shown. The current frame (1101) includes the current CTB (1115) being reconstructed and the previously reconstructed CTB (1110) which is the left neighbor of the current CTB (1115). The CTB in the current frame (1101) has a CTB size and a CTB width. The current CTB (1115) includes four regions (1116) to (1119), wherein the current region (1116) is being reconstructed. The current region (1116) includes multiple coded blocks (1121) to (1129). Similarly, the previously reconstructed CTB (1110) includes four regions (1111) to (1114). In the current region (1116), the current block (1121) being reconstructed is reconstructed first, and the coded blocks (1122) to (1129) are reconstructed. In the example, the CTB size is 128×128 samples, and each of the regions (1111) to (1114) and (1116) to (1119) contains 64×64 samples. The reference memory size is equal to the CTB size and is 128×128 samples, and therefore, when limited by the reference memory size, the search range includes three regions and a portion of an additional region.

[0142] With reference Figure 10 Similarly described, the current region (1116) has a juxtaposed region (i.e., region (1111) in the previously reconstructed CTB (1110)). According to the fourth condition above, the reference block of the current block (1121) can be in region (1111), and therefore the search range can include regions (1111) to (1114). For example, when the reference block is in region (1111), the juxtaposed region of the reference block is region (1116), where the samples in region (1116) were not reconstructed before the current block (1121) was reconstructed. However, as referenced Figure 10 As described in the fourth condition, for example, after reconstructing the coded block (1121), the region (1111) is no longer available to be included in the search range used for reconstructing the coded block (1122). Therefore, strict synchronization and timing control of the reference memory buffer will be used, and this strict synchronization and timing control may be challenging.

[0143] According to some implementations, when the current block is to be reconstructed first in the current region of the current CTB, the search range can exclude juxtaposed regions of the current region located in previously reconstructed CTBs, where the current CTB and the previously reconstructed CTBs are in the same current image. A block vector can be determined such that the reference block is within the search range excluding juxtaposed regions in previously reconstructed CTBs. In some implementations, the search range includes coded blocks reconstructed in decoding order after the juxtaposed regions and before the current block.

[0144] In the following description, the CTB size can vary, and the maximum CTB size is set to be the same as the reference memory size. In the example, the reference memory size or maximum CTB size is 128 × 128 samples. These descriptions may be adapted to other reference memory sizes or maximum CTB sizes.

[0145] In this implementation, the CTB size is equal to the reference memory size. The previously reconstructed CTB is the left neighbor of the current CTB, the position of the juxtaposed region is offset from the position of the current region by the CTB width, and the coded blocks within the search range are at least one of the following: the current CTB and the previously reconstructed CTB.

[0146] Figures 12A to 12D An example of intra-block copying according to an embodiment of this disclosure is shown. (Refer to...) Figures 12A to 12D The current image (1201) includes the current CTB (1215) being reconstructed and the previously reconstructed CTB (1210) which is the left neighbor of the current CTB (1215). The CTBs in the current image (1201) have a CTB size and a CTB width. The current CTB (1215) includes four regions (1216) to (1219). Similarly, the previously reconstructed CTB (1210) includes four regions (1211) to (1214). In the implementation, the CTB size is the maximum CTB size and is equal to the reference memory size. In the example, the CTB size and the reference memory size are 128 × 128 samples, and therefore, each of the regions (1211) to (1214) and (1216) to (1219) has a size of 64 × 64 samples.

[0147] exist Figures 12A to 12D In the example shown, the current CTB (1215) includes the upper left region, upper right region, lower left region, and lower right region corresponding to regions (1216) to (1219), respectively. The previously reconstructed CTB (1210) includes the upper left region, upper right region, lower left region, and lower right region corresponding to regions (1211) to (1214), respectively.

[0148] Reference Figure 12A The current region (1216) is being reconstructed. The current region (1216) may include multiple coding blocks (1221) to (1229). The current region (1216) has a juxtaposed region in the previously reconstructed CTB (1210), namely region (1211). The search range of one of the coding blocks (1221) to (1229) to be reconstructed may exclude the juxtaposed region (1211). The search range may include regions (1212) to (1214) of the previously reconstructed CTB (1210) that are reconstructed in decoding order after the juxtaposed region (1211) and before the current region (1216).

[0149] Reference Figure 12A The position of the juxtaposed region (1211) is offset from the position of the current region (1216) by the CTB width, for example, 128 samples. For example, the position of the juxtaposed region (1211) is shifted to the left by 128 samples from the position of the current region (1216).

[0150] Refer again Figure 12A When the current region (1216) is the upper left region of the current CTB (1215), the juxtaposed region (1211) is the upper left region of the previously reconstructed CTB (1210), and the search region excludes the upper left region of the previously reconstructed CTB.

[0151] Reference Figure 12B The current region (1217) is being reconstructed. The current region (1217) may include multiple coding blocks (1241) to (1249). The current region (1217) has a juxtaposed region (i.e., region (1212) in the previously reconstructed CTB (1210). The search range of one of the multiple coding blocks (1241) to (1249) can exclude the juxtaposed region (1212). The search range includes regions (1213) to (1214) of the previously reconstructed CTB (1210) and region (1216) in the current CTB (1215) that is reconstructed after the juxtaposed region (1212) and before the current region (1217). Due to the limitation of the reference memory size (i.e., one CTB size), the search range further excludes region (1211). Similarly, the position of the juxtaposed region (1212) is offset from the position of the current region (1217) by the CTB width, for example, 128 samples.

[0152] exist Figure 12B In the example, the current region (1217) is the upper right region of the current CTB (1215), the juxtaposed region (1212) is also the upper right region of the previously reconstructed CTB (1210), and the search region excludes the upper right region of the previously reconstructed CTB (1210).

[0153] Reference Figure 12CThe current region (1218) is being reconstructed. The current region (1218) may include multiple coding blocks (1261) to (1269). The current region (1218) has a juxtaposed region (i.e., region (1213)) in the previously reconstructed CTB (1210). The search range of one of the multiple coding blocks (1261) to (1269) may exclude the juxtaposed region (1213). The search range includes region (1214) of the previously reconstructed CTB (1210) and regions (1216) to (1217) of the current CTB (1215) that are reconstructed after the juxtaposed region (1213) and before the current region (1218). Similarly, due to the limitation of the reference memory size, the search range further excludes regions (1211) to (1212). The position of the juxtaposed region (1213) is offset from the position of the current region (1218) by the CTB width, for example, 128 samples. Figure 12C In the example, when the current region (1218) is the lower left region of the current CTB (1215), the juxtaposed region (1213) is also the lower left region of the previously reconstructed CTB (1210), and the search region excludes the lower left region of the previously reconstructed CTB (1210).

[0154] Reference Figure 12D The current region (1219) is being reconstructed. The current region (1219) may include multiple coded blocks (1281) to (1289). The current region (1219) has a juxtaposed region (i.e., region (1214)) in the previously reconstructed CTB (1210). The search range of one of the multiple coded blocks (1281) to (1289) may exclude the juxtaposed region (1214). The search range includes regions (1216) to (1218) in the current CTB (1215) that are reconstructed in decoding order after the juxtaposed region (1214) and before the current region (1219). Due to the limitation of the reference memory size, the search range excludes regions (1211) to (1213), and therefore, the search range excludes the previously reconstructed CTB (1210). Similarly, the position of the juxtaposed region (1214) is offset from the position of the current region (1219) by the CTB width, for example, 128 samples. Figure 12D In the example, when the current region (1219) is the lower right region of the current CTB (1215), the juxtaposed region (1214) is also the lower right region of the previously reconstructed CTB (1210), and the search region excludes the lower right region of the previously reconstructed CTB (1210).

[0155] Referring back to Figure 2, the MV associated with the five surrounding samples (or locations) (202 to 206, respectively) denoted as A0, A1, and B0, B1, B2 can be referred to as spatial merge candidates. A candidate list (e.g., a merged candidate list) can be formed based on the spatial merge candidates. Any suitable order can be used to form the candidate list according to location. In the example, the order could be A0, B0, B1, A1, and B2, where A0 is the first and B2 is the last. In the example, the order could be A1, B1, B0, A0, and B2, where A1 is the first and B2 is the last.

[0156] According to some implementations, motion information from previous encoded blocks used for the current block (e.g., coding block (CB) or current CU) can be stored in a history-based motion vector prediction (HMVP) buffer (e.g., a table) to provide motion vector prediction (MVP) candidates (also known as HMVP candidates) for the current block. The HMVP buffer may include one or more HMVP candidates and can be maintained during encoding / decoding processing. In the example, the HMVP candidates in the HMVP buffer correspond to the motion information of previous encoded blocks. The HMVP buffer can be used with any suitable encoder and / or decoder. HMVP candidates can be added to the merged candidate list after one or more spatial MVPs and TMVPs.

[0157] When a new CTU (or a new CTB) line is encountered, the HMVP buffer can be reset (e.g., cleared). When a non-sub-block inter-coded block is present, the associated motion information can be added as a new HMVP candidate to the last entry of the HMVP buffer.

[0158] In the example, such as in VTM3, the HMVP buffer size (denoted by S) is set to 6, indicating that up to 6 HMVP candidates can be added to the HMVP buffer. In some implementations, the HMVP buffer can operate according to a first-in-first-out (FIFO) rule, and therefore, for example, when the HMVP buffer is full, the motion information (or HMVP candidates) stored in the HMVP buffer first are removed from the HMVP buffer. When a new HMVP candidate is inserted into the HMVP buffer, a constrained FIFO rule can be used, where a redundancy check is first applied to determine if the same or similar HMVP candidate is in the HMVP buffer. If the same or similar HMVP candidate is determined to be in the HMVP buffer, the same or similar HMVP candidate can be removed from the HMVP buffer, and the remaining HMVP candidates can be moved forward in the HMVP buffer.

[0159] HMVP candidates can be used for the merge candidate list construction process, for example, in a merge pattern. The most recently stored HMVP candidates in the HMVP buffer can be checked sequentially, and these HMVP candidates can be inserted into the merge candidate list after the TMVP candidates. Redundancy checks can be applied to the HMVP candidates with respect to spatial or temporal merge candidates in the merge candidate list. These descriptions can be appropriately adapted to an AMVP pattern to construct an AMVP candidate list.

[0160] To reduce the number of redundant verification operations, the following simplification can be used.

[0161] (i) Can the number of HMVP candidates used to generate the merge candidate list be set to (N<=4)? M:(8–N). N indicates the number of existing candidates in the merge candidate list, and M indicates the number of available HMVP candidates in the HMVP buffer. When the number of existing candidates (N) in the merge candidate list is less than or equal to 4, the number of HMVP candidates used to generate the merge candidate list is equal to M. Otherwise, the number of HMVP candidates used to generate the merge candidate list is equal to (8-N).

[0162] (ii) When the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list construction process from the HMVP buffer terminates.

[0163] When IBC mode operates as a mode separate from inter-frame prediction mode, simplified BV derivation processing of IBC mode can be used. A history-based block vector prediction buffer (called an HBVP buffer) can be used to perform BV prediction. The HBVP buffer can be used to store BV information (e.g., BV) of previous encoded blocks for the current block (e.g., CB or CU) in the current picture. In the example, the HBVP buffer is a history buffer separate from other buffers (e.g., the HMVP buffer). The HBVP buffer can be a table.

[0164] An HBVP buffer can provide BV predictor (BVP) candidates (also known as HBVP candidates) for the current block. An HBVP buffer (e.g., a table) can include one or more HBVP candidates and can be maintained during encoding / decoding processing. In the example, the HBVP candidates in the HBVP buffer correspond to the BV information of previously encoded blocks in the current picture. HBVP buffers can be used with any suitable encoder and / or decoder. HBVP candidates can be added to a merge candidate list configured for BV prediction after the BV of spatially adjacent blocks of the current block. The merge candidate list configured for BV prediction can be used to merge BV prediction modes and / or non-merge BV prediction modes.

[0165] When a new CTU (or a new CTB) line is encountered, the HBVP buffer can be reset (e.g., cleared).

[0166] In the example, such as in VVC, the HBVP buffer size is set to 6, indicating that a maximum of 6 HBVP candidates can be added to the HBVP buffer. In some implementations, the HBVP buffer can operate according to a FIFO rule, and therefore, for example, when the HBVP buffer is full, the BV information (or HBVP candidates) stored in the HBVP buffer are removed first. When a new HBVP candidate is inserted into the HBVP buffer, a restricted FIFO rule can be used, where a redundancy check is first applied to determine if there is an identical or similar HBVP candidate in the HBVP buffer. If an identical or similar HBVP candidate is determined to be in the HBVP buffer, it can be removed from the HBVP buffer, and the remaining HBVP candidates can be shifted forward in the HBVP buffer.

[0167] HBVP candidates can be used for candidate list construction processing, such as merging BV prediction patterns. The most recently stored HBVP candidates in the HBVP buffer can be checked sequentially, and these HBVP candidates can be inserted into the merge candidate list after the spatial candidates. Redundancy checks can be applied to the HBVP candidates with respect to the spatial merge candidates in the merge candidate list.

[0168] In this implementation, an HBVP buffer is established to store one or more BV information entries for one or more previously encoded blocks encoded in IBC mode. The one or more BV information entries may include one or more BV entries for one or more previously encoded blocks encoded in IBC mode. Furthermore, each of the one or more BV information entries may include side information (or additional information), such as the block size, block position, etc., of each previously encoded block encoded in IBC mode.

[0169] In class-based and history-based block vector prediction (also known as CBVP), for the current block, one or more BV (Block Value) entries in the HBVP buffer that meet certain conditions can be classified into corresponding categories (also known as classes), thus forming the CBVP buffer. In the example, each BV entry in the HBVP buffer is used for, for example, a corresponding previously encoded block encoded in IBC mode. This BV entry for the previously encoded block can include BV, block size, block position, etc. The previously encoded block has a block width, block height, and block area. The block area can be the product of the block width and the block height. In the example, the block size is represented by the block area. The block position of the previously encoded block can be represented by the top-left corner of the previously encoded block (e.g., the top-left corner of a 4×4 region) or the top-left sample.

[0170] Figure 13 An example of a spatial class for IBCBV prediction of a current block (e.g., CB, CU) (1310) according to an embodiment of this disclosure is shown. A left region (1302) may be located to the left of the current block (1310). BV information of a previous coded block with the corresponding block position in the left region (1302) may be referred to as a left candidate or left BV candidate. An upper region (1303) may be located above the current block (1310). BV information of a previous coded block with the corresponding block position in the upper region (1303) may be referred to as an upper candidate or upper BV candidate. A top-left region (1304) may be located to the upper left of the current block (1310). BV information of a previous coded block with the corresponding block position in the top-left region (1304) may be referred to as a top-left candidate or top-left BV candidate. A top-right region (1305) may be located to the upper right of the current block (1310). The BV information of the previous encoded block with the corresponding block position in the upper right region (1305) can be referred to as the upper right candidate or upper right BV candidate. The lower left region (1306) can be located to the lower left of the current block (1310). The BV information of the previous encoded block with the corresponding block position in the lower left region (1306) can be referred to as the lower left candidate or lower left BV candidate. Other types of space classes can also be defined and used in the CBVP buffer.

[0171] If the BV information of the previously encoded block meets the following conditions, the BV information can be classified into the corresponding category (or class).

[0172] (i) Class 0: Block size (e.g., block area) is greater than or equal to the threshold (e.g., 64 pixels).

[0173] (ii) Class 1: The occurrence count (or frequency) of the BV is greater than or equal to 2. The occurrence count of the BV can refer to the number of times the BV is used to predict previous coded blocks. When using pruning to form the CBVP buffer, when the BV is used multiple times to predict previous coded blocks, the BV can be stored in one entry (instead of multiple entries with the same BV). The occurrence count of the BV can be recorded.

[0174] (iii) Class 2: The block is located in the left region (1302), wherein a portion of the previously encoded block (e.g., the top left corner of a 4×4 region) is to the left of the current block (1310). The previously encoded block may be within the left region (1302). Alternatively, the previously encoded block may span multiple regions, including the left region (1302), wherein the block is located in the left region (1302).

[0175] (iv) Class 3: The block is located in the upper region (1303), wherein a portion of the previously encoded block (e.g., the upper left corner of a 4×4 region) is above the current block (1310). The previously encoded block may be within the upper region (1303). Alternatively, the previously encoded block may span multiple regions, including the upper region (1303), wherein the block is located within the upper region (1303).

[0176] (v) Class 4: The block is located in the top-left region (1304), where a portion of the previously encoded block (e.g., the top-left corner of a 4×4 region) is located to the top left of the current block (1310). The previously encoded block may be within the top-left region (1304). Alternatively, the previously encoded block may span multiple regions, including the top-left region (1304), where the block is located within the top-left region (1304).

[0177] (vi) Class 5: The block is located in the upper right region (1305), where a portion of the previously encoded block (e.g., the upper left corner of a 4×4 region) is located to the upper right of the current block (1310). The previously encoded block may be within the upper right region (1305). Alternatively, the previously encoded block may span multiple regions, including the upper right region (1305), where the block is located within the upper right region (1305).

[0178] (vii) Class 6: The block is located in the lower left region (1306), where a portion of the encoded block (e.g., the upper left corner of a 4×4 region) is located to the lower left of the current block (1310). Previous encoded blocks may be located within the lower left region (1306). Alternatively, previous encoded blocks may span multiple regions, including the lower left region (1306), where the block is located within the lower left region (1306).

[0179] For each category (or class), the BV of the most recently encoded block can be derived as a BVP candidate. A CBVP buffer can be constructed by appending BV predictors for each category in order from class 0 to class 6. The above description of CBVP can be appropriately adapted to include fewer or additional classes not described above. One or more of classes 0 to 6 can be modified. In the example, each entry in the HBVP buffer is classified into one of the seven classes 0 to 6. An index can be signaled to indicate which class 0 to 6 has been selected. On the decoder side, the first entry in the selected class can be used to predict the BV of the current block.

[0180] This disclosure provides techniques for reference position constraints in string copy mode. String copy mode is also known as string matching (mode) or string prediction (mode). String matching mode is similar to intra block copy (IBC) and can reconstruct reconstructed regions based on sample strings within the same image. Furthermore, string matching mode offers greater flexibility regarding the shape of the sample strings. For example, blocks can have rectangular shapes and strings can form non-rectangular shapes.

[0181] Figure 14 An example of a string copying mode according to an embodiment of this disclosure is shown. The current image (1410) includes a reconstructed region (gray area) (1420) and a region being reconstructed (1421). The current block (1435) in region (1421) is being reconstructed. The current block (1435) can be CB, CU, etc. The current block (1435) can include multiple strings, for example... Figure 14 The example shows strings (1430) and (1431). In the example, the current block (1435) is divided into multiple consecutive strings, where one string is followed by the next in the scan order. The scan order can be any suitable scan order, such as raster scan order, transverse scan order, etc.

[0182] The reconstructed region (1420) can be used as a reference region to reconstruct strings (1430) and (1431).

[0183] For each of the multiple strings, a signal can be used to indicate the string offset vector (also called the string vector (SV)) and the length of the string (called the string length). SV can be a displacement vector indicating the displacement offset between the string to be reconstructed and a reference string located in the reference region (1420) and already reconstructed. The reference string can be used to reconstruct the string to be reconstructed. For example, SV0 is a displacement vector indicating the displacement offset between string (1430) and reference string (1400), and SV1 is a displacement vector indicating the displacement offset between string (1431) and reference string (1401). Therefore, SV can indicate where the corresponding reference string is located in the reference region (1420). The string length indicates the number of samples in the string. Typically, the string to be reconstructed has the same length as the reference string.

[0184] Reference Figure 14 The current block (1435) is an 8×8 CB containing 64 samples. The current block (1435) is divided into a string (1430) and a string (1431) using a raster scan sequence. String (1430) includes the first 29 samples of the current block (1435), and string (1431) includes the remaining 35 samples of the current block (1435). The reference string (1400) used to reconstruct string (1430) can be indicated by the corresponding string offset vector SV0, and the reference string (1401) used to reconstruct string (1431) can be indicated by the corresponding string offset vector SV1.

[0185] Typically, string size can refer to either the length of the string or the number of samples in the string. (See reference...) Figure 14 String (1430) contains 29 samples, and therefore the string size of string (1430) is 29. String (1431) contains 35 samples, and therefore the string size of string (1431) is 35. String position (or string location) can be represented by the sample position of a sample in the string (e.g., the first sample in decoding order).

[0186] The above description can be appropriately adapted to reconstruct a current block comprising any suitable number of strings. Alternatively, in the example, when a sample in the current block has no matching sample in the reference region, an escaped sample is signaled, and the value of the escaped sample can be directly encoded without referring to the reconstructed sample in the reference region.

[0187] In some examples, the available reference samples used in string matching can be aligned with the reference regions used in IBC mode. Furthermore, the reference sample availability logic, which in IBC mode can be applied to a combination of the reconstructed portion of the current CTU and a region of the left CTU, can also be similarly applied to string matching.

[0188] In some examples, a reference memory is used to store the most recently reconstructed samples, and this reference memory can be accessed in IBC mode to reconstruct the current block, or in string matching mode to reconstruct the current string. The string vector can then be constrained based on the reference memory to ensure that reference samples used for reconstructing the current block or string are stored in the reference memory. In the examples, it is assumed that the reference memory size is the same as the CTU size (e.g., 128 × 128 samples). In some examples, the reference memory is implemented for fast access.

[0189] Figure 15 The process for reference memory update in IBC mode is shown in some examples, and the reference memory update process can be similarly applied to string matching mode.

[0190] exist Figure 15 In the example, each CTU has a size of 128×128 samples, therefore the reference memory has a size for storing 128×128 samples. Figure 15 In this configuration, the reference memory is divided into four sub-parts, each with a size of 64×64 samples. In some examples, during the encoding of the current CTU, the reference memory is updated one sub-part after another. Figure 15 The state of the reference memory during encoding at the current CTU is shown (reference memory view), along with the corresponding encoding / decoding process based on the CTU (CTU view). Figure 15 In the diagram, the encoded portion is shown in gray, the uncoded portion is shown in white, the current encoded block is shown by a striped pattern, and the reference block of the current encoded block is shown by a dashed line.

[0191] For example, initially, the reference memory is in state (0) as shown in (1510) and stores reconstructed samples from the left CTU of the current CTU. For example, the left CTU can be divided into four sub-blocks labeled 0, 1, 2, and 3, and as shown in (1510), the reference memory stores reconstructed samples from sub-blocks 0 to 3 of the left CTU. The current CTU can be divided into four sub-blocks labeled 4, 5, 6, and 7.

[0192] In order to encode / decode subblock 4 in the current CTU, the reference memory enters state (1) as shown in (1520). The reference memory can still store reconstructed samples of subblocks 1, 2, and 3 of the left CTU. The sub-part of the reference memory used to store the reconstructed samples of subblock 0 of the left CTU will be used to store the reconstructed samples of subblock 4 of the current CTU. In the example, the sub-part of the reference memory used to store the reconstructed samples of subblock 0 can be cleared before encoding subblock 4 of the current CTU.

[0193] From the CTU view, as shown in (1525), sub-block 0 of the left CTU is not in the reference memory and is marked with an "X". Sub-blocks 1 to 3 in the left CTU are still in the reference memory.

[0194] In order to encode / decode subblock 5 in the current CTU, the reference memory enters state (2) as shown in (1530). The reference memory can still store reconstructed samples of subblocks 2 and 3 of the left CTU and subblock 4 of the current CTU. The sub-part of the reference memory used to store the reconstructed sample of subblock 1 of the left CTU will be used to store the reconstructed sample of subblock 5 of the current CTU.

[0195] From the CTU view, as shown in (1535), sub-blocks 0 and 1 of the left CTU are not in the reference memory and are marked with an "X". Sub-blocks 2 and 3 of the left CTU are still in the reference memory.

[0196] In order to encode / decode subblock 6 in the current CTU, the reference memory enters state (3) as shown in (1540). The reference memory can still store reconstructed samples of subblock 3 of the left CTU and subblocks 4 and 5 of the current CTU. The sub-part of the reference memory used to store the reconstructed samples of subblock 2 of the left CTU will be used to store the reconstructed samples of subblock 6 of the current CTU.

[0197] From the CTU view, as shown in (1545), sub-blocks 0, 1, and 2 of the left CTU are not in the reference memory and are marked with an "X". Sub-block 3 in the left CTU is still in the reference memory.

[0198] In order to encode / decode subblock 7 in the current CTU, the reference memory enters state (4) as shown in (1550). The reference memory can still store reconstructed samples of subblocks 4, 5 and 6 of the current CTU. The sub-part of the reference memory used to store the reconstructed samples of subblock 3 of the left CTU will be used to store the reconstructed samples of subblock 7 of the current CTU.

[0199] From the CTU view, as shown in (1555), sub-blocks 0, 1, 2 and 3 of the left CTU are not in the reference memory and are marked with "X".

[0200] exist Figure 15 In the example, in each state (at the beginning of encoding each 64×64 sub-block of the current CTU), the available reference area with the corresponding sample available in the reference memory is shown in gray without an “X” mark.

[0201] Based on some aspects of this disclosure, reference sample constraints can be relaxed for string matching patterns, thereby making more potential reference samples available and improving coding efficiency.

[0202] In the example, the IBC mode has a non-overlap constraint that requires the reference block to not overlap with the current block. The non-overlap constraint can be similarly applied to string matching modes. For example, the non-overlap constraint in string matching mode requires that samples in the reference string should not overlap with samples in the current string. The non-overlap constraint is used to avoid accessing unreconstructed samples in the reference string when copying reference samples to fill the current string. In a related example, the reconstruction of the current string can be performed by a memory copy operation, copying values ​​stored in a first memory space corresponding to the reference string to a second memory space corresponding to the current string. To perform the memory copy operation, samples of the reference string need to be reconstructed and stored in the first memory space, thus requiring the current string to not overlap with the reference string. The non-overlap constraint can restrict string vectors and limit coding efficiency.

[0203] In the following text, the term "block" can be interpreted as a prediction block, coding block, or coding unit, i.e., CU.

[0204] Some aspects of this disclosure provide techniques for string matching patterns that allow for overlapping samples between a current string and a reference string. The string vector of the current string is represented by SV(svx, svy), where svx is the horizontal component and svy is the vertical component. In some examples, the top-left corner of the image is considered the origin of the image's two-dimensional plane, with the horizontal values ​​(e.g., x-values) of the two-dimensional plane increasing from left to right, and the vertical values ​​(e.g., y-values) increasing from top to bottom.

[0205] According to some aspects of this disclosure, samples in the current string can be reconstructed, for example, in a part-by-part processing. In part-by-part processing, the first part of the current string is reconstructed before the second part of the current string is reconstructed. After the first part of the current string is reconstructed, the second part of the current string can be reconstructed based on the first part. In some examples, the reconstruction of the current string can be performed by multiple memory copy operations. For example, the reconstruction of the first part of the current string is performed by a first memory copy operation storing sample values ​​in a first memory space corresponding to the first part of the current string, and the reconstruction of the second part of the current string is performed by a second memory copy operation storing sample values ​​in a second memory space corresponding to the second part of the current string. The second memory copy operation copies at least the sample values ​​in the first memory space to be stored in the second memory space.

[0206] It should be noted that in the above description, the portions of the current string can each have any suitable number of samples and can have any suitable shape. In one example, a portion of the string is a sample row that includes rows containing samples from the string. In another example, a portion of the string is a sample column that includes columns containing samples from the string. In yet another example, a portion of the string is a sample from the string.

[0207] For ease of description, a horizontal scanning order is assumed for the following description of the technique for reference position constraints in string matching patterns. A corresponding technique for reference position constraints in string matching patterns with a vertical scanning order can be appropriately obtained in a similar manner.

[0208] According to one aspect of this disclosure, when the current string comprises multiple rows (e.g., sample rows in the horizontal direction, sample columns in the vertical direction), the reconstruction of the current string can be performed row by row (e.g., row by row, column by column).

[0209] In the example, the current string consists of multiple sample lines, each of which can be scanned in a horizontal scan order. The reconstruction of the current string can be performed line by line. Specifically, once a sample in one line of the current string has been reconstructed, the reconstruction of the samples in the next line of the current string (in scan order) can be performed.

[0210] Figure 16 An example of string reconstruction processing based on some examples is shown. Figure 16 The reconstruction process for the current string (1615) in block (1610) is shown. A string vector SV is determined for the current string (1615). The string vector SV points to a reference string that overlaps with the current string (1615). Figure 16 In the example, the current string (1615) includes samples from three rows that can be scanned according to the horizontal scan order, such as the first sample row (1611), the second sample row (1612), and the third sample row (1613).

[0211] The current string (1615) is reconstructed line by line. For example, in the first step (step 1), the first sample line (1611) of the current string (1615) is reconstructed based on the pixels (1621) of the reference string. In the example, a memory copy operation can be performed to copy the sample value stored in the memory space corresponding to the pixel (1621) to the memory space corresponding to the first sample line (1611).

[0212] Furthermore, in the second step (step 2), the second sample row (1612) of the current string (1615) is reconstructed based on the pixels (1622) of the reference string. In the example, a memory copy operation can be performed to copy the sample value stored in the memory space corresponding to the pixel (1622) to the memory space corresponding to the second sample row (1612).

[0213] Furthermore, in the third step (step 3), the third sample row (1613) of the current string (1615) is reconstructed based on the pixels (1623) of the reference string. In the example, a memory copy operation can be performed to copy the sample value stored in the memory space corresponding to the pixel (1623) to the memory space corresponding to the third sample row (1613).

[0214] According to aspects of this disclosure, non-overlap constraints can be relaxed without imposing them. In some examples, when at least one of the vertical and horizontal components of the string vector is negative, the string vector satisfies the requirement that the current string partially overlaps with the reference string, and no non-overlap constraint is imposed.

[0215] In the example using the horizontal scan order, when the vertical component svy of the string vector SV is negative (svy < 0), the reference string is above the current string. Then, regardless of the value of the horizontal component svx of the string vector SV, overlap between the current string and the reference string is allowed, thus not imposing overlap constraints.

[0216] However, when the vertical component svy of the string vector SV is not negative (svy≥0), the reference string is either on the same line as the current string or below the current string. Then, regardless of the value of the horizontal component svx, an overlap constraint needs to be imposed to prevent the current string from overlapping with its reference string.

[0217] Note that in the example above, the entire reference string should be within the allowed reference area.

[0218] Typically, reference memory is formed by multiple parts, such as multiple pages. In some examples, access to memory space within the same part (e.g., the same page) can be easily performed, for example, without needing to compute large offset values. In some examples, samples of the reference string are constrained to regions corresponding to pages of memory space.

[0219] According to aspects of this disclosure, samples of the reference string are allowed to span the boundaries of two regions corresponding to two pages in the memory space.

[0220] Figure 17 An example of a reference string that crosses a region boundary is shown. Figure 17 In the example, block (1700) can be a 128×128 (sample) block, such as CTU. Block (1700) is divided into four 64×64 sub-blocks A to D. Sub-blocks A, B, and C have already been reconstructed, and the current string to be reconstructed is in sub-block D. Figure 17 In the example, the current string (1710) to be reconstructed is in sub-block D, and the reference string (1720) spans the boundaries of sub-blocks A and B.

[0221] Figure 18Another example of a reference string that spans a region boundary is shown. Figure 18 In the example, block (1800) can be a 128×128 (sample) block, such as CTU. Block (1800) is divided into four 64×64 sub-blocks A to D. Sub-blocks A, B, and C are reconstructed, and the current string to be reconstructed is in sub-block D. Figure 18 In the example, the current string (1810) to be reconstructed is in sub-block D, and the reference string (1820) spans the boundaries of sub-blocks A and C.

[0222] In some examples, for memory reuse, the reference memory is divided into multiple regions of equal size. Multiple regions can be allocated to store samples of sub-blocks. For example, when reconstructing sub-block D, the first region of the reference memory can store the reconstructed samples of sub-block A; the second region of the reference memory can store the reconstructed samples of sub-block B; the third region of the reference memory can store the reconstructed samples of sub-block C; and a fourth region of the reference memory can be allocated to store the samples to be reconstructed from sub-block D.

[0223] In some implementations, boundary crossing constraints can be used. In one implementation, a vertical boundary crossing constraint is applied when the entire reference string is within an allowable reference region. The vertical boundary crossing constraint disallows the reference string of the current string from vertically crossing the boundaries of two sub-blocks, but allows the reference string to horizontally cross the boundaries of two sub-blocks. For example, reference string (1720) is allowed, but reference string (1820) is not. In this example, the starting position of the reference string is (px0, py0). For any sample (pxi, pii) in the reference string, the vertical boundary crossing constraint requires that (floor(py0 / 64)) equals (floor(pyi / 64)).

[0224] In another implementation, a horizontal cross-boundary constraint is applied when the entire reference string is within an allowable reference region. The horizontal cross-boundary constraint disallows the reference string of the current string from horizontally crossing the boundaries of two sub-blocks, but allows the reference string to vertically cross the boundaries of two sub-blocks. For example, reference string (1820) is allowed, but reference string (1720) is not. In the example, the starting position of the reference string is (px0, py0), and for any sample (pxi, pii) of the reference string, the horizontal cross-boundary constraint requires that (floor(px0 / 64)) equals (floor(pxi / 64)).

[0225] According to aspects of this disclosure, the constraint of limiting the reference string to the current CTU can be relaxed. For example, the sample row above the current CTU can also be used as a reference sample in the reference string.

[0226] Figure 19 An example of a reference string with a sample above the current CTU is shown. Figure 19 In the example, the current CTU(1901) is a 128×128 (sample) block. The current CTU(1901) is divided into four 64×64 sub-blocks A to D. Sub-blocks A, B, and C are reconstructed, and the current string to be reconstructed is in sub-block D. Figure 19 In the example, the current string (1910) to be reconstructed is in sub-block D, and the reference string (1920) has at least one sample in the sample line (1930) above the current CTU (1901). Overlap between the reference string (1920) and the sample line (1930) above the current CTU (1901) is allowed. In the example, the decoder includes a specific buffer (referred to as the line buffer in the example) to store samples in the sample line, such as the sample line (1930) above the current CTU (1901), and this specific buffer can be accessed during string matching mode. Therefore, when the reference string (1920) includes samples that overlap with the sample line (1930), the specific buffer can be accessed to copy the overlapping samples to reconstruct the current string (1910).

[0227] Figure 20 A flowchart of an overview process (2000) according to an embodiment of the present disclosure is shown. The process (2000) can be used to reconstruct blocks or strings in a picture of an encoded video sequence. The process (2000) can be used to reconstruct blocks to generate predicted blocks of the reconstructed blocks. The term "block" in this disclosure can be interpreted as a predicted block, CB, CU, etc. In various embodiments, the process (2000) is executed by a processing circuitry system, such as: the processing circuitry system in terminal devices (310), (320), (330), and (340); a processing circuitry system performing the function of a video encoder (403); a processing circuitry system performing the function of a video decoder (410); a processing circuitry system performing the function of a video decoder (510); a processing circuitry system performing the function of a video encoder (603), etc. In some embodiments, the process (2000) is implemented as software instructions, so that the processing circuitry system executes the process (2000) when the software instructions are executed. The process begins at (S2001) and proceeds to (S2010).

[0228] At (S2010), determine the (potential) string vector of the current string in the CTU. The (potential) string vector points to the reference string of the current string.

[0229] At (S2020), it is determined that the (potential) string vector satisfies the requirement that the current string partially overlaps with the reference string.

[0230] In some examples, in response to at least one of the vertical and horizontal components of the (potential) string vector being negative, it is determined that the (potential) string vector satisfies the requirement that the current string partially overlaps with the reference string.

[0231] In one example, the overlap constraint can be skipped in response to satisfying the requirement that the current string partially overlaps with the reference string. In another example, an overlap constraint is applied to the (potential) string vector in response to not satisfying the requirement that the current string partially overlaps with the reference string.

[0232] At (S2030), the current string is reconstructed based on the reference string.

[0233] In some examples, the current string is reconstructed in parts. For instance, the first part of the current string is reconstructed before the second part is reconstructed. Then, the second part of the current string can be reconstructed based on the first part.

[0234] In the example, a horizontal scan order is used for the string. Then, in response to the vertical component of the string vector being negative, the current string is reconstructed line by line. For example, the first sample line of the current string is reconstructed, and then the second sample line of the current string is reconstructed based on the first sample line.

[0235] In another example, a vertical scan order is used for the string. Then, in response to the horizontal component of the string vector being negative, the current string is reconstructed column by column. For example, the first sample column of the current string is reconstructed, and the second sample column of the current string is reconstructed based on the first sample column.

[0236] In some implementations, the reference string overlaps with the sample line above the current CTU, and then the buffer storing samples in the sample line can be accessed to reconstruct the current string.

[0237] Then, the process proceeds to (S2099) and terminates.

[0238] The processing (2000) can be adjusted as appropriate. One or more steps in the processing (2000) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used. For example, when it is determined that the current vector information is unique, as described above, the current vector information can be stored in a history buffer. In some examples, when the current vector information is stored in the history buffer, a pruning process is used and one of the vector information items in the history buffer is removed.

[0239] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 21 A computer system (2100) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0240] Computer software can be coded using any suitable machine code or computer language. Machine code or computer language can be subjected to mechanisms such as assembly, compilation, and linking to create code that includes instructions. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, etc.

[0241] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0242] Figure 21 The components shown for the computer system (2100) are exemplary in nature and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in the exemplary embodiments of the computer system (2100).

[0243] Computer system 2100 may include certain human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swiping, movement with a data glove), audio input (e.g., voice, tapping), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device can also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0244] The input human-machine interface device may include one or more of the following (only one of each is shown): keyboard (2101), mouse (2102), touchpad (2103), touch screen (2110), data glove (not shown), joystick (2105), microphone (2106), scanner (2107), and camera device (2108).

[0245] The computer system (2100) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touchscreen (2110), data gloves (not shown), or joystick (2105), but tactile feedback devices that are not used as input devices may also exist); audio output devices (e.g., speakers (2109), headphones (not depicted)); visual output devices (e.g., screens (2110), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability—some of which may be able to output two-dimensional or more than three-dimensional visual output in a manner such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and ashtrays (not depicted)); and printers (not depicted).

[0246] The computer system (2100) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (2120) having media such as CD / DVD (2121), thumb drives (2122), removable hard disk drives or solid-state drives (2123), conventional magnetic media (such as magnetic tape and floppy disks (not depicted)), devices based on dedicated ROM / ASIC / PLD (such as security dongles (not depicted)), etc.

[0247] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0248] The computer system (2100) may also include interfaces (2154) to one or more communication networks (2155). Networks may be, for example, wireless networks, wired networks, or optical networks. Networks may also be local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), vehicular and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of networks include: local area networks (e.g., Ethernet, wireless LANs), cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable television connections or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, and vehicular and industrial networks including CAN buses, etc. Some networks typically require connection to certain general-purpose data ports or peripheral buses (2149).

[0249] External network interface adapters (e.g., USB ports of computer system (2100)); other networks are typically integrated into the core of computer system (2100) via attachment to system buses as described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). Computer system (2100) can communicate with other entities using any of these networks. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., a CAN bus to some CAN bus device), or bidirectional (e.g., using a local area digital network or a wide area digital network to other computer systems). Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0250] The aforementioned human-machine interface device, human-accessible storage device, and network interface can be attached to the core (2140) of the computer system (2100).

[0251] The core (2140) may include one or more Central Processing Units (CPUs) (2141), Graphics Processing Units (GPUs) (2142), Dedicated Programmable Processing Units in the form of Field Programmable Gate Areas (FPGAs) (2143), hardware accelerators (2144) for certain tasks, graphics adapters (2150), etc. These devices, along with Read-Only Memory (ROM) (2145), Random Access Memory (2146), and internal mass storage devices (e.g., internal non-user-accessible hard disk drives, SSDs, etc.) (2147), may be connected via a system bus (2148). In some computer systems, the system bus (2148) may be accessed as one or more physical connectors to allow for expansion via additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (2149) to the core's system bus (2148). In this example, a screen (2110) may be connected to a graphics adapter (2150). Peripheral bus architectures include PCI, USB, etc.

[0252] The CPU (2141), GPU (2142), FPGA (2143), and accelerator (2144) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (2145) or RAM (2146). Transient data can also be stored in RAM (2146), while permanent data can be stored, for example, in an internal mass storage device (2147). Fast storage and retrieval of any storage device can be achieved using a cache memory, which can be closely associated with one or more CPUs (2141), GPUs (2142), mass storage devices (2147), ROMs (2145), RAMs (2146), etc.

[0253] Computer-readable media may contain computer code for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of types known and available to those skilled in the art of computer software.

[0254] By way of example and not limitation, a computer system (2100) with an architecture, and in particular a core (2140), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices as described above, as well as certain storage devices of the core (2140) having non-transitory characteristics, such as in-core mass storage (2147) or ROM (2145). Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core (2140). Depending on specific needs, the computer-readable media may include one or more memory devices or chips. The software can cause the core (2140)—and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.)—to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (2146) and modifying such data structures according to software-defined processes. Alternatively or additionally, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (2144)), which may operate in place of or with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) storing software for execution, circuitry implementing logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0255] Appendix A: Acronyms

[0256] JEM: Joint Exploration Model

[0257] VVC: Universal Video Coding

[0258] BMS: Benchmark Set

[0259] MV: Motion Vector

[0260] HEVC: High-Efficiency Video Coding

[0261] MPM: Most Likely Pattern

[0262] WAIP: Wide-angle Intra-frame Prediction

[0263] SEI: Supplemental Enhancement Information

[0264] VUI: Video Availability Information

[0265] GOP: Image Group

[0266] TU: Transformation Unit

[0267] PU: Prediction Unit

[0268] CTU: Coding Tree Unit

[0269] CTB: Coded Tree Block

[0270] PB: Prediction Block

[0271] HRD: Assuming a reference decoder

[0272] SDR: Standard Dynamic Range

[0273] SNR: Signal-to-noise ratio

[0274] CPU: Central Processing Unit

[0275] GPU: Graphics Processing Unit

[0276] CRT: Cathode Ray Tube

[0277] LCD: Liquid Crystal Display

[0278] OLED: Organic Light Emitting Diode

[0279] CD: Compact Disc

[0280] DVD: Digital Video Disc

[0281] ROM: Read-Only Memory

[0282] RAM: Random Access Memory

[0283] ASIC: Application-Specific Integrated Circuit

[0284] PLD: Programmable Logic Device

[0285] LAN: Local Area Network

[0286] GSM: Global System for Mobile Communications

[0287] LTE: Long Term Evolution

[0288] CANBus: Controller Area Network Bus

[0289] USB: Universal Serial Bus

[0290] PCI: Peripheral Component Interconnect

[0291] FPGA: Field Programmable Gate Domain

[0292] SSD: Solid State Drive

[0293] IC: Integrated Circuit

[0294] CU: Encoding Unit

[0295] PDPC: Location-Related Prediction Combination

[0296] ISP: Intra-Frame Sub-Partition

[0297] SPS: Sequence Parameter Set

[0298] While this disclosure has described several exemplary embodiments, variations, substitutions, and various alternative equivalents fall within the scope of this disclosure. It will therefore be appreciated that, although not expressly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that implement the principles of this disclosure and thus within its spirit and scope.

Claims

1. A method for performing video decoding in a decoder, characterized in that, The method includes: The processor determines the string vector of the current string in the first sub-block of the current coding tree unit (CTU). The string vector points to a reference string of the current string. The reference string is positioned across a boundary, such that the first part of the reference string is located in the second sub-block of the CTU, and the second part of the reference string is located in the third sub-block of the CTU. Determine whether a vertical or horizontal boundary crossing constraint is applied to the reference string; In response to the application of the vertical cross-boundary constraint, and the determination that the boundary between the second sub-block and the third sub-block is a vertical boundary, the processor determines that the reference string is not allowed; and, In response to the application of the horizontal cross-boundary constraint, and the determination that the boundary between the second sub-block and the third sub-block is a horizontal boundary, the processor determines that the reference string is not allowed; In response to at least one of the vertical and horizontal components of the string vector being negative, it is determined that the string vector satisfies the requirement that the current string partially overlaps with the reference string; In response to satisfying the requirement that the current string partially overlaps with the reference string, the overlap constraint is skipped; and In response to the failure to meet the requirement that the current string is allowed to partially overlap with the reference string, the overlap constraint is applied to the string vector; Reconstruct the first part of the current string before reconstructing the second part of the current string; Reconstruct the second part of the current string based on the first part of the current string.

2. The method for video decoding in a decoder according to claim 1, characterized in that, The method uses a horizontal scan order for the string, and further includes: The vertical component of the string vector is determined to be negative; Reconstruct the first sample line of the current string; and Reconstruct the second sample line of the current string based on the first sample line of the current string.

3. The method for video decoding in a decoder according to claim 1, characterized in that, The method uses a vertical scan order for the string, and further includes: The horizontal component of the string vector is determined to be negative; Reconstruct the first sample column of the current string; and The second sample column of the current string is reconstructed based on the first sample column of the current string.

4. The method for video decoding in a decoder according to claim 1, characterized in that, The method further includes: In response to the reference string overlapping with the sample row above the current coding tree unit (CTU), the buffer storing samples in the sample row is accessed to reconstruct the current string.

5. A method for performing video encoding in an encoder, characterized in that, The method includes: The processor generates the string vector of the current string in the first sub-block of the current coding tree unit (CTU); Specifically, when generating the string vector of the current string, if the string vector points to a reference string of the current string, and the reference string crosses a boundary for positioning, then: When a vertical cross-boundary constraint is applied, the first part of the reference string is located in the second sub-block of the coding tree unit (CTU), the second part of the reference string is located in the third sub-block of the CTU, and the boundary between the second sub-block and the third sub-block is not a vertical boundary. When a horizontal cross-boundary constraint is applied, the first part of the reference string is located in the second sub-block of the coding tree unit (CTU), the second part of the reference string is located in the third sub-block of the CTU, and the boundary between the second sub-block and the third sub-block is not a horizontal boundary. In response to at least one of the vertical and horizontal components of the string vector being negative, it is determined that the string vector satisfies the requirement that the current string partially overlaps with the reference string; In response to satisfying the requirement that the current string partially overlaps with the reference string, the overlap constraint is skipped; and In response to the failure to meet the requirement that the current string is allowed to partially overlap with the reference string, the overlap constraint is applied to the string vector; Reconstruct the first part of the current string before reconstructing the second part of the current string; Reconstruct the second part of the current string based on the first part of the current string.

6. An apparatus for video decoding, comprising a processing circuit system, characterized in that, The processing circuitry is configured as described in any one of claims 1-4.

7. An apparatus for video encoding, comprising a processing circuit system, characterized in that, The processing circuit system is configured as described in claim 5.

8. A computer device, characterized in that, The computer device includes: One or more computer-readable non-transitory storage media configured to store computer program code; and One or more computer processors configured to access the computer program code and execute, as instructed by the computer program code, the method for video decoding in a decoder according to any one of claims 1 to 4, or the method for video encoding in an encoder according to claim 5.

9. A non-transitory computer-readable medium storing instructions, characterized in that, When executed by a computer for video decoding, the instructions cause the computer to perform the method for video decoding in a decoder according to any one of claims 1 to 4, or the method for video encoding in an encoder according to claim 5.

10. A method for storing a bit stream, characterized in that, Generate a bit stream by performing the method of claim 5; and store the bit stream.

11. A method for transmitting a bit stream, characterized in that, The method of claim 5 is used to generate a bit stream; and the bit stream is transmitted.

12. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 5 to generate the bit stream.

Images