Subblock temporal motion vector prediction for video encoding and decoding

Abstract

To provide a method, a computing device, and a storage medium for decoding video data.SOLUTION: A method includes determining a collocated image of a current coding unit 805, locating spatial neighboring blocks of the current coding unit corresponding to the collocated image 810, and determining a motion shift vector of the current coding unit from one or more motion vectors for the spatial neighboring blocks in a predetermined fixed order 815. The method further includes reconstructing, for each sub-block of the plurality of sub-blocks in the current coding unit, a sub-block-based temporal motion vector from a corresponding sub-block in the collocated image based on the motion shift vector.SELECTED DRAWING: Figure 8A

Description

[Technical field]

[0001] The present invention relates generally to encoding and decoding video data, and more particularly to a method and apparatus for encoding and decoding video data. and system for predicting motion vectors of subblocks in encoding and decoding - Patents.com . [Background technology]

[0002] Digital television, laptop or desktop computer, tablet computer computers, digital cameras, digital recording devices, digital media players, Video game consoles, smartphones, video conferencing equipment, video streaming equipment, etc. All electronic devices in the world support digital video. Electronic devices include MPEG-4, ITU- T H.263, ITU-T H.264 / MPEG-4, Part 10, Advance d Video Coding (AVC), High Efficiency Video o Coding (HEVC) and Versatile Video Coding (V It performs video compression and decompression standards defined in the VC (Computer Video Coding) standard to digitally encode and encode digital video. - Receive, transmit, encode, decode and store data. Video compression is usually done spatially (frame Performs intraframe (intra-frame) prediction and / or temporal (inter-frame) prediction to account for the redundancy inherent in the video data In block-based video coding, the video frame is divided into two parts, which are called block-based video coding. The tree is divided into multiple parts called coding tree units (CTUs). Each CTU is partitioned into one or more slices, each containing a number of video blocks. CU or until a predefined minimum CU size is reached. It may be recursively partitioned into smaller CUs. Each CU (also called a leaf CU) has: One or more transform units (TUs) and one or Each CU contains multiple prediction units (PUs). , can be coded in either intra, inter, or IBC mode. A video block in an intra-coded (I) slice in a video frame The block is coded by spatial prediction with respect to reference samples in neighboring blocks in the same video frame. Inter-coded (P or B) slices in one video frame A video block in a video frame is compared with reference samples in adjacent blocks in the same video frame. The spatial prediction of the video frame or the reference samples in other previous and / or future reference video frames. Use time predictions for the pull.

[0003] Previously coded reference blocks, e.g., spatial prediction or temporal prediction based on neighboring blocks gives the prediction block of the current video block to be coded. The finding process can be realized by a block matching algorithm. Residual data indicating pixel differences between a current block to be coded and a predicted block is The difference block or prediction error is called the inter-coded block. The motion vector is a motion vector pointing to a reference block in the reference frame and a residual block. The process of determining motion vectors is usually called motion estimation. The updated block is coded according to the intra prediction mode and the residual block. For the compression to occur, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain. , resulting in residual transform coefficients that are then quantified. Then, first in a two-dimensional matrix The positioned and quantified transform coefficients are scanned to generate a one-dimensional vector of transform coefficients; It is then entropy coded into a video bitstream to achieve further compression. can be.

[0004] and storing the encoded video bitstream on a computer readable storage medium. (e.g. flash memory) and transmitted by electronic devices with digital video capabilities. The electronic device may then access the information, or may transmit the information directly to the electronic device via wire or wirelessly. The electronic device may, for example, analyze the encoded video bitstream to generate a and extracts a syntax element from the bitstream and extracts at least one of the syntax elements from the bitstream. Also, based in part on the digital video data from this encoded video stream, By reconstructing the video into a different format, video decompression (the opposite process to video compression described above) is achieved. This reconstructed digital video data is then displayed on the electronic device. Reproduce in I.

[0005] As digital video quality progresses from high definition to 4K x 2K and even 8K x 4K, As the number of bits increases, the amount of video data to be encoded / decoded increases exponentially. US20130233633A1 - Efficiently encoding / decoding video data while maintaining image quality of the video data - Google Patents That is always a challenge. Summary of the Invention

[0006] This application relates to the encoding and decoding of video data, and more particularly to the subblock motion. Implementations of a method and system for vector prediction are described.

[0007] According to a first aspect of the present application, there is provided a method for decoding a current coding unit in a current image, comprising: determining a collocated image of the current coding unit; locating spatially adjacent blocks of the current coding unit; Each of the motion vectors included in the first reference frame list (List0) of the block is sequentially Next, check whether each motion vector in List0 is in the collocated position. According to the decision to use the selected image as a reference image for the motion vector, the motion vector in the spatially adjacent block is set as the motion shift vector; The subsequent motion vectors in List0 of the block and the second reference frame list (List t1) and discard the motion vector check for each motion vector in List0. According to the decision of the vector not to use the collocated image as a reference image, Then, sequentially check each of the motion vectors included in List 1 of the spatially adjacent blocks. Then, which of the motion vectors in List1 is the one that moves the collocated image to the corresponding motion vector? According to the decision to use the image as a reference for the motion vector in List1, Set the vector to the motion shift vector, and set the motion vectors in List1 to the motion shift vector. The test is abandoned, and all the motion vectors in List1 are not included in the co-located image. According to the decision not to use the image as a reference image for the motion vector, the motion shift and setting the current vector in the current image to a zero value vector. and a corresponding co-located block in the co-located image. determining a motion shift vector of the current coding unit indicating a shift in spatial position between and each of a plurality of sub-blocks in the current coding unit. For a sub-block, based on the motion shift vector, A subblock-based temporal motion vector is reconstructed from the corresponding subblocks in the and

[0008] In accordance with a second aspect of the present application, a computing device includes one or more processors. The present invention includes a memory and a plurality of programs stored in the memory. When executed by the one or more processors, have the device perform the above operations.

[0009] In accordance with a third aspect of the present application, a non-transitory computer-readable storage medium includes one or Multiple programs executed by a computing device having multiple processors When the program is executed by the one or more processors, , causing the computing device to perform the above operations. [Brief description of the drawings]

[0010] The present invention is further understood to be practiced in accordance with the principles of the present invention. The accompanying drawings illustrate the implementations described above and together with the description serve to explain the underlying principles. In addition, the same symbols indicate the same or corresponding parts. [Figure 1] FIG. 1 is a block diagram illustrating an example video encoding and decoding system according to an embodiment of this disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example video encoder according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a block diagram illustrating an example video decoder according to an embodiment of this disclosure. [Figure 4] 4A-4E are block diagrams illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with one embodiment of this disclosure. [Diagram 5] FIG. 5 is a block diagram illustrating spatially adjacent and temporally aligned block positions of a current CU to be coded according to an embodiment of the present disclosure. [Figure 6] 6A-6D are block diagrams illustrating steps for deriving a temporal motion vector predictor for a current block or a sub-block temporal motion vector predictor for a sub-block of the current block according to one embodiment of the present disclosure. [Figure 7] FIG. 7 illustrates a block diagram for determining valid regions used in deriving a temporal motion vector predictor and a sub-block temporal motion vector predictor according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart illustrating an example process for implementing a technique for a video encoder to derive a sub-block temporal motion vector predictor, according to an embodiment of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In order to facilitate understanding of the teachings set forth herein, a number of non-limiting specific details are set forth. However, the present invention is not limited to the above and may be modified in various ways without departing from the scope and spirit of the claims. It will be apparent to those skilled in the art that the present invention can be implemented in various forms. The concept can be implemented in many types of electronic devices that have digital video capabilities.

[0012] FIG. 1 illustrates a block diagram of a video block encoding and decoding system according to an embodiment of the present disclosure. As shown in FIG. 1, the system 10 is a block diagram illustrating an example system 10 for is a source device that generates and encodes video data that will be decoded by a future target device 14. The source device 12 and the destination device 14 include a desktop or laptop Computers, tablet computers, smartphones, set-top boxes, Digital TVs, cameras, displays, digital media players, video game consoles The device may include any of a variety of electronic devices, including video streaming devices, etc. In one embodiment, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0013] In one embodiment, target device 14 receives an encoded video signal to be decoded via link 16. Link 16 receives encoded video data from source device 12. Target device 14 may include any type of communication medium or device that is mobile. In one example, link 16 may include a link that transmits encoded video data to a source device 12 and a link that transmits encoded video data to a destination device 14. 14 may include a communication medium that can transmit the encoded information directly to the The video data is modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the target device 14. The communication medium is radio frequency (RF) signal. This includes any wireless or wired communication medium, such as a spectrum or one or more physical transmission paths. The communication medium can be a local area network, a wide area network, network, or packet-based global networks such as the Internet. The communication medium may include routers, switches, base stations, etc. may include a station or any other device that facilitates communication from source device 12 to target device 14. .

[0014] In another embodiment, the encoded video data is streamed from the output interface 22. The encoded video is then sent to the storage device 32. The data is accessed by the target device 14 via the input interface 28. Storage device 32 includes hard drives, Blu-ray discs, DVDs, CD-RO M, flash memory, volatile or non-volatile memory, or encoded video data and other suitable digital storage media for storing In another example, the data storage medium may include any of the data storage media that is accessed by the storage device. The data storage device 32 may be a file server or a storage device for storing the encoded video data generated by the source device 12. The target device 14 may correspond to another intermediate storage device that can hold the data. The stored video data is transferred from the storage device 32 by streaming or downloading. The file server stores the encoded video data. , any type of video signal that can transmit this encoded video data to the target device 14. An exemplary file server may be a web server (e.g., a web site), FTP server, network attached storage (NAS) device, or The target device 14 includes a local disk drive. A wireless channel (e.g., Wi-Fi connection) suitable for accessing the encrypted video data, a wired Any connection (e.g., DSL, cable modem, etc.), or a combination thereof The encoded video data can be accessed via a standard data connection. The transmission of the encoded video data from the storage device 32 may be a streaming transmission, a download transmission, or the like. The content may be transmitted, downloaded, or a combination thereof.

[0015] As shown in FIG. 1, the source device 12 includes a video source 18, a video encoder 20, and a and an output interface 22. The video source 18 includes a video capture device ( video cameras), video archives containing previously captured video, video content A video feed interface for receiving video from the content provider, and / or is a computer program for generating computer graphics data as a source video. It can include sources such as a graphics system, or a combination of these. In one example, video source 18 may be a video camera in a security surveillance system. If so, the source device 12 and the destination device 14 may be a camera phone or a video However, the embodiments described herein are generally applicable to video coding. It is possible and applicable to wireless and / or wired applications.

[0016] The video encoder 20 may convert video to be captured, pre-captured, or generated by a computer. The encoded video data can be encoded using a software program. The received signal is then sent directly to the target device 14 via the output interface 22 of the source device 12. Additionally (or alternatively), the encoded video data may then be to be accessed by the target device 14 or other devices for decryption and / or playback. The output interface 22 may be a modem or The device may further include a transmitter and / or a receiver.

[0017] The target device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 includes a receiver and / or a modem to connect the link 16 to Receives encoded video data via link 16. The encoded video data provided to storage device 32 includes A number of syntax elements are generated and used by video decoder 30 to decode the video data. These encoded video data may be transmitted over a communications medium or stored in a Whether stored on a storage medium or on a file server, such It may contain syntax elements such as:

[0018] In some embodiments, target device 14 may include an integrated display device or a display capable of communicating with target device 14. The display device 34 may include an external display device configured to display the image. The encoded video data is displayed to the user on a liquid crystal display (LCD). , plasma display, organic light-emitting diode (OLED) display, or another type of display. The display device may include any of a variety of display devices, such as any of the above types of displays.

[0019] The video encoder 20 and the video decoder 30 are compatible with VVC, HEVC, MPEG-4, Part 10: Advanced Video Coding (AVC) ng), or an extension of such standards. This application is not limited to any particular video encoding / decoding standard and may be used in conjunction with other video encoding / decoding It should be understood that the present invention is also applicable to the video encoding of the source device 12. The DA20 encodes video data according to any of these current or future standards. Similarly, the video decoder 30 of the target device 14 may be configured to receive these current or It is configured to decode video data according to any of the future standards.

[0020] Each of the video encoder 20 and the video decoder 30 includes one or more microprocessors. Processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGA), Discrete Logic, Software , hardware, firmware, or any combination thereof. The present invention can be implemented by any suitable encoder circuit such as When implemented in part by a processor, the electronic device executes software instructions in a suitable non-transitory computer. stored on a computer-readable medium and executed in hardware by one or more processors. The instructions in the following may be executed to perform the video encoding / decoding operations described in this disclosure. The video encoder 20 and the video decoder 30 are combined in their respective devices. One or more encoders integrated as part of a coder / decoder (CODEC) Or it may be included in the decoder.

[0021] FIG. 2 is a block diagram illustrating a video encoder 20 according to an embodiment described herein. FIG. 1 is a diagram illustrating a video encoder 20 that performs an in-depth Intra-prediction coding and inter-prediction coding can be performed. It relies on spatial prediction to account for spatial redundancy in video data within a given video frame or image. Inter-prediction coding relies on temporal prediction and reduces or eliminates adjacent Reducing or removing temporal redundancy in video data within adjacent video frames or images .

[0022] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, and a , a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantification unit 54, The prediction processing unit 41 includes a motion estimation unit 42 and a motion compensation unit 56. 44, a partition unit 45, an intra prediction processing unit 46, and an intra block copy (BC) unit 48 In one embodiment, video encoder 20 also includes a video block replay function. The inverse quantification unit 58 for the composition, the inverse transformation processing unit 60, and the adder 62 are further included. Between the adder 62 and the DPB 64, a filter is provided to filter the boundaries between blocks from the reconstructed video. A deblocking filter (shown in figure) removes blocky artifacts by filtering the In addition, in order to filter the output of the adder 62, In addition to the deblocking filter, an inner ring filter (not shown) may be used. The video encoder 20 may be in the form of a fixed or programmable hardware unit. The present invention may be implemented by any of the following fixed or programmable hardware units: The space may be partitioned within one or more of the spaces.

[0023] The video data memory 40 stores the video data to be encoded by the components in the video encoder 20. The video data in the video data memory 40 is stored as, for example, video The DPB64 receives video data from a source 18 via a video encoder 20. Used when encoding (e.g., in intra-predictive or inter-predictive coding modes) This is a buffer for storing reference video data. The may be formed of any of a variety of memory devices. The video data memory 40 is integrated on-chip with the other components in the video encoder 20. It may be present or off-chip for those components.

[0024] As shown in FIG. 2, after receiving the video data, the partition unit 45 in the prediction processing unit 41: The video data is partitioned into video blocks, each of which contains The video frames are then sorted according to a predefined partition structure, such as a quad-tree structure. Partitioning into slices, tiles or other larger coding units (CUs) A video frame may be divided into multiple video blocks (also called tiles). The prediction processor 41 can divide the current video block into a set of blocks (locked sets). Based on the error results (e.g., coding rate and distortion level), multiple inputs are One of the inter-prediction coding modes or one of the inter-prediction coding modes To select one of a plurality of possible predictive coding modes, The processing unit 41 provides the resulting intra- or inter-prediction coded block to an adder 50. The residual block is generated by subtracting the encoded block from the original block and then used as part of the subsequent reference frame. The prediction processing unit 41 further reconstructs the motion vector, the intra mode image, The syntax elements such as indicators, partition information, and other syntax information are provided to the entropy coder 56. Provide.

[0025] The intra prediction unit 46 in the prediction unit 41 calculates an intra prediction code suitable for the current video block. To select the TrP coding mode, the same frame as the current block to be coded is selected. Intra prediction of the current video block relative to one or more neighboring blocks in the frame. By executing the encoding, spatial prediction can be performed. The motion compensation unit 42 and the motion compensation unit 44 are configured to calculate one or more prediction blocks in one or more reference frames. In relation to the clock, temporal prediction is performed by performing inter-prediction coding of the current video block. The video encoder 20 performs a multi-pass encoding process to produce, for example, a video An appropriate encoding mode may be selected for each block in the data.

[0026] In one embodiment, the motion estimator 42 estimates a predetermined pattern of a sequence of video frames. In turn, for the current video frame, the predicted block in the reference video frame is calculated. A prediction unit (PU) of a video block in the current video frame associated with the block The inter prediction mode is determined by generating a motion vector that indicates the displacement of the motion vector. The motion estimation performed by the determination unit 42 is performed using motion vectors that estimate the motion of video blocks. A motion vector is, for example, the distance between the current video frame or image. A reference frame (or other coding unit) for the current video block being coded. For a prediction block in the current video frame (or other coding unit ) can indicate the displacement of the PU of the video block within a predetermined part of the sequence. The turn determines which video frames in this sequence are P-frames or B-frames. The intra BC unit 48 can specify the motion vector for inter prediction by the motion estimation unit 42. In a similar manner to vector determination, vectors for intra BC coding, e.g., block Alternatively, the motion estimator 42 may be used to determine the block vector. may be determined.

[0027] The predicted block is then computed using the sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metric. The pixel difference between the PU and the video block to be coded is determined by the pixel difference. In one embodiment, the video encoding The imager 20 calculates the values ​​of the sub-integer pixel positions of the reference frame stored in the DPB 64. For example, video encoder 20 may select a quarter pixel position of the reference frame. , 1 / 8 pixel positions, or other fractional pixel positions may be interpolated. The estimation unit 42 performs a motion search process for all pixel and fractional pixel positions. , it is possible to output motion vectors with fractional pixel accuracy.

[0028] The motion estimation unit 42 determines the position of the PU of the video block in the inter-prediction coded frame and A first reference frame that identifies one or more reference frames, each of which is stored in the DPB64. Select from the frame list (List0) or the second reference frame list (List1) The motion vector for this PU is calculated by comparing it with the position of the predicted block in the reference frame. The motion estimation unit 42 transmits the calculated motion vector to the motion compensation unit 44. , and then transmits it to the entropy coding unit 56.

[0029] The motion compensation performed by the motion compensation unit 44 includes the motion estimation unit 42. The motion compensation unit 4 may include obtaining or generating a prediction block based on the motion vector. 4. upon receiving the motion vector for the PU of the current video block, locating the predicted block to which this motion vector points in one of the DPB6 4 and transfers this predicted block to adder 50. Calculator 50 calculates the pixel values ​​calculated by motion compensation unit 44 from the pixel values ​​of the current video block being coded. A residual video block of pixel difference values ​​is obtained by subtracting pixel values ​​of the prediction block provided by the The pixel difference values ​​forming the residual video block may be luma or chroma difference components, or The motion compensation unit 44 may also include a motion compensation unit for a video block of a video frame. It is possible to generate further syntax elements for the video decode. It is used by decoder 30 in decoding video blocks of a video frame. For example, a syntax element that defines a motion vector for identifying this prediction block, Any flag indicating a prediction mode or any other syntactic information described herein. It should be noted that the motion estimator 42 and the motion compensator 44 are shown separately for conceptual purposes. However, it may be highly integrated.

[0030] In one embodiment, the intra BC unit 48 is Then, a vector can be generated in the same manner as described above to obtain a predicted block. Here, the predicted block is in the same frame as the current block being coded, and the vector The vectors are called block vectors, not motion vectors. can determine the intra prediction mode to be used to encode the current block. In one example, the intra BC unit 48 may use various methods, such as in encoding the individual passes. Encode the current block using intra prediction mode and use rate distortion analysis to Next, the intra BC unit 48 performs various tests. From the intra prediction modes, one suitable intra prediction is selected and used to perform the corresponding For example, the intra BC unit 48 may generate an intra mode indicator based on the rate distortion solution. The rate-distortion values ​​of the various tested intra-prediction modes were calculated by the analysis. The intra prediction mode with the best rate-distortion performance is selected as the appropriate intra prediction mode. Rate-distortion analysis usually involves determining the block being coded and the The distortion between the encoded block and the original block ( or error) and the number of bits used to generate this coded block. The intra BC unit 48 determines the bit rate (i.e., the number of bits) of the various coding The ratio is calculated from the distortion and rate for the block to be processed, and the intra prediction mode is determined. It may then be determined which represents the optimal rate-distortion value for this block.

[0031] In another example, the intra BC unit 48 may be the entirety of the motion estimation unit 42 and the motion compensation unit 44 or A part of the BC prediction function is performed according to the embodiment described herein. In either case, for intra block copies, the predicted block may be Determined by sum of absolute differences (SAD), sum of squared differences (SSD) or other difference metric The pixel difference between the two blocks is considered to be an exact match for the block being coded. where identifying the prediction block may involve calculating values ​​for sub-integer pixel positions.

[0032] The video encoder 20 may further include a processor 22 for generating a prediction block based on intra prediction of a block from the same frame. Whether the code is from the same frame or from a different frame based on inter prediction, The pixel values ​​of the prediction block are subtracted from the pixel values ​​of the current video block being coded to obtain the pixel By generating the raw difference values, a residual video block can be generated. The pixel difference values ​​forming the block may include both luminance and chrominance component differences.

[0033] The intra prediction processing unit 46 is implemented by the motion estimation unit 42 and the motion compensation unit 44 described above. Inter prediction, or intra block prediction, performed by the intra BC unit 48 Instead of copy prediction, intra prediction can be performed for the current video block. In particular, the intra prediction processor 46 determines one intra prediction mode and applies it to the current block. In order to achieve this, the intra prediction processing unit 46 can encode, for example, In the encoding process of each pass, various intra prediction modes are used to encode the current block. and the intra prediction unit 46 (or in some examples, the mode selector) One appropriate intra prediction mode may be selected and used from the selected intra prediction modes. The intra prediction processing unit 46 receives information indicating the intra prediction mode selected for this block. The entropy coding unit 56 may provide the selected In one embodiment, information indicating the selected intra-prediction mode may be encoded into the bitstream.

[0034] A prediction processor 41 generates a prediction block of a current video block by inter prediction or intra prediction. After determining the lock, adder 50 subtracts this predicted block from the current video block. The residual video block is generated by subtracting the residual video data from the Alternatively, the signal is provided to the conversion processing unit 52 in a form included in a plurality of conversion units (TUs). 52 is a method for reconstructing residual video using a discrete cosine transform (DCT) or a conceptually similar transform. The digital data is converted to residual transform coefficients.

[0035] The conversion processing unit 52 transmits the obtained conversion coefficients to the quantification unit 54. These transform coefficients are then quantified to further reduce the bit rate. The bit depth associated with some or all of the coefficients of can be reduced. can be changed by adjusting the quantification parameters. Then, the quantification unit 54 may perform a scan on a matrix containing the quantified transform coefficients. This scanning may be performed by the entropy coding unit 56.

[0036] Following quantification, an entropy coding unit 56 may, for example, use a context-adaptive variable length coding (CAVC) to (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), Syntax-Based Coding Text-Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partition Entropy (PIPE ) coding or another entropy coding method or technique to convert the quantified transform coefficients into binomials. Then, the encoded bitstream is The frame may be transmitted to the video decoder 30 or subsequently or to a storage device 32 for retrieval by the video decoder 30. The entropy coding unit 56 may also archive the current The motion vectors and other syntax elements for the video frames may be entropy coded. stomach.

[0037] The inverse quantification unit 58 and the inverse transformation processing unit 60 respectively perform inverse quantification and inverse transformation to obtain the other A residual binarization in the pixel domain for generating a reference block used to predict a video block of interest. As described above, the motion compensation unit 44 reconstructs the video blocks stored in the DPB64. generating a motion compensated prediction block from one or more reference blocks of the frame; The motion compensation unit 44 may also add one or more interpolation filters to the prediction block. The filter may be applied to calculate sub-integer pixel values ​​used for motion estimation.

[0038] The adder 62 adds the reconstructed residual block to the motion compensation block generated by the motion compensation unit 44. This is added to the predicted block to generate a reference block to be stored in the DPB64. The reference block is a predicted block, and is processed by the intra BC unit 48, the motion estimation unit 42, and the motion is used by compensation unit 44 to interpolate another video block in a subsequent video frame. It is possible to predict.

[0039] FIG. 3 is a block diagram illustrating an example video decoder 30 according to an embodiment of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit, and a The image processing unit 80 includes a calculation unit 81, an inverse quantification unit 86, an inverse transformation processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 includes a motion compensation unit 82, an intra prediction processing unit 84, and an intra BC unit 85. The video decoder 30 includes the above-described components with respect to the video encoder 20 with reference to FIG. A decoding process can be performed that is roughly the reverse of the encoding process described above. For example, The motion compensation unit 82 performs prediction based on the motion vector received from the entropy decoding unit 80. The intra prediction unit 84 generates the intra prediction data received from the entropy decoding unit 80. Prediction data may be generated based on the prediction mode indicator.

[0040] In one example, one component in video decoder 30 may be responsible for carrying out the implementation of the present invention. Also, in some examples, implementations of the present disclosure may include one or more For example, the intra BC unit 85 may be It may be realized alone, or a motion compensation unit 82, an intra-prediction processing unit 84 and an ent 3. The above-mentioned embodiment is realized in combination with other components of the video decoder 30, such as the tropy decoder 80. In one example, video decoder 30 may not include intra BC section 85 and may The function of the BC unit 85 is transferred to other components in the prediction processing unit 81, such as the motion compensation unit 82. This may be realized thereby.

[0041] The video data memory 79 is a digital video data storage device that stores video data decoded by other components in the video decoder 30. The video data may be stored in a video memory, such as a coded video bitstream. The video data stored in the data memory 79 may be, for example, a wired or wireless Network communications and physical data storage media (e.g. flash drives and hard disks) access to the storage device 32 and local video sources such as cameras The video data memory 79 stores the coded video bitstream. The video signal may include a coded picture buffer (CPB) for storing the coded video data. The decoded picture buffer (DPB) 92 in the video decoder 30 is Decoding video data (e.g., in intra-prediction or inter-prediction coding modes) The video data memory 79 and the DPB 92 store the reference video data used for , Synchronous DRAM (SDRAM), Magnetoresistive RAM (MRAM), Resistive RAM (RRAM Dynamic Random Access Memory (DRAM), including DDR3, or other types of memory The memory device may be formed by any of a variety of memory devices, such as a memory cell. For convenience of explanation, the video data memory 79 and the DPB 92 are shown in FIG. 0, but those skilled in the art will recognize The data memory 79 and the DPB 92 may be in the same memory device or in separate memory devices. It is apparent that in one embodiment, the video data memory 79 is provided by a video It may be on-chip with other components in the decoder 30, It may be off-chip for the element.

[0042] During the decoding process, the video decoder 30 decodes the video of the encoded video frames. Receiving an encoded video bitstream indicating blocks and associated syntax elements Video decoder 30 may operate at a video frame level and / or a video block level. The entropy decoding unit 80 of the video decoder 30 may receive the syntax element. The data stream is entropy decoded to obtain quantified coefficients, motion vectors, or inverse and generate the entropy prediction mode indicator and other syntax elements. The encoding unit 80 transfers the motion vectors and other syntax elements to a prediction processing unit 81 .

[0043] The video frames are coded as intra-predictive (I) frames or other types of When used for intra-coded predicted blocks in a frame, the prediction processing unit 81 The intra prediction processing unit 84 performs the intra prediction process based on the intra prediction mode notified by the signal and the current frame. The image of the current video frame is generated based on reference data from previously decoded blocks of the current video frame. It is possible to generate prediction data for a video block.

[0044] The video frame was coded as an inter-predictive (i.e., B or P) frame. In this case, the motion compensation unit 82 in the prediction processing unit 81 receives the Based on the motion vectors and other syntax elements, a video block of the current video frame is It is possible to generate one or more prediction blocks for each of the prediction blocks. , generated from a reference frame in one of the reference frame lists. 0 is based on the reference frame stored in the DPB92 and is constructed using the default construction technique. It is possible to construct reference frame lists, List0 and List1, from these.

[0045] In one example, if a video block is coded according to the intra BC mode described herein, In this case, the intra BC unit 85 in the prediction processing unit 81 receives the entropy decoding unit 80 Based on the received block vector and other syntax elements, The prediction block is generated based on the determined prediction block size by the video encoder 20. The video block may be in the same reconstruction region of the image as the current video block selected.

[0046] The motion compensation unit 82 and / or the intra BC unit 85 process the motion vectors and other syntax elements determining prediction information for a video block of the current video frame by analyzing the and uses this prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 generates this block using a part of the received syntax element. A prediction mode for encoding a video block of a video frame (e.g., intra prediction). or inter-predicted), the inter-predicted frame type (e.g., B or P), Structural information of one or more reference frame lists for the frame, the motion vectors of each inter-prediction video block of this frame, The inter prediction state of the video block in the current video frame is Determine other information to decode.

[0047] Similarly, the intra BC unit 85 uses some of the received syntax elements, such as flags, to The current video block is predicted in intra BC mode, and Structural information regarding whether a video block is in the reconstruction domain and should be stored in DPB92 , the block vector of each intra BC predicted video block in this frame, The intra BC prediction state of each intra BC predicted video block in the frame, and the current Other information for decoding a video block in a video frame can be determined. Cut.

[0048] In addition, motion compensation unit 82 may be used by video encoder 20 to encode a video block. Calculate the sub-integer pixel interpolated value of the reference block by performing interpolation using the interpolation filter. In this case, the motion compensation unit 82 outputs the video encoding from the received syntax elements. and determining the interpolation filter used by the predictor 20 and using the interpolation filter to generate a predictive block. A lock may be created.

[0049] Inverse quantification unit 86 uses this information to determine the degree of quantification by video encoder 20. The same quantification parameters were calculated for each video block in the video frame. The entropy is provided to the bitstream and decoded by the entropy decoder 80. The inverse transform processor 88 inversely quantifies the transform coefficients of the pixel-decoded quantification. To reconstruct the residual block, an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptual A substantially similar inverse transform process is applied to these transform coefficients.

[0050] The motion compensation unit 82 or the intra BC unit 85 calculates the current After generating the prediction block for the video block, adder 90 outputs the prediction block to inverse transform processor 88. and the corresponding residual block generated by the motion compensation unit 82 and the intra BC unit 85. The decoded video block is calculated by adding the predicted block to the current video block. The block is reconstructed. Between the adder 90 and the DPB 92, an in-loop filter (not shown) is provided. It is possible to further process this decoded video block by placing Then, these decoded video blocks in a given frame are used to decode the next video block. The reference frames are stored in the DPB 92, which stores the reference frames used for future motion compensation of the block. In addition, the DPB 92 or a memory device separate from the DPB 92 may include a display device such as the display device 34 of FIG. The decoded video may also be stored for subsequent display on any display device. It is possible.

[0051] In a typical video coding process, a video sequence is typically generated in an ordered It contains a set of frames or images. Each frame has SL, SCb and SCr. It can contain three sample matrices: SL is a two-dimensional matrix of luma samples SCb is a two-dimensional matrix of Cb chroma samples. SCr is a two-dimensional matrix of Cr chroma samples. In another example, the frame may be monochrome, in which case the luminance sample is It contains only one two-dimensional matrix of pulls.

[0052] As shown in FIG. 4A, video encoder 20 (or, more specifically, partition unit 45) First, we partition the frame into a set of coding tree units, A video frame is generated in raster scan order, from left to right and top to bottom. Each CTU contains an integer number of CTUs that are ordered in a logical order. The width and height of all CTUs in the video sequence are 128x128. to have the same size, being one of 64x64, 32x32 and 16x16, The sequence parameter set is notified by the video encoder 20. As shown in FIG. 4B, each CTU is a set of luminance samples. One coding tree block (CTB) of chroma samples, and two corresponding coding tree blocks of chroma samples. , and syntax elements used to encode samples of the coding tree block. The syntax elements describe the attributes of different types of units of a coding block of pixels, and how 1 describes how a video sequence is reconstructed in a video decoder 30. , e.g., inter- or intra-prediction, intra-prediction mode, motion vectors, and Other parameters include: For monochrome images or images with three separate color planes, one A CTU encodes a single coding tree block and samples of this coding tree block. A coding tree block may include syntax elements used for N×N sample blocks. It is possible that the

[0053] To achieve better performance, the video encoder 20 may Re-tree partitioning for blocks, such as binary tree partitioning, quad tree partitioning, or a combination of both. We can then recursively partition this CTU into smaller coding units (CUs). To achieve better performance, video encoder 20 may For each tree block, a binary tree partition, a ternary tree partition, a quad tree partition, or a combination of these Then, we divide this CTU into smaller coding units (CUs) by recursively partitioning the tree. As shown in FIG. 4C, the 64×64 CTU 400 can be first It is partitioned into four smaller CUs of 32 block size. These four smaller C Of U, CU410 and CU420 are four C The two CUs 430 and 440 with a block size of 16×16 are The image is further partitioned into four CUs of 8×8 block size. 4 shows a quadtree data structure representing the final result of the partitioning process of CTU400; The leaf nodes correspond to one CU for each size from 32x32 to 8x8. Like the CTU shown in Fig. 1, each CU is a single symbol of the same-sized luma samples of the frame. The coding block (CB) and the two corresponding coding blocks of chroma samples and their coding and syntax elements used to encode the samples of the coding block. For an image or an image with three separate color planes, one CU is assigned to a single coding block. and the syntax structure used to encode the samples of this coding block. Note that the quadtree division shown in FIG. 4C and FIG. 4D is merely an example. Based on the ternary / binary tree partition, the data can be divided into CUs that suit various local characteristics. In the multi-type tree structure, one CTU is divided according to the quadtree structure, and each quadtree leaf is The CU can be further divided according to a binary tree and a ternary tree structure. As shown, there are five possible partition types of a coding block with width W and height H: These include four-way partitions, two-way horizontal partitions, two-way vertical partitions, three-way horizontal partitions, and three-way vertical partitions. .

[0054] In one embodiment, video encoder 20 further divides the coding blocks of the CU into one or It can be partitioned into multiple M×N prediction blocks (PBs). A prediction block can be divided into multiple prediction blocks (PBs) with the same prediction ( A rectangular (square or non-square) sample on which inter- or intra-prediction is applied. The prediction unit (PU) of the CU is a prediction block of one luma sample. The two corresponding prediction blocks of chroma samples, and the It may contain syntax elements used to represent monochrome images or images with three separate color planes. In a given image, the PU has a single predicted block and the PU is used to predict this predicted block. The video encoder 20 may include a syntax structure for determining a luminance prediction block of each PU of a CU, Predictive luminance block for Cb prediction block and Cr prediction block, predictive Cb A block and a predictive Cr block can be generated.

[0055] The video encoder 20 predicts these PUs by intra prediction or inter prediction. The video encoder 20 can generate a prediction block of When generating a prediction block for a PU, the decoded sample of the frame associated with this PU is used. Based on the rule, the video encoder can generate a predictive block for this PU. 20, when generating a predictive block of a PU by inter prediction, This P is based on the decoded samples of one or more frames other than the frame being A predictive block of U can be generated.

[0056] The video encoder 20 may further include a predictive luminance block for one or more PUs of a CU, a predictive After generating the predictive Cb block and the predictive Cr block, the original luminance coding block of the CU is The CU’s predictive luminance block is subtracted from the CU’s luminance residual block. where each sample in the luma residual block of this CU is a predictive The luma samples in one of the predictive luma blocks and the original luma samples in this CU Similarly, the difference between the corresponding sample in the video encoding block is The digital decoder 20 generates a Cb residual block and a Cr residual block for the CU, respectively, where Each sample in the Cb residual block of this CU is the The Cb samples in one of the predictive Cb blocks and the original Cb coding block of this CU. Each sample in the Cr residual block of this CU is the difference between the corresponding sample in the The sample is in one of the predictive Cr blocks of this CU. It indicates the difference between the Cr sample and the corresponding sample in the original Cr coded block of this CU. vinegar.

[0057] Furthermore, as shown in FIG. 4C, the video encoder 20 may divide the luminance of the CU by a quadtree partition. The residual block, the Cb residual block, and the Cr residual block are transformed into one or more luma transforms. It can be decomposed into a block, a Cb transform block, and a Cr transform block. Locking is a rectangular (square or non-square) block of samples to which the same transformation is applied. The transform unit (TU) of the CU is a transform block of luma samples and two transform blocks of chroma samples. The corresponding transform blocks and the structure used to transform the transform block samples. Therefore, each TU of a CU has a luminance transform block, a Cb transform block, and Cr transform blocks. The filtered luma transform block may be a sub-block of the luma residual block of the CU. The transform block may be a sub-block of the Cb residual block of the CU. can be a sub-block of the Cr residual block of a CU. For an image with 1 color plane, a TU consists of a single transform block and a sample of this transform block. and syntax structures used to transform the rules.

[0058] The video encoder 20 applies one or more transforms to the luma transform block of the TU to A block of luminance coefficients for this TU can be generated. The block of coefficients is a quadratic representation of the transform coefficients. The transform coefficients may be scalar quantities. , apply one or more transforms to the Cb transform block of the TU to obtain the Cb coefficient block of the TU. Video encoder 20 may perform one or more transforms on the TU to generate a block. It can be applied to the Cr transform block to generate the Cr coefficient block for this TU.

[0059] The video encoder 20 may generate a coefficient block (e.g., a luminance coefficient block, a Cb coefficient block, After generating the Cr coefficient block, the coefficient block may be quantified. In general, it is desirable to quantify the conversion coefficients and reduce the amount of data representing these conversion coefficients. , which means further compression can be achieved. The video encoder 20 quantifies the coefficient blocks After that, it is possible to entropy code the syntax elements representing the quantified transform coefficients. For example, video encoder 20 may apply a code to a syntax element that indicates a quantified transform coefficient. Finally, the video may be encoded using a context-adaptive binary arithmetic coding (CABAC). The audio encoder 20 receives a bit sequence that constitutes a representation of an encoded frame and associated data. outputting a bitstream containing the sequence for storage in storage device 32; or Transmit to target device 14.

[0060] The video decoder 30 converts the bitstream generated by the video encoder 20 into Once received, this bitstream is parsed to obtain syntax elements from the bitstream. The video decoder 30 may then extract at least some of the syntax elements from the bitstream. Based on the above, a frame of the video data can be reconstructed. The process of performing this is generally the same as the encoding process performed by video encoder 20. For example, video decoder 30 may Then, we can perform an inverse transform to reconstruct the residual block associated with the TU of the current CU. In addition, the video decoder 30 receives a sample of a prediction block for the PU of the current CU. and the corresponding samples of the transform block of the TU of the current CU. Reconstruct the coding blocks of U. After the coding blocks of each CU in the frame are reconstructed, , video decoder 30 is able to reconstruct this frame.

[0061] As mentioned above, there are two main modes of video coding: intraframe prediction (or video using intraframe prediction (or intra prediction) and interframe prediction (or inter prediction) Note that IBC can be considered as intra-frame prediction or the third mode. Comparing these two modes, interframe prediction uses motion vectors. Intra-frame prediction is used to predict the current video block from the reference video block. This contributes more to coding efficiency than the conventional method.

[0062] However, with the advancement of video data capture technology and the need to retain details of video data, As the video block size becomes finer, the motion vector for the current frame is represented as The amount of data required to perform this task has also increased dramatically. is that a pair of adjacent CUs in the spatial and temporal domains have the same metric for prediction purposes. Not only does it contain video data, but it also has the same motion vectors between these adjacent CUs. Therefore, the fact that spatially adjacent CUs and / or temporally adjacent CUs are By exploring the spatial and temporal correlations of the motion information of the CUs, The motion information of adjacent CUs and / or temporally co-located CUs is used as the “motion The motion vector predictor (MVP) is also used as an approximation of motion information (e.g., motion vectors). It is possible to use.

[0063] The actual motion vector of the current CU determined by the motion estimation unit 42 described above with reference to FIG. Instead of encoding the motion vectors into the video bitstream, the actual motion vectors of the current CU are used to The motion vector difference (MVD) of the current CU is calculated by subtracting the motion vector predictor of the current CU. As a result, the motion estimation unit 42 generates a motion vector for each CU of the frame. There is no need to encode the vectors into the video bitstream, and the This can significantly reduce the amount of data required to represent motion information in the image.

[0064] Selecting a prediction block from within a reference frame during interframe prediction of a coding block - Patents.com Similar to the process, the video encoder 20 and the video decoder 30 are controlled by a set of rules. and related to spatially adjacent CUs and / or temporally colocated CUs of the current CU. The potential candidate motion vectors are used to create a motion vector candidate list (called a "merge list"). (also called a motion vector candidate list), and then select one item from these motion vector candidate lists. This requires the video encoder 20 to use the motion vector predictor for the current CU. The motion vector candidate list itself does not need to be transmitted between the video decoder 30 and the video decoder 30. The index of the selected motion vector predictor in the motion vector candidate list is and the video decoder 30 uses the same motion vector predictor in the motion vector candidate list. It is sufficient to encode and decode the current CU using

[0065] In one embodiment, each inter-prediction CU may include an inter prediction CU for constructing a motion vector candidate list. Three types of motion vector prediction are available: In each mode, the motion vector prediction is performed according to the algorithm described below. You can add one or more motion vector candidates to the motion vector candidate list by Finally, one of those motion vector candidates in the candidate list is selected as the video encoder. The video signal is encoded into a video bitstream by a video decoder 20 or 30, the optimal motion vector of the inter-predicted CU decoded from the video bitstream. To find the best motion vector predictor from the candidate list, To achieve this, a motion vector competition (MVC) scheme is introduced to optimize spatial and temporal motion vectors. A motion vector candidate list is selected from a predetermined motion vector candidate set, i.e., a motion vector candidate list including a motion vector candidate. A motion vector is selected.

[0066] Candidate motion vector predictors are chosen from spatially adjacent or temporally co-located CUs. In addition to being derived, so-called "history motion vector prediction" (HMVP) tables The HMVP table contains the same number of CTUs. The predetermined number of bits used to encode / decode a given CU in a row (or the same CTU) is The spatial / temporal proximity of these CUs determines the number of motion vector predictors that can be used to predict a given CU. Therefore, one of the motion vector predictors in the HMVP table matches the motion vector in the same row of the CTU. It is highly likely that a motion vector will be reused to encode / decode different CUs. By using the HMVP table in the process of constructing vector candidate lists, It is possible to achieve high coding efficiency.

[0067] In one embodiment, the HMVP table has a fixed length (e.g., 5) and is first-in, first-out ( For example, to decode one inter-coded block of a CU, When the CU is reconstructed, the motion vector for this CU is reconstructed. Since the motion vector of the next CU may be a predictor of the motion vector of the next CU, the HMVP table The HMVP table is updated on the fly by the vector. There are two situations: (i) the reconstructed motion vector does not match an existing motion vector in the HMVP table. or (ii) the reconstructed motion vector is different from one of the existing motion vectors in the HMVP table. In the first situation, if the HMVP table is less than The reconstructed motion vector is added to the HMVP table as the latest one. If the P table is already full, the reconstructed motion vectors are considered to be the latest. Before being added, the oldest motion vector in the HMVP table is first removed from the HMVP table. In other words, in this case, the HMVP table needs to be deleted from the FIFO buffer. Similarly, the FIFO buffer is at the top of another block that was previously inter-coded. The motion information associated with the block is removed from this buffer, and the reconstructed motion vectors are It is added to the end of the FIFO buffer as the newest entry in the HMVP table. In this situation, the reconstructed motion vector is added to the HMVP table as the latest one. Before reconstructing a motion vector, the HMVP table must first find an existing motion vector that is substantially the same as the reconstructed motion vector. The vector is deleted from the HMVP table. The HMVP table also uses a FIFO buffer method. If the next motion vector prediction for the same motion vector in the HMVP table is The child is moved forward one item to occupy the space left by the deleted motion vector. and the reconstructed motion vector is the latest one in the HMVP table. It is added to the end of the FO buffer.

[0068] The motion vectors in the HMVP table are used for different predictions such as AMVP, Merge, and Skip. In the HMVP table, the motion vectors can be added to the motion vector candidate list in the estimation mode. The stored motion information of the previously inter-coded blocks is stored as the motion information of the blocks adjacent to the current block. Even if it does not match, we found that it can be used for more efficient motion vector prediction.

[0069] After one MVP candidate is selected within a given motion vector candidate set for the current CU, The video encoder 20 generates one or more syntax elements for the corresponding MVP candidates. The syntax elements are then used by the video decoder 30 to encode the We can search for this MVP candidate from this video bitstream. Depending on the specific mode you configure, different modes (e.g. AMVP, merge, spooling) In the case of AMVP mode, the syntax elements are , inter prediction indicator (List0, List1, or bi-prediction), reference search The motion vector prediction residual signal is included. In the merge mode, the current CU is merged with the neighboring CU pointed to by the encoded merge index. ,It inherits other syntax elements such as inter prediction indicator,,reference index, motion vector, etc. In this case, only the merge index is coded into the bitstream. In this case, the motion vector prediction residual signal is also omitted.

[0070] FIG. 5A illustrates a spatially neighboring CU of a current CU to be encoded / decoded according to an embodiment of the present disclosure. FIG. 1 is a block diagram showing coincident and temporally collocated block positions in a given mode; First, the motion vectors related to the spatially left adjacent block position and the spatially upper adjacent block position are calculated. the availability of motion vectors related to temporally collocated block positions; by checking the possibility and then checking the availability of motion vectors in the HMVP table. Then, a motion vector prediction (MVP) candidate list is constructed. The process includes removing some redundant MVP candidates from the candidate list and reorganizing the candidate list as necessary. Zero-valued motion vectors are added so that the stream has a fixed length (note that this depends on the mode). After constructing the MVP candidate list, Video Encoder 2 0 selects the best motion vector predictor from this candidate list and points to the selected candidate. The corresponding index can be encoded into the video bitstream.

[0071] In one example, the candidate list (also called the merge candidate list) is divided into five types of candidates: The method may be constructed by including, in order: 1. Spatial MVP (i.e., motion vector predictor) from spatially neighboring CUs 2. Temporal MVP from Colocated CU 3. MVP based on history from FIFO table 4. Pairwise average MVP 5. Zero MV

[0072] In one embodiment, the size of the candidate list is signaled in the slice header, and the first The maximum allowable size is 6 (for example, in the case of VVC). For each CU code in merge mode Then, the index of the best merge candidate is encoded by truncated unary binarization (TU). The first bin of the merge index is coded with the context, while the bypass code In the following context of this disclosure, this extended merge mode is the same as the normal merge mode, since the concept is the same as the merge mode used for HEVC. Also called.

[0073] Using FIG. 5A as an example, and assuming the candidate list has a fixed length of 2, The motion vector predictor (MVP) candidate list for U is generated in the following steps in AMVP mode: It can be constructed by executing the steps in order. 1) Selection of MVP candidates from spatially adjacent CUs a) One of the two spatially adjacent CUs on the left starting at A0 and ending at A1 Derive at most one non-scaling MVP candidate from the set of b) If there is no non-scale MVP candidate available to the left in the previous step, then At most one from one of the two spatially adjacent CUs to the left that start and end with A1 Derive a scale MVP candidate; c) The three spatially adjacent Cs on the upper side starting from B0, passing through B1 and ending at B2 Derive at most one non-scale MVP candidate from one of U; d) If both A0 and A1 are unavailable or if they are both intranet When coded in mode, the top three start with B0 and end with B2 through B1. Derive at most one scale MVP candidate from one of the spatially neighboring CUs of; 2) If two MVP candidates are found in the previous step and they are identical, remove one of these two candidates from the list of MVP candidates; 3) Selection of MVP candidates from temporally collocated CUs a) If after the previous step, the MVP candidate list does not contain two MVP candidates In this case, we derive at most one MVP candidate from a temporally collocated CU (e.g., T0). ; 4) Select MVP candidates from the HMVP table a) If after the previous step, the MVP candidate list does not contain two MVP candidates derives the MVP with up to two histories from the HMVP table; 5) After the previous step, if the MVP candidate list does not contain two MVP candidates adds up to two zero-valued MVPs to the MVP candidate list.

[0074] Since there are only two candidates in the constructed AMVP mode MVP candidate list, which of the two MVP candidates in the list is currently used to decode the CU. Encode relevant syntax elements, such as binary flags, into the bitstream.

[0075] In one embodiment, in skip or merge mode, the sequence of steps as described above is By executing the steps in order, a list of MVP candidates for the current CU can be constructed. One special kind of merge candidate, called an "awise merge candidate", can also be used in skip mode or is included in the MVP candidate list for the merge mode. Pairwise merge candidates are previously The MV of the two derived merge mode motion vector candidates is averaged. The size of the merge MVP candidate list (for example, 1 to 6) is determined by the slice of the current CU. The index of the best merge candidate for each CU in the merge mode is signaled in the header. The first bin of the merge index is the context and bypass coding is used for the other bins.

[0076] As mentioned above, the history-based MVP is the AMVP mode after the spatial MVP and the temporal MVP. Candidates can be added to the MVP Candidate List or the Merge MVP Candidate List. - The motion information of the coded CU is stored in the HMVP table, and the MV for the current CU is The HMVP table is maintained during the encoding / decoding process. For each non-subblock inter coded CU, the associated motion vector information is added as a new candidate to the last entry in the HMVP table, while If the table is already full and there is no duplicate copy of the associated motion vector information in this table, ) The motion vector information stored in the first entry of the HMVP table is deleted from it. Instead, the associated motion vector information is stored in the last entry of the HMVP table. Before adding a motion vector to the list, the same copy of the associated motion vector information is removed from this table. Good too.

[0077] As mentioned above, Intra Block Copy (IBC) is the coding of screen content material. The IBC mode is a block-level coding mode that can significantly improve the coding efficiency. Therefore, the video encoder 20 performs block matching (BM). Then, we find the optimal block vector for each CU. Here, the block vector is , to indicate the displacement of the current block to an already reconstructed reference block in the current image. IBC coded CUs can be coded in either intra or inter prediction mode. This is treated as the third prediction mode other than the first prediction mode.

[0078] At the CU level, IBC mode can be set to IBC AMVP mode or IBC ST mode as follows: Can be signaled as skip / merge mode. -IBC AMVP mode: The actual block vector of a CU and the block vector of this CU The block vector difference between the block vector predictor of this CU selected from the block vector candidates (BVD) is coded in the same way as the motion vector difference is coded in the AMVP mode described above. In the case of IBC coding, the block vector prediction method uses two blocks. Two candidate vectors are used as predictors, one from the left neighbor and one from the upper neighbor. If none of the neighbors are available, the default block vector is The binary flag indicates the block vector predictor index. The IBC AMVP candidate list includes spatial candidates and HMVP Includes candidates. -IBC skip / merge mode: The merge candidate index is the adjacent IBC coded block Which block vectors in the merge candidate list (also called the "merge list") from to indicate whether the candidate block is used to predict the block vector for the current block. The IBC merge candidate list includes spatial candidates, HMVP candidates, and pairwise candidates. Includes supplementary.

[0079] 6A-6D are diagrams illustrating a temporal motion vector prediction of a current block according to an embodiment of the present disclosure. child (TMVP) or subblock temporal motion vector predictor (Sb FIG. 1 is a block diagram showing the steps for deriving a TMVP.

[0080] In one embodiment, one temporal motion vector predictor (T Only MVP) candidates are added to the merge candidate list. This TMVP candidate is valid or invalid. As instructed, the first flag (sps_temporal_mvp_enabled_flag) is set to the image sequence parameters. It is notified in the meter set (SPS) and the second flag (slice_temporal_mvp_enabled_flag) In particular, the derivation of temporal merge candidates takes into account the scale motion. The input vector is a collocated image that is a previously encoded and decoded image in the reference image list. In this derivation of the temporal motion candidates, the MV of the collocated image is used. The image is in the first reference frame list (List0) or the second reference frame list (List1 ) is selected from the slice header. _from_l0_flag) is sent to the decoder first to determine which image in the list is used. to indicate which image is selected as a collocated image for deriving temporal motion candidates. In addition, the co-located reference index (co-located_ref_idx) is transmitted. List0 (also called L0) and List1 (also called L1) MVs of motion candidates To extract different images in a collocated block of a collocated image, follow the pseudocode below: The MVs in the list are derived individually in a predetermined order. [Table 1]

[0081] The scale motion vector 602 of the temporal merging candidate is the POC distance, as shown by the dotted line in FIG. 6A. The distance tb604 and the POC distance td606 are used to determine the selected collocated block. where tb is the reference vector of the current image. Between a reference image (e.g., current reference 608) and a current image (e.g., current image 610). is defined as the POC difference of the reference image of the collocated image (the collocated reference 6 It is defined as the POC difference between the image (colocated image 612) and the image (colocated image 14). The reference image index of the temporal merge candidate is set to zero. The practical implementation is described in the HEVC specification. For B slices, one is the reference image L Two motion vectors are obtained, one for ist0 and the other for reference image List1. The bi-predictive merge candidates are then combined to create the bi-predictive merge candidates.

[0082] A collocated block belonging to a reference frame (e.g., collocated block 620 ), the position of the temporal candidate is selected between candidates C0 and C1, as shown in FIG. 6B. The block at position C0 is unavailable, is intra-coded, and If is outside the current CTU, then position C1 is used; otherwise, position C0 is Used to derive temporal merge candidates.

[0083] Some coding / decoding standards (e.g., VVC Test Model 1) use subblock-based time division Supports the temporal motion vector predictor (SbTMVP) method. Similar to motion vector prediction (TMVP), SbTMVP uses the motion filter of collocated images. Use the filter to improve the motion vector prediction and merge mode of the CU in the current picture. SbTMVP uses the same collocated images as TMVP. SbTMVP differs from TMVP in two main respects: 1. TMVP predicts motion at the CU level, while SbTMVP predicts motion at the sub-CU level. Predict. 2. TMVP extracts temporal motion vectors from collocated blocks in collocated images. The period during which the block is selected (the colocated block is the lower right or center block for the current CU). In the case of the current CU, SbTMVP selects the spatially adjacent blocks of the current CU. The motion shift obtained from the motion vector from one of the collocated images is The method is applied to the temporal motion information obtained.

[0084] The SbTMVP process is shown in Figures 6C-6D. MVP632) in two steps, the sub-CU in the current CU (current CU636 in Figure 6D) In the first step, the motion vector of a sub-CU (e.g., sub-CU 634) is predicted. Examine spatial neighbor A1 (e.g., spatial neighbor 638) in FIG. 6C. A motion vector is calculated using a co-located image (e.g., the co-located image 612 in FIG. 6A) as a reference image. If the motion vector has a motion shift, then the motion vector is adjusted based on the motion shift applied (e.g., the motion shift in FIG. 6D). If no such motion vector is identified, then the motion vector is selected as the motion vector 630. The shift is set to the zero-valued vector (0, 0). List0MV and The first available motion vector from List1MV is set as the motion shift. Thus, in SbTMVP, the current CU is always located at the bottom right or center. The corresponding blocks (sometimes called collocated blocks) are compared to TMVP. The pseudo code for determining the motion shift is as follows: As follows.

number

[0085] The variables and functions used in the above table are denoted as follows: ColFromL0Flag: Is the collocated image from the List0 reference image list? Syntax to indicate whether: · LDC: indicates whether the POC values ​​of all reference images are smaller than the current image; · CurrentSliceType: type of the current slice (image); · count: available number of already derived merge candidates; · interDirA1: interDir of Nth merge candidate (1:L0, 2:L1, or 3:Bi); refIdxA1[0]: L0 motion information of the Nth merging candidate (e.g., MV, ref. index) refIdxA1[1]: L1 motion information of the Nth merging candidate (e.g., MV, ref. index) getRefPic(M, I): Gets a reference image from the reference image ListM by reference index I. A function to obtain.

[0086] In the second step, the motion shift identified in step 1 is applied, as shown in Figure 6D. (i.e., added to the coordinates of the current block) and the sub-C Obtain U-level motion information (motion vectors and reference indexes). In the example of FIG. 6D Assume that the motion shift is set to the motion of block A1. The software can be set to either block A1, A2, B1, or B2 movement. It is.

[0087] First, a representative sub-CU is selected, and the motion information of the corresponding block of the representative sub-CU is calculated. In the existing scheme of SbTMVP, the current The sub-CU located at the lower right of the center position of the CU is selected as the representative sub-CU. If valid motion information cannot be derived from the corresponding block of the sub CU as default motion information, If the default motion information is available, the SbTMVP candidate is considered unavailable. If yes, proceed to the next step to obtain the motion information of each sub-CU in the current CU. Whenever there is no motion information available for the corresponding block of the sub-CU, the default motion information is used as the derived temporal motion of that sub-CU.

[0088] Then, for each sub-CU, the corresponding block in the collocated image (the central sample (the smallest motion grid including After the motion information of the co-located sub-CU is identified, a temporal motion scale is applied to TMVP in HEVC: Aligning the reference image of the intermediate motion vector with the reference image of the current CU The motion vector and reference index of the current sub-CU are converted in the same manner as in the process. can be.

[0089] In the current design, the motion filter in the collocated CTU in the collocated image is + Only one column to the right of this colocated CTU has SbTMVP and As shown in Figure 6, the motion information in the collocated CTU + A column of movement information to the right of this collocated CTU (in this example, CTU2 is the current Only the collocated CTU of CU) is the temporal mv derivation of SbTMVP and TMVP. For the sake of convenience, we will refer to this collocated CTU+1 column as , which we call the “valid region” for SbTMVP / TMVP derivation. In this context, is outside the valid region of the corresponding N × N block in the collocated image of the sub-CU. Each time a block is created, the corresponding N × N block is assigned to an alternate block in the collocated CTU. The location of the replacement NxN block is placed within the valid region using the following formula: It is derived by clipping the original position of the corresponding N × N block to be positioned. In the formula (position clip process of each sub CU), CurPicWidthInSamplesY and CurPicHei ghtInSamplesY is the width and height of the encoded image, and CTUWidthInSamplesX and CTUWidthInSamplesY are the width and height of the encoded image. TUWidthInSamplesY is the width and height of the CTU, and xCtb and yCtb are the coordinates of the collocated CTU. The xColCtrCb and yColCtrCb are the horizontal and vertical positions of the top-left sample of the sub-CU. are the horizontal and vertical positions of the representative sample, and MotionShiftX and MotionShiftY are the The functions Clip3(x, y, z) and Min(x, y) are as follows: is defined as follows.

number

[0090] In VVC, we use subblock-based affine merge candidates, including both SbTMVP candidates and affine merge candidates. The merge list based on subblocks is used to signal the merge mode based on subblocks. The SbTMVP mode is specified by the sequence parameter set (SPS) flag. Enabled / Disabled. If SbTMVP mode is enabled, SbTMVP prediction The child is added as the first entry in the list of subblock-based merge candidates, This is followed by affine merge candidates. The size of the subblock-based merge list is SPS. In VVC, the maximum allowed size of the merge list based on the subblock is is 5.

[0091] The sub-CU size used in SbTMVP is fixed at 8 × 8, and the affine merge modulus As in the case of the mode, the SbTMVP mode supports CUs with both width and height ≥ 8. In addition, the current VVC does not support the For temporal motion field storage, motion field compression is used in HEVC. It runs at an 8x8 granularity as opposed to a 16x16 granularity.

[0092] In one embodiment, the motion shift is always derived from the List0mv of the neighboring blocks. If List0mv is unavailable, List1mv from the adjacent block is used, Derive the motion shift of SbTMVP. The pseudocode is explained below.

number

[0093] In one embodiment, the motion shift is always derived from the List1mv of neighboring blocks. If List1mv is unavailable, List0mv from the adjacent block is used. ,We derive the motion shift of SbTMVP.,The pseudocode is explained below.

number

[0094] In one embodiment, any of the corresponding blocks of the sub-CUs that are located outside the valid area may Each time, the zero vector is used as the motion shift vector to derive SbTMVP. In this way, all the corresponding blocks of the sub-CUs of the current CU are located within the valid region. Therefore, the position clipping process for each sub-CU is not required. Any of the corresponding blocks of the sub-CU in the current CU is outside the valid region. There are many ways to determine whether the corresponding block in the top-left N×N sub-CU is The corresponding blocks of the lock and the lower right N × N sub-CU are are checked to determine if they are within the valid region. If the lock is outside the valid region, the zero vector is used as the motion shift vector. Otherwise (both corresponding blocks are in the valid region), the derived Motion shifting is used for SbTMVP.

[0095] In one embodiment, any of the corresponding blocks of the sub-CUs located outside the valid area At any time, SbTMVP is considered unavailable for the current CU.

[0096] In one embodiment, any of the corresponding blocks of the sub-CUs located outside the valid area Each time a motion shift occurs, the corresponding blocks of all sub-CUs are positioned within the valid region. Therefore, the position clipping for each sub-CU is modified to ensure that No process is required.

[0097] In one embodiment, the zero vector is always used for the motion shift for SbTMVP derivation. can be.

[0098] In one embodiment, the MV of a sub-CU that has a corresponding block located outside the valid region As a result, it is proposed to use a default MV derived from a representative sub-CU.

[0099] FIG. 7 illustrates a current image (e.g., current image 704) according to an embodiment of the present disclosure. TMVP and SbTMVP for the coding blocks (e.g., the current CU702) FIG. 1 shows a block diagram for determining a valid region used to derive The region is the current CU for TMVP or SbTMVP (e.g., current CU702) A collocated image for which a corresponding CU (e.g., corresponding CU 702′) is being searched. (e.g., a region in the collocated image 704′). The region is the CTU (e.g., CTU2) + 1 for deriving TMVP and SbTMVP. (e.g., one column of the TMV buffer 706). This is a design to reduce memory usage. The effective area is divided into collocated CTUs + 1 column. By limiting the range, only the motion information within the valid range is stored in the internal memory (e.g., cache). and the average cost of accessing the temporal motion data from external memory (in time or Energy) needs to be reduced. Currently, the maximum CTU size is 128x128 in VVC. Yes (maximum CTU size may be determined in a later stage of the VVC profile) , the CTU size is set to less than 128x128 (e.g., 64x64 or 32x32). In one example, if the CTU size is set to 64x64, The active area is constrained to a collocated 64x64 block + 1 column. Max CTU Since the design of the temporal MV buffer for the Therefore, it may not be wise to use a valid region smaller than the maximum CTU size. In one embodiment, regardless of the CTU size in use, the available region is always the maximum CTU allowed. Fixed at TU size + 1 column.

[0100] In one embodiment, the valid region is modified to be only collocated CTUs.

[0101] According to the tenth embodiment, if the CTU size is equal to the maximum CTU size, the valid area is a collocated CTU + 1 column. If the CTU size is smaller than the maximum CTU size, In this case, the effective area is the Colocated CTU + one CTU to the right of the Colocated CTU. The columns and collocated CTUs are changed to one row below.

[0102] 8A-8B are diagrams illustrating a video encoder for performing sub-block temporal segmentation in accordance with an embodiment of the present disclosure. 8 is a flow chart illustrating an example process 800 for implementing a technique for deriving a motion vector predictor. The process 800 may be a decoding process or an encoding process, but may be conveniently Above, the process 800 is performed by a video decoder (e.g., the video decoder 30 of FIG. 3). This is described as the decoding process that is performed.

[0103] As a first step, the decoder determines the collocated image of the current coding unit. (e.g., the co-located image of the current frame from the bitstream is added to the first list. receiving a first syntax element indicating whether the first or second list is from the first or second list; Which frames from the selected list from the bitstream are to be considered as co-located frames? Receive a second syntax element indicating whether the second syntax element is to be used (805). See, for example, FIG. 6A. Then, the current CU 601 in the current image 610 is Compatible with collocated CU601'.

[0104] Next, the decoder locates (810) the spatial neighboring blocks of the current coding unit. For example, referring to FIG. 6D, a current coding unit (e.g., current CU 636) is , and spatial neighbor 638 (block A1). In one embodiment, the spatial neighbor blocks are , a coding unit or sub-block.

[0105] After locating the spatial neighboring blocks, the decoder then calculates the motion of the current coding unit. A shift vector is determined (815). The motion shift vector is a vector of the current image (e.g., 6D) in the current coding unit (e.g., the current image 610 in FIG. 6D). The current CU636) and the collocated image (e.g., the collocated image in FIG. 6D) 6D) in the corresponding collocated block (e.g., spatial neighbors in FIG. 6D). 638' (block A1').

[0106] To determine the motion shift vector, the decoder uses List0 of spatially neighboring blocks. Each of the included motion vectors is examined in turn (820). Any of the motion vectors uses a collocated image as a reference image for that motion vector. In accordance with the decision (825), the decoder assigns the motion vector in List0 to the motion vector. 830 as a shift vector (e.g., motion shift vector 630) and The subsequent motion vectors in List0 and the motion vectors in List1 of the adjacent blocks As a result, the motion vector search is terminated and the list is The first matching motion vector in is used as the motion shift vector. , the decoder always first checks the spatial neighbors List1. Then, the motion vectors in List0 of the target adjacent blocks are checked.

[0107] On the other hand, each motion vector in List0 refers to a collocated image. Following the decision not to use (840) the image, the decoder Each motion vector in List1 is examined in turn (845). The controller searches for spatial neighbors only if the search for a motion vector in List0 returns a negative result. Check the motion vectors in List1 of the neighboring blocks.

[0108] While searching for a motion vector in List1 of spatially neighboring blocks, Each of the motion vectors uses the collocated image as the reference image for that motion vector. In accordance with the decision to use the motion in List1 as the motion vector (850), the decoder Set the vector as the motion shift vector (855) and the subsequent motions in List1 Abandon the vector search (860) and search for the first matching motion vector in List1. The vectors in List1 are used as motion shift vectors. However, none of them use the collocated image as a reference image for the motion vector. Pursuant to the determination (865), the decoder sets the motion shift vector to a zero value (870 As a result, the corresponding coding unit and the current coding unit are collocated. the same relative position with respect to the previous image and the current image (e.g., the current coding unit and the corresponding coding unit).

[0109] Finally, the decoder selects each of the sub-blocks in the current coding unit. For each sub-block, the motion shift vector is used to find the corresponding sub-block in the collocated image. Reconstruct subblock-based temporal motion vectors from the subblocks (875). For example, referring to FIG. 6D, after scaling, sub-block temporal motion vector prediction The motion shift vector 630 is used to calculate the corresponding sub-block temporal motion vector. 6A. In one embodiment, the sub-blocks are List0 and List Contains one or two temporal motion vectors from t1.

[0110] In one embodiment, the motion shift vector is used to determine the motion of the corresponding sub-image in the co-located image. From the block, each of the sub-blocks in the current coding unit is Reconstructing temporal motion vectors based on subblocks of the current coding unit is Based on each of the subblocks among the plurality of subblocks in the unit and estimating a temporal motion vector, and Within a given region (e.g., a valid region) in the image, each subblock Search for the collocated sub-block corresponding to a given According to the determination that a collocated subblock exists in the region, Identify one or two motion vectors for each sub-block. A sub-block-based temporal motion vector for the current image and the reference image for the current image is calculated. The first image order count (POC) distance between the ) and the second POC distance between the collocated image and the collocated image's reference image (e.g., one or two scaled based on the POD distance td in FIG. 6A) In one embodiment, the motion vector is set as a motion vector of a predetermined region in the collocated image. Following the determination that there are no colocated sub-blocks in the region, The sub-block based temporal motion vectors of the sub-blocks are set to zero-valued motion vectors. In another embodiment, the collocated image is imaged within a predetermined region. According to the determination that the co-located image does not have a red sub-block, Alternative subblocks within a region are set as corresponding subblocks. For example, The sub-block is a border sub-block in a given region that is closest to the co-located sub-block. It is a block.

[0111] In one embodiment, regardless of the size of the CTU that contains the collocated coding unit, A given region has a size equal to the maximum allowed CTU size plus one column.

[0112] In one embodiment, before checking List0 of spatial neighboring blocks, the decoder: First, the motion vectors in List1 of spatially adjacent blocks are checked.

[0113] In one or more examples, the functionality described above may be implemented in hardware, software, firmware, or other similar devices. If implemented in software, These functions may be implemented as one or more instructions or code on a computer-readable medium. and executed by a hardware processing unit. A computer-readable medium is a computer program that corresponds to a tangible medium such as a data storage medium. A computer-readable storage medium or a computer-readable storage medium, for example, according to a communication protocol, Includes communication media, including any medium that facilitates the transfer of a computer program from one location to another. Thus, computer-readable media generally includes: (1) non-transitory or (2) communications, such as signals or carrier waves. The data storage medium may correspond to one or more computers. or accessed by one or more processors to implement the embodiments described herein. Any program that can retrieve instructions, code, and / or data structures to achieve the The computer program product may be a computer-readable medium. The recording medium may include a recording medium that can be used for recording.

[0114] The terminology used to describe the embodiments herein is intended to describe specific embodiments only. The present invention is intended to be illustrative, not limiting, of the scope of the claims. As used in the description and the appended claims, the singular forms "one," "one," and "an" are intended to be used interchangeably. and "this" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" refers to one or more of the following: It should also be understood that the term means and includes any and all possible combinations of the listed items. As used herein, the term "comprising" means any combination of the stated features, elements, and / or indicates the presence of one or more other features, elements, components, and / or It should further be understood that the presence or addition of such sets is not excluded.

[0115] Herein, the terms first, second, etc. have been used to describe various elements. It should also be understood that these terms are not limiting. The elements are used only to distinguish one element from another. For example, the elements may deviate from the scope of the embodiment. Unless otherwise specified, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first The first electrode and the second electrode are both electrodes, but they may be called electrodes. do not have.

[0116] The present description has been presented for purposes of illustration and description and is not intended to be exhaustive or to be construed in a disclosed form. The present invention is not limited to the above. Various modifications, variations, and alternative implementations are possible in accordance with the above description. It will be apparent to one skilled in the art having the benefit of the teachings presented herein and the associated drawings. The embodiments are intended to best explain the principles and practical applications of the present invention, and will allow those skilled in the art to easily understand the present invention for various implementations. Understand the underlying principles and various implementations with various modifications to suit specific applications. The following examples have been selected and described to facilitate their general use: The claims are not limited to the specific examples of implementations disclosed, but rather include modifications and other implementations. It should be understood that this falls within the scope of the appended claims.

Claims

1. 1. A method for decoding a current coding unit in a current image, comprising: determining a collocated image of the current coding unit; and Position the spatially neighboring blocks of the current coding unit corresponding to the collocated image. Deciding on the placement and Each of the motion vectors included in List0 of the spatially adjacent blocks is examined in sequence. To do, Which of the motion vectors in the List0 corresponds to the co-located image? According to the decision to use the image as a reference for the motion vector: The motion vector in List0 is set as a motion shift vector; Examination of subsequent motion vectors in List0 and motion vectors in List1 and Each motion vector in the List0 refers to the co-located image. Following the decision not to use it as an image: Sequentially checking each of the motion vectors in List 1 of the spatially neighboring blocks; Which of the motion vectors in List1 corresponds to the co-located image? According to the decision to use as a reference image for the motion vector: The motion vector in the List1 is set as the motion shift vector; Abandon examination of subsequent motion vectors in List1; Each motion vector in the List1 corresponds to the co-located image. According to the decision not to use the image as a reference image for the motion vector, setting the motion shift vector to a zero value vector; Including, The current coding unit in the current image and its counterpart in the collocated image the current coding unit indicating a spatial position shift between the corresponding co-located block and the current coding unit; determining the motion shift vector of the set; Each subblock of the plurality of subblocks in the current coding unit Based on the motion shift vector, reconstructing a sub-block based temporal motion vector from the sub-blocks; A decoding method including:

2. Each subblock of the plurality of subblocks in the current coding unit Based on the motion shift vector, Reconstructing a sub-block-based temporal motion vector from the sub-blocks Based on the motion shift vector, Then, the colocated sub-blocks corresponding to each sub-block are searched for. 、 The collocated sub-block exists within a predetermined region in the collocated image. Pursuant to the decision to: identifying one or two motion vectors for the collocated sub-block; a sub-block based temporal motion vector for each of the sub-blocks in the A first picture order count (POC) distance between a current image and a reference image of the current image. and a second PO between the collocated image and a reference image of the collocated image. C. Setting as one or two motion vectors scaled based on the distance; Including, Each of the sub-blocks in the current coding unit predicting a temporal motion vector based on sub-blocks of the block; The method of claim 1 , comprising:

3. Each subblock of the plurality of subblocks in the current coding unit Predicting the temporal motion vector based on the sub-blocks of The collocated sub-block is located within the predetermined region in the collocated image. Following the non-existence decision: The sub-block-based temporal motion vector of the corresponding sub-block is expressed as a zero-value motion vector. Tor The method of claim 2 , comprising:

4. Each subblock of the plurality of subblocks in the current coding unit Predicting the temporal motion vector based on the sub-blocks of The collocated sub-block is located within the predetermined region in the collocated image. Following the non-existence decision: The collocated sub-blocks in the given region in the collocated image The alternative sub-block is a border sub-block in the predetermined region that is closest to the Setting it as the corresponding sub-block; The method of claim 2 , comprising:

5. The blocks adjacent to the current coding unit are coding units or adjacent blocks of coding units. The method of claim 1 , wherein the first and second subblocks are

6. The predetermined area is independent of the size of the CTU that includes the collocated coding unit.

3. The method of claim 2, wherein the first and second columns have a size such that the first column is equal to the maximum allowed CTU size plus one column. 。

7. The method of claim 5 , wherein the maximum allowed CTU size is 128×128.

8. one or more processors; a memory coupled to the one or more processors; When stored in the memory and executed by the one or more processors, the A plurality of programs for causing a computing device to perform the operations according to any one of claims 1 to 7. Ram and 23. A computing device comprising:

9. A method for implementing a method for computing a computer system having one or more processors, comprising: A non-transitory computer-readable storage medium for storing a program, The plurality of programs, when executed by the one or more processors, A non-transitory method for causing a computing device to perform the operations of any one of claims 1 to 7. A computer-readable storage medium.

Images