Methods, electronic devices, non-temporary computer-readable storage media, and computer programs

Abstract

The computing device performs a method for decoding video data by generating a multi-model linear model (MMLM) from a group of reference luma samples and a group of reference chroma samples, the MMLM including a first linear model between a minimum luma value and a threshold luma value and a second linear model between a threshold luma value and a maximum luma value, and reconstructing each sample value of a chroma block from a weighted combination of corresponding respective first sample values ​​of the luma block reconstructed using the multi-model linear model and respective second sample values ​​of adjacent chroma blocks reconstructed from an intra prediction mode.

Description

[Technical Field]

[0001] Cross-reference of related applications This application is based on U.S. Provisional Patent Application No. 63 / 176,140, ​​“Video Coding Using a Multi-Model Linear Model,” filed on 16 April 2021, and claims priority to U.S. Provisional Patent Application No. 63 / 176,140, ​​the contents of which are incorporated in their entirety by reference.

[0002] This application relates to video coding and compression, and more specifically, to a method for improving coding efficiency, simplifying complexity, and improving the accuracy of intra-prediction, and to electronic devices. , non This relates to temporary computer-readable storage media and computer programs. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, including digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconference devices, and video streaming devices. These electronic devices transmit, receive, or communicate digital video data over communication networks and / or store digital video data in storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video coding may be used to compress video data according to one or more video coding standards before it is transmitted or stored. Examples of video coding standards include Universal Video Coding (VVC), Joint Exposure Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding generally utilizes predictive methods (e.g., inter-prediction, intra-prediction, etc.) that leverage the inherent redundancy of video data. Video coding aims to compress video data into a format that uses a lower bitrate while avoiding or minimizing a decrease in video quality. [Overview of the project]

[0004] This application relates to video data coding and decoding, more specifically to a method for improving coding efficiency, simplifying complexity, and improving the accuracy of intra-prediction using a multi-model linear model (MMLM) and intra-prediction mode, and to electronic devices. , non Embodiments relating to temporary computer-readable storage media and computer programs are described.

[0005] According to a first aspect of this application, a method for constructing a chroma block of a video signal includes receiving a bitstream encoding a chroma block, a corresponding luma block, a plurality of adjacent luma samples surrounding the luma block, and the plurality of adjacent chroma samples surrounding the chroma block; decoding the luma block, the plurality of adjacent luma samples, and the plurality of adjacent chroma samples in order to obtain a plurality of reconstructed luma samples of the luma block, a plurality of reconstructed adjacent luma samples, and a plurality of reconstructed adjacent chroma samples, respectively; and obtaining from the plurality of reconstructed adjacent luma samples and the plurality of reconstructed adjacent chroma samples a group of reference luma samples and a group of reference chroma samples, where each reference luma sample corresponds to a respective reference chroma sample. standard Select a group of chromatic samples and a group of reference chromatic samples. Toto This includes: calculating a threshold luma value from a group of reference luma samples, calculating a corresponding threshold chroma value from a group of reference chroma samples; determining the maximum and minimum luma values ​​from the group of reference luma samples such that the threshold luma value lies between the minimum and maximum luma values; generating a multi-model linear model that includes a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value; and reconstructing each sample value of a chroma block from a weighted combination of the corresponding first sample values ​​derived from the luma block and reconstructed using the multi-model linear model, and the respective second sample values ​​of adjacent chroma blocks reconstructed from the intra-prediction mode. Multiple adjacent luma samples are divided into a predetermined number of reference sets, multiple adjacent chromatomorphic samples are divided into a predetermined number of reference sets, a group of reference luma samples is selected from one of the predetermined number of reference sets, a predetermined number is determined based on the coded information of the current block, which includes chromatomorphic and luma blocks, and the groups of reference luma samples and reference chromatomorphic samples are selected based on the coded information of the current block. .

[0006] According to a second aspect of this application, the electronic device comprises one or more processing units, a memory coupled to the one or more processing units, and a plurality of programs stored in the memory. When these programs are executed by one or more processing units, they cause the electronic device to perform the video signal encoding method described above.

[0007] According to a third aspect of the present application, a non - transient computer - readable storage medium stores a plurality of programs to be executed by an electronic device having one or more processing units. When these programs are executed by one or more processing units, the electronic device is caused to execute the method of encoding a video signal as described above.

[0008] According to a fourth aspect of the present application, a computer - readable storage medium stores a bitstream including video information generated by the method of video decoding as described above.

[0009] It should be understood that both the foregoing general description and the following detailed description are exemplary only and do not limit the present disclosure.

[0010] Included to provide a further understanding of the embodiments, the accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate the described embodiments and are useful for explaining the underlying principles together with the description. Like reference numerals refer to corresponding parts.

Brief Description of the Drawings

[0011] [Figure 1] It is a block diagram showing a typical video encoding and decoding system according to some implementations of the present disclosure. [Figure 2] It is a block diagram illustrating a typical video encoder according to some implementations of the present disclosure. [Figure 3] It is a block diagram showing a typical video decoder according to some implementations of the present disclosure. [Figure 4A] It is a block diagram showing how a frame is recursively quad - tree divided into a plurality of video blocks of different sizes according to some implementations of the present disclosure. [Figure 4B] It is a block diagram showing how a frame is recursively quad - tree divided into a plurality of video blocks of different sizes according to some implementations of the present disclosure. [Figure 4C]This block diagram shows how a frame is recursively quadtree-partitioned into multiple video blocks of different sizes according to some implementations of this disclosure. [Figure 4D] This block diagram shows how a frame is recursively quadtree-partitioned into multiple video blocks of different sizes according to some implementations of this disclosure. [Figure 5A] This block diagram illustrates spatially adjacent and temporally collated block locations of the current CU to be encoded in some implementations of the present disclosure. [Figure 5B] Block diagram showing multithreaded encoding of multiple lines of a picture CTlJ using wavefront parallel processing according to some implementations of this disclosure. [Figure 5C] An intra-mode block diagram as defined in the VVC standard relating to some implementations of this disclosure. [Figure 5D] This block diagram shows a set of reconstructed samples adjacent to the current block, above and to the left, as a reference for intra-prediction related to some implementations of this disclosure. [Figure 5E] A block diagram showing a selected set of pixels on which gradient analysis is performed, relating to some implementations of this disclosure. [Figure 5F] This block diagram shows a 3x3 Sobel gradient filter convolution process using templates relating to some implementations of this disclosure. [Figure 6A] A block diagram showing a typical pre-reconstructed Lumablock 602 to be decrypted, relating to some implementations of this disclosure. [Figure 6B] This is a block diagram showing a typical relevant chroma block 620 to be decrypted in some implementations of this disclosure. [Figure 7A] This plot shows a typical process in which a videocoder performs a technique relating to some implementations of this disclosure, which involves deriving a multi-model linear model and applying the multi-model linear model to predict chroma samples of coding units. [Figure 7B]This plot shows a typical process in which a videocoder performs a technique relating to some implementations of this disclosure, which involves deriving a multi-model linear model and applying the multi-model linear model to predict chroma samples of coding units. [Figure 7C] This plot shows a typical process in which a videocoder performs a technique relating to some implementations of this disclosure, which involves deriving a multi-model linear model and applying the multi-model linear model to predict chroma samples of coding units. [Figure 7D] This plot shows a typical process in which a videocoder performs a technique relating to some implementations of this disclosure, which involves deriving a multi-model linear model and applying the multi-model linear model to predict chroma samples of coding units. [Figure 8] This flowchart shows a typical process in which a videocoder performs a technique for some implementations of this disclosure, which involves deriving a multi-model linear model and applying the multi-model linear model to predict chroma samples of coding units. [Figure 9] This block diagram shows the positions of adjacent samples (shown as gray circles) used in MMLM in some implementations of this disclosure. [Figure 10] A block diagram showing the locations of four sets of samples used in MMLM in some implementations of this disclosure. [Figure 11] This flowchart shows a typical process in which a videocoder implements a technique for predicting or constructing chroma blocks in a video signal by combining MMLM and intra-prediction, as described in some implementations of this disclosure. [Figure 12] This figure shows a computing environment coupled with a user interface, relating to some implementations of this disclosure. [Modes for carrying out the invention]

[0012] Here, we will refer in detail to a specific embodiment shown in the accompanying drawings. The following detailed description includes numerous non-limiting specific details to aid in understanding the subject matter presented herein. However, as those skilled in the art will see, various alternative forms can be used without departing from the claims, and the subject matter can be implemented without these specific details. For example, as those skilled in the art will see, the subject matter presented herein can be implemented in many types of electronic devices having digital video capabilities.

[0013] Figure 1 is a block diagram showing a typical system 10 for encoding and decoding video blocks in parallel according to some implementations of the present disclosure. As shown in Figure 1, the system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. The source device 12 and destination device 14 may include any of the wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, and video streaming devices. In some implementations, the source device 12 and destination device 14 have wireless communication capabilities.

[0014] In some implementations, the destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device that can move the encoded video data from the source device 12 to the destination device 14. In one example, link 16 may include a communication medium to enable the source device 12 to directly transmit the encoded video data to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device 12 to the destination device 14.

[0015] In some other implementations, encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 can then be accessed by destination device 14 via input interface 28. Storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, storage device 32 may correspond to a file server or other intermediate storage device that may hold encoded video data generated by source device 12, and destination device 14 may access the stored video data from storage device 32 via streaming or download. The file server may be any type of computer capable of storing and transmitting encoded video data to destination device 14. Typical file servers include web servers (e.g., for websites), FTP servers, network-attached storage (NAS) devices, or local disk drives. The destination device 14 may access the encoded video data stored on the file server via any standard data connection, including a wireless channel suitable for accessing the encoded video data (e.g., Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both. Transmission of the encoded video data from the storage device 32 may be via streaming transmission, download transmission, or a combination of both.

[0016] As shown in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources such as a video imaging device such as a video camera, a video archive containing pre-recorded video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. For example, if the video source 18 is a video camera in a security surveillance system, the source device 12 and the destination device 14 may form a cameraphone or video cameraphone. However, the embodiments described in this application may be applicable to video coding in general and may be applicable to wireless and / or wired applications.

[0017] Captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video data may be transmitted directly to the destination device 14 via the output interface 22 of the source device 12, or the encoded video data may be stored in the storage device 32 for later access by the destination device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0018] The destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 may include a receiver and / or a modem and may receive encoded video data via link 16. The encoded video data communicated via link 16 or provided on the storage device 32 may include various syntax elements generated by the video encoder 20 for use by the video decoder 30 when decoding the video data. Such syntax elements may be included in the encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0019] In some implementations, the destination device 14 may include a display device 34 which can be an integrated display device and an external display device configured to communicate with the destination device 14. The display device 34 displays the decoded video data to the user and may include any of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0020] The video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards such as VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions of such standards. It should be understood that this application is not limited to any particular video coding / decoding standard and may be applicable to other video coding / decoding standards. It is generally conceivable that the video encoder 20 of the source device 12 may be configured to encode video data in accordance with any of these current or future standards. Similarly, it is generally conceivable that the video decoder 30 of the destination device 14 may be configured to decode video data in accordance with any of these current or future standards.

[0021] The video encoder 20 and video decoder 30 may each be implemented as one of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. If partially implemented in software, the electronic device may store instructions for the software in a suitable non-temporary computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed herein. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be incorporated as part of a composite encoder / decoder (CODEC) in the respective device.

[0022] Figure 2 is a block diagram showing a typical video encoder 20 according to several embodiments described in this application. The video encoder 20 may perform intra-predictive coding and inter-predictive coding of video blocks within a video frame. Intra-predictive coding relies on spatial prediction to reduce or eliminate spatial redundancy of video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or eliminate temporal redundancy of video data within adjacent video frames or pictures in a video sequence.

[0023] As shown in Figure 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a splitting unit 45, an intra-prediction processing unit 46, and an intra-block copy (BC) unit 48. In some implementations, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A deblocking filter (not shown) may be positioned between the adder 62 and the DPB 64 to filter block boundaries and remove blockness artifacts from the reconstructed video. In addition to the deblocking filter, an in-loop filter (not shown) may be used to filter the output of the adder 62. The video encoder 20 may take the form of a fixed or programmable hardware unit, or it may be divided into one or more of the exemplified fixed or programmable hardware units.

[0024] The video data memory 40 may store video data to be encoded by the components of the video encoder 20. The video data in the video data memory 40 may be obtained, for example, from the video source 18. The DPB 64 is a buffer that stores reference video data used for encoding video data by the video encoder 20 (for example, in intra or interpredictive coding mode). The video data memory 40 and the DPB 64 may be formed by any of various memory devices. In various examples, the video data memory 40 may be on-chip with the other components of the video encoder 20 or off-chip with respect to those components.

[0025] As shown in Figure 2, the splitting unit 45 within the prediction processing unit 41, after receiving the video data, splits the video data into video blocks. This splitting may also include dividing the video frame into slices, tiles, or other larger coding units (CUs) according to a predetermined splitting structure, such as a quadtree structure associated with the video data. A video frame may be split into multiple video blocks (or sets of video blocks called tiles). Based on error results (e.g., coding rate and distortion level), the prediction processing unit 41 may select one of several possible predictive coding modes for the current video block, such as one of several intra-predictive coding modes or one of several inter-predictive coding modes. The prediction processing unit 41 may feed the resulting intra- or inter-predictive coded blocks to the adder 50 to generate residual blocks, and to the adder 62 to reconstruct the coded blocks, which may then be used as part of a reference frame. Furthermore, the prediction processing unit 41 supplies syntax elements such as motion vectors, intra-mode indicators, segmentation information, and other such syntax information to the entropy coding unit 56.

[0026] To select an intra-predictive coding mode suitable for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 may perform intra-predictive coding of the current video block for one or more adjacent blocks in the same frame as the current block that are to be coded to give a spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform interpredictive coding of the current video block for one or more prediction blocks within one or more reference frames to provide a time prediction. The video encoder 20 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0027] In some embodiments, the motion estimation unit 42 determines the inter-prediction mode for the current video frame by generating motion vectors that show the displacement of the prediction unit (PU) of the video block in the current video frame relative to the prediction block in the reference video frame, according to a predetermined pattern in the sequence of video frames. Motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors that estimate motion in the video block. The motion vectors may, for example, show the displacement of the PU of the video block in the current video frame or picture relative to the prediction block in the reference frame (or other coding unit) relative to the current block coded in the current frame (or other coding unit). The predetermined pattern may specify the video frames in the sequence as P frames or B frames. The intra-BC unit 48 may determine vectors for intra-BC coding, such as block vectors, in a manner similar to the determination of motion vectors by the motion estimation unit 42 for inter-prediction, or may determine block vectors using the motion estimation unit 42.

[0028] The prediction block is a block of the reference frame that is considered to closely match the PU of the video block to be coded with respect to pixel differences, which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 may calculate values ​​for the subinteger pixel positions of the reference frame stored in the DPB64. For example, the video encoder 20 may interpolate values ​​for the quarter-pixel, eighth-pixel, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 may perform motion search for all pixel positions and fractional pixel positions and output motion vectors with fractional-pixel precision.

[0029] The motion estimation unit 42 calculates a motion vector for the PU of a video block in an interpredicted coding frame by comparing the position of the PU with the position of the predicted block in a reference frame selected from a first reference frame list (List0) or a second reference frame list (List1), each of which identifies one or more reference frames stored in the DPB64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0030] Motion compensation performed by the motion compensation unit 44 may include fetching or generating a prediction block based on a motion vector determined by the motion estimation unit 42. Upon receiving the motion vector in the PU of the current video block, the motion compensation unit 44 may find the prediction block pointed to by the motion vector in one of the reference frame lists, search for the prediction block in the DPB 64, and transfer the prediction block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block given by the motion compensation unit 44 from the pixel values ​​of the current video block being coded. The pixel difference values ​​forming the residual video block may include luma or chroma difference components or both. The motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, a syntax element defining a motion vector used to identify the prediction block, an optional flag indicating a prediction mode, or any other syntax information described herein. The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes.

[0031] In some implementations, the intraBC unit 48 may generate vectors and fetch prediction blocks in a manner similar to those described above in relation to the motion estimation unit 42 and the motion compensation unit 44, but the prediction blocks are in the same frame as the currently coded block, and the vectors are called block vectors, as opposed to motion vectors. In particular, the intraBC unit 48 may determine which intraprediction mode should be used to encode the current block. In some examples, the intraBC unit 48 may encode the current block using various intraprediction modes, for example, in separate encoding passes, and test their performance by rate distortion analysis. The intraBC unit 48 may then select an appropriate intraprediction mode from among the various tested intraprediction modes and generate it accordingly using an intramode indicator. For example, the intraBC unit 48 may calculate rate distortion values ​​using rate distortion analysis of various tested intraprediction modes and select the intraprediction mode with the best rate distortion characteristics among the tested modes as the appropriate intraprediction mode to use. Rate distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to produce the encoded block, as well as the bit rate (i.e., number of bits) used to produce the encoded block. The intraBC unit 48 may calculate a ratio from the distortion and rate for various encoded blocks to determine which intraprediction mode shows the best rate distortion value for the block.

[0032] In other examples, the intraBC unit 48 may use the motion estimation unit 42 and the motion compensation unit 44 in whole or in part to perform such functions for intraBC prediction in the implementation described herein. In any case, with respect to the intrablock copy, the predicted block may be a block that is considered to closely match the block to be coded with respect to a pixel difference which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics, and the identification of the predicted block may involve the calculation of a value with respect to a partial integer pixel position.

[0033] Regardless of whether the prediction block is from the same frame relating to intra-prediction or from a different frame relating to inter-prediction, the video encoder 20 may form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the currently encoded video block to form a pixel difference value. The pixel difference value forming the residual video block may include both luminous component difference and chroma component difference.

[0034] The intra-prediction processing unit 46 may intra-predict the video block to be processed instead of intra-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, or intra-block copy prediction performed by the intra-BC unit 48, as described above. In particular, the intra-prediction processing unit 46 may determine the intra-prediction mode to use for encoding the current block. To this end, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, in separate encoding passes, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) may select an intra-prediction mode suitable for use from the tested intra-prediction modes. The intra-prediction processing unit 46 may provide the entropy encoding unit 56 with information indicating the selected intra-prediction mode for the block. The entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into a bitstream.

[0035] After the prediction processing unit 41 determines the prediction block for the current video block via either interpretation or intraprediction, the adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be contained in one or more transformation units (TUs) and provided to the transformation processing unit 52. The transformation processing unit 52 transforms the residual video data into residual transformation coefficients using a transformation such as a discrete cosine transform (DCT) or a conceptually similar transformation.

[0036] The conversion processing unit 52 may send the resulting conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be changed by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0037] Following quantization, the entropy coding unit 56 entropy-codes the quantization transformation coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), stochastic interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be transmitted to the video decoder 30, or archived in the storage device 32 for later transmission to the video decoder 30 or retrieval by the video decoder 30. The entropy coding unit 56 may also entropy-code motion vectors and other syntax elements for the current video frame being coded.

[0038] The inverse quantization unit 58 and the inverse transformation processing unit 60 apply inverse quantization and inverse transformation, respectively, to reconstruct the residual video block in the pixel region to generate a reference block for predicting other video blocks. As previously mentioned, the motion compensation unit 44 may generate a motion-compensated prediction block from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate partial integer pixel values ​​for use in motion estimation.

[0039] The adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by the motion compensation unit 44 to generate a reference block for storage in the DPB 64. The reference block may then be used by the intraBC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block for interpreting other video blocks in subsequent video frames.

[0040] Figure 3 is a block diagram showing a typical video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transformation processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-prediction processing unit 84, and an intra-BC unit 85. The video decoder 30 may perform a decoding process that is substantially the inverse of the encoding process described above with respect to the video encoder 20 in relation to Figure 2. For example, the motion compensation unit 82 may generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-prediction unit 84 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 80.

[0041] In some examples, a unit of the video decoder 30 may be assigned to perform the embodiments of the present application. Also, in some examples, the embodiments of the present disclosure may be divided among one or more units of the video decoder 30. For example, the intraBC unit 85 may perform the embodiments of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra predictive processing unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intraBC unit 85, and the function of the intraBC unit 85 may be performed by other components of the predictive processing unit 81, such as the motion compensation unit 82.

[0042] The video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of the video decoder 30. Video data stored in the video data memory 79 may be retrieved, for example, from a storage device 32, from a local video source such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coding picture buffer (CPB) that stores the encoded video data from the encoded video bitstream. The decoding picture buffer (DPB) 92 of the video decoder 30 stores reference video data (e.g., in intra or inter-predictive coding mode) used when the video decoder 30 decodes the video data. The video data memory 79 and DPB 92 may be formed by any of various memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM®), or other types of memory devices. For illustrative purposes, the video data memory 79 and DPB92 are shown as two separate components of the video decoder 30 in Figure 3. However, as those skilled in the art will see, the video data memory 79 and DPB92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip with the other components of the video decoder 30 or off-chip with respect to those components.

[0043] During the decoding process, the video decoder 30 receives an encoded video bitstream and associated syntax elements representing video blocks of the encoded video frames. The video decoder 30 may receive syntax elements at the video frame level and / or video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-predictive mode indicators, and other syntax elements. The entropy decoding unit 80 then transfers the motion vectors and other syntax elements to the prediction processing unit 81.

[0044] When a video frame is coded as an intra-predictive coded (I) frame, or with respect to an intra-coded predictive block within another type of frame, the intra-predictive processing unit 84 of the predictive processing unit 81 may generate predictive data in the video block of the current video frame based on the signaled intra-predictive mode and reference data from a pre-decoded block of the current frame.

[0045] When a video frame is coded as an interpredicted (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each prediction block may be generated from a reference frame in one of the reference frame lists. The video decoder 30 may configure the reference frame lists, list 0 and list 1, using default configuration techniques based on the reference frames stored in the DPB 92.

[0046] In some examples, when a video block is coded according to the intraBC mode described herein, the intraBC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block may be within the same reconfigured region of the picture as the current video block defined by the video encoder 20.

[0047] The motion compensation unit 82 and / or intra-BC unit 85 determine prediction information about the video block of the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate prediction blocks about the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode used to encode the video block of the video frame (e.g., intra or inter-predict), the inter-predict frame type (e.g., B or P), configuration information in one or more of the reference frame lists for the frame, the motion vector in each inter-predict coded video block of the frame, the inter-predict state in each inter-predict coded video block of the frame, and other information for decoding the video block in the current video frame.

[0048] Similarly, the intraBC unit 85 may use some of the received syntax elements, for example, flags, to determine that the current video block was predicted using intraBC mode, configuration information indicating which video blocks in the frame are in the reconstructed region and should be stored in the DPB 92, block vectors in each intraBC predicted video block of the frame, the intraBC prediction state in each intraBC predicted video block of the frame, and other information for decoding the video blocks in the current video frame.

[0049] Furthermore, the motion compensation unit 82 may perform interpolation during the encoding of the video block using an interpolation filter similar to that used by the video encoder 20 in order to calculate interpolation values ​​in partial integer pixels of the reference block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the video encoder 20 from the received syntax elements and generate a predicted block using the interpolation filter.

[0050] The inverse quantization unit 86 inverse quantizes the quantized transformation coefficients that are given to the bitstream and entropy-decoded by the entropy decoding unit 80, using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame, and determines the degree of quantization. The inverse transformation processing unit 88 applies an inverse transformation, such as an inverse DCT, an inverse integer transformation, or a conceptually similar inverse transformation process, to the transformation coefficients in order to reconstruct the residual blocks in the pixel region.

[0051] After the motion compensation unit 82 or intraBC unit 85 generates a predicted block in the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block in the current video block by adding the residual block from the inverse processing unit 88 with the corresponding predicted block generated by the motion compensation unit 82 and intraBC unit 85. An in-loop filter (not shown) may be positioned between the adder 90 and the DPB 92 for further processing of the decoded video block. The decoded video block in a given frame is stored in the DPB 92, which stores a reference frame to be used for subsequent motion compensation of the next video block. Alternatively, the DPB 92 or a separate memory device may store the decoded video for later presentation on a display device such as the display device 34 in Figure 1.

[0052] In a typical video coding process, a video sequence generally contains an ordered set of frames or pictures. Each frame may contain three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore contain only one two-dimensional array of luma samples.

[0053] As shown in Figure 4A, the video encoder 20 (or, more specifically, the splitting unit 45) first generates an encoded representation of a frame by splitting the frame into a set of coding tree units (CTUs). A video frame may contain an integer number of CTUs that are sequentially ordered from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTUs are signaled by the video encoder 20 in the sequence parameter set such that all CTUs in the video sequence have the same size, one of 128x128, 64x64, 32x32, and 16x16. However, it should be noted that this application is not necessarily limited to specific sizes. As shown in Figure 4B, each CTU may contain one coding tree block (CTB) for a chroma sample, two corresponding coding tree blocks for a chroma sample, and syntax elements used to encode the samples in the coding tree blocks. The syntax elements describe how the video sequence can be reconstructed by the video decoder 30, including the appropriateness of different types of units of coded pixel blocks, and inter- or intra-prediction, intra-prediction mode, motion vector, and other parameters. For a monochrome picture or a picture with three separate color planes, the CTU may include a single coding tree block and syntax elements used to code samples of the coding tree block. The coding tree block may be an NxN block of samples.

[0054] To achieve better performance, the video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, quadtree partitioning, or a combination of both, in the coding tree block of the CTU to divide the CTU into smaller coding units (CUs). As shown in Figure 4C, the 64x64 CTU400 is first divided into four smaller CUs, each with a block size of 32x32. Of the four smaller CUs, CU410 and CU420 are each divided into four 16x16 CUs by their block size. The two 16x16 CUs, CU430 and CU440, are each further divided into four 8x8 CUs by their block size. Figure 4D shows a quadtree data structure representing the final result of the partitioning process of the CTU400 as shown in Figure 4C, where each leaf node of the quadtree corresponds to one CU of each size ranging from 32x32 to 8x8. Similar to the CTU shown in Figure 4B, each CU may include a coding block (CB) for a luminous sample and two corresponding coding blocks for a chroma sample of the same size frame, along with syntax elements used to encode the samples in the coding blocks. For monochrome pictures or pictures with three distinct color planes, the CU may include a single coding block and a syntax structure used to encode the samples in the coding block.

[0055] In some implementations, the video encoder 20 may further divide the coding block of the CU into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of a sample to which the same prediction, i.e., inter-prediction or intra-prediction, is applied. A prediction unit (PU) of the CU may include a prediction block for a luminous sample, two corresponding prediction blocks for a chroma sample, and syntax elements used to predict the prediction block. For a monochrome picture or a picture with three distinct color planes, the PU may include a single prediction block and a syntax structure used to predict the prediction block. The video encoder 20 may generate prediction luminous, Cb and Cr blocks in the luminous, and Cb and Cr prediction blocks for each PU of the CU.

[0056] The video encoder 20 may use intra-prediction or inter-prediction to generate prediction blocks in the PU. If the video encoder 20 uses intra-prediction to generate prediction blocks in the PU, the video encoder 20 may generate prediction blocks in the PU based on decoded samples of frames associated with the PU. If the video encoder 20 uses inter-prediction to generate prediction blocks in the PU, the video encoder 20 may generate prediction blocks in the PU based on decoded samples of one or more frames other than the frames associated with the PU.

[0057] After the video encoder 20 has generated predicted luma and Cb and Cr blocks in one or more PUs of the CU, the video encoder 20 may generate luma residual blocks in the CU by subtracting the predicted luma blocks of the CU from its original luma coding block, such that each sample in the luma residual block of the CU represents the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate Cb residual blocks and Cr residual blocks in the CU, respectively, so that each sample in the Cb residual block of the CU represents the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU represents the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0058] Furthermore, as illustrated in Figure 4C, the video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of the CU into one or more luma, Cb, and Cr transformation blocks. A transformation block is a rectangular (square or non-square) block of a sample to which the same transformation is applied. A transformation unit (TU) of the CU may include a transformation block for a luma sample, two corresponding transformation blocks for a chroma sample, and a syntax element used to transform the transformation block sample. Thus, each TU of the CU may be associated with a luma transformation block, a Cb transformation block, and a Cr transformation block. In some examples, a luma transformation block associated with a TU may be a subblock of the luma residual block of the CU. A Cb transformation block may be a subblock of the Cb residual block of the CU. A Cr transformation block may be a subblock of the Cr residual block of the CU. In a monochrome picture or a picture having three separate color planes, the TU may include a single transformation block and a syntax structure used to transform samples of the transformation block.

[0059] The video encoder 20 may apply one or more transformations to the Luma transform block of TU in order to generate a Luma coefficient block in TU. The coefficient block may be a two-dimensional array of transformation coefficients. The transformation coefficients may be scalar quantities. The video encoder 20 may apply one or more transformations to the Cb transform block of TU in order to generate a Cb coefficient block in TU. The video encoder 20 may apply one or more transformations to the Cr transform block of TU in order to generate a Cr coefficient block in TU.

[0060] After generating a coefficient block (e.g., a Luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to the process by which the transformation coefficients are quantized to potentially reduce the amount of data used to represent the transformation coefficients, resulting in further compression. After the video encoder 20 has quantized the coefficient block, the video encoder 20 may entropy encode the syntax elements representing the quantized transformation coefficients. For example, the video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements representing the quantized transformation coefficients. Finally, the video encoder 20 may output a bitstream containing a sequence of bits that form a representation of the encoded frame and associated data, which is stored in the storage device 32 or transmitted to the destination device 14.

[0061] After receiving the bitstream generated by the video encoder 20, the video decoder 30 may parse the bitstream to obtain syntax elements from it. The video decoder 30 may reconstruct frames of video data based at least partially on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 may perform an inverse transform on the coefficient blocks associated with the TU of the current CU to reconstruct the residual blocks associated with the TU of the current CU. The video decoder 30 also reconstructs the coding blocks of the current CU by adding samples of the prediction blocks in the PU of the current CU to the corresponding samples of the transform blocks in the TU of the current CU. After reconstructing the coding blocks for each CU of the frame, the video decoder 30 may reconstruct the frame.

[0062] As mentioned earlier, video coding primarily uses two modes to achieve video compression: intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction). IBC can be considered either intra-frame prediction or a third mode. Between the two modes, intra-frame prediction contributes to coding efficiency more than intra-frame prediction because it uses motion vectors to predict the current video block from a reference video block.

[0063] However, improvements in video data acquisition techniques for preserving video data details and more sophisticated video block sizes also significantly increase the amount of data required to represent the motion vector in the current frame. One way to overcome this challenge is to benefit from the fact that groups of adjacent CUs in both the spatial and temporal domains not only have similar video data to predict the target, but also similar motion vectors between these adjacent CUs. Therefore, by exploring their spatial and temporal correlations, also called "motion vector predictors (MVPs) of the current CU," it is possible to use the motion information of spatially adjacent CUs and / or temporally collated CUs as an approximation of the motion information (e.g., motion vector) of the current CU.

[0064] As mentioned above in relation to Figure 2, instead of encoding the actual motion vector of the current CU determined by the motion estimation unit 42 into a video bitstream, the motion vector difference (MVD) of the current CU is generated by subtracting the motion vector predictor of the current CU from the actual motion vector of the current CU. By doing so, the motion estimation unit 42 no longer needs to encode the motion vector it determines for each CU of a frame into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0065] Similar to the process of selecting a predicted block in a reference frame during interframe prediction of a code block, both the video encoder 20 and the video decoder 30 must employ a set of rules to select one member from the motion vector candidate list (also known as a "merge list") in the current CU as the motion vector predictor for the current CU, after constructing a motion vector candidate list (also known as a "merge list") in the current CU using potential candidate motion vectors associated with spatially adjacent CUs and / or temporally collated CUs of the current CU. In this way, the motion vector candidate list itself does not need to be transmitted between the video encoder 20 and the video decoder 30, and the index of the selected motion vector predictor in the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to use the same motion vector predictor in the motion vector candidate list to encode and decode the current CU.

[0066] In some implementations, each interpretation CU has three motion vector prediction modes, including interpretation (also called "Advanced Motion Vector Prediction" (AMVP)), skipping, and merging, to construct a motion vector candidate list. Under each mode, one or more motion vector candidates may be added to the motion vector candidate list according to the algorithms described below. Finally, one of the motion vector candidates in the candidate list is used as the best motion vector predictor of the interpretation CU to be encoded into a video bitstream by the video encoder 20 or decoded from the video bitstream by the video decoder 30. To find the best motion vector predictor from the candidate list, a motion vector competition (MVC) scheme is introduced to select a motion vector from a given set of motion vector candidates, i.e., a motion vector candidate list containing spatial and temporal motion vector candidates.

[0067] In addition to deriving motion vector prediction candidates from spatially adjacent or temporally collated CUs, motion vector prediction candidates may also be derived from a so-called “history-based motion vector prediction” (HMVP) table. The HMVP table contains a predetermined number of motion vector predictors, each predictor used to encode / decode a specific CU in the same row of a CTU (or possibly the same CTU). Because these CUs are spatially / temporally close, it is likely that one of the motion vector predictors in the HMVP table may be reused to encode / decode a different CU in the same row of a CTU. Therefore, including an HMVP table in the process of constructing the motion vector candidate list can result in higher coding efficiency.

[0068] In some implementations, the HMVP table has a fixed length (e.g., 5) and is managed in a quasi-first-in, first-out (FIFO) manner. For example, a motion vector is reconstructed with respect to a CU when decoding one intercoded block of the CU. The HMVP table is updated on the fly with the reconstructed motion vector, as such a motion vector may be a motion vector predictor for a subsequent CU. When updating the HMVP table, there are two scenarios: (i) the reconstructed motion vector is different from other existing motion vectors in the HMVP table, or (ii) the reconstructed motion vector is the same as one of the existing motion vectors in the HMVP table. In the first scenario, if the HMVP table is not full, the reconstructed motion vector is added to the HMVP table as the latest. If the HMVP table is already full, the oldest motion vectors in the HMVP table must first be removed from the HMVP table before the reconstructed motion vector is added as the latest. In other words, the HMVP table in this case is similar to a FIFO buffer, so that motion information located at the beginning of the FIFO buffer and associated with another pre-intercoded block is shifted out of the buffer, and as a result the reconstructed motion vector is appended to the end of the FIFO buffer as the latest member in the HMVP table. In the second scenario, before the reconstructed motion vector is added to the HMVP table as the latest, existing motion vectors in the HMVP table that are substantially identical to the reconstructed motion vector are removed from the HMVP table. If the HMVP table is also maintained in the form of a FIFO buffer, the motion vector predictor after the identical motion vector in the HMVP table is shifted forward by one element to occupy the space left by the removed motion vector, and the reconstructed motion vector is then appended to the end of the FIFO buffer as the latest member in the HMVP table.

[0069] Motion vectors in the HMVP table may be added to the motion vector candidate list under different prediction modes such as AMVP, merge, and skip. It has been found that motion information of pre-intercoded blocks stored in the HMVP table can be used for more efficient motion vector prediction, even if they are not adjacent to the current block.

[0070] After one MVP candidate is selected from a given set of motion vector candidates in the current CU, the video encoder 20 may generate one or more syntax elements for the corresponding MVP candidate and encode these syntax elements into the video bitstream so that the video decoder 30 can use these syntax elements to search for the MVP candidate from the video bitstream. Depending on the specific mode used to construct the motion vector candidate set, different modes (e.g., AMVP, merge, skip, etc.) have different sets of syntax elements. In AMVP mode, the syntax elements include an inter-prediction indicator (list 0, list 1, or bidirectional prediction), a reference index, a motion vector candidate index, and a motion vector prediction residual signal. In skip and merge modes, the current CU inherits other syntax elements, including the inter-prediction indicator, reference index, and motion vector, from an adjacent CU referenced by the encoded merge index, so only the merge index is encoded into the bitstream. In the case of a skip-coded CU, the motion vector prediction residual signal is also omitted.

[0071] Figure 5A is a block diagram showing spatially adjacent and temporally collated block locations of the current CU to be encoded / decoded in some implementations of the present disclosure. For a given mode, the motion vector prediction (MW) candidate list is constructed by first checking the availability of motion vectors associated with spatially adjacent block locations to the left and above, and the availability of motion vectors associated with temporally collated block locations, and then checking motion vectors in the HMVP table. During the process of constructing the MVP candidate list, some redundant MVP candidates are removed from the candidate list, and zero-value motion vectors are added as needed so that the candidate list has a fixed length (note that different modes may have different fixed lengths). After constructing the MVP candidate list, the video encoder 20 can select the best motion vector predictor from the candidate list and encode the corresponding index indicating the selected candidate into the video bitstream.

[0072] As an example, using Figure 5A and assuming the candidate list has a fixed length of 2, the motion vector predictor (MVP) candidate list in the current CU may be constructed by sequentially performing the following steps under AMVP mode. 1) Selection of MVP candidates from spatially adjacent CUs a) Derive at most one unscaled MVP candidate from one of two left spatially adjacent CUs starting at A0 and ending at A1; b) If no noise-scaled MVP candidates are available from the left in the previous step, derive at most one scaled MVP candidate from one of two left spatially adjacent CUs starting at A0 and ending at A1; c) Derive at most one unscaled MVP candidate from one of the three spatially adjacent CUs described above, starting at B0, then passing through B1, and ending at B2; d) If neither A0 nor A1 is available, or if they are coded in intra-mode, derive at most one scaled MVP candidate from one of the three spatially adjacent CUs listed above, starting at B0, then passing through B1, and ending at B2; 2) If two MVP candidates were found in the previous step and they are identical, remove one of the two candidates from the MVP candidate list; 3) Selection of MVP candidates from CUs (Central Units) that have been time-coordinated. a) If the list of MVP candidates after the previous step does not contain two MVP candidates, derive at most one MVP candidate from the temporally collated CU (e.g., T0). 4) Selection of MVP candidates from the HMVP table a) If the MVP candidate list after the previous step does not contain two MVP candidates, derive up to two history-based MVPs from the HMVP table; and 5) If the MVP candidate list after the previous step does not contain two MVP candidates, add up to two zero-value MVPs to the MVP candidate list.

[0073] Since the AMVP mode MVP candidate list constructed above contains only two candidates, relevant syntax elements, such as binary flags, are encoded into the bitstream to indicate which of the two MVP candidates in the candidate list is used to decode the current CU.

[0074] In some implementations, the MVP candidate list in the current CU under skip mode or merge mode may be constructed by sequentially performing a set of steps similar to those described above. The MVP candidate list in skip mode or merge mode also includes a special merge candidate called a "pairwise merge candidate." The pairwise merge candidate is generated by averaging the MVs of two pre-derived merge mode motion vector candidates. The size of the merge MVP candidate list (e.g., 1-6) is signaled in the slice header of the current CU. For each CU in merge mode, the index of the best merge candidate is encoded using shortened unary binarization (TU). The first bin of the merge index is coded in context, and bypass coding is used for the other bins.

[0075] As mentioned above, history-based MVPs may be added to either the AMVP mode MVP candidate list or the merge MVP candidate list after the spatial MVP and temporal MVP. Pre-intercoded CU motion information is stored in the HMVP table and used as an MVP candidate for the current CU. The HMVP table is maintained during the encoding / decoding process. Whenever there are CUs that have not been subblock intercoded, the associated motion vector information is added as a new candidate to the last entry of the HMVP table, while the motion vector information stored in the first entry of the HMVP table is removed from it (if the HMVP table is already full and there are no identical copies of the associated motion vector information in the table). Alternatively, an identical copy of the associated motion vector information is removed from the table before the associated motion vector information is added to the last entry of the HMVP table.

[0076] As mentioned earlier, intra-block copying (IBC) can significantly improve the coding efficiency of screen content materials. Since IBC mode is implemented as a block-level coding mode, block matching (BM) is performed in the video encoder 20 to find the optimal block vector for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block that has been pre-reconstructed within the current picture. IBC-coded CUs are treated as a third prediction mode other than intra or inter-prediction mode.

[0077] At the CU level, the IBC mode may be signaled as either the IBC AMVP mode or the IBC skip merge mode, as follows: -IBC AMVP mode: The block vector difference (BVD) between the actual block vector of the CU and the block vector predictor of the CU selected from the block vector candidates of the CU is encoded in the same way that the motion vector difference is encoded in the AMVP mode described above. The block vector prediction method uses two block vector candidates as predictors, one from the left neighbor and the other from the upper neighbor (if IBC coded). If either neighbor is unavailable, the default block vector is used as the block vector predictor. A binary flag is signaled to indicate the block vector predictor index. The IBC AMVP candidate list consists of spatial and HMVP candidates. -IBC Skip / Merge Mode: Uses a merge candidate index to indicate which block vector candidate from the merge candidate list (also called the "merge list") from adjacent IBC coding blocks will be used to predict the block vector in the current block. The IBC merge candidate list consists of spatial, HMVP, and pairwise candidates.

[0078] Another technique used by cutting-edge coding standards to improve coding efficiency is to introduce parallel processing into the video encoding / decoding process, for example, by using multi-core processors. Wavefront parallelism (WPP), for instance, is already implemented in HEVC as a feature that uses multiple threads to encode or decode multiple rows of CTUs in parallel.

[0079] Figure 5B is a block diagram illustrating multithreaded encoding of multiple rows of a picture's Coding Tree Unit (CTU) using wavefront parallel processing (WPP) according to some implementations of the present disclosure. When WPP is enabled, it is possible to process multiple rows of a CTU in parallel using a wavefront method, in which case there may be a delay of two CTUs between the start of two adjacent wavefronts. For example, to encode picture 500 using WPP, a video coder such as a video encoder 20 and a video decoder 30 may divide the coding tree unit (CTU) of picture 500 into multiple wavefronts, each wavefront corresponding to each row of the CTU in the picture. The video coder may, for example, use a first coder core or thread to start coding the upper wavefront. After the video coder has coded two or more CTUs of the upper wavefront, the video coder may, for example, use a second parallel coder core or thread to start coding a second upper wavefront in parallel with the coding of the upper wavefront. After the video coder has coded two or more CTUs of the second upper wavefront, the video coder may, for example, use a third parallel coder core or thread to begin coding the third upper wavefront in parallel with coding the higher wavefronts. This pattern may continue down the wavefront in picture 500. In this disclosure, a set of CTUs that the video coder is coding simultaneously using WPP is referred to as a “CTU group”. Thus, when the video coder codes a picture using WPP, each CTU in the CTU group may belong to an intrinsic wavefront of the picture, and the CTU may be offset from each CTU above the wavefront by at least two rows of CTUs in the picture.

[0080] The video coder may initialize the context for the current wavefront to perform context-adaptive binary arithmetic coding (CABAC) of the current wavefront, based on the data of the first two blocks of the wavefront described above, as well as one or more elements of the slice header in the slice containing the first code block of the current wavefront. The video coder may use the context state to perform CABAC initialization of the subsequent wavefront (or CTU row) after coding two CTUs in the CTU row above the subsequent CTU row. In other words, before starting to code the current wavefront, the video coder (or more specifically, the video coder's thread) may code at least two blocks of the wavefront above the current wavefront, assuming that the current wavefront is not the top row of the picture's CTUs. The video coder may then initialize the CABAC context for the current wavefront after coding at least two blocks of the wavefront above the current wavefront. In this example, each CTU row in picture 500 is a separate partition and has associated threads (WPP thread 1, WPP thread 2, ...) that allow the number of CTU rows in picture 500 to be encoded in parallel.

[0081] The current implementation of the HMVP table uses a global motion vector (MV) buffer to store pre-reconstructed motion vectors, and therefore this HMVP table cannot be implemented in the WPP-enabled parallel encoding scheme described in relation to Figure 5B. In particular, the fact that the global MV buffer is shared by all threads in the video coder's encoding / decoding process prevents subsequent WPP threads from starting after the first WPP thread (i.e., WPP thread 1), because these WPP threads must wait for the HMVP table update from the last CTU (i.e., the rightmost CTU) of the first WPP thread (i.e., the first CTU row) to be completed.

[0082] To overcome this problem, it is proposed to replace the global MV buffer shared by WPP threads with a buffer dedicated to multiple CTU rows, such that when WPP is enabled in the video coder, each wavefront of a CTU row has its own buffer for storing the HMVP table corresponding to the CTU row being processed by the corresponding WPP thread. Note that each CTU row having its own HMVP table is equivalent to resetting the HMVP table before coding the first CU of the CTU row. HMVP table reset is to clear all motion vectors in the HMVP table resulting from the coding of another CTU row. In one embodiment, the reset operation is to set the size of available motion vector predictors in the HMVP table to 0. In yet another implementation, the reset operation may be to set the base index of all entries in the HMVP table to an invalid value such as -1. In this way, the construction of the MVP candidate list in the current CTU within a particular wavefront depends on the HMVP table associated with the WPP thread processing that particular wavefront, regardless of whether it is in one of the three modes: AMVP, merge, or skip. Aside from the aforementioned 2-CTU delay, there is no interdependence between different wavefronts, and the construction of candidate motion vector lists associated with different wavefronts may proceed in parallel, as shown in the WPP process in Figure 5B. In other words, at the start of processing a particular wavefront, the HMVP table is reset to be empty without affecting the coding of other wavefronts of the CTU by another WPP thread. In some cases, the HMVP table may be reset to be empty before the coding of each individual CTU. In this case, the motion vectors in the HMVP table are restricted to a particular CTU, and it is likely that the motion vectors in the HMVP table will be selected as the motion vectors for the current CU within that particular CTU.

[0083] In some examples, intra-prediction modes with wide-angle intra-directions are further described herein. A set of pre-decoded samples adjacent to one current CU (i.e., above or to the left) is used to predict a sample in a CU. However, to capture finer edge directions present in natural video (especially in high-resolution video content, e.g., 4K video content), the number of angular intra-modes is expanded from 33 in HEVC to 93 in VVC. In addition to angular directions, planar modes (assuming a gently changing surface with horizontal and vertical slopes derived from the boundary) and DC modes (assuming a plane) are also applied. Figure 5C is a block diagram showing intra-modes as defined in a VVC standard in some implementations of this disclosure. Figure 5D is a block diagram showing a set of reconstructed samples adjacent to the above and left of the current block as a criterion for intra-prediction in some implementations of this disclosure. All intra-modes (i.e., planar, DC, and angular directions) utilize a set of reconstructed samples adjacent to the above and left of the prediction block as a criterion for intra-prediction. However, in some embodiments, which differ from the method where only the nearest row / column of the reconstructed sample (i.e., line 0 in Figure 5D) is used as the reference, a multi-reference line (MRL) is introduced in which two additional rows / columns (e.g., lines 1 and 3 in Figure 5D) are used for intra-prediction. The index of the selected reference row / column is signaled from the encoder to the decoder. In some examples, when the least nearest row / column is selected, the planar mode and DC mode are excluded from the set of intra-modes that can be used to predict the current block.

[0084] In some cases, in Decoder-Side Mode Derivation (DIMD) mode, the intra-prediction mode is no longer retrieved by the encoder, but rather derived using pre-encoded adjacent pixels via gradient analysis. DIMD is signaled for intra-coded blocks using a simple flag. In the decoder, if the DIMD flag is true, the intra-prediction mode is derived in the reconstruction process using the same pre-encoded adjacent pixels. Otherwise, the intra-prediction mode is parsed from the bitstream as that of the classical intra-coding mode.

[0085] To derive the intra-prediction mode in a block, a set of adjacent pixels must first be selected, and gradient analysis is performed on them. For normalization purposes, these pixels should be in a pool of decoded / reconstructed pixels, and Figure 5E is a block diagram showing the selected set of pixels on which gradient analysis is performed, according to some implementations of this disclosure. As shown in Figure 5E, the template is selected by surrounding the current block by T pixels to the left and T pixels above (at the top edge). Next, gradient analysis is performed on the pixels of the template. This allows for the determination of the principal angular direction of the template that is assumed to be likely identical to the current block. This assumption is central to this method. Thus, a simple 3x3 Sobel gradient filter is used, which is defined by the following matrix that is convolved with the template.

number

[0086] For each pixel in the template, each of these two matrices is multiplied point by point by a 3x3 window centered on the current pixel and consisting of eight directly adjacent pixels, and the results are summed. Thus, two values ​​Gx (the value obtained by multiplying with Mx) and Gy (the value obtained by multiplying with My) are obtained in both the horizontal and vertical directions, corresponding to the gradient at the current pixel.

[0087] Figure 5F is a block diagram showing the convolution process of a 3x3 Sobel gradient filter using a template (as shown in Figure 5E) relating to some implementations of this disclosure. Black pixels are the current pixels. White (and black) pixels are pixels for which gradient analysis is possible. Pixels with diagonal gradients are pixels for which gradient analysis is not possible because there are no adjacent pixels. Pixels with dashed lines are available (reconstructed) pixels outside the template being considered, and these are used for gradient analysis of the red pixels. If a pixel with a dashed line is unavailable (for example, because the block is too close to the picture boundary), gradient analysis of the white pixels using this pixel is not performed. For each white pixel, the gradient intensity (G) and direction (O) are calculated using Gx and Gy as they are.

number

[0088] Subsequently, the gradient direction is converted to an intra-angle prediction mode used to index the histogram. When initialized to 0, the histogram value in the intra-angle mode is increased by G. Once all red pixels in the template have been processed, the histogram contains the cumulative gradient intensity for each intra-angle mode. The mode showing the highest peak in the histogram is selected as the intra-prediction mode for the current block. If the maximum value in the histogram is 0 (meaning that gradient analysis could not be performed or the region constituting the template is flat), the DC mode is selected as the intra-prediction mode for the current block.

[0089] Figures 6A and 6B are block diagrams showing a typical pre-reconstructed luma block 602 and a typical chroma block 620 to be reconstructed, respectively, according to some implementations of the present disclosure. In this example, luma samples from the pre-reconstructed luma block 602 (e.g., including luma sample 604), luma samples from the adjacent luma group 606 (e.g., including luma sample 608), and luma samples from the adjacent luma group 610 (e.g., including luma sample 615) are predicted during the video coding process. Chroma samples from chroma block 620 (e.g., including chroma sample 622) are predicted, while chroma samples from the adjacent chroma group 624 (e.g., including chroma sample 626) and chroma samples from the adjacent chroma group 628 (e.g., including chroma sample 630) are pre-reconstructed during the video coding process. In some embodiments, when the luma block 602 and the chroma block 620 are of different sizes and shapes, the chroma samples of the chroma block 620 may be predicted by applying a multi-model linear model (MMLM) to a pre-reconstructed corresponding downsampled luma sample of the luma block 602 (e.g., downsampled luma sample 605), along with the chroma samples of the upper adjacent chroma group 624 (e.g., including chroma sample 626) and the chroma samples of the left adjacent chroma group 628 (e.g., including chroma sample 630). The derivation and application of the MMLM are given below in relation to Figures 7A-7D.

[0090] In some embodiments, the pre-reconstructed luma block 602 and chroma block 620 each represent different components of a portion of the video frame. For example, in the YCbCr color space, an image is represented by a luma component (Y), a blue-differential chroma component (Cb), and a red-differential chroma component (Cr). The pre-reconstructed luma block 602 represents the luma component (i.e., luminance) of a portion of the video frame, and the chroma block 620 represents the chroma component (i.e., color) of the same portion of the video frame. A luma sample of the pre-reconstructed luma block 602 (e.g., luma sample 604) has a luma value representing the luminance at a particular pixel of the video frame, and a chroma sample (e.g., chroma sample 622) has a chroma value representing the color at a particular pixel of the video frame.

[0091] In some embodiments, the pre-reconstructed Luma block 602 is a 2Mx2N block with 2M Luma samples across the block width and 2N Luma samples across the block height. For example, "M" and "N" may be the same value (e.g., the pre-reconstructed Luma block 602 is a square block) or different values ​​(e.g., the pre-reconstructed Luma block 602 is a non-square block).

[0092] Chroma subsampling is a common compression technique because the human visual system is less sensitive to color differences than to luminance differences. As a result, pre-reconstructed luma block 602 and chroma block 620 may represent the same portion of a video frame but are encoded at different resolutions. For example, a video frame may be encoded using a chroma subsampling scheme (e.g., 4:2:0 or 4:2:2) such that chroma information is encoded at a lower resolution than luma information. As shown in Figures 6A and 6B, the pre-reconstructed luma block 602 is encoded at a resolution of 2M2N, while chroma block 620 is encoded at a lower resolution of MxN. In practice, chroma block 620 can have other resolutions such as 2Mx2N (e.g., 4:4:4 full sampling), 2MxN (e.g., 4:4:0 subsampling), Mx2N (e.g., 4:2:2 subsampling), and 1 / 2Mx2N (e.g., 4:1:1 subsampling).

[0093] The pre-reconstructed luma block 602 is adjacent to the upper neighbor luma group 606 and the left neighbor luma group 610. The sizes of the upper neighbor luma group 606 and the left neighbor luma group 610 may be explicitly signaled or depend on the size of the pre-reconstructed luma block 602. For example, the upper neighbor luma group 606 may have a width of 2M samples (e.g., the same as the width of the pre-reconstructed luma block 602) or a width of 4M samples (e.g., twice the width of the pre-reconstructed luma block 602) and a height of 2 samples. The left neighbor luma group 610 may have a height of 2N or 4N samples and a width of 2 samples. In some embodiments, the upper neighbor luma group 606 and the left neighbor luma group 610 are each part of one or more other luma blocks of the same reconstructed video frame.

[0094] Chromablock 620 is adjacent to the upper neighbor chromagroup 624 and the left neighbor chromagroup 628. The sizes of the upper neighbor chromagroup 624 and the left neighbor chromagroup 628 may be explicitly signaled or depend on the size of chromablock 620. For example, the upper neighbor chromagroup 624 may have a size of 1xM or 1x2M, and the left neighbor chromagroup 628 may have a size of Nx1 or 2Nx1.

[0095] In some embodiments, the chroma value (e.g., the chroma value of a chroma sample in chroma block 620) can be predicted based on the luma value of the reconstructed corresponding luma sample (e.g., the luma value of a luma sample in a pre-reconstructed luma block 602). For example, under the assumption that a linear or quasi-linear relationship exists between the luma value of a coding unit and the corresponding chroma value, the video coder can predict the chroma value based on the corresponding luma value reconstructed using MMLM. By doing so, the video coder can save considerable time and bandwidth for encoding the chroma value, transmitting the encoded chroma value, and decoding the encoded chroma value. To predict an unknown chroma value of a chroma sample from a known luma value of a luma sample using MMLM, the videocoder (1) derives a group (e.g., two or more) of linear relationships between the known chroma value of a chroma sample and the known luma value of the corresponding luma sample within the coding block (each linear relationship is applicable to luma and / or chroma values ​​within a specific range), and (2) predicts the unknown chroma value of a chroma sample by applying the appropriate linear relationships to the known luma value of the corresponding luma sample, which has been reconstructed in advance. For details on how the videocoder predicts an unknown chroma value from a known luma value of a corresponding luma sample, which has been reconstructed in advance, see Figures 7A–7D, Figure 8, and related explanations.

[0096] In some embodiments, since the luma blocks and chroma blocks have different resolutions (for example, the chroma blocks may be subsampled), the video coder first performs downsampling on the luma samples to generate downsampled luma samples (e.g., downsampled luma samples 605, 609, and 613) that uniquely correspond to each chroma sample. In some embodiments, when predicting unknown chroma values ​​of chroma samples using MMLM, the video coder applies a linear relationship to the known luma values ​​of the downsampled luma samples (e.g., each of which uniquely corresponds to each chroma sample) instead of the known luma values ​​of the actual luma samples. In some embodiments, six adjacent reconstructed luma samples in both the height and width directions of the video frame are used to generate downsampled chroma samples using weighted averaging schemes known in the art, including, for example, 6-tap downsampling. For example, six reconstructed luma samples (represented by small boxes in the figure) within region 611 of the adjacent luma group above are used to generate a chroma sample 609 that has been downsampled by a weighted average of their corresponding luma values, and six reconstructed luma samples (represented by small boxes in the figure) within region 607 of the pre-reconstructed luma block 602 are used to generate a downsampled chroma sample 605.

[0097] For example, the application of MMLM with two linear relationships can be expressed as follows:

number

[0098] In another example, the application of MMLM with three linear relationships can be expressed as follows:

number

[0099] Equation 2 differs from Equation 1 in that the MMLM in Equation 2 includes three distinct linear relationships, each having two thresholds that define three separate ranges. For the derivation of the MMLM including the three linear relationships (e.g., the derivation of the parameters (α1, α2, α3, β1, β2, β3, Threshold1, Threshold2)), please refer to Figures 7A to 7D, Figure 8, and related explanations.

[0100] Figures 7A to 7D are plots illustrating examples of a process in which a videocoder performs a technique to derive a multi-model linear model (MMLM) and apply the MMLM to predict unknown chroma values ​​of chroma samples in a coding unit, according to some embodiments of the present disclosure. For convenience, this process is described as being performed by a videocoder. In each plot, the horizontal axis represents the chroma value of a chroma sample, the vertical axis represents the chroma value of a chroma sample, and each data point in the plot represents a pair of a chroma sample and its corresponding chroma sample. In some embodiments, the corresponding chroma sample is a downsampled chroma sample. For example, a data point on the plot may represent a pair of a pre-reconstructed chroma sample in the upper neighboring chroma group 624 (Figure 6B) (e.g., chroma sample 626 in Figure 6B) and a downsampled corresponding chroma sample in the upper neighboring chroma group 606 (Figure 6A) (e.g., downsampled chroma sample 613 in Figure 6A).

[0101] Figure 7A shows plot 702a with a first group 704 of data points, where each data point (also known as, for example, a reference sample pair) lies on the plot. 702a represents pairs of pre-reconstructed chroma samples (e.g., chroma sample 626 in Figure 6B) and pre-reconstructed corresponding luma samples (e.g., downsampled luma sample 613 in Figure 6A). The pre-reconstructed chroma samples and their corresponding luma samples are known as “reference chroma samples” and “reference luma samples,” respectively. In some embodiments, the video coder selects a reference luma sample in the current luma coding block from an adjacent group of reconstructed luma samples (e.g., the upper adjacent luma group 606, the left adjacent luma group 610, or both) and also selects a reference chroma sample from an adjacent group of reconstructed chroma samples (e.g., the upper adjacent chroma group 624, the left adjacent chroma group 628, or both). The reference lunar samples and their corresponding reference chromatic samples are used to derive linear model parameters in MMLM, such as the (α1, α2, β1, β2, Threshold) parameter in Equation 1 or the (α1, α2, α3, β1, β2, β3, Threshold1, Threshold2) parameter in Equation 2.

[0102] In some embodiments (also known as “MMLM_A mode”), the video coder selects a luma reference sample from the upper neighbor luma group (e.g., a luma sample downsampled from the upper neighbor luma group 606 in Figure 6A) and a chroma reference sample from the corresponding upper neighbor chroma group (e.g., the upper neighbor chroma group 624 in Figure 6B). Downsampled luma samples from the left neighbor group (e.g., the left neighbor luma group 610 in Figure 6A) and their corresponding chroma samples from the left neighbor chroma group (e.g., the left neighbor chroma group 628 in Figure 6B) are ignored. For example, in Figures 6A and 6B, the video coder may select M downsampled luma samples (e.g., the number of downsampled luma samples per row in the reconstructed luma block 602) and M chroma reference samples (e.g., the number of chroma reference samples per row in the chroma block 620) as the reference luma sample and reference chroma sample, or 2M downsampled luma samples (e.g., twice the number of downsampled luma samples per row in the reconstructed luma block 602) and 2M chroma samples (e.g., twice the number of chroma samples per row in the chroma block 620) as the reference luma sample and reference chroma sample. Generally, the more reference luma samples and reference chroma samples used, the more accurately the prediction of chroma values ​​based on luma values ​​can be made (e.g., more accurately the determination of MMLM parameters), but at a higher computational cost.

[0103] In some embodiments (also known as “MMLM_L mode”), the video coder selects a luma reference sample from the left-next luma group (left-next luma group 610 in Figure 6A) and a chroma reference sample from the corresponding left-next chroma group (left-next chroma group 628 in Figure 6B). Downsampled luma samples from the upper-next luma group (e.g., upper-next luma group 606 in Figure 6A) and their corresponding chroma samples from the upper-next chroma group (e.g., upper-next chroma group 628 in Figure 6B) are ignored. For example, in Figures 6A and 6B, the video coder may select N downsampled luma samples (e.g., the number of downsampled luma samples per column of the reconstructed luma block 602) and N chroma reference samples (e.g., the number of chroma reference samples per column of the chroma block 620) as the reference luma sample and reference chroma sample, or 2N downsampled luma samples (e.g., twice the number of downsampled luma samples per column of the reconstructed luma block 602) and 2M chroma samples (e.g., twice the number of chroma samples per column of the chroma block 620) as the reference luma sample and reference chroma sample.

[0104] In some embodiments, the video coder uses both MMLM_L mode and MMLM_A mode to select luma and chroma reference samples.

[0105] Figure 7B shows plot 702b with a second group 706 of data points. The videocoder uses the second group 706 of data points to derive the MMLM parameters. In some embodiments, the second group 706 of data points is a subset of the first group 704 of data points. Reducing the number of luma and chroma reference samples reduces the computational complexity when deriving the MMLM parameters. The videocoder determines the second group 706 of data points from the first group 704 of data points in the following manner:

[0106] In some embodiments, the number of data points in a second group 706 of data points (also known, for example, as reference sample pairs) is limited to a predetermined value based on the size and / or shape of the chroma block (e.g., chroma block 620 in Figure 6B) from which the chroma sample is to be predicted. Table 1 below shows four different examples (labeled as Methods 1, 2, 3, and 4), where n may be 2, 4, and / or 8, depending on the size and shape of the chroma block of the current coding unit. [Table 1]

[0107] For example, if the chroma block has a size of 4x8 or 8x4 and the video coder selects method 1, the number of data points in the second group 706 of data points is limited to 4. In another example, if the chroma block has a size of 32x32 and the video coder selects method 2, the number of data points in the second group 706 of data points is limited to 8.

[0108] In some embodiments, MMLM is applicable only to blocks having a block size greater than or equal to a predetermined threshold. For example, chroma blocks smaller than the threshold are not predicted using MMLM. In one example, the maximum number of reference sample pairs used to derive MMLM is limited to 8, and the block size threshold is limited to 8x8 or 16x16. As a result, smaller chroma blocks that may not have sufficiently associated reference sample pairs are not predicted using MMLM.

[0109] In some embodiments, the video coder selects a reference sample pair within a second group 706 of data points by fixed downsampling. For example, the video coder may use a fixed downsampling method in which a luma or chroma reference sample is selected at a specific indexed position (e.g., an odd-indexed position) in the vertical (e.g., MMLM_L mode) or horizontal (e.g., MMLM_A mode) direction for MMLM parameter derivation.

[0110] In some embodiments, the video coder selects reference sample pairs within a second group 706 of data points by adaptive downsampling. For example, the video coder may select an adaptive downsampling method in which reference samples are selected according to (1) a predetermined sampling interval and (2) a vertical or horizontal start offset. More specifically, the video coder may determine the sampling interval and start offset in the following manner based on the number of original reference sample pairs (e.g., in a first group 704 of data points) and the number of reduced reference sample pairs (e.g., in a second group 706 of data points). 1. Determine the number of original reference sample pairs: L (e.g., 16) 2. Determine the number of reduced reference sample pairs: N (e.g., 8) 3. Determine the sampling interval: Δ = L / N (for example, 2) 4. Determine the starting offset: Offset = Δ / 2 (for example, 1)

[0111] The video coder selects a first reference sample (e.g., a luma or chroma sample) at a predetermined start position (e.g., a second reference sample) + start offset (e.g., an adjacent luma or chroma group). The positions of other reference samples are the position from the previous point + sampling interval.

[0112] Figure 7C shows plot 702c with a second group 708 of data points separated into two subgroups by a luma threshold 710 based on luma values, the first subgroup from the minimum reference luma value 712 to the luma threshold 710, and the second subgroup from the luma threshold to the maximum luma value 712. Within each subgroup, the video coder then derives a corresponding linear model of MMLM that maps luma values ​​to chroma values.

[0113] In some embodiments, the videocoder calculates a luma threshold 710 by selecting all luma samples (or downsampled luma samples) within the left-next luma group (e.g., left-next luma group 610 in Figure 6A) and ignoring all other luma samples. The videocoder then performs actions on the selected luma samples, such as determining the mean luma value, median luma value, and modal luma value, or by a custom-defined expression.

[0114] In some embodiments, the videocoder calculates a luma threshold 710 by selecting all luma samples (or downsampled luma samples) within an upper neighbor luma group (e.g., upper neighbor luma group 606 in Figure 6A) and ignoring all other luma samples. The videocoder then performs actions on the selected luma samples, such as determining the mean luma value, median luma value, and mode luma value, or by a custom-defined expression.

[0115] In some embodiments, the videocoder calculates a luma threshold 710 by selecting all luma samples (or downsampled luma samples) from both the upper and left adjacent luma groups (e.g., the left adjacent luma group 610 and the upper adjacent luma group 606 in Figure 6A) and ignoring all other luma samples. The videocoder then performs actions on the selected luma samples, such as determining the mean luma value, median luma value, and mode luma value, or by a custom-defined expression.

[0116] In some embodiments, the video coder calculates a luma threshold 710 by selecting all luma samples (or downsampled luma samples) within the current coding unit (e.g., luma block 602 in Figure 6A) and ignoring all other luma samples. The video coder then performs actions on the selected luma samples, such as determining the mean luma value, median luma value, and modal luma value, or by a custom-defined expression.

[0117] In some embodiments, the video coder calculates a luma threshold 710 by selecting all luma samples (or downsampled luma samples) within the current coding unit (e.g., luma block 602 in Figure 6A), as well as the adjacent groups above and to the left (e.g., the left adjacent luma group 610 and the adjacent luma group 606 in Figure 6A). The video coder then performs actions on the selected luma samples, such as determining the mean luma value, median luma value, and modal luma value, or by a custom-defined expression.

[0118] In some embodiments, the custom definition formula is the minimum and maximum luma value (L min and L max ) Finding,

number

[0119] Similarly, a video coder can determine the chroma threshold 711 by applying the above technique to chroma adjacency groups.

[0120] Plot 702c includes only one luma threshold 710 that divides the luma value into two separate groups (e.g., there are two linear relationships to be derived for MMLM), but in practice, there may be multiple luma thresholds that divide the luma value into three or more separate groups (e.g., there are three or more linear relationships to be derived for MMLM). For example, if there are three linear relationships for MMLM, the video coder may determine two luma thresholds based on the maximum criterion luma value 712 and the minimum criterion luma value 714 in the following way:

number

[0121] In another example, all adjacent (upper or left) reconstructed luma samples (or downsampled luma samples) are separated into two groups based on the mean values ​​of the adjacent reconstructed luma samples. Luma samples with values ​​less than the mean belong to one group, and luma samples with values ​​not less than the mean belong to the other group. Threshold1 and Threshold2 may then be calculated as the mean values ​​for each group.

[0122] Figure 7D shows plot 702d in which two linear relationships of MMLM (linear relationships 716 and 718) are derived based on the luma threshold 710, the minimum criterion luma value 712, and the maximum criterion luma value 714. As previously mentioned with reference to Figure 7C, the video coder first separates the reference samples of the second group 708 of data points into two subgroups based on the luma threshold 710. Within each subgroup, the video coder determines the respective linear relationships (see Equation 1) that map the luma values ​​to the chroma values. In some embodiments, the video coder uses regression methods to determine each linear relationship (e.g., taking into account all data points in the group). However, performing regressions is computationally intensive and often impractical for purposes such as real-time video coding / decoding. Therefore, a more efficient implementation for deriving linear relationships (e.g., determining the linear parameters of Equation 1) is desired.

[0123] In some embodiments, the video coder derives the linear relationships 716 and 718 using the Max-Min method. The video coder uses (1) a reference sample having a minimum reference luma value 712 (e.g., A(X A ,Y A ), where X A is the minimum reference luma value 712), (2) a data point having a luma threshold 710 and a chroma threshold 711 (e.g., the threshold (X T ,Y T ), where X T and Y T are the luma threshold 710 and the chroma threshold 711, respectively), and (3) a reference sample having a maximum reference luma value 714 (e.g., B(X B ,Y B ) to determine the linear model parameters (α1 β1 α2 β2) of Equation 1. Note that X A and X B are the minimum and maximum luma values, but Y A and Y B are not necessarily the minimum and maximum chroma values. The video coder determines the linear model parameters in the following manner. [Equation]

[0124] In some embodiments, for a coding block having a square shape, the video coder directly applies the above technique. For a non-square coding block, in some embodiments, the video coder first subsamples the adjacent reference samples of the longer boundary so that they have the same number of samples as the shorter boundary.

[0125] If the MMLM includes three linear relationships (e.g., represented by Equation 3), the videocoder can derive the linear model parameters in the linear relationships in a similar manner to that described above for an MMLM with two linear relationships. For example, if two threshold data points are given by Threshold(X T1 , Y T1 ) and Threshold(X T2 , Y T2 It can be expressed as Y T2 >Y T1 Assuming this is the case, the video coder will use A(X A , Y A ) and Threshold(X T1 , Y T1 The linear model parameters α1 and β1 can be determined from the linear relationship between ) and Threshold(X). The linear model parameters α2 and β2 are given by the linear relationship Threshold(X). T1 , Y T1 ) and Threshold(X T2 , Y T2 The linear model parameters α3 and β3 are derived from Threshold(X T2 , Y T2 ) and B(X B ,Y B This can be derived from the linear relationship between them.

[0126] After deriving the linear relations 716 and 718 of MMLM, the video coder can predict the chroma sample value (for example, the chroma sample value of chroma sample 622 in Figure 6B) by applying an appropriate linear model to the corresponding luma value (or subsampled luma value).

[0127] Figure 8 is a flowchart of a typical process 800 in which a video coder performs a technique to derive a multi-model linear model (MMLM) and apply the MMLM to predict chroma samples of coding units, as per some implementations of this disclosure. For convenience, process 800 is described as being performed by a video decoder on a destination device, such as the video decoder 30 in Figure 3.

[0128] As a first step, the video decoder receives (e.g., transmitted by the video encoder 20 in Figure 2) a bitstream encoding a chroma block (e.g., chroma block 620 in Figure 6B), a luma block (e.g., luma block 602 in Figure 6A) (e.g., the chroma block and luma block belong to the same coding unit), a plurality of adjacent luma samples surrounding a luma block (e.g., the upper adjacent luma group 606 and / or left adjacent luma group 610 in Figure 6A), and a plurality of adjacent chroma samples surrounding a chroma block (e.g., the upper adjacent chroma group 624 and / or left adjacent chroma group 628) (e.g., one or more luma samples in a plurality of adjacent luma samples correspond to chroma samples in adjacent chroma samples) (810). In some embodiments, the luma blocks and chroma blocks are sampled at different sample rates and have different block sizes and / or shapes. For example, a luma block may be larger than a chroma block, and subsampling of the luma block is performed to find a subsampled luma sample (e.g., a luma sample calculated by averaging adjacent luma samples) that corresponds to a chroma sample.

[0129] The video decoder then decodes the luma block, multiple adjacent luma samples, and multiple adjacent chroma samples to obtain multiple reconstructed luma samples of the luma block, multiple reconstructed adjacent luma samples, and multiple reconstructed adjacent chroma samples, respectively (820). For example, the video decoder may use intermode prediction or intramode prediction to decode the luma block, multiple adjacent luna samples, and multiple adjacent chroma samples. Each reconstructed adjacent luma sample (or reconstructed subsampled adjacent luma sample) and its corresponding reconstructed adjacent chroma sample may be represented as a reference data point showing the correspondence between luma values ​​and chroma values ​​(for example, as one data point in the first group 704 of the data points in Figure 7A).

[0130] Next, the video decoder selects a group of reference luma samples and a group of reference chroma samples from a plurality of reconstructed adjacent luma samples (or reconstructed subsampled adjacent luma samples) and a plurality of reconstructed adjacent chroma samples, corresponding groups (for example, represented by the second group 706 of data points in Figure 7B) (830). In some embodiments, the reference luma samples and the corresponding reference chroma samples are subsets of the plurality of reconstructed adjacent luma samples and the plurality of reconstructed adjacent chroma samples, respectively. For details of the selection mechanism, see Figure 7B and the related description.

[0131] Next, the video decoder calculates a threshold chroma value (e.g., mean chroma value, median chroma value, or chroma value calculated from other predetermined operations) from multiple reconstructed adjacent chroma samples, and also calculates a threshold chroma value (e.g., mean chroma value, median chroma value, or chroma value calculated from chroma samples in other ways) (for example, the data points (threshold chroma value, threshold chroma value) represent the "knee points" in MMLM and are used to separate the first linear model in MMLM from the second linear model, see Figure 7D and related explanations) (840).

[0132] After determining the threshold luma value and threshold chroma value, the video decoder determines the maximum luma value and minimum luma value from the group of reference luma samples (850). For example, in plot 702d of Figure 7D, the maximum luma value is value X B Therefore, the minimum luma value is value X A The reference samples containing the maximum and minimum luma values ​​are B(X B ,Y B ) and A(X A , Y A ) is the minimum luma value (for example, X in Figure 7D). A ), threshold lumen value (for example, X in Figure 7D) T ), and the maximum luma value (for example, X in Figure 7D) BThe `Luma` value defines two separate regions: the first region extends from the minimum Luma value to the threshold Luma value, and the second region extends from the threshold Luma value to the maximum Luma value. The threshold Luma value lies between the minimum Luma value and the maximum Luma value. In some embodiments, if the maximum Luma value is different from the minimum Luma value, the threshold Luma value is greater than or equal to the minimum Luma value and less than or equal to the maximum Luma value. In some embodiments, if the maximum Luma value is equal to the minimum Luma value (for example, the region for calculating the maximum and minimum Luma values ​​includes a uniform Luma sample), then the maximum, minimum, and threshold Luma values ​​are all equal to each other. Therefore, if the threshold Luma value is greater than or equal to the minimum Luma value and less than or equal to the maximum Luma value, the threshold Luma value lies between the minimum Luma value and the maximum Luma value.

[0133] The video decoder then generates a multi-model linear model that includes a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value (860). The first linear model includes a reference sample (e.g., A(X) in Figure 7D) that contains the minimum luma value. A , Y A )) and the threshold value (for example, Threshold(X in Figure 7D) T , Y T The second linear model is defined by a reference sample including )). The second linear model is defined by a threshold value (e.g., Thresholdf(X) in Figure 7D). T , Y T A reference sample including )) and a reference sample including the maximum luma value (for example, B(X) in Figure 7D). B ,Y B )) are defined by and . For example, plot 702d in Figure 7D shows the first and second linear models as linear relation 716 and linear relation 718, respectively.

[0134] Finally, the video decoder reconstructs the chroma blocks from the luma blocks using a multi-model linear model (870). In some embodiments, the video decoder may process each luma sample (or subsampled luma sample) in the luma block in raster scan order and reconstruct the corresponding chroma sample by applying an appropriate linear relationship of MMLM (for example, applying a first linear relationship if the luma value of the luma sample is below the luma threshold, and applying a second linear relationship if the luma value of the luma sample is above the luma threshold).

[0135] In some embodiments, generating a multi-model linear model involves determining a first chroma value of a first reference chroma sample corresponding to a first reference chroma sample having the maximum luma value, and a second chroma value of a second reference chroma sample corresponding to a second reference chroma sample having the minimum luma value, wherein the first linear model links (minimum luma value, first chroma value) to (threshold luma value, threshold chroma value), and the second linear model links (threshold luma value, threshold chroma value) to (maximum luma value, second chroma value).

[0136] In some embodiments, constructing a chroma block from a chroma block using a multi-model linear model includes, for each chroma sample in the chroma block, determining the respective chroma value of each chroma sample in the reconstructed chroma block corresponding to each chroma sample, applying a first linear model to each chroma value to obtain the respective chroma value if it is determined to be less than or equal to a threshold chroma value, and applying a second linear model to each chroma value to obtain the respective chroma value if it is determined to be greater than or equal to a threshold chroma value.

[0137] In some embodiments, calculating a threshold luma value involves finding the mean luma value from multiple reconstructed adjacent luma samples, and calculating a threshold chroma value involves finding the mean chroma value from multiple reconstructed adjacent chroma samples.

[0138] In some embodiments, selecting a group of reference luma samples and a group of reference chromatic samples includes determining an upper limit on the number of reference luma samples and reference chromatic samples to be used.

[0139] In some embodiments, selecting a group of reference luma samples and a group of reference chromato samples involves selecting every other luma sample from a plurality of reconstructed adjacent luma samples and every other chromato sample from a plurality of reconstructed adjacent chromato samples.

[0140] In some embodiments, a second threshold luma value greater than the threshold luma value and a corresponding second threshold chroma value greater than the threshold chroma value are calculated, the second linear model is applicable to luma values ​​between the threshold luma value and the second threshold luma value, and the third linear model is applicable to luma values ​​between the second threshold luma value and the maximum luma value.

[0141] In some embodiments, calculating a threshold luma value involves finding a weighted average luma value between a maximum luma value and a minimum luma value from a plurality of reconstructed adjacent luma samples, and calculating a threshold chroma value involves finding a weighted average chroma value between a maximum chroma value and a minimum chroma value from a plurality of reconstructed adjacent chroma samples.

[0142] In some embodiments, calculating a threshold luma value involves finding the mean luma value from multiple reconstructed luma samples of a luma block.

[0143] In some embodiments, constructing a chroma block from a chroma block using a multi-model linear model includes determining the average chroma value of each block of chroma samples in the decoded chroma block corresponding to each block of chroma samples in the chroma block, and, in response to the determination that the respective average chroma value is less than or equal to a threshold chroma value, applying a first linear model to each chroma value in each block of chroma samples to obtain the respective chroma value in each block of chroma samples, and, in response to the determination that the respective average chroma value is greater than or equal to a threshold chroma value, applying a second linear model to each chroma value in each block of chroma samples to obtain the respective chroma value in each block of chroma samples.

[0144] Figure 9 is a block diagram showing the positions of adjacent samples (shown as gray circles) used in MMLM in some embodiments of this disclosure. In some embodiments, the derivation of multimodel linear model parameters is from multiple lines, as shown in Figure 9. In some embodiments, the model parameters in MMLM are generated using multiple lines of a reference sample.

[0145] In one embodiment, multiple lines of a reference sample are divided into N sets, where N is a positive number, and its value may be dynamically changed based on specific coded information of the current block, such as the quantization parameters or size of the coded blocks associated with the TB (transformation block) / CB (coding block) and / or slice / profile. Figure 10 is a block diagram illustrating the locations of four sets of samples used in MMLM according to some embodiments of the present disclosure. In Figure 10, the four sets of samples are represented by different symbols within gray circles, for example, a gray circle without a symbol, a gray circle with a cross symbol, a gray circle with a triangle symbol, and a gray circle with a check symbol. In another embodiment, multiple lines of a reference sample are divided into N reference sets, where N is a positive number, and the codeword is dynamically changed to indicate a specific one of the N reference sets based on specific coded information of the current block, such as the quantization parameters, the number or size of the coded blocks associated with the TB / CB and / or slice / profile.

[0146] In one example, a control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether signaling of the reference set in an MMLM block is enabled or disabled. If the control flag is enabled, a syntax element is further signaled for each CB to indicate that a specific reference set is used for deriving the linear model parameters within that CB. If the control flag is disabled (e.g., the flag is set to "0"), no further syntax elements are signaled at lower levels to indicate a specific reference set for deriving the linear model parameters, and the default reference set (e.g., the available upper and left reconstructions) is used for deriving the linear model parameters.

[0147] In some embodiments, MMLM and intra-prediction are used in combination to form a prediction. In some embodiments, MMLM and intra-prediction with derived weights are combined to form the final prediction. In one embodiment, weights are derived from the prediction modes of two adjacent blocks, left (top) and top, and are combined to form the final prediction. Only planar modes are used as intra-prediction modes for the weight combination. In another embodiment, the intra-prediction mode of the combination may be the same mode as the collated MMLM intra-prediction. In yet another embodiment, the weights and / or intra-prediction modes of the combination may be signaled at the TB / CB / slice / picture / sequence level to indicate the weights and / or type of intra-prediction mode used in the combination.

[0148] In one example, a control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether the combined MMLM and intra-prediction modes are enabled or disabled. When the control flag is enabled, a syntax element is further signaled for each CB to indicate that a specific intra-prediction is used as the intra-prediction mode for the combination within that CB. In another example, a control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether the combined MMLM and intra-prediction modes are enabled or disabled. When the control flag is enabled, a syntax element is further signaled for each CB to indicate that weights are used for the combined MMLM and intra-prediction within that CB.

[0149] In yet another embodiment, the intra-mode type and weights for combining the final predictions are derived from the prediction modes of two adjacent blocks, left and top (upper end).

[0150] In one example, a control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether the combined MMLM and intra-prediction modes are enabled or disabled. If the control flag is enabled, the intra-prediction mode is derived by gradient analysis using pre-encoded adjacent pixels, for example, as used in DIMD. The mode showing the highest peak in the histogram is selected as the intra-prediction mode for the combination at that CB. The weights may be dynamically changed based on the ratio of the highest peaks in the histogram; for example, the higher the ratio of the highest peaks, the higher the weight of the intra-mode.

[0151] Figure 11 is a flowchart of a typical process 800 in which a video coder implements a technique for predicting or constructing chroma blocks of a video signal by combining MMLM and intra-prediction, as described in some implementations of this disclosure. For convenience, the process 800 is described as being performed by a video decoder on a destination device, such as the video decoder 30 in Figure 3.

[0152] As the first step, the video decoder receives a bitstream (for example, transmitted by the video encoder 20 in Figure 2) which encodes a chroma block, a corresponding luma block, several adjacent luma samples surrounding the luma block, and several adjacent chroma samples surrounding the chroma block (1110).

[0153] The video decoder then decodes the luma block, multiple adjacent luma samples, and multiple adjacent chroma samples to obtain multiple reconstructed luma samples of the luma block, multiple reconstructed adjacent luma samples, and multiple reconstructed adjacent chroma samples, respectively (1120).

[0154] Subsequently, the video decoder selects a group of reference luma samples and a group of reference chroma samples from multiple reconstructed adjacent luma samples and multiple reconstructed adjacent chroma samples, such that each reference luma sample corresponds to its respective reference chroma sample (1130).

[0155] Subsequently, the video decoder calculates a threshold luma value from a group of reference luma samples and a corresponding threshold chroma value from a group of reference chroma samples (1140).

[0156] Subsequently, the video decoder determines the maximum and minimum lumen values ​​from the group of reference lumen samples, where the threshold lumen value is between the minimum and maximum lumen values ​​(1150).

[0157] The video decoder then generates a multi-model linear model that includes a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value (1160).

[0158] The video decoder further reconstructs each sample value of a chroma block from a weighted combination of the corresponding first sample values ​​of the chroma block, reconstructed using a multi-model linear model, and the respective second sample values ​​of the adjacent chroma block, reconstructed from an intra-predictive mode (1170).

[0159] In some embodiments, multiple adjacent luma samples are selected from at least two left lines and at least two upper lines surrounding a luma block, and multiple adjacent chroma samples are selected from at least two left lines and at least two upper lines surrounding a chroma block. For example, the derivation of multi-model linear model parameters is from multiple lines as shown in Figure 9.

[0160] In some embodiments, multiple adjacent luma samples are divided into a predetermined number of reference sets, multiple adjacent chroma samples are divided into a predetermined number of reference sets, a group of reference luma samples is selected from one of the predetermined number of reference sets, and a group of reference chroma samples is selected from one of the predetermined number of reference sets. For example, as shown in Figure 10, multiple lines of a reference sample are divided into N reference sets, where N is a positive number, and a dynamically changing codeword is used to indicate a particular one of the reference sets based on the number or size of the number of coded blocks associated with specific coded information of the current block, such as quantization parameters, TB / CB and / or slice / profile.

[0161] In some embodiments, a group of reference luma samples and a group of reference chromato samples are selected based on coded information of the current block, including the chromato and luma blocks.

[0162] In some embodiments, the step of selecting a group of reference luma samples and a group of reference chroma samples from a plurality of reconstructed adjacent luma samples and a plurality of reconstructed adjacent chroma samples (1130) includes determining from a control flag whether a predetermined number of reference sets are signaled at one of a group consisting of TB (transformation block), CB (coding block), slice, picture, and sequence levels, and determining from syntax that a particular set of the predetermined number of reference sets is selected as a group of reference luma samples and a group of reference chroma samples, in accordance with the determination that the signaling of a predetermined number of reference sets is enabled from the control flag. In some examples, one control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether the signaling of reference sets in an MMLM block is enabled or disabled. If the control flag is signaled as enabled, one syntax element is further signaled for each CB to indicate that a particular set of references is used for the derivation of linear model parameters within that CB.

[0163] In some embodiments, reconstructing each sample value of a chroma block from a weighted combination of corresponding first sample values ​​of the chroma block, reconstructed using a multi-model linear model, and corresponding second sample values ​​of the adjacent chroma block, reconstructed from an intra-predictive mode (1170), includes receiving at least one of a first signal transmission indicating the weights of the reconstructed corresponding first sample values ​​and a second signal transmission indicating the type of intra-predictive mode at the TB / CB / slice / picture / sequence level. For example, the weights and / or intra-predictive mode of a combination may be signaled at the TB / CB / slice / picture / sequence level to indicate the weights and / or type of intra-predictive mode used in the combination. The corresponding weights of the reconstructed second sample values ​​may be derived from the weights of the reconstructed corresponding first sample values.

[0164] In some embodiments, reconstructing each sample value of a chroma block from a weighted combination of the corresponding first sample values ​​of the chroma block, reconstructed using a multi-model linear model, and the corresponding second sample values ​​of the adjacent chroma block, reconstructed from an intra-prediction mode (1170), is performed when a control flag signaled at the TB / CB / slice / picture / sequence level is enabled. In one example, one control flag is signaled at the TB / CB / slice / picture / sequence level to indicate whether the combined MMLM and intra-prediction modes are enabled or disabled.

[0165] In some embodiments, the intra-prediction mode is derived using neighboring samples pre-encoded by gradient analysis.

[0166] In some embodiments, multiple adjacent luma samples are selected from a single left line and a single top line surrounding a luma block, and multiple adjacent chromato samples are selected from a single left line and a single top line surrounding a chromato block. For example, as shown in Figures 6A and 6B, a single left line and a single top line are shown surrounding a luma block or a chromato block.

[0167] In some embodiments, reconstructing each sample value of a chroma block from each corresponding first sample value of a luma block, reconstructed using a multi-model linear model, includes applying a first linear model to each reconstructed corresponding first sample value of a luma block to obtain each sample value of a chroma block in response to a determination that each reconstructed corresponding first sample value of a luma block is less than or equal to a threshold luma value, and applying a second linear model to each reconstructed corresponding first sample value of a luma block to obtain each sample value of a chroma block in response to a determination that each reconstructed corresponding first sample value of a luma block is greater than a threshold luma value.

[0168] Figure 12 shows a computing environment 1210 coupled with a user interface 1250. The computing environment 1210 can be part of a data processing server. The computing environment 1210 includes a processor 1220, memory 1230, and an input / output (I / O) interface 1240.

[0169] The processor 1220 generally controls the overall operation of the computing environment 1210, including operations related to display, data acquisition, data communication, and image processing. The processor 1220 may include one or more processors that execute instructions to perform all or part of the steps in the method described above. Furthermore, the processor 1220 may include one or more modules that facilitate interaction between the processor 1220 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphics processing unit (GPU), etc.

[0170] Memory 1230 is configured to store various types of data to support the operation of the computing environment 1210. Memory 1230 may include certain software 1232, examples of such data include instructions for any application or method operating on the computing environment 1210, video datasets, image data, etc. Memory 1230 may be implemented using any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks, etc.

[0171] The I / O interface 1240 provides an interface between the processor 1220 and peripheral interface modules such as a keyboard, click wheel, and buttons. The buttons may include, but are not limited to, a home button, a scan start button, and a scan stop button. The I / O interface 1240 may be coupled with an encoder and a decoder.

[0172] In one embodiment, a non-temporary computer-readable storage medium is also provided that contains multiple programs in, for example, memory 1230, which can be executed by a processor 1220 in a computing environment 1210, in order to carry out the method described above. Alternatively, the non-temporary computer-readable storage medium may store a bitstream or datastream containing encoded video information (e.g., video information including one or more syntax elements) generated by an encoder (e.g., video encoder 20 in Figure 2) using the encoding method described above for use by a decoder (e.g., video decoder 30 in Figure 3) when decoding video data. The non-temporary computer-readable storage medium may be, for example, a ROM, random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, an optical data storage device, etc.

[0173] In one embodiment, a computing device is also provided, which comprises one or more processors (e.g., processor 1220) and a non-temporary computer-readable storage medium or memory 1230 that internally stores a plurality of programs executable by the one or more processors, and the one or more processors are configured to perform the method described above when executing the plurality of programs.

[0174] In one embodiment, a computer program product is also provided, which includes multiple programs in memory 1230, for example, that can be executed by a processor 1220 in a computing environment 1210, in order to carry out the method described above. For example, the computer program product may include a non-temporary computer-readable storage medium.

[0175] In one embodiment, the computing environment 1210 may be implemented using one or more ASICs, DSPs, digital signal processing units (DSPDs), programmable logic units (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the method described above.

[0176] Further embodiments include various subsets of the above embodiments that are combined with or otherwise reconfigured in various other embodiments.

[0177] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored in a computer-readable medium or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. Thus, the computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is not transient, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for carrying out the embodiments described in this application, and the computer program product may include a computer-readable medium.

[0178] The descriptions in this disclosure are provided for illustrative purposes only and are not intended to be exhaustive or limitful to this disclosure. Many modifications, variations, and alternative embodiments will become apparent to those skilled in the art who are interested in the teachings presented in the foregoing description and the accompanying drawings.

[0179] Unless otherwise specifically stated, the order of the steps in the methods relating to this disclosure is intended to be illustrative only, and the steps in the methods relating to this disclosure are not limited to the order specifically described above and may be modified in accordance with the actual conditions. Furthermore, at least one of the steps in the methods relating to this disclosure may be adjusted, combined, or deleted in accordance with the actual requirements.

[0180] The examples provided illustrate the principles of the Disclosure and have been selected and described to enable those skilled in the art to understand the Disclosure in relation to various embodiments and to make the best use of the fundamental principles and various embodiments with various modifications suitable for specific intended uses. Therefore, it should be understood that the scope of the Disclosure is not limited to specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the Disclosure.

Claims

1. A method for constructing a chroma block of a video signal, Receiving the chroma block, the corresponding luma block, a plurality of adjacent luma samples surrounding the luma block, and a bitstream encoded with the plurality of adjacent chroma samples surrounding the chroma block, In order to obtain multiple reconstructed luma samples, multiple reconstructed adjacent luma samples, and multiple reconstructed adjacent chromatic samples of the luma block, the luma block, the multiple adjacent luma samples, and the multiple adjacent chromatic samples are decoded. From the plurality of reconstructed adjacent luma samples and the plurality of reconstructed adjacent chromato samples, select a group of reference luma samples and a group of reference chromato samples, where each reference luma sample corresponds to its respective reference chromato sample. The process involves calculating a threshold chroma value from the group of reference chroma samples and calculating a corresponding threshold chroma value from the group of reference chroma samples. From the group of reference luma samples, determine the maximum luma value and the minimum luma value such that the threshold luma value lies between the minimum luma value and the maximum luma value. To generate a multi-model linear model that includes a first linear model between the minimum luma value and the threshold luma value, and a second linear model between the threshold luma value and the maximum luma value, Reconstructing each sample value of a chroma block from a weighted combination of the corresponding first sample values ​​derived from the chroma block and reconstructed using the multi-model linear model, and the corresponding second sample values ​​of adjacent chroma blocks reconstructed from the intra-prediction mode, Includes, A method comprising: dividing the plurality of adjacent luma samples into a predetermined number of reference sets; dividing the plurality of adjacent chroma samples into the predetermined number of reference sets; selecting the group of reference luma samples from one of the predetermined number of reference sets; selecting the group of reference chroma samples from one of the predetermined number of reference sets; determining the predetermined number based on coded information of the current block including the chroma block and the luma block; and selecting the group of reference luma samples and the group of reference chroma samples based on coded information of the current block.

2. The method according to claim 1, wherein the plurality of adjacent luma samples are selected from at least two left lines and at least two upper lines surrounding the luma block, and the plurality of adjacent chromato samples are selected from at least two left lines and at least two upper lines surrounding the chromato block.

3. Selecting the group of reference luma samples and the group of reference chromatic samples from the plurality of reconstructed adjacent luma samples and the plurality of reconstructed adjacent chromatic samples is, Determining whether the signal transmission of a predetermined number of reference sets is enabled based on a control flag in one selected from a group consisting of TB (transformation block), CB (coding block), slice, picture, and sequence level, Based on the control flag, it is determined that the signal transmission of the predetermined number of reference sets is enabled, and in accordance with this determination, it is determined from the syntax that a specific set among the predetermined number of reference sets is selected as the group of reference luma samples and the group of reference chroma samples. The method according to claim 1, including the method described in claim 1.

4. Reconstructing each of the sample values ​​of a chroma block from the weighted combination of the corresponding first sample values ​​of the chroma block, reconstructed using the multi-model linear model, and the second sample values ​​of the adjacent chroma block, reconstructed from the intra-prediction mode, Receiving at least one of a first signal transmission indicating the weight of each corresponding reconstructed first sample value at the TB / CB / slice / picture / sequence level, and a second signal transmission indicating the type of the intra-prediction mode. The method according to claim 1, including the method described in claim 1.

5. The method according to claim 1, wherein the reconstruction of each of the sample values ​​of the chroma block from the weighted combination of the corresponding first sample values ​​derived from the chroma block and reconstructed using the multi-model linear model, and the second sample values ​​of the adjacent chroma block reconstructed from the intra-prediction mode, is performed when a control flag signaled at the TB / CB / slice / picture / sequence level is enabled.

6. The method according to claim 5, wherein the intra prediction mode is derived by gradient analysis using pre-encoded adjacent samples.

7. The method according to claim 1, wherein the plurality of adjacent luma samples are selected from a single left line and a single upper line surrounding the luma block, and the plurality of adjacent chromato samples are selected from a single left line and a single upper line surrounding the chromato block.

8. Reconstructing each sample value of a chroma block from a weighted combination of the corresponding first sample values ​​derived from the chroma block and reconstructed using the multi-model linear model, and the respective second sample values ​​of adjacent chroma blocks reconstructed from the intra-prediction mode, In response to the determination that each of the luma values ​​in the luma block is less than or equal to the threshold luma value, the first linear model is applied to each of the luma values ​​in the luma block to obtain the corresponding reconstructed first sample values. In response to the determination that each of the luma values ​​in the luma block is greater than the threshold luma value, the second linear model is applied to each of the luma values ​​in the luma block to obtain the corresponding reconstructed first sample values. The method according to claim 1, including the method described in claim 1.

9. One or more processing units, A memory connected to one or more processing units, A plurality of programs stored in the memory, which, when executed by one or more processing units, cause an electronic device to perform the method described in any one of claims 1 to 8, An electronic device equipped with the following features.

10. A non-temporary computer-readable storage medium for storing a plurality of programs executed by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, cause the electronic device to execute the method according to any one of claims 1 to 8.

11. A computer program having instructions, when executed by a processor, that perform steps of the method according to any one of claims 1 to 8.

12. A method for storing the bitstream decoded by the method according to any one of claims 1 to 8.

13. A method for receiving the bitstream decoded by the method described in any one of claims 1 to 8.

Images