Compound prediction for video coding

Abstract

Composite prediction for video coding is disclosed. A method for generating a composite predictor block for a current block includes generating, for the current block, a predictor block comprising a first predictor block of first predictor pixels. The method further includes determining, for a first one of the first predictor pixels and using at least a subset of the first predictor pixels, a first adjustment value for adjusting a first weight to be applied to the first one of the first predictor pixels. The composite predictor block is generated using the first predictor pixels, the first weight, and the first adjustment value.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese Invention Patent Application No. 201780082759.0, filed on October 31, 2017.

[0003] Cross-references to related applications

[0004] This application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 445,999, filed January 13, 2017, the entire disclosure of which is incorporated herein by reference. Technical Field

[0005] This application generally relates to composite prediction of video coding. Background Technology

[0006] A digital video stream can represent video using a series of frames or still images. Digital video can be used in a variety of applications, including, for example, video conferencing, high-definition video entertainment, video advertising, or sharing user-generated video. Digital video streams can include large amounts of data and consume significant computing or communication resources of computing devices for processing, transmitting, or storing the video data. Various methods have been proposed to reduce the amount of data in video streams, including compression and other coding techniques.

[0007] Motion estimation and compensation-based coding can be performed by segmenting a frame or image into blocks that are predicted based on one or more predictor blocks of a reference frame. The differences between the blocks and the predictor blocks (i.e., residual errors) are compressed and encoded in the bitstream. The decoder uses the differences and the reference frame to reconstruct the frame or image. Summary of the Invention

[0008] This disclosure generally relates to video coding and, in particular, to composite prediction patterns for video coding.

[0009] One aspect of the disclosed implementation is a method for generating a composite predictor block for a current block of video. The method includes generating a predictor block for the current block that includes a first predictor block of first predictor pixels. The method further includes determining, for the first predictor pixels and using at least a subset of the first predictor pixels, a first adjustment value for adjusting a first weight to be applied to the first predictor pixels, and generating the composite predictor block using the first predictor pixels, the first weight, and the first adjustment value.

[0010] Therefore, when decoding blocks encoded in this way, the distortion between the original block and the reconstructed block can be reduced.

[0011] Optionally, the first weight is selected from a weighting scheme including weights of 0.75, 0.6, 0.5 and 0.25, and the method further includes encoding the first weight in the encoded bitstream.

[0012] Optionally, the predictor block further includes a second predictor block of second predictor pixels. Generating a composite predictor block using the first predictor pixels and the first adjustment value includes, for the second predictor pixels in the second predictor pixels, using a second adjustment value to adjust the complement of the first weight, wherein the second adjustment value is determined using at least a subset of the second predictor pixels.

[0013] Optionally, the first adjustment value is determined based on the difference between the first predictor pixel and the second predictor pixel.

[0014] Optionally, the first adjustment value is determined using the smoothness in the window surrounding the first predicted pixel.

[0015] Optionally, the window is centered on the first predicted pixel and the window size is 3×3.

[0016] Optionally, the first adjustment value is determined using a decay function that has a maximum value at a predetermined pixel value in the first predictor block, and the first predicted pixel is input to the decay function to determine the first adjustment value.

[0017] Optionally, the method may further include decoding the first weight from the encoded bitstream.

[0018] On the other hand, according to one implementation of the present disclosure, there is an apparatus for generating a composite predictor block, including a memory and a processor. The processor is configured to execute instructions stored in the memory to generate a first predictor block and a second predictor block, the first predictor block including first predictor pixels and the second predictor block including second predictor pixels; determine a corresponding first adjustment value for a corresponding first predictor pixel of the first predictor block; determine a corresponding second adjustment value for a corresponding second pixel of the second predictor block; and use the first predictor pixel, the first adjustment value, the second predictor pixel, and the second adjustment value to determine pixel values ​​of pixels in the composite predictor block. Each first adjustment value is determined using at least some of the first predictor pixels. Each second adjustment value is determined using at least some of the second predictor pixels.

[0019] Optionally, the instructions further include instructions for decoding baseline weights from the encoded bitstream, wherein the baseline weights indicate a mask including a first baseline weight and the complement of the first baseline weight.

[0020] Optionally, each first adjustment value is used to adjust the first baseline weight and each second adjustment value is used to adjust the complement of the first baseline weight.

[0021] Optionally, the baseline weights indicate weights from a weighting scheme including 0.75, 0.6, 0.5, and 0.25.

[0022] Optionally, the instructions further include instructions for decoding the complementary mask indicator, and based on the complementary mask indicator, adjusting the complement of the first baseline weight using each first adjustment value and adjusting the first baseline weight using each second adjustment value.

[0023] Optionally, the first adjustment value of the first predictor pixel and the second adjustment value of the second predictor pixel are based on the characteristics of the first predictor pixel and the second predictor pixel, wherein the first predictor pixel and the second predictor pixel are co-located.

[0024] Optionally, the feature includes the difference between the first predictor pixel and the second predictor pixel.

[0025] Optionally, the instructions further include instructions for selecting a first baseline weight and a complement of the first baseline weight, wherein each first adjustment value is used to adjust the first baseline weight and each second adjustment value is used to adjust the complement of the first baseline weight; and selecting an adjustment direction having an upward or downward value, wherein when the adjustment direction is an upward value, the first baseline weight is adjusted upward as the characteristic increases, and wherein when the adjustment direction is a downward value, the first baseline weight is adjusted downward as the characteristic increases.

[0026] Optionally, the features include a first smoothness with respect to the first predictor pixel and a second smoothness with respect to the second predictor pixel.

[0027] Optionally, determining the corresponding first adjustment value of the corresponding first predictor pixel of the first predictor block includes identifying the peak value of the first predictor block and determining the first adjustment value of the first predictor pixel based on the difference between the peak value and the first predictor pixel.

[0028] On the other hand, according to an implementation of this disclosure, there is an apparatus for decoding a current block, including a memory and a processor. The processor is configured to execute instructions stored in the memory to generate a first predictor block and a second predictor block, decode a weight mask from an encoded bitstream, determine from the weight mask a baseline weight for a first weight used as a pixel in the first predictor block and a complementary baseline weight for a second weight used as a pixel in the second predictor block, determine a first adjustment value for a first predictor pixel in the first predictor block using at least some of the pixels in the first predictor block, determine a second adjustment value for a second predictor pixel in the second predictor block using at least some of the pixels in the second predictor block, and generate a composite predictor including a third predictor pixel using the first predictor pixel, the second predictor pixel, the baseline weight, the complementary baseline weight, the first adjustment value, and the second adjustment value. The first adjustment value is used to adjust the baseline weight. The second adjustment value is used to adjust the complementary baseline weight. The second predictor pixel is co-located with the first pixel predictor.

[0029] Optionally, the baseline weights have a first value and the complementary baseline weights have a second value, and the instruction further includes an indication to decode using a complementary mask, and in response to the indication to use a complementary mask being the first value, the second value is used as the value of the baseline weights and the first value is used as the value of the complementary baseline weights. The first value and the second value are added together to reach 1.

[0030] It should be noted that any of the above features can be used with any particular aspect or embodiment of the invention.

[0031] These and other aspects of this disclosure are disclosed in the following detailed description of embodiments, appended claims and drawings. Attached Figure Description

[0032] The description herein refers to the accompanying drawings, wherein the same reference numerals in all the drawings refer to the same parts.

[0033] Figure 1 This is a schematic diagram of a video encoding and decoding system.

[0034] Figure 2 This is a block diagram of an example computing device that can implement a sending station or a receiving station.

[0035] Figure 3 This is a diagram of the video stream that is to be encoded and subsequently decoded.

[0036] Figure 4 This is a block diagram of an encoder implemented according to this disclosure.

[0037] Figure 5 This is a block diagram of a decoder implemented according to this disclosure.

[0038] Figure 6 This is a flowchart of the process for predicting the composite motion of the current pixel block according to the present disclosure.

[0039] Figure 7 This is an example of adjusting baseline weights using pixel differences according to an implementation of this disclosure.

[0040] Figure 8 This is a flowchart illustrating the process of using a first predictor block and a second predictor block to predict a composite prediction of the current pixel block according to an implementation of this disclosure.

[0041] Figure 9 This is a flowchart of the process for generating a composite predictor block for the current block of a video according to a second implementation of this disclosure.

[0042] Figure 10 This is a flowchart of the process of generating a composite predictor block according to the third implementation of this disclosure.

[0043] Figure 11 This is a flowchart of the process of decoding the current block according to the second implementation of the present invention. Detailed Implementation

[0044] As described above, compression schemes related to coded video streams can include segmenting images (i.e., the original or source image) into blocks and using one or more techniques to restrict the information included in the output to generate a digital video output bitstream. The received encoded bitstream can be decoded to reconstruct the blocks and the source image from the limited information. Encoding a video stream or a portion thereof (such as a frame or block) can include using temporal or spatial similarities within the video stream to improve coding efficiency. For example, the current block of the video stream can be encoded based on identifying the difference (residual) between previously coded pixel values ​​and pixel values ​​in the current block. In this way, only the residual and the parameters used to generate the residual need to be added to the encoded bitstream. The residual can be encoded using a lossy quantization step. Decoding (i.e., reconstructing) the encoded block from such residuals often results in distortion between the original block and the reconstructed block.

[0045] Encoding using spatial similarity can be called intra-frame prediction. Using intra-frame prediction, the algorithm attempts to predict the pixel values ​​of the current block using pixels surrounding the current block in the current frame of the video stream. Pixels surrounding the current block are pixels within the current frame but outside the current block. These pixels can be pixels adjacent to the current block. Which pixels surrounding the block are used can depend on the intra-frame prediction mode and / or the scan order of the frames' blocks. For example, in a raster scan order, pixels above and / or to the left of the current block (i.e., the block being encoded or decoded) can be used.

[0046] Encoding using temporal similarity can be called inter-frame prediction. Inter-frame prediction uses motion vectors representing the temporal displacement of previously coded blocks relative to the current block. Motion vectors can be identified using motion estimation methods such as motion search. In motion search, a portion of a reference frame can be translated to a series of succession positions to form a predictor block, which can be subtracted from a portion of the current frame to form a series of residuals. Horizontal and / or vertical translations corresponding to positions with, for example, minimum residuals can be selected as motion vectors. The motion vectors, along with indications of the reference frame, can be encoded in the encoded bitstream.

[0047] In some cases, and to further minimize the residuals, more than one predictor block can be combined to predict the current block. This is called composite prediction. Composite prediction can sometimes significantly reduce the residual error signal to be coded.

[0048] Composite predictors can be created by combining two or more predictors determined using inter-frame prediction modes and intra-frame prediction modes. For example, a composite predictor can be generated by combining intra-frame generated predictors with inter-frame generated predictors (i.e., intra + inter), by combining two intra-frame generated predictor blocks (i.e., intra + intra), or by combining two inter-frame generated predictor blocks (i.e., inter + inter). For example, composite inter-frame prediction can employ a first motion vector to obtain a first predictor block from a first block of a first frame and a second motion vector to obtain a second predictor block from a second block of a second frame. The first and second frames are referred to as reference frames. Reference frames can both be past, both be future, or some combination thereof. The second motion vector can be independent of the first motion vector or obtained from the first motion vector. The encoder conveys the first motion vector, the second motion vector, the first reference, and the second reference frame (e.g., encoded in the encoded bitstream) to the decoder. A composite predictor can use two or more predictor blocks.

[0049] When forming a composite predictor (e.g., in the case of two predictor blocks), the video codec (i.e., the encoder and / or decoder) combines pixels from a co-located first predictor block and pixels from a second predictor block. For example, to obtain the pixel value of a pixel in the composite predictor block located at pixel position (row = r, column = c), the video codec combines the first pixel value of the first pixel at position (r, c) in the first predictor block with the second pixel value of the second pixel at position (r, c) in the second predictor block.

[0050] Combining allows the same weight mask to be used for all pixels in the composite predictor block. That is, using equation (1), with the mask {w1, (1-w1)}, weight w1 is applied to each pixel value of the first predictor block, and weight (1-w1) is applied to each pixel value of the second predictor block:

[0051] current[r][c]=p1[r][c]*w1+p2[r][c]*(1-w1) (1)

[0052] In equation (1), current[r][c] is the pixel value of the pixel in the composite predictor block at position (r, c), p1[r][c] is the co-located pixel value in the first predictor block, and p2[r][c] is the co-located pixel value in the second predictor block. In the example, the pixels from the first and second predictor blocks are weighted equally using a mask {1 / 2, 1 / 2}. In the mask {w1, (1-w1)}, weight w1 may be referred to herein as the baseline weight, and weight (1-w1) may be referred to herein as the complementary baseline weight. When using more than two predictor blocks, the mask can contain more weights. For example, the mask may explicitly or implicitly include weights for each of the predictors. The weights of the mask can be added together to reach 1.

[0053] In another example, the weights can vary based on the segmentation of the first predictor block and the second predictor block. For example, the segmentation could be based on detecting edges in the first and second predictor blocks. In another example, the segmentation could be based on dividing the first and second predictor blocks into four quadrants or two equal parts. The weights can be assigned based on which segment within the first predictor block the first pixel value falls into, which segment within the second predictor block the second pixel value falls into, or a combination thereof. Thus, the weighting is spatial (i.e., based on pixel position). For example, in the case where the predictor block is segmented into left and right sides, it is possible that the left side of the first predictor block is a better predictor for the left side of the current block than the left side of the second predictor block. Therefore, pixel positions within the left side of the first predictor block can be weighted more. In any case, the mask indication (i.e., the weights) can be sent from the encoder to the decoder in the encoded bitstream. In the example, the mask indication can be sent implicitly. For example, the encoder and decoder can be configured a priori to use a specific mask (e.g., mask {0.75, 0.25}). Therefore, the encoder does not need to encode the mask in the encoded bitstream. Encoding a mask can mean encoding the mask value or the quantized value of the mask. Encoding a mask can mean encoding the mask value or the quantized value of the mask. Encoding a mask can also mean an indicator of the mask. For example, as described below, an indicator can be a mask index.

[0054] In an implementation of the present invention, compression performance can be improved by obtaining weights from the pixel values ​​of the predictor block itself and / or by adjusting the weights of the pixels to be applied to the predictor block (i.e., when generating a composite predictor).

[0055] For example, as further explained below, comparisons (e.g., differences) of pixels in the predictor block can be used to generate modifier values ​​and / or weights for pixels to be applied to the first predictor and the second predictor.

[0056] For example, as further explained below, if the baseline weight w1 will be used with the first predictor block, the adjustment value can be determined using at least a subset of the pixels of the first predictor block. When combining predictor blocks to generate a composite predictor block, the adjustment value can be applied to the baseline weight w1. Different adjustment values ​​can be generated for each pixel in the first predictor block (and the second predictor block) using pixels from the first predictor block (and pixels from the second predictor block). Using pixels from the first predictor block means using the values ​​of the pixels from the first predictor block.

[0057] The codec (i.e., encoder and / or decoder) can determine the weights to be applied to the composite prediction based at least in part on the pixel values ​​of the predictor block. Because the decoder can accurately recreate the predictor block (used by the encoder) using the information conveyed by the encoder in the encoded bitstream, the decoder can generate the same weights (i.e., the acquired weights) and the same final composite predictor block without additional information regarding the weights of the pixels to be applied to the predictor block. Baseline weights (e.g., default, initial, average weights) can be conveyed in the encoded bitstream, and the decoder can adjust (i.e., modify) the baseline weights using the acquired weights (i.e., modulated values).

[0058] As described above, different weights can be used for each pixel, and the weights can depend on the prediction signal (e.g., the pixel values ​​of the predictor block) itself. The "prediction signal" includes information indicating at least one pixel value. "Pixel values" include values ​​associated with the pixel, such as color or luminance. The different weights can be weights of some adjusted (i.e., modified) base weights. Baseline weights can be weights applied to each pixel in the predictor block. In contrast, adjusted weights are weights calculated per pixel. The disclosure herein also applies to the chroma and luminance components and / or red-green-blue (RGB) components of a pixel. That is, for example, adjusted weights can be calculated for the chroma component; and different or the same adjusted weights can be determined for the chroma component.

[0059] In the case of inter-frame composite predictor blocks, given a first motion vector and a second motion vector, a first predictor block and a second predictor block can be generated from a first reference frame and a second reference frame, respectively. The weights applied to pixels in the first predictor block and the weights applied to pixels in the second predictor block may depend on the properties of the first and second predictor blocks themselves (e.g., pixel values). Although this document uses two predictor blocks (i.e., a first predictor block and a second predictor block), the teachings of this disclosure are not limited thereto. Any number of predictor blocks can be used for correlated composite prediction modes in video codification.

[0060] As further described below, the encoder can convey additional information in the encoded bitstream (described further below), and the decoder can receive and use this additional information to further guide the decoder's weight generation process. The weight generation process is the process of determining which corresponding weights to apply to the pixels of the predictor block when combining pixels via composite prediction. The additional information may include the adjustment direction of the baseline weights, whether a complementary mask is used, and one or more of the peak pixel values ​​of at least some of the predictor blocks.

[0061] The adjustment direction can indicate how to adjust the baseline weights for predictor pixels when calculating composite pixels using predictor pixels. For example, the adjustment direction can indicate whether the baseline weights should be adjusted upwards (i.e., increased) or downwards (i.e., decreased). For instance, the adjustment direction can indicate whether the baseline weights are adjusted upwards or downwards for predictor pixels in a predictor block, as a characteristic of the predictor pixel increases or decreases. As further described below, this characteristic can be related to pixel difference, pixel smoothness, or peak pixel value. The adjustment direction can indicate whether the baseline weights are increased or decreased as the pixel value difference or relative smoothness difference between the pixels co-located in the first and second predictor blocks increases or decreases.

[0062] The complementary mask indicator can indicate that the weight mask {(1-w1),w1} instead of the standard weight mask {w1,(1-w1)} will be applied to the first predictor block and the second predictor block accordingly.

[0063] The peak pixel value indicates how the baseline weights are adjusted based on the decay function. The decay function can have a maximum value at the peak pixel value and decays as the difference between the pixel values ​​of the predictor block and the peak increases. After first describing the environment in which this disclosure can be implemented, further details are described herein.

[0064] This paper first describes relevant composite prediction patterns for video coding, referring to systems that can be incorporated into this teaching.

[0065] Figure 1This is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 may, for example, have features such as... Figure 2 The internal configuration of the hardware described herein is that of a computer. However, other suitable implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed across multiple devices.

[0066] Network 104 can connect sending station 102 and receiving station 106 for encoding and decoding video streams. Specifically, the video stream can be encoded in sending station 102 and the encoded video stream can be decoded in receiving station 106. Network 104 can be, for example, the Internet. Network 104 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile phone network, or any other means used in this example to transmit the video stream from sending station 102 to receiving station 106.

[0067] In one example, receiving station 106 may have features such as Figure 2 The internal configuration of the hardware described herein is that of a computer. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed across multiple devices.

[0068] Other implementations of the video decoding and encoding system 100 are possible. For example, network 104 may be omitted from the implementation. In another implementation, the video stream may be encoded and stored for later transmission to receiving station 106 or any other device with memory. In one implementation, receiving station 106 receives the encoded video stream (e.g., via network 104, a computer bus, and / or some communication path) and stores the video stream for subsequent decoding. In one example implementation, Real-Time Transport Protocol (RTP) is used to transmit the encoded video over network 104. In another implementation, a transport protocol other than RTP may be used, such as a video streaming protocol based on Hypertext Transfer Protocol (HTTP).

[0069] For example, when used in a video conferencing system, sending station 102 and / or receiving station 106 may include the ability to encode and decode video streams as described below. For example, receiving station 106 may be a video conferencing participant who receives an encoded video bitstream from a video conferencing server (e.g., sending station 102) for decoding and viewing, and further encodes and transmits its own video bitstream to the video conferencing server for decoding and viewing by other participants.

[0070] Figure 2 This is a block diagram illustrating an example of a computing device 200 that can function as a transmitting or receiving station. For instance, the computing device 200 can implement... Figure 1One or both of the transmitting station 102 and the receiving station 106. The computing device 200 can be in the form of a computing system including multiple computing devices or a single computing device, such as a mobile phone, tablet computer, laptop computer, desktop computer, etc.

[0071] The CPU 202 in computing device 200 can be a central processing unit. Alternatively, the CPU 202 can be any other type of device or multiple devices capable of operating or processing existing or future-developed information. Although the disclosed implementation can be implemented with a single processor such as the CPU 202 shown, advantages in speed and efficiency can be achieved by using more than one processor.

[0072] In one implementation, the memory 204 in computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 204. Memory 204 may include code and data 206 accessed by CPU 202 using bus 212. Memory 204 may further include an operating system 208 and an application 210, which includes at least one program that allows CPU 202 to perform the methods described herein. For example, application 210 may include applications 1 to N, which further include video coded applications that perform the methods described herein. Computing device 200 may also include secondary storage 214, which may be, for example, a memory card used with a mobile computing device 200. Because video communication sessions may involve a large amount of information, it may be stored, in whole or in part, in secondary storage 214 and loaded into memory 204 as needed for processing.

[0073] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element capable of operating to sense touch input. The display 218 may be coupled to the CPU 202 via a bus 212. In addition to or as a replacement for the display 218, other output devices may be provided that allow the user to program or otherwise use the computing device 200. When the output device is a display or includes a display, the display may be implemented in a variety of ways, including by a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light-emitting diode (LED) display, such as an organic LED (OLED) display.

[0074] The computing device 200 may also include or communicate with an image sensing device 220, such as a camera or any other image sensing device 220 that can sense images (such as images of a user operating the computing device 200) whether it is present or developed in the future. The image sensing device 220 may be positioned such that it is directed toward the user operating the computing device 200. In one example, the position and optical axis of the image sensing device 220 may be configured such that the field of view includes the area directly adjacent to and from which the display 218 can be seen.

[0075] The computing device 200 may also include or communicate with a sound sensing device 222, such as a microphone or any other existing or future sound sensing device that can sense sound in the vicinity of the computing device 200. The sound sensing device 222 may be positioned such that it is directed toward a user operating the computing device 200 and may be configured to receive sounds generated by the user, such as speech or other words, when the user operates the computing device 200.

[0076] although Figure 2 The CPU 202 and memory 204 of computing device 200 are depicted as being integrated into a single unit, but other configurations can be utilized. The operation of CPU 202 can be distributed across multiple machines (each machine having one or more processors) that can be directly coupled or coupled across a local area network or other networks. Memory 204 can be distributed across multiple machines, such as internet-based memory or memory across multiple machines performing the operations of computing device 200. Although depicted herein as a single bus, bus 212 of computing device 200 can consist of multiple buses. Furthermore, secondary storage 214 can be directly coupled to other components of computing device 200 or can be accessed via the internet and can include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Therefore, computing device 200 can be implemented in a variety of configurations.

[0077] Figure 3This is a schematic diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a large number of adjacent frames 304. Although three frames are described as adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individual frames, such as frame 306. At the next level, frame 306 can be divided into a series of segments 308 or planes. For example, segments 308 can be subsets of frames that allow for parallel processing. Segments 308 can also be subsets of frames that can separate video data into individual colors. For example, frame 306 of color video data can include a luma plane and two chroma planes. Segments 308 can be sampled at different resolutions.

[0078] Regardless of whether frame 306 is segmented into segments 308, frame 306 can be further subdivided into blocks 310, which may include data corresponding to, for example, 16x16 pixels in frame 306. Blocks 310 may also be arranged to include data from one or more segments 308 of pixel data. Blocks 310 may also have any other suitable size, such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger.

[0079] Figure 4 This is a block diagram of an encoder 400 implemented according to the present disclosure. As described above, the encoder 400 can be implemented in the transmitting station 102, for example, by providing a computer software program stored in a memory such as memory 204. The computer software program can include machine instructions that, when executed by a processor such as CPU 202, cause the transmitting station 102 to encode video data in the manner described herein. The encoder 400 can also be implemented as specific hardware included, for example, in the transmitting station 102. The encoder 400 has the following levels that perform various functions (shown by solid connecting lines) in a forward path using video stream 300 as input to produce an encoded or compressed bitstream 420: intra / inter-frame prediction level 402, transform level 404, quantization level 406, and entropy coding level 408. The encoder 400 can also include (shown by dashed connecting lines) a reconstruction path to reconstruct frames for encoding subsequent blocks. Figure 4 In the encoder 400, the following stages perform various functions in the reconstruction path: dequantization stage 410, inverse transform stage 412, reconstruction stage 414, and loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0080] When presenting the video stream 300 for encoding, adjacent frames 306 can be processed in blocks. At the intra / inter-frame prediction level 402, blocks can be encoded using intra-frame prediction (also called intra-prediction), inter-frame prediction (also called inter-prediction), or a combination of both. In any case, a predictor block can be formed. In the case of intra-frame prediction, all or part of the predictor block can be formed from samples in the current frame that have already been encoded and reconstructed. In the case of inter-frame prediction, all or part of the predictor block can be formed from samples in one or more previously constructed reference frames determined using motion vectors.

[0081] The following reference will still be made. Figure 4 At intra / inter prediction stage 402, a prediction block can be subtracted from the current block to produce a residual block (also called a residual). Transform stage 404 uses a block-based transform to transform the residual into transform coefficients in, for example, the frequency domain. Such block-based transforms include, for example, the Discrete Cosine Transform (DCT) and the Asymmetric Discrete Sine Transform (ADST). Other block-based transforms are possible. Furthermore, combinations of different transforms can be applied to a single residual. In one example of transform application, the DCT transforms the residual block to the frequency domain, where the transform coefficient values ​​are based on spatial frequencies. The lowest frequency (DC) coefficients are located at the top left corner of the matrix, and the highest frequency coefficients are located at the bottom right corner. It is worth noting that the size of the predictor block and the resulting residual block can be different from the size of the transform block. For example, the predictor block can be split into smaller blocks to which individual transforms are applied.

[0082] Quantization stage 406 uses quantizer values ​​or quantization levels to convert transform coefficients into discrete values, referred to as quantized transform coefficients. For example, transform coefficients can be divided by the quantizer value and truncated. The quantized transform coefficients are then entropy encoded by entropy coding stage 408. Entropy coding can be performed using any number of techniques, including tokens and binary trees. The entropy-coded coefficients, along with other information used for decoding blocks, including, for example, the prediction type used, transform type, motion vector, and quantizer values, are output to compressed bitstream 420. The information used for decoding blocks can be entropy-coded into blocks, frames, stripes, and / or headers within compressed bitstream 420. Compressed bitstream 420 may also be referred to as an encoded video stream or an encoded video bitstream, and these terms are used interchangeably herein.

[0083] Figure 4The reconstruction path (shown by the dashed connecting line) is used to ensure that both encoder 400 and decoder 500 (described below) decode the compressed bitstream 420 using the same reference frames and blocks. The reconstruction path performs functions similar to those occurring during the decoding process discussed in more detail below, including dequantizing the quantized transform coefficients at dequantization stage 410 and inverse transforming the dequantized transform coefficients at inverse transform stage 412 to produce a derivative residual block (also called the derivative residual). At reconstruction stage 414, the predictor block predicted at the intra / inter-frame prediction stage 402 can be added to the derivative residual to create a reconstruction block. A loop filtering stage 416 can be applied to the reconstruction block to reduce distortion such as block artifacts.

[0084] Other variations of encoder 400 can be used to encode the compressed bitstream 420. For example, a non-transform-based encoder 400 can directly quantize the residual signal for certain blocks or frames without a transform stage 404. In another implementation, encoder 400 may have a quantization stage 406 and a dequantization stage 410 combined into a single stage.

[0085] Figure 5 This is a block diagram of a decoder 500 implemented according to the present disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program may include machine instructions that, when executed by a processor such as CPU 202, cause the receiving station 106 to... Figure 6 The decoder 500 decodes video data in the manner described above. The decoder 500 can also be implemented in hardware included, for example, in a transmitting station 102 or a receiving station 106. Similar to the reconstruction path of the encoder 400 described above, in one example, the decoder 500 includes the following stages that perform various functions from the compressed bitstream 420 to produce an output video stream 516: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter-frame prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0086] When presenting the compressed bitstream 420 for decoding, the data elements in the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and the inverse transform stage 506 performs an inverse transform on the dequantized transform coefficients using a selected transform type to produce a derivative residual that is identical to the derivative residual created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 can create a predictor block using the intra / inter-frame prediction stage 508 that is identical to the predictor block created in the encoder 400, for example, at the inter / intra-frame prediction stage 402. In the reconstruction stage 510, the predictor block is added to the derivative residual to create a reconstruction block. The loop filtering stage 512 can be applied to the reconstruction block to reduce block artifacts. Other filters can be applied to the reconstruction block. In the example, deblocking filter stage 514 is applied to the reconstructed block to reduce block distortion, and the result is output as output video stream 516. Output video stream 516 can also be referred to as the decoded video stream, and these terms will be used interchangeably here.

[0087] Other variations of decoder 500 can be used to decode compressed bitstream 420. For example, decoder 500 can produce output video stream 516 without deblocking filter stage 514. In some implementations of decoder 500, deblocking filter stage 514 is applied before loop filter stage 416. Alternatively, encoder 400 may include deblocking filter stage in addition to loop filter stage 416.

[0088] Figure 6 This is a flowchart of a process 600 for composite motion prediction of a current pixel block, implemented according to the present invention. Process 600 can be implemented in an encoder such as encoder 400 or a decoder such as decoder 500.

[0089] Process 600 can be implemented, for example, as a software program executable by a computing device such as transmitting station 102. The software program may include machine-readable instructions that can be stored in a memory such as memory 204 or secondary memory 214, and these machine-readable instructions can be executed by a processor such as CPU 202 to cause the computing device to perform process 600. In at least some implementations, process 600 may be executed wholly or partially by the intra-frame / inter-frame prediction stage 402 of encoder 400.

[0090] Process 600 can be executed by a decoder such as decoder 500. Process 600 can be implemented as a software program, for example, executed by a computing device such as receiver 106. Process 600 can be executed wholly or partially by the intra / inter-frame prediction stage 508 of decoder 500. For example, the implementation of process 600 can be executed by storing instructions in memory such as memory 204 of receiver 106 for execution by a processor such as CPU 202.

[0091] Process 600 can be implemented using dedicated hardware or firmware. Some computing devices may have multiple memories, multiple processors, or both. Different processors, memories, or both can be used to distribute the steps or operations of process 600. The terms "processor" or "memory" used in the singular include computing devices having one processor or one memory that can be used to perform some or all of the steps, as well as devices having multiple processors or multiple memories.

[0092] Process 600 determines the pixel value of a pixel location in a composite predictor block based on a weighted average. The weighting may be based on a first pixel value from a first predictor block and a second pixel value from a second predictor block. Process 600 can determine a corresponding pixel value for each location in the composite predictor block. In the example, process 600 determines at least a portion of the composite predictor block as a weighted average of pixel values ​​from two or more predictor blocks. The weighting of at least one predictor pixel value may be based on the pixel value of a pixel in at least one predictor block.

[0093] At 602, process 600 determines a first predictor block and a second predictor block to predict the current pixel block. As used herein, “determine” means to select, construct, identify, specify, receive, or determine in any way. For example, process 600 may receive a first motion vector, a second motion vector, and reference frame information as described above with respect to intra / inter-frame prediction level 402 of encoder 400 or intra / inter-frame prediction level 508 of decoder 500. Process 600 determines the first predictor block and the second predictor block accordingly. Alternatively, process 600 determines the first predictor block and the second predictor block by receiving the first predictor block and the second predictor block determined by the previous steps. The first predictor block and the second predictor block can be any combination of inter-frame + inter-frame predictor blocks, inter-frame + intra-frame predictor blocks, or intra-frame + intra-frame predictor blocks. Any number (e.g., more than two) of predictor blocks can be determined at 602. The teachings herein apply to any number of predictor blocks.

[0094] At 604, process 600 determines the pixel values ​​of the composite predictor pixel block based on a weighted average of the pixel values ​​of the first predictor block and the pixel values ​​of the second predictor block. As indicated above, the weighting of each pixel depends on the prediction signal itself. That is, the weighting depends on the pixel values ​​of the pixels in the first predictor block and the second predictor block. A non-limiting example of a weighted average based on the first pixel values ​​and the second pixel values ​​of the first predictor block is now provided.

[0095] To minimize decoder complexity, the weight generation process is expected to be simple. The weights of predictor pixels (i.e., pixels in the first or second predictor block) can be adjusted based on their characteristics. Examples of characteristics are provided below. Each predictor pixel (i.e., the predictor pixel to be combined to form the composite predictor pixel) can have a corresponding feature value for the characteristic. For example, in the case of two predictor blocks, a first feature value of the characteristic of the first predictor pixel in the first predictor block can be used to adjust the baseline weights, and a second feature value of the characteristic of the co-located second predictor pixel in the second predictor block can be used to adjust the baseline weights. The adjusted baseline weights are then applied to the first and second predictor pixels in the second predictor block to generate the composite predictor pixel. As indicated above, the first and second predictor pixels are co-located.

[0096] In the first example, the weight generation process proceeds as follows: Given a first predictor block and a second predictor block, and for each pixel location in the composite predictor block, process 600 determines the absolute difference (e.g., characteristic) between the pixels co-located in the first and second predictor blocks. This absolute difference can be used to set the weights. In the implementation, the baseline weights can be adjusted (e.g., adjusted, modified, scaled) based on the difference. Therefore, the weighting is further based on the baseline weights.

[0097] Figure 7 Example 700 is an implementation of this disclosure that uses pixel differences to adjust baseline weights. In this example, the difference between a first pixel of a first predictor block and a second pixel of a second predictor block is a characteristic used to adjust the weighting. Example 700 includes a first predictor block 710, a second predictor block 720, and a composite predictor block 730. The pixels of the composite predictor block 730 are generated by combining the co-located pixels of the predictor blocks (e.g., the first predictor block 710 and the second predictor block 720). The pixel values ​​of the composite predictor block 730 are obtained based on a weighted average of the pixel values ​​of the first predictor block 710 and the second predictor block 720. The adjusted weights for the pixel located at pixel position (r, c) in the composite predictor block 730 are calculated according to equation (2).

[0098]

[0099] In Equation 2, modulated[r][c] is the weight (i.e., the adjusted or modulated baseline weight) of the pixel value to be applied to the composite predictor block at position (r,c), p1[r][c] is the pixel value at position (r,c) in the first predictor block, p2[r][c] is the pixel value at position (r,c) in the second predictor block, weight is the baseline weight, and maxvalue is the highest possible pixel value. Although this paper uses "pixel value," the teachings of this paper also apply to the color components of a pixel. Color components may include luma and chroma components. Color components may include the RGB color components of a pixel.

[0100] Using the adjustment function of equation (2), as the difference in pixel values ​​increases, the first predictor block is weighted more and its baseline weight approaches 1. Furthermore, as the difference in pixel values ​​increases, the second predictor block is weighted less (i.e., 1-weight) and its baseline weight approaches 0. This is because positive values ​​are added to the baseline weight used with the first predictor block. When the corresponding pixels of the first and second predictor blocks have the same value, the baseline weight (e.g., 0.75) is applied to the pixels of the first predictor block.

[0101] The adjustment function in Example 700 is just one example of an adjustment function. Other functions can be used. For example, another adjustment function could adjust the baseline weights up or down based on the difference rather than the absolute value of the difference.

[0102] The encoder can encode and the decoder can decode the adjustment direction from the encoded bitstream. The adjustment direction can indicate an upward or downward value. For example, an upward value can be indicated by a bit value of 0 and a downward value by a bit value of 1. Other indications are possible. When the adjustment direction is upward, the first baseline weight is adjusted upward as the difference between the corresponding pixel values ​​in the predictor block and the second predictor block increases. As the difference increases, this adjustment function will weight the pixels of the first predictor block more heavily. When the adjustment direction is downward, the first baseline weight is adjusted downward as the difference increases. On the other hand, this adjustment benefits the pixels of the second predictor block—that is, since the sum of the first baseline weight and the second baseline weight is 1, by decreasing the first baseline weight of the first predictor block, the second baseline weight of the second predictor block increases accordingly.

[0103] When indicating baseline weights in the encoded bitstream, it can be assumed that the corresponding values ​​of the weighting scheme will be used with the first predictor block. For example, the baseline weights can be selected from the weighting scheme. Examples of weighting schemes include baseline weights {0.75, 0.6, 0.5, 0.25}. The weighting scheme is the set of baseline weights that the encoder, decoder, or encoder / decoder pair can use. If bit 01 is indicated, the baseline weight of the weighting scheme, 0.6, will be used with the first predictor block and the complementary baseline weight, 0.4 (i.e., (1-weight)), will be used with the second predictor block. However, in some instances, the encoder can determine that the complementary baseline weight will be used with the first predictor block. Therefore, the encoder encodes in the encoded bitstream and the decoder decodes from the encoded bitstream a complementary mask indicator (i.e., a syntax element) indicating that the first baseline weight will be used with the second predictor block and the second baseline weight (i.e., the complementary baseline weight) will be used with the first predictor block. For example, if baseline scheme 01 (i.e., decimal value 1) is indicated and complementary mask signal is indicated, weight 0.4 is used with the first predictor block and weight 0.6 is used with the second predictor block.

[0104] In Example 700, assume the baseline weight is 0.75 and the maximum possible pixel value (maxvalue) is 255. Using pixel 712 (i.e., pixel value 112) of the first predictor block and pixel 722 (i.e., pixel value 111) of the second predictor block, the adjusted weight for pixel 732 at pixel position (0,0) of the composite predictor block is calculated as follows:

[0105]

[0106] And using pixel 714 (i.e., pixel value 248) of the first predictor block and pixel 724 (i.e., pixel value 57) of the second predictor block, the adjusted weights for pixel 732 at pixel position (1,2) of the composite predictor block are calculated as follows:

[0107]

[0108] Using adjusted weights, the pixel values ​​of the composite prediction block can be calculated. Equation (1) can be used to calculate the pixel values. For example, the pixel values ​​of pixels 732 and 734 can be calculated as follows:

[0109] current[0][0]=112*0.75098+722*(1-0.75098)=112

[0110] current[1][2]=248*0.937255+57*(1-0.937255)=236

[0111] Note that since pixel values ​​are integer values, the calculated pixel values ​​can be rounded or truncated—in the example above, the calculated value is rounded to the nearest integer.

[0112] Baseline weights can be provided as configuration to both the encoder and decoder. Alternatively, the baseline weights can be set in both the encoder and decoder. The encoder can encode the indices of the baseline weights that will be used by the decoder.

[0113] Referring again to the weighting scheme {0.75, 0.6, 0.5, 0.25}, the encoder can use 2 bits to indicate which baseline weight to use. That is, the encoder can indicate the index of the weight based on the weighting scheme. Bit 00 can be encoded in the encoded bitstream to indicate a baseline weight of 0.75; and bit 10 can be encoded to indicate that a baseline weight of 0.5 should be used. The baseline weight can indicate that the baseline weight will be used for the first predictor block. Therefore, the decoder can determine that the baseline weight (1-weight) will be used for the second predictor block. Thus, the baseline weight represents a mask of the first and second baseline weights, i.e., the mask {weight, (1-weight)}. Alternatively, the baseline weight can be encoded (instead of encoding the index). For example, the weight can be coarsely transmitted in the bitstream and several different values ​​can be used based on the coarse transmission.

[0114] In the second example, the smoothness of each predictor block surrounding each pixel of the predictor block can be used to determine the weighting or adjust the baseline weights. Therefore, the property used to adjust the weighting could be a first smoothness with respect to the first pixel of the first predictor block and a second smoothness with respect to the second co-located pixels of the second predictor block.

[0115] Smoothness around a pixel can indicate noise around that pixel. For example, higher smoothness indicates lower noise, and lower smoothness indicates higher noise. If the local smoothness around a pixel location of one predictor block is higher than the local smoothness at the same pixel location of another predictor block, the former predictor block can have a higher weight. Alternatively, when using baseline weights, the baseline weights of smoother predictor pixels can be adjusted upwards or downwards. Smoothness at a pixel location can be determined by examining surrounding pixels. For example, a 3×3 window centered on the pixel location can be used to determine the smoothness at that pixel location. Any window size can be used. Smoothness around a pixel can be determined using statistical values ​​(e.g., range, standard deviation, etc.) of the 3×3 window around the pixel. Other methods for determining smoothness can be used. The relative values ​​of the first and second smoothness can be used to adjust the baseline weights of the predictor blocks. For example, Equation 3 can be used to adjust the baseline weights:

[0116]

[0117] In Equation 3, modulated1[r][c], smoothness1[r][c], modulated2[r][c], and smoothness2[r][c] are, respectively, the adjusted weight of the baseline weight of the first pixel at position (r, c) of the first predictor block, the smoothness at the first pixel position, the adjusted weight of the second pixel at position (r, c) of the second predictor block, and the smoothness at the second pixel position. weight is the baseline weight used with the first predictor block, and (1-weight) is the baseline weight used with the second predictor block.

[0118] For reference Figure 7 As described, baseline weights can be conveyed in the encoded bitstream. Baseline weights can be selected based on the weighting scheme. (See reference...) Figure 7 The adjustment direction can also be encoded by the encoder and decoded and used by the decoder. (See reference...) Figure 7 The complementary mask indicator can also be encoded by the encoder and decoded and used by the decoder.

[0119] In the third example, one (or more) peak pixel values ​​from each predictor block can be used to determine the adjustment value. In this example, the encoder can send one peak value from each predictor block for composite prediction. The peak values ​​can be encoded by the encoder with low precision (i.e., a coarse approximation of the peak value). For example, the peak values ​​could be grayscale pixel values, each transmitted with a small number of bits (e.g., 2 or 3 bits). This implementation can be useful when a color is predicted better than other colors based on the predictor block. For each predictor block, pixels from the predictor block that are closer in value to the peak value can be weighted more than pixels from the predictor block that are farther in value from the peak value.

[0120] The adjusted weights used for the pixel positions of the predictor block can be obtained by a function that has a maximum value when the pixel value is equal to or approximately equal to the peak value and decays as the difference between the pixel value and the peak pixel value increases. If the first pixel value of the first predictor block is closer to (i.e., approximately equal in value to) the peak pixel value of the first predictor block than the second pixel value of the second predictor block, it can be assumed that the first predictor is a more accurate predictor for the pixels of the current block than the second predictor block.

[0121] The final weighted values ​​of the pixels used to determine the composite predictor block can be obtained using the relative weighting of the adjusted weights using equation (4):

[0122]

[0123] In equation (4), w1 and w2 are the adjusted baseline weights determined using the described decay function, p1[r][c] and p2[r][c] are the pixel values ​​at positions (r, c) of the first and second predictor blocks, respectively, and current[r][c] is the pixel at position (r, c) of the composite predictor block (i.e. the pixel to be predicted).

[0124] As described above, process 600 may include decoding baseline weights, such that weighting can be further based on baseline weights. The baseline weights may indicate a mask including a first baseline weight and a second baseline weight. The first baseline weight may be used with a first prediction block and the second baseline weight may be used with a second prediction block. The baseline weights may indicate weights from a weighting scheme including weights of 0.75, 0.6, 0.5, and 0.25. As described above, the weighting may be adjusted based on characteristics of a first pixel of the first prediction block and a second pixel co-located with the second prediction block. Characteristics may be the difference between the first pixel of the first prediction block and the second pixel of the second prediction block. Characteristics may be a first smoothness with respect to the first pixel and a second smoothness with respect to the second pixel. Furthermore, as described above, process 600 may decode a complementary mask indicator, and based on the complementary mask indicator, the first baseline weight may be used with the second prediction block and the second baseline weight may be used with the first prediction block.

[0125] As described above, process 600 can decode the adjustment direction having an upward or downward value. When the adjustment direction is upward, the first baseline weight is adjusted upward as the difference increases. When the adjustment direction is downward, the second baseline weight is adjusted downward as the difference increases.

[0126] In implementation, determining the pixel value of the current pixel block based on a weighted average of the pixel values ​​of the first prediction block and the pixel values ​​of the second prediction block may include: identifying a first peak in the first prediction block, determining a first weight of the first pixel in the first prediction block, and determining the pixel value based at least on the first weight and the first pixel.

[0127] Figure 8 This is a flowchart of a process 800 for predicting a composite prediction of a current pixel block using a first predictor block and a second predictor block, according to an implementation of this disclosure. Process 800 can be implemented in an encoder such as encoder 400.

[0128] Process 800 can be implemented as a software program, for example, executable by a computing device such as transmitting station 102. The software program may include machine-readable instructions that can be stored in a memory such as memory 204 or secondary memory 214, and the software program can be executed by a processor such as CPU 202 to cause the computing device to perform process 800. In at least some implementations, process 800 may be executed wholly or partially by the intra-frame / inter-frame prediction stage 402 of encoder 400.

[0129] Process 800 can be implemented using dedicated hardware or firmware. Some computing devices may have multiple memories, multiple processors, or both. Different processors, memories, or both can be used to distribute the steps or operations of process 800. The terms "processor" or "memory" used in the singular form include computing devices having one processor or one memory that can be used to perform some or all of the steps, as well as devices having multiple processors or multiple memories.

[0130] Process 800 can receive or determine a first predictor block and a second predictor block. Any number of predictor blocks can be used, not just the first and second predictor blocks.

[0131] At 802, process 800 encodes the baseline weights. These baseline weights can be decoded and used by the decoder to generate a composite predictor pixel block by weighting the first predictor block and the second predictor block. At 804, process 800 encodes the peak value of the first predictor block. The peak value of the first predictor block and the first pixel value are used to adjust the baseline weights of the co-located pixel values ​​of the composite predictor pixel block. The peak value is as described above with reference to the third example.

[0132] Process 800 may further include encoding the adjustment direction of the baseline weights. Process 800 may further include encoding an indication to use a complementary mask. In response to the indication being a first value, the decoder uses the first baseline weights with the first predictor block and uses the second baseline weights with the second predictor block. In response to the indication being a second value, the decoder uses the first baseline weights with the second predictor block and uses the first baseline weights with the second predictor block, and wherein the sum of the first baseline weights and the second baseline weights is one (1). The adjustment direction and the indication to use a complementary mask are referenced above. Figure 7 As stated above.

[0133] Figure 9 This is a flowchart of a process 900 for generating a composite predictor block of the current block of video according to a second implementation of this disclosure. Process 900 may be implemented in an encoder such as encoder 400 or a decoder such as decoder 500.

[0134] Process 900 can be implemented as a software program, for example, executable by a computing device such as transmitting station 102. The software program may include machine-readable instructions that can be stored in a memory such as memory 204 or secondary memory 214, and the software program can be executed by a processor such as CPU 202 to cause the computing device to perform process 900. In at least some implementations, process 900 may be executed wholly or partially by the intra / inter-frame prediction stage 402 of encoder 400.

[0135] Process 900 can be executed by a decoder such as decoder 500. Process 900 can be implemented as a software program, for example, executable by a computing device such as receiver 106. Process 900 can be executed wholly or partially by intra / inter-frame prediction stage 508 of decoder 500. For example, the implementation of process 900 can be executed by storing instructions in memory such as memory 204 of receiver 106 for execution by a processor such as CPU 202.

[0136] Process 900 can be implemented using dedicated hardware or firmware. Some computing devices may have multiple memories, multiple processors, or both. Different processors, memories, or both can be used to distribute the steps or operations of process 900. The terms "processor" or "memory" used in the singular form include computing devices having one processor or one memory that can be used to perform some or all of the steps, as well as devices having multiple processors or multiple memories.

[0137] At 902, process 900 generates a predictor block for the current block, including a first predictor block of first predictor pixels. The predictor block can be generated as described above with reference to process 600. At 904, process 900 determines a first adjustment value for adjusting a first weight to be applied to the first predictor pixels, using at least a subset of the first predictor pixels. At 906, process 900 generates a composite predictor block using the first predictor pixels, the first weight, and the first adjustment value.

[0138] The first weight can be selected from a weighting scheme including weights of 0.75, 0.6, 0.5, and 0.25. Other weights are also possible. When implemented by the encoder, process 900 can be implemented in ways such as... Figure 4 The first weight is encoded from the compressed bitstream 420 encoded bitstream. When implemented by a decoder, process 900 can extract weights from, for example... Figure 5 The first weight is decoded from the encoded bitstream of the compressed bitstream 420. Process 900 can encode the first weight by encoding the index of the weight mask (i.e., encoded when implemented by the encoder and decoded when implemented by the decoder). Process 900 can also encode the first weight by encoding a coarse value of the first weight.

[0139] In the example, the predictor block includes a second predictor block comprising second predictor pixels. Generating a composite predictor block using the first predictor pixels and a first adjustment value may include applying a second adjustment value to the second predictor pixels within the second predictor pixels, using a complement to adjust the first weights. The second adjustment value may be determined using at least a subset of the second predictor pixels. In the example, the first adjustment value may be determined based on the difference between the first and second predictor pixels. In the example, the first adjustment value may be determined using the smoothness within a window surrounding the first predictor pixel. The window may be a window centered on the first predictor pixel and have a window size of 3×3.

[0140] In the example, a decay function that has a maximum value at a predetermined pixel value in the first predictor block can be used to determine the first adjustment value. The first predicted pixel can be input into the decay function to determine the first adjustment value.

[0141] Figure 10 This is a flowchart of a process 1000 for generating a composite predictor block according to a third implementation of this disclosure. Process 1000 can be implemented by an encoder or decoder as described in reference process 900. Process 1000 can be implemented as a software program as described in reference process 900. Process 1000 can be implemented using dedicated hardware or firmware as described in reference process 900.

[0142] At 1002, process 1000 generates a first predictor block and a second predictor block. The first predictor block includes first predictor pixels. The second predictor block includes second predictor pixels. At 1004, process 1000 determines a corresponding first adjustment value for a corresponding first predictor pixel of the first predictor block. Each first adjustment value is determined using at least some of the first predictor pixels. At 1006, process 1000 determines a corresponding second adjustment value for a corresponding second pixel of the second predictor block. Each second adjustment value is determined using at least some of the second predictor pixels. At 1008, process 1000 uses the first predictor pixel, the first adjustment value, the second predictor pixel, and the second adjustment value to determine the pixel value of a pixel in the composite predictor block.

[0143] In the example processing 1000, the first adjustment value of the first predictor pixel and the second adjustment value of the second predictor pixel can be based on characteristics of the first and second predictor pixels. The first and second predictor pixels are located at one point. In the example, the characteristic can be the difference between the first and second predictor pixels. The difference can be an absolute difference. In the example, the characteristic can be a first smoothness with respect to the first predictor pixel and a second smoothness with respect to the second predictor pixel. In one example, more than one characteristic can be combined. For example, the mask weights can be adjusted based on any combination of pixel difference, smoothness, peaking, or any other characteristic.

[0144] In the example, process 1000 can decode baseline weights from an encoded bitstream. The baseline weights can indicate a mask comprising a first baseline weight and its complement. In the example, each first adjustment value can be used to adjust the first baseline weight, and each second adjustment value can be used to adjust the complement of the first baseline weight. The baseline weights can indicate weights from a weighting scheme comprising weights of 0.75, 0.6, 0.5, and 0.25.

[0145] In one example, process 1000, when implemented by a decoder, includes decoding a complementary mask indicator. Based on the complementary mask indicator, process 1000 can adjust the complement of the first baseline weights using each first adjustment value and adjust the first baseline weights using each second adjustment value.

[0146] In the example, process 1000 may include selecting a first baseline weight and a complement of the first baseline weight. Each first adjustment value can be used to adjust the first baseline weight, and each second adjustment value can be used to adjust the complement of the first baseline weight. Process 1000 may also include selecting an adjustment direction with an upward or downward value. When the adjustment direction is upward, the first baseline weight is adjusted upward as the characteristic increases. When the adjustment direction is downward, the first baseline weight is adjusted downward as the characteristic increases.

[0147] Figure 11 This is a flowchart of a process 1100 for decoding the current block according to a second implementation of this disclosure. Process 1100 generates a composite prediction block for the current block.

[0148] Process 1100 can be executed by a decoder such as decoder 500. Process 1100 can be implemented as a software program, for example, executable by a computing device such as receiver 106. Process 1100 can be executed wholly or partially by the intra / inter-frame prediction stage 508 of decoder 500. For example, implementation of process 1100 can be executed by a processor such as CPU 202 by storing instructions in memory such as memory 204 of receiver 106.

[0149] Process 1100 can be implemented using dedicated hardware or firmware. Some computing devices may have multiple memories, multiple processors, or both. Different processors, memories, or both can be used to distribute the steps or operations of process 1100. The terms "processor" or "memory" used in the singular form include computing devices having one processor or one memory that can be used to perform some or all of the steps, as well as devices having multiple processors or multiple memories.

[0150] At 1102, process 1100 generates a first predictor block and a second predictor block. At 1104, process 1100 obtains data from sources such as... Figure 5 The compressed bitstream 420 is encoded and then decoded using a weight mask. At 1106, process 1100 determines from the weight mask a baseline weight for a first weight of pixels used as a first predictor block and a complementary baseline weight for a second weight of pixels used as a second predictor block. At 1108, process 1100 determines a first adjustment value for a first predictor pixel of the first predictor block using at least some pixels of the first predictor block. The first adjustment value can be used to adjust the baseline weight. At 1110, process 1100 determines a second adjustment value for a second predictor pixel of the second predictor block using at least some pixels of the second predictor block. The second adjustment value can be used to adjust the complementary baseline weight. The second predictor pixel is co-located with the first pixel predictor. At 1112, process 1100 uses the first predictor pixel, the second predictor pixel, the baseline weight, the complementary baseline weight, the first adjustment value, and the second adjustment value to generate a composite predictor including a third predictor pixel.

[0151] In the example, if the baseline weights have a first value and the complementary baseline weights have a second value, process 1100 may include decoding an indication to use a complementary mask, and in response to the indication to use a complementary mask being the first value, using the second value as the value of the baseline weights and using the first value as the complementary baseline weights, and adding the first value and the second value to reach 1.

[0152] The above-described aspects of encoding and decoding illustrate some encoding and decoding techniques. However, it should be understood that encoding and decoding, as the terms used in the claims, can refer to compression, decompression, transformation, or any other processing or alteration of data.

[0153] The terms “example” or “implementation” are used herein to indicate something used as an example, instance, or illustration. No aspect or design described herein as an “example” or “implementation” should be construed as being more preferred or advantageous than other aspects or designs. Rather, the use of the terms “example” or “implementation” is intended to present concepts in a specific manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise stated or clear from the context, “X comprises A or B” is intended to mean any natural inclusive arrangement. That is, if X comprises A; X comprises B; or X comprises A and B, then “X comprises A or B” is satisfied in any of the foregoing examples. Additionally, unless otherwise stated or clear from the context, the articles “a” and “an” used herein and in the appended claims should generally be understood to mean “one or more” as referring to the singular form. Furthermore, the use of the terms “implementation” or “an implementation” throughout the document is not intended to represent the same embodiment or implementation unless so described.

[0154] The implementation of transmitting station 102 and / or receiving station 106 (and algorithms, methods, instructions, etc. stored thereon and / or executed thereon, including those executed by encoder 400 and decoder 500) can be implemented in hardware, software, or any combination thereof. Hardware may include, for example, a computer, intellectual property (IP) core, application-specific integrated circuit (ASIC), programmable logic array, optical processor, programmable logic controller, microcode, microcontroller, server, microprocessor, digital signal processor, or any other suitable circuitry. In the claims, the term "processor" should be understood to include any of the foregoing hardware, individually or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, portions of transmitting station 102 and receiving station 106 need not necessarily be implemented in the same manner.

[0155] Furthermore, in one aspect, for example, transmitting station 102 or receiving station 106 may be implemented using a general-purpose computer or a general-purpose processor having a computer program that, when executed, performs any of the corresponding methods, algorithms, and / or instructions described herein. Alternatively or alternatively, for example, a dedicated computer / processor may be used, which may include other hardware for performing any of the methods, algorithms, or instructions described herein.

[0156] Sending station 102 and receiving station 106 can be implemented, for example, on a computer in a video conferencing system. Alternatively, sending station 102 can be implemented on a server, and receiving station 106 can be implemented on a device separate from the server, such as a handheld communication device. In this example, sending station 102 can use encoder 400 to encode content into an encoded video signal and send the encoded video signal to the communication device. The communication device can then use decoder 500 to decode the encoded video signal. Alternatively, the communication device can decode content stored locally on the communication device, such as content not sent by sending station 102. Other suitable implementations of sending station 102 and receiving station 106 are available. For example, receiving station 106 can be a typically stationary personal computer instead of a portable communication device, and / or the device including encoder 400 can also include decoder 500.

[0157] Furthermore, all or part of the implementation of this disclosure may take the form of a computer program product accessible from, for example, a tangible computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be any device capable of, for example, tangibly including, storing, communicating, or transmitting a program for use by or in connection with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

[0158] The embodiments, implementations, and aspects described above are intended to facilitate understanding of this disclosure and do not limit it. Rather, this disclosure is intended to cover a variety of modifications and equivalent arrangements included within the scope of the appended claims, which should be given the broadest interpretation to include all such modifications and equivalent structures permitted by law.

Claims

1. A method for generating a composite predictor block of a current block in a video, comprising: A first predictor block is generated for the current block using either inter-frame prediction or intra-frame prediction. The first predictor block includes a first pixel located at position (r, c) in the first predictor block, where r indicates the row number and c indicates the column number. Wherein, the first predictor block includes a corresponding first predicted pixel for each pixel of the current block, and Wherein, the first predictor block is different from the current block; A second predictor block is generated for the current block, including a second pixel co-located with the first pixel, wherein the second pixel is located in the second predictor block at position (r, c). The second predictor block includes a corresponding second predicted pixel for each pixel of the current block, and Wherein, the second predictor block is different from the first predictor block and different from the current block; The first weight of the first pixel is determined by the difference between the value of the first pixel and the value of the second pixel; The first weight is used to determine the second weight of the second pixel; and The composite predictor block is generated by combining the first predictor block and the second predictor block, wherein the composite predictor block includes weighted pixels, which are determined using the first weight and the second weight, and by weighting the first pixel and the second pixel.

2. The method according to claim 1, wherein, The first weight is determined by including the steps of multiplying the absolute value of the difference by a first constant to obtain a result and adding a second constant to the result.

3. The method according to claim 2, wherein, The first constant is calculated as a ratio having a numerator obtained using baseline weights and a denominator obtained using the highest possible pixel value.

4. The method according to claim 3, wherein, The second constant is equal to the baseline weight.

5. The method according to claim 1, wherein, The second weight is determined to be 1 minus the first weight.

6. The method of claim 1, further comprising: Decoding instructions from the compressed bitstream to obtain the prediction mode of the composite predictor block using the difference.

7. The method of claim 1, further comprising: The prediction mode of the composite predictor block, which indicates the use of the difference, is encoded in a compressed bitstream.

8. An apparatus for generating a composite predictor block for a current block, comprising: Processor, the processor being configured to: Generate a first predictor block for the current block. The first predictor block includes a first pixel located at position (r, c) in the first predictor block, where r indicates the row number and c indicates the column number. Wherein, the first predictor block includes a corresponding first predicted pixel for each pixel of the current block, and Wherein, the first predictor block is different from the current block; Generate a second predictor block for the current block. The second predictor block includes a second pixel co-located with the first pixel, and the second pixel is located in the second predictor block at position (r,c). The second predictor block includes a corresponding second predicted pixel for each pixel of the current block, and Wherein, the second predictor block is different from the first predictor block and different from the current block; The first weight of the first pixel is determined by the difference between the value of the first pixel and the value of the second pixel; The first weight is used to determine the second weight of the second pixel; and The composite predictor block is generated by combining the first predictor block and the second predictor block, wherein the composite predictor block includes weighted pixels, which are determined by using the first weight and the second weight, respectively, and by utilizing the weighted sum of the first pixel and the second pixel.

9. The apparatus according to claim 8, wherein, The first weight is determined by including the steps of multiplying the absolute value of the difference by a first constant to obtain a result and adding a second constant to the result.

10. The apparatus according to claim 9, wherein, in, The first constant is calculated as a ratio having a numerator obtained using baseline weights and a denominator obtained using the highest possible pixel value.

11. The apparatus according to claim 10, wherein, The second constant is equal to the baseline weight.

12. The apparatus according to claim 8, wherein, The second weight is determined to be 1 minus the first weight.

13. The apparatus according to claim 8, wherein, The processor is further configured to: Decoding instructions from the compressed bitstream to obtain the prediction mode of the composite predictor block using the difference.

14. The apparatus according to claim 8, wherein, The processor is further configured to: The prediction mode of the composite predictor block, which indicates the use of the difference, is encoded in a compressed bitstream.

15. A non-transitory computer-readable storage medium including executable instructions that, when executed by a processor, facilitate the execution of operations, said operations including: A first predictor block is generated for the current block, wherein the first predictor block is generated using either inter-frame prediction or intra-frame prediction, and wherein the first predictor block includes a first pixel in the first predictor block located at position (r, c), wherein r indicates a row number and c indicates a column number, and wherein the first predictor block is different from the current block. A second predictor block is generated for the current block, wherein the second predictor block includes a second pixel co-located with the first pixel, the second pixel is located in the second predictor block at the position (r,c), and wherein the second predictor block is different from the first predictor block and different from the current block; The first weight of the first pixel is determined by the difference between the value of the first pixel and the value of the second pixel; The first weight is used to determine the second weight of the second pixel; and A composite predictor block is generated by combining the first predictor block and the second predictor block, wherein the composite predictor block includes weighted pixels, which are determined by using the first weight and the second weight, respectively, and by utilizing the weighted sum of the first pixel and the second pixel.

16. The non-transitory computer-readable storage medium according to claim 15, wherein, The first weight is determined by including the steps of multiplying the absolute value of the difference by a first constant to obtain a result and adding a second constant to the result.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, The first constant is calculated as a ratio having a numerator obtained using baseline weights and a denominator obtained using the highest possible pixel value.

18. The non-transitory computer-readable storage medium according to claim 17, wherein, The second constant is equal to the baseline weight.

19. The non-transitory computer-readable storage medium according to claim 17, wherein, The baseline weights are obtained from a weighting scheme that includes weights of 0.75, 0.6, 0.5, and 0.

25.

20. The non-transitory computer-readable storage medium of claim 15, further comprising: Decoding instructions from the compressed bitstream to obtain the prediction mode of the composite predictor block using the difference.

Images