Local illumination compensation based on adjustments

Abstract

Aspects of the present disclosure provide a method and apparatus for video decoding. The apparatus includes processing circuitry that decodes prediction information for one or more blocks including a current block from an encoded video bitstream. The prediction information indicates that local illumination compensation (LIC) is applied to the one or more blocks and includes LIC information of the one or more blocks. The processing circuitry determines a final scaling factor [alpha] f1 and a final offset [beta] f1 for the first sub-block in the current block based on the LIC information. The processing circuitry determines an updated first predictor sub-block of the predictors corresponding to the current block based on the final scaling factor [alpha] f1, the final offset [beta] f1, and the first predictor sub-block. The update value of the first sample in the first predictor sub-block is equal to [alpha] f1 * pi1 + [beta] f1. Pi1 is a value of the first sample.

Description

[0001] This application is a divisional application. The application number of the original application is 2022800087012, and the application date is September 8, 2022.

[0002] Incorporated by Reference

[0003] This application claims the benefit of priority to U.S. Patent Application No. 17 / 903,697, filed on September 6, 2022, for "ADJUSTMENT BASED LOCAL ILLUMINATION COMPENSATION," which claims the benefit of priority to U.S. Provisional Application No. 63 / 298,788, filed on January 12, 2022, for "ADJUSTMENT BASED LOCAL ILLUMINATION COMPENSATION." The disclosures of the prior applications are incorporated herein by reference in their entirety. Technical Field

[0004] This disclosure describes implementations that generally relate to video encoding. Background Art

[0005] The purpose of the background description provided herein is to generally present the context of the present disclosure. To the extent the work is described in this background section, the work of the presently named inventors and aspects of the description that may not otherwise be described as prior art at the time of filing are neither explicitly nor implicitly admitted as prior art to the present disclosure.

[0006] An uncompressed digital image and / or video may include a series of pictures, each picture having, for example, spatial dimensions of 1920×1080 luminance samples and associated chrominance samples. This series of pictures may have a fixed or variable picture rate (also informally referred to as a frame rate) of, for example, 60 pictures per second or 60 Hz. Uncompressed images and / or videos have specific bit rate requirements. For example, 1080p60 4:2:0 video (1920×1080 luminance sample resolution at 60 Hz frame rate) with 8 bits per sample requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires over 600 Gigabytes of storage space.

[0007] One purpose of image and / or video encoding and decoding can be to reduce redundancy in input image and / or video signals by compression. Compression can help reduce the bandwidth and / or storage space requirements mentioned above, in some cases by two orders of magnitude or more. Although the description herein uses video encoding / decoding as an illustrative example, the same technology can be applied to image encoding / decoding in a similar manner without departing from the spirit of the present disclosure. Both lossless compression and lossy compression, as well as combinations thereof, can be employed. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may be different from the original signal, but the distortion between the original signal and the reconstructed signal is small enough to enable the reconstructed signal to be used for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect that a higher allowable / tolerable distortion can produce a higher compression ratio.

[0008] Video encoders and decoders may utilize techniques from several broad categories including, for example, motion compensation, transform processing, quantization, and entropy coding.

[0009] Video codec techniques may include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures are spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-frame mode, the picture may be an intra-frame picture. Intra-frame pictures and their derivatives (e.g., independent decoder refresh pictures) may be used to reset decoder states, and may therefore be used as the first picture in a coded video bitstream and video session, or as a still image. Samples of intra-frame blocks may be subjected to transformation, and transform coefficients may be quantized before entropy coding. Intra-frame prediction may be a technique for minimizing sample values ​​in a pre-transform domain. In some cases, the smaller the DC value after transformation and the smaller the AC coefficient, the fewer bits are required to represent the block after entropy coding at a given quantization step size.

[0010] Traditional intra-frame coding, such as that used in MPEG-2 generation coding techniques, does not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to perform predictions based on metadata and / or surrounding sample data obtained, for example, during encoding and / or decoding of a data block. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses reference data only from the current picture under reconstruction, and does not use reference data from reference pictures.

[0011] There can be many different forms of intra-frame prediction. When more than one such technique can be used in a given video coding technique, the specific technique used can be encoded as a specific intra-frame prediction mode using that specific technique. In some cases, the intra-frame prediction mode can have sub-modes and / or parameters, where these sub-modes and / or parameters can be encoded separately or included in a mode codeword, which defines the prediction mode used. Which codeword is used for a given mode, sub-mode and / or parameter combination can affect the coding efficiency gain through intra-frame prediction, and therefore the entropy coding technique used to convert the codeword into a bitstream can also affect the coding efficiency gain through intra-frame prediction.

[0012] Some modes of intra prediction were introduced with H.264, refined in H.265, and further refined in newer coding techniques such as the joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). Using the values ​​of neighboring samples of the already available samples, a predictor block can be formed. Sample values ​​of neighboring samples are copied into the predictor block according to the direction. A reference to the used direction can be encoded in the bitstream or can be predicted itself.

[0013] Reference Figure 1A , depicted at the bottom right is a subset of nine predictor directions known from the 33 possible predictor directions defined in H.265 (corresponding to the 33 angular modes for the 35 intra modes). The point where the arrows intersect (101) represents the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted based on one or more samples at an angle of 45 degrees to the horizontal to the upper right. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more samples at an angle of 22.5 degrees to the horizontal to the lower left of sample (101).

[0014] Still refer to Figure 1A, depicted at the top left is a square block (104) of 4×4 samples (indicated by bold dashed lines). The square block (104) includes 16 samples, each of which is labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y dimension and the X dimension. Since the size of the block is 4×4 samples, S44 is at the bottom right. Also shown are reference samples that follow a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index) and X position (column index) relative to the block (104). In both H.264 and H.265, the prediction samples are adjacent to the block under reconstruction; therefore, there is no need to use negative values.

[0015] Intra-picture prediction can work by copying reference sample values ​​from neighboring samples indicated by a signaled prediction direction. For example, assume that the coded video bitstream includes signaling that indicates a prediction direction consistent with arrow (102) for this block - that is, the sample is predicted from the sample to the upper right at a 45 degree angle from the horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

[0016] In some cases, the values ​​of multiple reference samples may be combined, for example, by interpolation, in order to compute a reference sample; in particular, when the directions are not evenly divisible at 45 degrees.

[0017] As video coding technology has developed, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. In H.265 (2013), this increased to 33. Currently, JEM / VVC / BMS can support up to 65 directions. Experiments have been conducted to identify the most likely directions, and certain techniques in entropy coding are used to represent these possible directions with a small number of bits, at the expense of fewer possible directions. In addition, the direction itself can sometimes be predicted based on the neighboring directions used in adjacent decoded blocks.

[0018] Figure 1B A schematic diagram (110) is shown depicting 65 intra prediction directions according to JEM to illustrate that the number of prediction directions increases over time.

[0019] The mapping of the intra-frame prediction direction bits representing directions in the coded video bitstream can be different according to different video coding techniques. The range of such mapping can be, for example, from simple direct mapping to codewords, to complex adaptive schemes involving the most likely mode and similar techniques. However, in most cases, there may be certain directions that are statistically less likely to appear in the video content than some other directions. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less likely directions will be represented by a larger number of bits than more likely directions.

[0020] Image and / or video encoding and decoding may be performed using inter-picture prediction with motion compensation. Motion compensation may be a lossy compression technique and may involve a technique in which a block of sample data from a previously reconstructed picture or portion thereof (reference picture) is used to predict a reconstructed picture or picture portion after being spatially shifted in a direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be the same as the picture under current reconstruction. The MV may have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference picture in use (indirectly, the third dimension may be a temporal dimension).

[0021] In some video compression techniques, an MV applicable to a particular region of sample data can be predicted based on other MVs, for example, based on an MV related to another region of sample data that is spatially adjacent to the region under reconstruction and precedes the MV in decoding order. The above prediction can significantly reduce the amount of data required to encode the MV, thereby eliminating redundancy and increasing compression. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera (called natural video), there is a statistical probability that a larger region than the region to which a single MV is applicable moves in a similar direction, and therefore in some cases similar motion vectors derived from MVs of adjacent regions can be used to predict the larger region. This makes the MV obtained for a given region similar or identical to the MV predicted from the surrounding MVs, and in turn, the MV can be represented after entropy coding with a smaller number of bits than would be used if the MV was encoded directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, MV prediction itself can be lossy, for example due to rounding errors when calculating a predictor based on several surrounding MVs.

[0022] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T H.265 Recommendation, "High Efficiency Video Coding", December 2016). Among the various MV prediction mechanisms provided by H.265, refer to Figure 2 Described is a technique referred to hereinafter as "spatial merging".

[0023] Reference Figure 2 , the current block (201) includes samples obtained by the encoder during the motion search process that can be predicted from a previous block of the same size that has been spatially shifted. Instead of encoding the MV directly, the MV associated with any of the five surrounding samples represented by A0, A1 and B0, B1, B2 (corresponding to 202 to 206, respectively) can be used to derive the MV from the reference picture associated with one or more reference pictures, such as from the most recent (in decoding order) reference picture. In H.265, MV prediction can use a predictor from the same reference picture being used by a neighboring block. Summary of the invention

[0024] Aspects of the present disclosure provide methods and apparatus for video encoding and decoding. In some examples, an apparatus for video decoding includes a processing circuit system. The processing circuit system is configured to decode prediction information for one or more blocks from an encoded video bitstream. The prediction information indicates that local illumination compensation (LIC) is applied to the one or more blocks and includes LIC information for the one or more blocks. The one or more blocks include a current block to be reconstructed. The processing circuit system determines a final scaling factor α for a first sub-block in the current block based on the LIC information. f1 and the final offset β of the first sub-block in the current block f1 The processing circuitry is based on the final scaling factor α f1 , final offset β f1 and the first predictor sub-block in the predictor to determine the updated first predictor sub-block in the predictor corresponding to the current block. In the example, the update value of the first sample in the first predictor sub-block in the predictor is equal to α f1 ×p i1 +β f1 , where p i1 is the value of a first sample in a first predictor sub-block in the predictor. The processing circuitry reconstructs a first sub-block in the current block based on the updated first predictor sub-block in the predictor.

[0025] In an embodiment, an initial scaling factor α of a first sub-block in the current block is determined based on a first portion of a current template of the current block and a first portion of a reference template of a reference block. i1 and the initial offset β of the first sub-block in the current block i1The predictor may be based on a reference block. The LIC information for the one or more blocks is signaled in the coded video bitstream and indicates at least one of the first parameter μ1 for the one or more blocks or the second parameter μ2 for the one or more blocks. The processing circuitry calculates the initial scaling factor α for the first sub-block in the current block based on the first parameter μ1 for the one or more blocks and the initial scaling factor α for the first sub-block in the current block. i1 To determine the final scaling factor α of the first sub-block in the current block f1 The processing circuit system is based on the second parameter μ2 of the one or more blocks and the initial offset β of the first sub-block in the current block. i1 To determine the final offset β of the first sub-block in the current block f1 .

[0026] In an example, LIC information of one or more blocks indicates one of (i) a first parameter μ1 and (ii) a second parameter μ2, and the other of (i) the first parameter μ1 and (ii) the second parameter μ2 is determined based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0027] In an example, one of (i) the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0028] In an example, the LIC information of the one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0029] In an example, at least one of the first parameter μ1 or the second parameter μ2 is a floating point value and is quantized with a plurality of bits, and the LIC information of the one or more blocks includes the plurality of bits.

[0030] In an example, the processing circuitry sets the final scaling factor α for the first sub-block in the current block to f1 Determined as a first parameter μ1 of one or more blocks and an initial scaling factor α of the first sub-block in the current block i1 The sum of .

[0031] In an example, the processing circuitry determines a parameter T of a first sub-block in the current block based on at least one of a first portion of a current template or a first portion of a reference template. avg，1 , and the final offset β of the first sub-block in the current block f1 Determined as (β i1 +μ2×T avg，1 ).

[0032] In an example, LIC information of one or more blocks is signaled at a coding unit (CU) level.

[0033] In an example, LIC information of one or more blocks is signaled at a level higher than the CU level.

[0034] In an example, the first subblock comprises the current block, the first predictor subblock comprises the predictor, the first portion of the current template comprises the current template, and the first portion of the reference template comprises the reference template.

[0035] In an example, the current block includes a first subblock and a second subblock, at least one of the first parameter μ1 or the second parameter μ2 indicated in the LIC information of one or more blocks is applied to the first subblock and the second subblock, and the second subblock is scaled with another initial scaling factor α i2 and another initial offset β i2 Associated, another initial scaling factor α i2 The initial scaling factor α of the first sub-block i1 Different, another initial offset β i2 Different from the initial offset of the first sub-block.

[0036] Aspects of the present disclosure also provide a non-transitory computer-readable storage medium storing a program executable by at least one processor to perform a method for video decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0038] Figure 1A is a schematic illustration of an exemplary subset of intra prediction modes.

[0039] Figure 1B is a diagram of exemplary intra prediction directions.

[0040] Figure 2 An example of a current block (201) and surrounding samples is shown.

[0041] Figure 3 is a schematic illustration of an exemplary block diagram of a communication system (300).

[0042] Figure 4 is a schematic illustration of an exemplary block diagram of a communication system (400).

[0043] Figure 5 is a schematic illustration of an exemplary block diagram of a decoder.

[0044] Figure 6 is a schematic illustration of an exemplary block diagram of an encoder.

[0045] Figure 7 A block diagram of an exemplary encoder is shown.

[0046] Figure 8 A block diagram of an exemplary decoder is shown.

[0047] Fig. 9 The positions of spatial merging candidates according to an embodiment of the present disclosure are shown.

[0048] Fig.10 Candidate pairs considered for redundancy checking of spatial merging candidates according to an embodiment of the present disclosure are shown.

[0049] Fig.11 Exemplary motion vector scaling of temporal merging candidates is shown.

[0050] Fig.12 Exemplary candidate positions of temporal merging candidates for the current coding unit are shown.

[0051] FIG. 13A to FIG. 13B An example of an affine model is shown.

[0052] Fig.14 An example of sub-block based affine prediction is shown.

[0053] Fig.15 An example of determining control point motion vector (CPMV) candidates in the affine merge list of the current CU is shown.

[0054] Fig.16 Examples of spatial and temporal neighbors of the current block are shown.

[0055] Fig.17 An exemplary LIC with parameter adjustments is shown.

[0056] Fig.18 A flow chart outlining an encoding process according to an embodiment of the present disclosure is shown.

[0057] Fig.19 A flow chart outlining a decoding process according to an embodiment of the present disclosure is shown.

[0058] Fig. 20 is a schematic illustration of a computer system according to an embodiment. DETAILED DESCRIPTION

[0059] Figure 3 An exemplary block diagram of a communication system (300) is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3In the example of , a first pair of terminal devices (310) and (320) perform unidirectional transmission of data. For example, the terminal device (310) can encode video data (e.g., a video picture stream captured by the terminal device (310)) for transmission to another terminal device (320) via a network (350). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) can receive the encoded video data from the network (350), decode the encoded video data to recover the video picture, and display the video picture based on the recovered video data. Unidirectional data transmission may be common in media service applications, etc.

[0060] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, for example, during a video conference. For the bidirectional transmission of data, in the example, each of the terminal devices (330) and (340) can encode video data (e.g., a video picture stream captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) via the network (350). Each of the terminal devices (330) and (340) can also receive the encoded video data transmitted by the other of the terminal devices (330) and (340), and can decode the encoded video data to restore the video picture, and can display the video picture at an accessible display device based on the restored video data.

[0061] exist Figure 3 In the example of , terminal devices (310), (320), (330) and (340) are respectively shown as servers, personal computers and smart phones, but the principles of the present disclosure may not be so limited. Implementations of the present disclosure are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. Network (350) represents any number of networks that transmit encoded video data between terminal devices (310), (320), (330) and (340), including, for example, wired (wired) and / or wireless communication networks. Communication network (350) can exchange data in circuit switching channels and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purposes of this discussion, unless otherwise described herein below, the architecture and topology of network (350) may be irrelevant to the operation of the present disclosure.

[0062] As examples of applications of the disclosed subject matter, Figure 4A video encoder and a video decoder in a streaming environment are shown. The disclosed subject matter can be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0063] The streaming system may include a capture subsystem (413) that may include a video source (401), such as a digital camera, that creates, for example, an uncompressed video picture stream (402). In an example, the video picture stream (402) includes samples captured by the digital camera. The video picture stream (402) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (404) (or encoded video bitstream), which may be processed by an electronic device (420) coupled to the video source (401) and including a video encoder (403). The video encoder (403) may include hardware, software, or a combination thereof to implement or embody aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video bitstream) is depicted as a thin line to emphasize the lower data volume when compared to the video picture stream (402), and the encoded video data (404) (or encoded video bitstream) can be stored on the streaming server (405) for future use. One or more streaming client subsystems, such as Figure 4 The client subsystems (406) and (408) in the streaming server (405) can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) can include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video picture stream (411) that can be presented on a display (412) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (404), (407) and (409) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. In the example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter can be used in the context of VVC.

[0064] Note that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0065] Figure 5 An exemplary block diagram of a video decoder (510) is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may replace Figure 4 The video decoder (410) in the example is used.

[0066] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510). In an embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequence can be received from a channel (501), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (531) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which can be forwarded to their respective use entities (not depicted). The receiver (531) can separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other applications, the buffer memory (515) can be external to the video decoder (510) (not depicted). In still other applications, there may be a buffer memory (not depicted) external to the video decoder (510) to, for example, prevent network jitter, and in addition there may be another buffer memory (515) internal to the video decoder (510) to, for example, handle playout timing. When the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory (515) may not be needed, or the buffer memory (515) may be small. For use over a best effort packet network such as the Internet, a buffer memory (515) may be needed, which may be relatively large and may advantageously have an adaptive size, and may be implemented at least in part in an operating system or similar element (not depicted) external to the video decoder (510).

[0067] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. The categories of these symbols include: information for managing the operation of the video decoder (510), and possibly information for controlling a rendering device such as a rendering device (512) (e.g., a display screen), which is not part of the electronic device (530) but can be coupled to the electronic device (530), such as Figure 5 . The control information of the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser (520) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may comply with a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set for at least one subgroup of the pixel subgroups in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include: a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the coded video sequence.

[0068] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0069] The reconstruction of the symbol (521) may involve a number of different units depending on the type of coded video picture or portion thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser (520). For clarity, such subgroup control information flow between the parser (520) and the following multiple units is not depicted.

[0070] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, it is appropriate to conceptually subdivide into the following functional units.

[0071] The first unit is a sealer / inverse transform unit (551). The sealer / inverse transform unit (551) receives quantized transform coefficients and control information (including which transform to use, block size, quantization factor, quantization scaling matrix, etc.) from the parser (520) as (one or more) symbols (521). The sealer / inverse transform unit (551) can output blocks including sample values, which can be input into the aggregator (555).

[0072] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed picture but can use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates a block of the same size and shape as the block in reconstruction using surrounding reconstructed information obtained from a current picture buffer (558). For example, the current picture buffer (558) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (555) adds the prediction information already generated by the intra-prediction unit (552) to the output sample information as provided by the scaler / inverse transform unit (551) on a per-sample basis.

[0073] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to an inter-frame coded and possibly motion compensated block. In such a case, the motion compensated prediction unit (553) may access the reference picture memory (557) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the block, these samples may be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (in this case referred to as residual samples or residual signal), thereby generating output sample information. The address within the reference picture memory (557) from which the motion compensated prediction unit (553) obtains the predicted samples may be controlled by a motion vector, which may be obtained by the motion compensated prediction unit (553) in the form of a symbol (521), which may have, for example, an X component, a Y component and a reference picture component. Motion compensation may also include interpolation of sample values ​​obtained from the reference picture memory (557) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0074] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream), which may be obtained by the loop filter unit (556) as symbols (521) from the parser (520). The video compression may also be responsive to meta-information obtained during decoding of a previous portion (in decoding order) of a coded picture or coded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0075] The output of the loop filter unit (556) may be a sample stream that may be output to a rendering device (512) and stored in a reference picture memory (557) for use in future inter-picture prediction.

[0076] Once fully reconstructed, certain coded pictures can be used as reference pictures for use in future predictions. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557) and a new current picture buffer can be reallocated before starting to reconstruct a subsequent coded picture.

[0077] The video decoder (510) may perform decoding operations according to a predetermined video compression technology or standard such as ITU-T H.265 Recommendation. The coded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the coded video sequence follows both the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as tools available only under the profile. For compliance, it is also required that the complexity of the coded video sequence is within the limits defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (measured in, for example, millions of samples per second), the maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the coded video sequence.

[0078] In an embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence (one or more). The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0079] Figure 6 An exemplary block diagram of a video encoder (603) is shown. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit system). The video encoder (603) may replace Figure 4 The video encoder (403) in the example is used.

[0080] The video encoder (603) can be used to obtain the video source (601) (not Figure 6 In an example of an electronic device (620) receiving video samples, a video source (601) can capture (one or more) video images to be encoded by a video encoder (603). In another example, the video source (601) is a part of the electronic device (620).

[0081] The video source (601) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (603), the digital video sample stream may have any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (601) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (601) may be a camera device that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The picture itself may be organized as a spatial pixel array, wherein each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on samples.

[0082] According to an embodiment, the video encoder (603) can encode and compress the pictures of the source video sequence into an encoded video sequence (643) in real time or under any other time constraints required. Implementing the appropriate encoding speed is a function of the controller (650). In some embodiments, the controller (650) controls other functional units as described below and is functionally coupled to the other functional units. The coupling is not depicted for simplicity. The parameters set by the controller (650) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology...), picture size, picture group (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions that belong to the video encoder (603) optimized for a specific system design.

[0083] In some embodiments, the video encoder (603) is configured to operate in an encoding loop. As an extremely simplified description, in an example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and (one or more) reference pictures) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder would also create sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces a bit-accurate result that is independent of the decoder location (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local encoder and the remote encoder. In other words, the prediction part of the encoder "sees" the same sample values ​​as the sample values ​​that the decoder "sees" when using prediction during decoding as reference picture samples. This basic principle of reference picture synchronization (and the offset that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0084] The operation of the "local" decoder (633) can be combined with the "remote" decoder such as has been described above. Figure 5 The operation of the video decoder (510) described in detail is the same. However, in addition, briefly refer to Figure 5 , since the symbols are available and the encoding of the symbols into a coded video sequence by the entropy encoder (645) and the decoding of the symbols by the parser (520) can be lossless, the entropy decoding portion of the video decoder (510) including the buffer memory (515) and the parser (520) may not be fully implemented in the local decoder (633).

[0085] In an embodiment, any decoder technology other than the parsing / entropy decoding present in the decoder is present in the corresponding encoder in the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on the decoder operation. The description of the encoder technology can be simplified because the encoder technology is opposite to the decoder technology described comprehensively. In some aspects, a more detailed description is provided below.

[0086] In some examples, during operation, the source encoder (630) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as “reference pictures.” In this manner, the encoding engine (632) encodes the differences between pixel blocks of an input picture and pixel blocks of reference picture(s) that may be selected as prediction reference(s) for the input picture.

[0087] The local video decoder (633) can decode the encoded video data of the picture that can be designated as the reference picture based on the symbol created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 6 When decoded at a remote video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (634). In this way, the video encoder (603) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture that will be obtained by the remote video decoder.

[0088] The predictor (635) may perform a prediction search for the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that may be used as suitable prediction references for the new picture. The predictor (635) may operate on a sample-block-by-pixel-block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (634).

[0089] The controller (650) may manage encoding operations of the source encoder (630), including, for example, setting parameters and sub-group parameters for encoding video data.

[0090] The outputs of all the above-mentioned functional units may be subjected to entropy coding in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0091] The transmitter (640) can buffer the encoded video sequence(s) created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which can be a hardware / software link to a storage device storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0092] The controller (650) may manage the operation of the video encoder (603). During encoding, the controller (650) may assign a certain encoding picture type to each encoded picture, which may affect the encoding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned one of the following picture types:

[0093] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of those variations of I pictures and their corresponding applications and features.

[0094] A predictive picture (P picture) may be a picture that can be encoded and decoded using inter prediction or intra prediction to predict sample values ​​of each block using at most one motion vector and a reference index.

[0095] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using inter-prediction or intra-prediction that uses up to two motion vectors and reference indices to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0096] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively), and coded block by block. These blocks may be predictively coded with reference to other (coded) blocks, which are determined by the coding allocation applied to the corresponding picture of the block. For example, blocks of an I picture may be non-predictively coded, or may be predictively coded (spatial prediction or intra-prediction) with reference to coded blocks of the same picture. Pixel blocks of a P picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0097] The video encoder (603) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0098] In an embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0099] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified as intra-prediction) exploits spatial correlation in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an example, a particular picture being encoded / decoded (which is referred to as the current picture) is divided into blocks. In the case where a block in the current picture is similar to a reference block in a reference picture that was previously encoded and buffered in the video, the block in the current picture can be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case of using multiple reference pictures, the motion vector may have a third dimension that identifies the reference picture.

[0100] In some embodiments, bidirectional prediction techniques may be used for inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.

[0101] In addition, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0102] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, a picture in a video picture sequence is divided into coding tree units (CTUs) for compression, and the CTUs in the picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), namely a luminance CTB and two chrominance CTBs. Each CTU can be recursively divided into one or more coding units (CUs) in a quadtree. For example, a 64×64 pixel CTU can be divided into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In the example, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. Depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in the codec (encoding / decoding) is performed in units of prediction blocks. Using the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values) such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0103] Figure 7 An exemplary diagram of a video encoder (703) is shown. The video encoder (703) is configured to receive a processed block (e.g., a prediction block) of sample values ​​within a current video picture in a sequence of video pictures, and encode the processed block into an encoded picture that is part of an encoded video sequence. In an example, the video encoder (703) replaces Figure 4 The video encoder (403) in the example is used.

[0104] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​of a processing block, such as a prediction block of 8×8 samples. The video encoder (703) uses, for example, rate-distortion optimization to determine whether to best encode the processing block using intra mode, inter mode, or bidirectional prediction mode. In the case where the processing block is to be encoded in intra mode, the video encoder (703) may encode the processing block into a coded picture using intra prediction techniques; and in the case where the processing block is to be encoded in inter mode or bidirectional prediction mode, the video encoder (703) may encode the processing block into a coded picture using inter prediction or bidirectional prediction techniques, respectively. In some video coding techniques, the merge mode may be an inter-picture prediction submode, where the motion vector is derived from the predictor without the aid of an encoded motion vector component external to one or more motion vector predictors. In some other video coding techniques, there may be a motion vector component applicable to the object block. In the example, the video encoder (703) includes other components, such as a mode decision module (not shown) that determines the mode of the processing block.

[0105] exist Figure 7 In the example of FIG. 7 , the video encoder ( 703 ) includes Figure 7 An inter-frame encoder (730), an intra-frame encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), an overall controller (721), and an entropy encoder (725) are shown coupled together.

[0106] The inter-frame encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-frame prediction information (e.g., description of redundant information, motion vectors, merge mode information according to an inter-frame coding technique), and calculate an inter-frame prediction result (e.g., a prediction block) based on the inter-frame prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the encoded video information.

[0107] The intra encoder (722) is configured to: receive samples of a current block (e.g., a processing block); compare the block with an already encoded block in the same picture in some cases; generate quantization coefficients after transformation; and in some cases also generate intra prediction information (e.g., generate intra prediction direction information according to one or more intra coding techniques). In an example, the intra encoder (722) also calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same picture.

[0108] The overall controller (721) is configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In an example, the overall controller (721) determines a mode of a block and provides a control signal to a switch (726) based on the mode. For example, when the mode is an intra-frame mode, the overall controller (721) controls the switch (726) to select an intra-frame mode result for use by a residual calculator (723), and controls the entropy encoder (725) to select intra-frame prediction information and include the intra-frame prediction information in a bitstream; and when the mode is an inter-frame mode, the overall controller (721) controls the switch (726) to select an inter-frame prediction result for use by a residual calculator (723), and controls the entropy encoder (725) to select inter-frame prediction information and include the inter-frame prediction information in a bitstream.

[0109] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (722) or the inter encoder (730). The residual encoder (724) is configured to operate based on the residual data to encode the residual data to generate a transform coefficient. In an example, the residual encoder (724) is configured to convert the residual data from the spatial domain to the frequency domain and generate a transform coefficient. Then, the transform coefficient is subjected to quantization to obtain a quantized transform coefficient. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) can generate a decoded block based on the decoded residual data and the inter prediction information, and the intra encoder (722) can generate a decoded block based on the decoded residual data and the intra prediction information. In some examples, the decoded blocks are processed appropriately to generate decoded pictures, and these decoded pictures can be buffered in memory circuits (not shown) and used as reference pictures.

[0110] The entropy encoder (725) is configured to format the bitstream to include the coded blocks. The entropy encoder (725) is configured to include various information in the bitstream according to a suitable standard, such as the HEVC standard. In an example, the entropy encoder (725) is configured to include overall control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Note that according to the disclosed subject matter, when the block is encoded in the inter-frame mode or the merge sub-mode of the bidirectional prediction mode, there is no residual information.

[0111] Figure 8An exemplary diagram of a video decoder (810) is shown. The video decoder (810) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In the example, the video decoder (810) replaces Figure 4 The video decoder (410) in the example uses.

[0112] exist Figure 8 In the example of FIG. 8 , the video decoder ( 810 ) includes Figure 8 An entropy decoder (871), an inter-frame decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-frame decoder (872) are shown coupled together.

[0113] The entropy decoder (871) may be configured to reconstruct certain symbols from the coded picture, which represent the syntax elements constituting the coded picture. Such symbols may include, for example, a mode for encoding a block (e.g., an intra-frame mode, an inter-frame mode, a bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), and prediction information (e.g., intra-frame prediction information or inter-frame prediction information) of certain samples or metadata that may be identified for prediction by an intra-frame decoder (872) or an inter-frame decoder (880). The symbol may also include, for example, residual information in the form of quantized transform coefficients, etc. In the example, when the prediction mode is an inter-frame mode or a bidirectional prediction mode, the inter-frame prediction information is provided to the inter-frame decoder (880); and when the prediction type is an intra-frame prediction type, the intra-frame prediction information is provided to the intra-frame decoder (872). The residual information may be subjected to inverse quantization and provided to the residual decoder (873).

[0114] The inter-frame decoder (880) is configured to receive the inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information.

[0115] The intra decoder (872) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0116] The residual decoder (873) is configured to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert residual information from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include quantizer parameters (Quantizer Parameter, QP)), and the information may be provided by the entropy decoder (871) (data path is not depicted because this may only be a small amount of control information).

[0117] The reconstruction module (874) is configured to combine the residual information output by the residual decoder (873) with the prediction result (output by the inter-frame prediction module or the intra-frame prediction module as appropriate) in the spatial domain to form a reconstructed block, which can be part of a reconstructed picture, which can be part of a reconstructed video. Note that other suitable operations such as deblocking operations can be performed to improve visual quality.

[0118] Note that the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using any suitable technology. In an embodiment, the video encoders (403), (603) and (703) and the video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603) and (603) and the video decoders (410), (510) and (810) may be implemented using one or more processors that execute software instructions.

[0119] Various inter prediction modes can be used in VVC. For an inter-predicted CU, motion parameters may include (one or more) MVs, one or more reference picture indexes, a reference picture list usage index, and additional information of certain coding features to be used for inter-prediction sample generation. Motion parameters may be signaled explicitly or implicitly. When a CU is encoded in skip mode, the CU may be associated with a PU and may not have valid residual coefficients, may not have coded motion vector increments or MV differences (e.g., MVDs), or reference picture indexes. A merge mode may be specified in which the motion parameters of the current CU are obtained from adjacent (one or more) CUs, including spatial candidates and / or temporal candidates, and optional additional information, such as that introduced in VVC. The merge mode may be applied to inter-predicted CUs, not just to skip mode. In the example, an alternative to the merge mode is the explicit transmission of motion parameters, in which (one or more) MVs, corresponding reference picture indexes for each reference picture list, and reference picture list usage flags and other information are explicitly signaled according to the CU.

[0120] In an embodiment, for example in VVC, the VVC Test model (VTM) reference software includes one or more refined inter-frame prediction coding tools, and the refined inter-frame prediction coding tools include: extended merge prediction, merge motion vector difference (MMVD) mode, adaptive motion vector prediction (AMVP) mode with symmetric MVD signaling, affine motion compensation prediction, subblock-based temporal motion vector prediction (SbTMVP), adaptive motion vector resolution (AMVR), motion field storage (1 / 16 luminance sample MV storage and 8×8 motion field compression), bi-prediction with CU-level weight (BCW), bi-directional optical flow (BDOF), prediction refinement using optical flow (PROF), decoder side motion vector refinement (DEC) refinement, DMVR), combined inter-frame and intra-frame prediction (CIIP), geometric partitioning mode (GPM), etc. The inter-frame prediction and related methods are described in detail below.

[0121] In some examples, extended merge prediction can be used. In an example, such as in VTM4, a merge candidate list is constructed by including the following five types of candidates in order: spatial motion vector predictor (MVP) from spatially adjacent CUs, temporal MVP from co-located CUs, history-based MVP from a first-in-first-out (FIFO) table, pairwise average MVP, and zero MV.

[0122] The size of the merge candidate list can be signaled in the slice header. In an example, in VTM4, the maximum allowed size of the merge candidate list is 6. For each CU encoded in merge mode, the index of the best merge candidate (e.g., merge index) can be encoded using truncated unary binarization (TU). The first bin of the merge index can be encoded with context (e.g., context-adaptive binary arithmetic coding (CABAC)), and bypass coding can be used for other bins.

[0123] Some examples of the generation process of different categories of merge candidates are provided below. In an embodiment, the spatial candidates are derived as follows. For example, the derivation of spatial merge candidates in VVC can be the same as the derivation of spatial merge candidates in HEVC. In the example, at Fig. 9 Select up to four merge candidates from the candidates for the positions depicted in . Fig. 9 FIG. 2 shows the positions of spatial merging candidates according to an embodiment of the present disclosure. Fig. 9 , the resulting order is B1, A1, B0, A0 and B2. Position B2 is considered only when any CU at positions A0, B0, B1 and A1 is unavailable (for example, because the CU belongs to another slice or another tile) or is intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates is subject to redundancy check, which ensures that candidates with the same motion information are excluded from the candidate list, thereby improving coding efficiency.

[0124] In order to reduce computational complexity, in some embodiments, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only certain pairs are considered, such as Fig.10 The pairs linked by arrows in , and the candidate is added to the candidate list only when the corresponding candidate for redundancy check does not have the same motion information. Fig.10 FIG. 4 shows candidate pairs considered for redundancy check of spatial merging candidates according to an embodiment of the present disclosure. Fig.10 , pairs linked by corresponding arrows include A1 and B1, A1 and A0, A1 and B2, B1 and B0, and B1 and B2. Thus, candidates at positions B1, A0, and / or B2 can be compared with candidates at position A1, and candidates at positions B0 and / or B2 can be compared with candidates at position B1.

[0125] In an embodiment, the temporal candidates are derived as follows: In the example, only one temporal merging candidate is added to the candidate list. Fig.11An example of motion vector scaling for temporal merge candidates is shown. To derive a temporal merge candidate for a current CU (1111) in a current picture (1101), a scaled MV (1121) may be derived based on a co-located CU (1112) belonging to a co-located reference picture (1104) (e.g., by Fig.11 The reference picture list used to derive the co-located CU (1112) may be explicitly signaled in the slice header. Fig.11 As shown by the dotted line in , a scaled MV (1121) of the temporal merge candidate can be obtained. The scaled MV (1121) can be scaled from the MV of the co-located CU (1112) using the picture order count (POC) distances tb and td. The POC distance tb can be defined as the POC difference between the current reference picture (1102) of the current picture (1101) and the current picture (1101). The POC distance td can be defined as the POC difference between the co-located reference picture (1104) of the co-located picture (1103) and the co-located picture (1103). The reference picture index of the temporal merge candidate can be set to zero.

[0126] Fig.12 Exemplary candidate positions (e.g., C0 and C1) of the temporal merge candidate for the current CU are shown. The position of the temporal merge candidate can be selected between candidate positions C0 and C1. Candidate position C0 is positioned at the lower right corner of the co-located CU (1210) of the current CU. Candidate position C1 is positioned at the center of the co-located CU (1210) of the current CU (1210). If the CU at candidate position C0 is not available, is intra-coded, or is outside the current row of the CTU, candidate position C1 is used to derive the temporal merge candidate. Otherwise, for example, if the CU at candidate position C0 is available, is intra-coded, and is in the current row of the CTU, candidate position C0 is used to derive the temporal merge candidate.

[0127] In some examples, the translational motion model is applied to motion compensation prediction (MCP). However, the translational motion model may not be suitable for modeling other types of motion (e.g., zooming in / out, rotation, perspective motion, and other irregular motions). In some embodiments, block-based affine transformation motion compensation prediction is applied. Fig.13A In , when a 4-parameter affine model is used, the affine motion field of the block is described by two control point motion vectors CPMV0 and CPMV1 of two control points (CP) CP0 and CP1. Fig. 13BIn , when a 6-parameter affine model is used, the affine motion field of the block is described by three CPMVs (CPMV0, CPMV1, and CPMV3) of CPs (CP0, CP1, and CP2).

[0128] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is given by:

[0129]

[0130] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is obtained as:

[0131]

[0132] In equations 1 to 2, (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right control point, and (mv 2x ,mv 2y ) is the motion vector of the lower left control point. In addition, the coordinates (x, y) are relative to the upper left corner of the corresponding block, and W and H represent the width and height of the corresponding block.

[0133] In order to simplify motion compensation prediction, in some embodiments, sub-block based affine transformation prediction is applied. Fig.14 In this paper, a 4-parameter affine motion model is used and two CPMVs are determined. and To derive the motion vector of each 4×4 (sample) luma subblock (1402) partitioned from the current block (1410), the motion vector (1401) of the center sample of each subblock (1402) is calculated according to Equation 1 and rounded to 1 / 16 fractional precision. Then, a motion compensated interpolation filter is applied to generate a prediction of each subblock (1402) using the derived motion vector (1401). The subblock size of the chroma component is set to 4×4. The MV of a 4×4 chroma subblock is calculated as the average of the MVs of the four corresponding 4×4 luma subblocks.

[0134] Similar to translational motion inter prediction, two affine motion inter prediction modes are adopted in some embodiments, namely, affine merge mode and affine AMVP mode.

[0135] In some embodiments, an affine merge mode may be applied to a CU whose width and height are both greater than or equal to 8. An affine merge candidate for the current CU may be generated based on motion information of spatially neighboring CUs. There may be up to five affine merge candidates, and an index is signaled to indicate which affine merge candidate to use for the current CU. For example, the following three types of affine merge candidates are used to form an affine merge candidate list:

[0136] (i) Inherited affine merge candidates inferred from the CPMV of neighboring CUs;

[0137] (ii) constructing affine merge candidates using the translated MVs of neighboring CUs; ​​and

[0138] (iii) Zero MV.

[0139] In some embodiments, there may be up to two inherited affine candidates derived from the affine motion models of neighboring blocks, one affine candidate from the left neighboring CU and one affine candidate from the top neighboring CU. For example, the candidate block may be located at Fig. 9 The positions shown in . For the left predictor, the scan order is A0>A1, while for the upper predictor, the scan order is B0>B1>B2. Only the first successor candidate from each side is selected. No pruning check is performed between two successor candidates.

[0140] When an adjacent affine CU is identified, the CPMV of the identified adjacent affine CU is used to derive the CPMV candidates in the affine merge list of the current CU. Fig.15 As shown, the adjacent lower left block A of the current CU (1510) is encoded in affine mode. The motion vectors of the upper left corner, upper right corner, and lower left corner of the CU (1520) containing block A are and When encoding block A using a 4-parameter affine model, according to and Calculate the two CPMVs of the current CU (1510) and When encoding block A with a 6-parameter affine model, according to and Three CPMVs of the current CU are calculated (not shown).

[0141] Affine candidates are constructed by combining the adjacent translational motion information of each control point. The motion information of the control point is obtained from Fig.16CPMVk (k=1, 2, 3, 4) represents the kth control point. For CPMV1, B2>B3>A2 blocks are checked in order, and the MV of the first available block is used. For CPMV2, B1>B0 blocks are checked, and for CPMV3, A1>A0 blocks are checked. If available, the TMVP at block T is used as CPMV4.

[0142] After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used in order: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}.

[0143] Combinations of 3 CPMVs construct 6-parameter affine merge candidates, while combinations of 2 CPMVs construct 4-parameter affine merge candidates. To avoid the motion scaling process, the relevant combinations of control point MVs are discarded if the reference indices of the control points are different.

[0144] After checking both the inherited affine merge candidates and the constructed affine merge candidates, if the list is still not full, a zero MV is inserted to the end of the merge candidate list.

[0145] In some embodiments, affine AMVP mode can be applied to CUs with width and height greater than or equal to 16. A CU-level affine flag is signaled in the bitstream to indicate whether to use affine AMVP mode, and then another flag is signaled to indicate whether to use 4-parameter affine or 6-parameter affine. The difference between the CPMV of the current CU and their predictors is signaled in the bitstream. The affine AVMP candidate list size is 2 and can be generated by using the following four types of CPVM candidates in order:

[0146] (i) Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs;

[0147] (ii) constructing affine AMVP candidates using the translated MVs of neighboring CUs; ​​and

[0148] (iii) the translated MV from the neighboring CU; and

[0149] (iv) Zero MV.

[0150] The order of checking the inherited affine AMVP candidates is similar to that of checking the inherited affine merge candidates in the example. The difference is that for AVMP candidates, affine CUs with the same reference picture as in the current block are considered. When the inherited affine motion predictor is inserted into the candidate list, the pruning process is not applied.

[0151] from Fig.16 The AMVP candidate is constructed by deriving the specified spatial neighbors shown in . The same check order as that performed in the affine merge candidate construction is used. In addition, the reference picture index of the neighboring blocks is also checked. The first block in the check order that is inter-coded and has the same reference picture as the current CU is used. When the current CU is encoded with a 4-parameter affine model and both CPMV0 and CPMV1 are available, the available CPMV is added as a candidate in the affine AMVP list. When the current CU is encoded with a 6-parameter affine model and all three CPMVs (CPMV0, CPMV1, and CPMV2) are available, the available CPMV is added as a candidate in the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable.

[0152] If there are still less than 2 candidates in the affine AMVP list after checking the inherited affine AMVP candidates and the constructed AMVP candidates, the translation motion vectors adjacent to the control points (if available) are added to predict all control point MVs of the current CU. Finally, if the affine AMVP list is still not full, the affine AMVP list is filled with zero MVs.

[0153] In an embodiment, an affine merge with motion vector difference (affine MMVD) mode may be used. An available affine merge candidate may be selected from a sub-block based merge list as a base predictor. A motion vector offset of the motion vector value of each control point with the motion vector of the base predictor may be applied. If no affine merge candidate is available, affine MMVD is not used. If affine MMVD is used, the distance index and offset direction index may then be signaled.

[0154] A distance index (IDX) may be signaled to indicate which distance offset from an offset table such as shown in Table 1 to use.

[0155] Table 1. Default distance offset table

[0156] Distance from IDX 0 1 2 3 4 Distance Offset 1 / 2 pixel 1 pixel 2 pixels 4 pixels 8 pixels

[0157] The direction index may represent four directions as shown in Table 2, where only the x-direction or the y-direction may have an MV difference, rather than having an MV difference in both directions.

[0158] Table 2. Default offset direction table

[0159] Offset direction IDX 00 01 10 11 x-direction factor +1 –1 0 0 Y-direction factor 0 0 +1 –1

[0160] If the inter prediction is unidirectional, a signaled distance offset may be applied in the offset direction of each control point predictor. The result may be the MV value for each control point.

[0161] If the inter prediction is bidirectional, the signaled distance offset may be applied in the signaled offset direction of the L0 motion vector of the control point predictor, and the offset to be applied on the L1 MV is applied on a mirroring or scaling basis, as described below.

[0162] If the inter prediction is bi-directional, the signaled distance offset is applied in the signaled offset direction of the L0 motion vector of the control point predictor. For L1 CPMV, the offset may be applied on a mirrored basis, where the same amount of distance offset is applied with opposite direction.

[0163] In an embodiment, an offset mirroring method based on POC distance may be used for bidirectional prediction. When the base candidate is bidirectionally predicted, the offset applied to L0 is signaled, while the offset on L1 may depend on the temporal position of the reference pictures on list 0 and list 1. If both reference pictures are on the same temporal side of the current picture, the same distance offset and the same offset direction may be applied to the CPMV of L0 and L1. When the two reference pictures are on different sides of the current picture, the CPMV of L1 may have a distance offset applied in the opposite offset direction.

[0164] In an embodiment, an offset scaling method based on POC distance is used for bi-directional prediction. When the base candidate is bi-directionally predicted, the offset applied to L0 is signaled, while the offset applied to L1 can be scaled based on the temporal distance of the reference pictures on list 0 and list 1.

[0165] In an embodiment, the range of distance offset values ​​is extended to three offset tables, such as the three different entries shown in Table 3. Each offset table (e.g., each row in Table 3) is adaptively selected based on the picture resolution. In the example, the offset table is selected based on the picture resolution. Referring to Table 3, when the picture height is greater than or equal to 1080, the first offset table (e.g., "distance offset 1") is selected. When the picture height is less than 1080 and greater than or equal to 720, the second offset table (e.g., "distance offset 2") is selected. When the picture height is less than 720, the third offset table (e.g., "distance offset 3") is selected.

[0166] Table 3. Extended distance offset table

[0167]

[0168] Local illumination compensation (LIC) is a prediction technique that models local illumination changes between a current block and a predictor (also referred to as a predictor block) of the current block by using a linear function, such as an inter-frame prediction technique. LIC can be applied to unidirectional prediction mode, but is not limited to unidirectional prediction mode. The predictor can be determined based on a reference block of the current block. The reference block is in a reference picture. In an example, for example in unidirectional prediction mode, the MV of the current block points from the current block to the reference block, and the predictor is the reference block of the current block. In an example, a function such as the linear function shown in Equation 3 is used to modify the value p of a sample (e.g., a reference sample) in the predictor (e.g., a reference block). i [x,y], to determine the updated value p of the sample in the updated predictor f [x,y].

[0169] p f [x, y] = α × p i [x, y] + β Equation 3

[0170] The parameters of the linear function can be represented by a scaling factor (also called a scale) α and an offset β to compensate for lighting changes. i [x, y] may be a reference sample at a certain position [x, y] on a reference picture, and the reference sample may be pointed to by the MV of the current block. The scaling factor α and the offset β may be derived based on a current template of the current block (also referred to as a current block template) and a reference template of the reference block (also referred to as a reference block template) using any suitable method (e.g., least squares method). In various embodiments, the scaling factor α and the offset β are not signaled, so that the scaling factor α and the offset β do not require signaling overhead. In an example, a LIC flag is signaled to indicate the use of LIC. LIC may be used under any suitable standard, such as under VVC and under standards other than VVC.

[0171] The current block may be encoded based on the updated predictor, for example, in merge mode, skip mode, etc. The current block may be encoded based on the updated predictor and additional information (eg, residual data) in AMVP mode.

[0172] In the related art, the LIC only uses a flag (eg, a LIC flag) to indicate whether the LIC is enabled, and does not have parameter adjustment for improving the accuracy of illumination compensation when the LIC is enabled.

[0173] According to aspects of the present disclosure, one or more parameters of a parameter set of a LIC (e.g., including a scaling factor α and an offset β) may be adjusted. One or more parameters may be adjusted using a single index or multiple indexes. In an example, the scaling factor α and / or the offset β are adjusted by using one or more indexes.

[0174] Fig.17 An exemplary LIC with parameter adjustment is shown. A current block (1701) in a current picture has a current template (also referred to as a neighboring reconstructed template) (1721). The current template (1721) can have any suitable shape and any suitable size. The current template (1721) can include samples (e.g., reconstructed samples) in neighboring reconstructed blocks of the current block (1701).

[0175] exist Fig.17 In the example shown in , the current template (1721) of the current block (1701) includes a top template (1722) and a left template (1723). Each of the top template (1722) and the left template (1723) can have any suitable shape and any suitable size. The top template (1722) can include samples in one or more top adjacent blocks of the current block (1701). In the example, the top template (1722) includes four rows of samples in one or more top adjacent blocks of the current block (1701). The left template (1723) can include samples in one or more left adjacent blocks of the current block (1701). In the example, the left template (1723) includes four columns of samples in one or more left adjacent blocks of the current block (1701).

[0176] In the example, the current template of the current block (1701) includes only the top template (1722) or only the left template (1723). In the example, the current template of the current block (1701) includes the top template (1722), the left template (1723) and the upper left template (1731).

[0177] The MV (1702) of the current block (1701) may point to a reference block (1703) in a reference picture. The reference block (1703) may have a reference template corresponding to the current template (1721), such as a reference template (1725).

[0178] The reference template (1725) may have the same shape and the same size as those of the current template, respectively. Fig.17 In the example in FIG. 1 , the reference template (1725) of the reference block (1703) includes a top template (1726) and a left template (1727). The top template (1726) corresponding to the top template (1722) may include samples from one or more top adjacent blocks of the reference block (1703). The left template (1727) corresponding to the left template (1723) may include samples from one or more left adjacent blocks of the reference block (1703).

[0179] The predictor of the current block (1701) may be determined based on the reference block (1703). In an example, such as in a unidirectional prediction mode, the predictor is the reference block (1703), such as having sample values ​​of the predictor equal to corresponding sample values ​​of the reference block (1703).

[0180] The LIC for compensating for local illumination changes may be used to modify the sample values ​​in the predictor of the current block (1701). According to an embodiment of the present disclosure, the updated value p of the sample (eg, reference sample) in the predictor is f [x,y] may be based on a linear function of the values ​​pi[x,y] of the samples in the predictor, such as shown in Equation 4.

[0181] p f [, y] = (α + μ1) × p i [x,y]+(β+μ2|×T avg ) Equation 4

[0182] Equation 4 can be written as Equation 5 below.

[0183] p f [x, y] = α f ×p i [x,y]+β f Equation 5

[0184] In Equation 5, the final scaling factor α f is equal to the sum of the scaling factor α and the first parameter μ1, and the final offset β f Based on the offset β and the second parameter μ2. In the example, the final offset β f =(β+μ2×T avg ). Equation 5 shows that the final scaling factor α can be f , final offset β f and predictor (eg, reference block (1703)) to determine an updated predictor.

[0185] When the first parameter μ1 and the second parameter μ2 in Equation 4 are zero, parameter adjustment is not applied to the LIC.

[0186] As described above, the scaling factor α and the offset β can be determined based on the current template (e.g., current template (1721)) of the current block (1701) and the reference template (e.g., reference template (1725)) of the reference block (1703) by using any suitable method (e.g., least squares method).

[0187] Parameter T avgIt can be determined based on the current template (e.g., current template (1721)) and / or the reference template (e.g., reference template (1725)) of the current block (1701). In the example, the parameter T avg is the average value of the reference template (e.g., reference template (1725)). For example, parameter T avg is the average of the sample values ​​of the entire reference template (eg, reference template (1725)) or a subset thereof. In the example, parameter T avg is the average value of the current template (e.g., current template (1721)). For example, parameter T avg is the average of the sample values ​​of the entire current template or its subset.

[0188] In the example, it is used to determine the parameter T avg The reference template used to determine the scaling factor α and the offset β is different from the reference template used to determine the parameter T avg The current template of is different from the current template used to determine the scaling factor α and the offset β.

[0189] In the example, the parameter T avg is 1, and thus the final offset β f is equal to (β+μ2). Therefore, the updated value p f [x,y] equals (α+μ1)×p i [x, y]+(β+μ2).

[0190] The first parameter μ1 and / or the second parameter μ2 may be signaled, for example indicated by LIC information of one or more blocks including the current block ( 1701 ). In an embodiment, the LIC information of one or more blocks is signaled in the video bitstream.

[0191] The first parameter μ1 and / or the second parameter μ2 may be signaled within a limited range of values ​​(e.g., from -5 to +5). In an example, the first parameter μ1 is signaled within a limited positive range or a limited negative range. In an example, the second parameter μ2 is signaled within a limited positive range or a limited negative range.

[0192] In an embodiment, the LIC information of one or more blocks indicates one of the first parameter μ1 and the second parameter μ2, and the other of the first parameter μ2 and the second parameter μ2 may be determined based on the one of the first parameter μ1 and the second parameter μ2.

[0193] In an example, μ2=-μ1. LIC information of one or more blocks indicates a first parameter μ1, and a second parameter μ2 is determined to be -μ1 according to the first parameter μ1. In an example, LIC information of one or more blocks indicates a second parameter μ2, and the first parameter μ1 is determined to be -μ2 according to the second parameter μ2.

[0194] When the relationship between the first parameter μ1 and the second parameter μ2 (e.g., μ2=-μ1) is known, for example, for a decoder, information indicating only one of the first parameter μ1 and the second parameter μ2 (e.g., LIC information) is signaled in the bitstream. For example, μ2=-μ1, LIC information indicating only the first parameter μ1 is signaled in the bitstream. In the example, only the first parameter μ1 is signaled in the bitstream.

[0195] In an embodiment, one of (i) the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and one of (i) the first parameter μ1 and (ii) the second parameter μ2 is not signaled. In an embodiment, the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2. For example, the first parameter μ1 is equal to zero, and the LIC information of one or more blocks does not indicate the first parameter μ1. In an example, the first parameter μ1 is equal to zero and is not signaled in the bitstream. In an example, the second parameter μ2 is equal to zero and is not signaled in the bitstream.

[0196] The LIC information of the one or more blocks may include (i) the first parameter μ1 and / or the second parameter μ2 or (ii) one or more indexes indicating the first parameter μ1 and / or the second parameter μ2.

[0197] The first parameter μ1 may be a floating point value and may be quantized with a first plurality of bits. The LIC information of the one or more blocks may include the first plurality of bits. The second parameter μ2 may be a floating point value and may be quantized with a second plurality of bits. The LIC information of the one or more blocks may include the second plurality of bits.

[0198] The LIC information of the one or more blocks may include at least one index indicating at least one of the first parameter μ1 or the second parameter μ2. The first parameter μ1 and / or the second parameter μ2 may be signaled by one or more indexes pointing to a lookup table.

[0199] In an example, the lookup table indicates a relationship between an index and a corresponding parameter pair (eg, a first parameter μ1 and a second parameter μ2), and the LIC information includes a single index indicating the first parameter μ1 and the second parameter μ2.

[0200] In an example, the first lookup table indicates a first relationship between the first index and the first parameter μ1, the second lookup table indicates a second relationship between the second index and the second parameter μ2, and thus the LIC information includes a first index indicating the first parameter μ1 and a second index indicating the second parameter μ2. When the first parameter μ1 or the second parameter μ2 is zero, a single index is used to indicate the first parameter μ1 or the second parameter μ2 that is not zero. When the relationship between the first parameter μ1 and the second parameter μ2 (e.g., μ2=-μ1) is known, a single index may be used to indicate the first parameter μ1 or the second parameter μ2.

[0201] LIC information of one or more blocks may be signaled at any suitable level, such as a coding unit (CU) level or a higher level than the CU level (eg, a CTU level, a slice level, etc.).

[0202] In an example, LIC information of one or more blocks is signaled at the CU level, the one or more blocks are in the same CU, and the first parameter μ1 and the second parameter μ2 are applicable to the CU. Different CUs may have different first parameters and / or different second parameters. For example, the first parameter μ1 and the second parameter μ2 of the first CU are different from the first parameter μ1 and the second parameter μ2 of the second CU, respectively.

[0203] In an example, LIC information of one or more blocks is signaled at a high level (e.g., a slice level). The one or more blocks include blocks in multiple CUs in a slice. The first parameter μ1 and the second parameter μ2 are applicable to multiple CUs in a slice. Multiple CUs in a slice may have the same first parameter μ1. Multiple CUs in a slice may have the same second parameter μ2.

[0204] In an example, one or more indexes indicating the first parameter μ1 and / or the second parameter μ2 are signaled at different levels (eg, CU level, CTU level, slice level, etc.).

[0205] When LIC operates at the sub-block level within the CU, for example, when LIC is performed at the sub-block level within the current block, the above embodiments may be appropriately adjusted. The current block may include multiple sub-blocks. Each sub-block may include one or more samples in the current block. For example, each of the multiple sub-blocks is associated with the MV of the corresponding sub-block. The MVs associated with the corresponding multiple sub-blocks may be different. Each of the MVs may point to a corresponding reference sub-block in a reference picture. Multiple sub-blocks may be predicted based on multiple reference sub-blocks associated with the corresponding MVs, respectively. In the example, multiple sub-blocks are predicted using an affine merge mode, an affine AMVP mode, an affine MMVD mode, or the like.

[0206] The plurality of subblocks may include a first subblock and a second subblock. LIC information indicating a first parameter μ1 and / or a second parameter μ2 of one or more blocks may be applied to one or more of the plurality of subblocks in the current block. The first parameter μ1 and the second parameter μ2 may be applied to the plurality of subblocks. For example, the first subblock and the second subblock have the same first parameter μ1, and the first subblock and the second subblock have the same second parameter μ2.

[0207] In an embodiment, each sub-block has a corresponding derived set of parameters used in LIC.

[0208] For the first subblock in the current block, Equation 4 can be rewritten as Equation 6. The updated value p for the first sample in the first predictor subblock (eg, the first reference subblock) is f1 [x1, y1] can be the value p of the first sample i1 A linear function of [x1, y1].

[0209] p f1 [x1, y1] = (α i1 +μ1)×p i1 [x1,y1]+(β i1 +μ2×T avg，1 ) Equation 6

[0210] The first MV of the first sub-block points to the first reference sub-block. The first initial scaling factor α i1 and the first initial offset β i1 Associated with the first sub-block to compensate for the local illumination variation of the first sub-block. Parameter T avg，1 The first initial scaling factor α may be associated with the first sub-block to compensate for local illumination variations of the first sub-block. The first initial scaling factor α may be determined based on the first portion of the current template and the first portion of the reference template. i1 and the first initial offset β i1 For example, the first part of the current template includes samples of the reconstructed spatial neighboring blocks of the first subblock. The first part of the reference template includes samples of the reconstructed spatial neighboring blocks of the first reference subblock. The parameter T may be determined based on the first part of the current template and / or the first part of the reference template. avg，1 .

[0211] For the second sub-block in the current block, Equation 4 can be rewritten as Equation 7. The updated value p of the second sample in the second predictor sub-block (eg, the second reference sub-block) in the predictor is f2 [x2, y2] may be the value p of the second sample i2 Linear function of [x2, y2].

[0212] p f2 [x2, y2] = (α i2 +μ1)×pi2 [x2,y2]+(β i2 +μ2×T avg，2 ) Equation 7

[0213] The second MV of the second sub-block points to the second reference sub-block. The second initial scaling factor α i2 and the second initial offset β i2 Associated with the second sub-block to compensate for the local illumination variation of the second sub-block. Parameter T avg，2 The second initial scaling factor α may be associated with the second sub-block to compensate for local illumination changes of the second sub-block. The second initial scaling factor α may be determined based on the second portion of the current template and the second portion of the reference template. i2 and the second initial offset β i2 For example, the second part of the current template includes samples of the reconstructed spatial neighboring blocks of the second subblock. The second part of the reference template includes samples of the reconstructed spatial neighboring blocks of the second reference subblock. The parameter T may be determined based on the second part of the current template and / or the second part of the reference template. avg，2 .

[0214] In an embodiment, multiple sub-blocks share the same derived parameter set (eg, including a first initial scaling factor α i1 , the first initial offset β i1 and / or parameter T avg，1 ). For example, the first initial scaling factor α i1 Equal to the second initial scaling factor α i2 , and the first initial offset β i1 Equal to the second initial offset β i2 In the example, the parameter T avg，1 Equal to parameter T avg，2 . A shared parameter set for multiple sub-blocks may be determined based on a current template and a reference template. In an example, a reference template may be determined based on boundary sub-blocks of a current block, which are neighbors of other blocks outside the current block. For example, an MV associated with a boundary sub-block may point to multiple sub-blocks in a reference block, and the reference template includes samples of reconstructed spatially neighboring blocks of the multiple sub-blocks in the reference block. In an example, the reference template includes a first portion of the reference template that may be determined based on a first MV and a second portion of the reference template that may be determined based on a second MV.

[0215] Fig.18A flow chart outlining an encoding process (1800) according to an embodiment of the present disclosure is shown. In various embodiments, the process (1800) is performed by a processing circuit system, such as a processing circuit system in a terminal device (310), (320), (330), and (340), a processing circuit system that performs the functions of a video encoder (e.g., (403), (603), (703), etc. In some embodiments, the process (1800) is implemented as software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1800). The process starts at (S1801) and proceeds to (S1810).

[0216] At (S1810), at least one of the first parameter μ1 and the second parameter μ2 of one or more blocks, an initial scaling factor α of the first sub-block in the current block, and i1 , and the initial offset β of the first sub-block i1 The one or more blocks may include a current block to be encoded using LIC. The first MV of the first subblock points to a first reference block of a reference block in a reference picture. The reference block corresponds to the current block.

[0217] The initial scaling factor α of the first subblock in the current block may be determined based on the first portion of the current template of the current block and the first portion of the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0218] In an example, an initial scaling factor α of the first sub-block in the current block may be determined based on a current template of the current block and a reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0219] The first parameter μ1 of the one or more blocks and / or the second parameter μ2 of the one or more blocks may be determined based on the one or more blocks.

[0220] In an example, a higher level than a CU (e.g., a CTU or a slice) includes a first block encoded with LIC and a second block not encoded with LIC. The first block may have the same first parameter μ1 and the same second parameter μ2, for example, determined based on one or more of the first block. Alternatively, the first block may have different first parameters μ1 and different second parameters μ2.

[0221] At (S1820), in an example, based on the first parameter μ1 and the initial scaling factor α i1 Determine the final scaling factor α for the first sub-block f1 For example, the final scaling factor α of the first sub-block in the current block is f1The first parameter μ1 of one or more blocks and the initial scaling factor α of the first sub-block in the current block are determined as i1 The sum of .

[0222] In the example, based on the second parameter μ2 and the initial offset β i1 Determine the final offset β of the first sub-block f1 For example, the parameter T of the first sub-block in the current block is determined based on at least one of the first part of the current template or the first part of the reference template. avg，1 The final offset β of the first sub-block in the current block f1 It is determined as (β i1 +μ2×T avg，1 ).

[0223] In an example, the parameter T may be determined based on the current template of the current block and the reference template of the reference block. avg，1 .

[0224] At (S1830), the final scaling factor α may be used to f1 , final offset β f1 and a first predictor subblock (eg, first reference subblock) in the predictor to determine an updated first predictor subblock (eg, updated first reference subblock) in the predictor (eg, reference block) corresponding to the current block.

[0225] At (S1840), the first subblock in the current block may be encoded based on the updated first predictor subblock.LIC information indicating the first parameter μ1 and / or the second parameter μ2 of one or more blocks may be encoded.

[0226] The encoded LIC information of one or more blocks may be signaled in the video bitstream.The LIC information of one or more blocks may be signaled at a coding unit (CU) level or at a level higher than the CU level.

[0227] In an example, one of (i) the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0228] In an example, the LIC information of the one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0229] In an example, at least one of the first parameter μ1 or the second parameter μ2 is a floating point value and is quantized with a plurality of bits, and the LIC information of the one or more blocks includes the plurality of bits.

[0230] Then, the process (1800) proceeds to (S1899) and ends.

[0231] The process (1800) can be appropriately adapted to various scenarios, and the steps in the process (1800) can be adjusted accordingly. One or more steps in the process (1800) can be adjusted, omitted, repeated, and / or combined. The process (1800) can be implemented in any suitable order. Additional steps can be added.

[0232] In an example, LIC information of one or more blocks indicates one of (i) a first parameter μ1 and (ii) a second parameter μ2, and the other of (i) the first parameter μ1 and (ii) the second parameter μ2 is determined based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0233] In an embodiment, the first subblock in the current block is the current block, the first predictor subblock in the predictor is the predictor (e.g., a reference block), the first portion of the current template is the current template, and the first portion of the reference template is the reference template. In an example, the updated predictor is determined using Equation 4 or Equation 5.

[0234] In an embodiment, the current block includes a first sub-block and a second sub-block, and at least one of the first parameter μ1 or the second parameter μ2 indicated in the LIC information of one or more blocks is applied to the first sub-block and the second sub-block. i2 and another initial offset β i2 Associated, the other initial scaling factor α i2 The initial scaling factor α of the first sub-block can be i1 The same or different, the other initial offset β i2 The initial offset β of the first sub-block can be i1 The same or different. An updated second predictor subblock (eg, an updated second reference subblock) in the predictor corresponding to the second subblock may be determined using Equation 7. The second subblock may be encoded based on the updated second predictor subblock.

[0235] Fig.19A flowchart outlining a decoding process (1900) according to an embodiment of the present disclosure is shown. In various embodiments, the process (1900) is performed by a processing circuit system, such as a processing circuit system in a terminal device (310), (320), (330) and (340), a processing circuit system that performs the functions of a video encoder (403), a processing circuit system that performs the functions of a video decoder (410), a processing circuit system that performs the functions of a video decoder (510), a processing circuit system that performs the functions of a video encoder (603), etc. In some embodiments, the process (1900) is implemented in software instructions, so when the processing circuit system executes the software instructions, the processing circuit system performs the process (1900). The process starts at (S1901) and proceeds to (S1910).

[0236] At (S1910), prediction information of one or more blocks may be decoded from the encoded video bitstream. The prediction information may indicate that local illumination compensation (LIC) is applied to the one or more blocks, and the prediction information includes LIC information of the one or more blocks, the one or more blocks including the current block to be reconstructed.

[0237] In an example, LIC information of one or more blocks is signaled at a coding unit (CU) level.

[0238] In an example, LIC information of one or more blocks is signaled at a level higher than the CU level.

[0239] At (S1920), a final scaling factor α of the first sub-block in the current block may be determined based on the LIC information. f1 and the final offset β of the first sub-block in the current block f1 .

[0240] In an embodiment, an initial scaling factor α of a first sub-block in the current block is determined based on a first portion of a current template of the current block and a first portion of a reference template of a reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0241] In an example, an initial scaling factor α of the first sub-block in the current block may be determined based on a current template of the current block and a reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0242] The LIC information of the one or more blocks may be signaled in the coded video bitstream and may indicate at least one of the first parameter μ1 of the one or more blocks or the second parameter μ2 of the one or more blocks. The initial scaling factor α of the first sub-block in the current block may be calculated based on the first parameter μ1 of the one or more blocks and the initial scaling factor α of the first sub-block in the current block. i1 To determine the final scaling factor α of the first sub-block in the current block f1 . It can be based on the second parameter μ2 of one or more blocks and the initial offset β of the first sub-block in the current block i1 To determine the final offset β of the first sub-block in the current block f1 .

[0243] In this example, the final scaling factor α of the first sub-block in the current block is f1 The first parameter μ1 of one or more blocks and the initial scaling factor α of the first sub-block in the current block are determined as i1 The sum of .

[0244] In an example, a parameter T of a first sub-block in a current block is determined based on at least one of a first portion of a current template or a first portion of a reference template. avg，1 , and the final offset β of the first sub-block in the current block f1 It is determined as (β i1 +μ2×T avg，1 ).

[0245] In an example, the parameter T may be determined based on the current template of the current block and the reference template of the reference block. avg，1 .

[0246] In an example, LIC information of one or more blocks indicates one of (i) a first parameter μ1 and (ii) a second parameter μ2, and the other of (i) the first parameter μ1 and (ii) the second parameter μ2 is determined based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0247] In an example, one of (i) the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0248] In an example, the LIC information of the one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0249] In an example, at least one of the first parameter μ1 or the second parameter μ2 is a floating point value and is quantized with a plurality of bits, and the LIC information of the one or more blocks includes the plurality of bits.

[0250] At (S1930), the final scaling factor α may be used to f1 , final offset β f1 The first predictor subblock in the predictor and the first predictor subblock in the predictor are used to determine an updated first predictor subblock in the predictor corresponding to the current block. As described above, the predictor may be determined based on the reference block. In an example, the predictor is a reference block, for example, the sample values ​​of the predictor are equal to the corresponding sample values ​​of the reference block. The updated value of the first sample in the first predictor subblock in the predictor may be equal to α f1 ×p i1 +β f1 . p i1 It may be the value of the first sample in the first predictor sub-block in the predictor.

[0251] At (S1940), a first sub-block in the current block may be reconstructed based on the updated first predictor sub-block in the predictor.

[0252] In an example, sample values ​​in a first sub-block in the current block are equal to corresponding sample values ​​in an updated first predictor sub-block in the predictor.

[0253] In an example, a first sub-block in the current block is reconstructed based on the updated first predictor sub-block in the predictor and additional information (eg, residual data).

[0254] The process (1900) can be appropriately adapted to various scenarios, and the steps in the process (1900) can be adjusted accordingly. One or more steps in the process (1900) can be adjusted, omitted, repeated, and / or combined. The process (1900) can be implemented in any suitable order. Additional steps can be added.

[0255] In an embodiment, the first subblock in the current block is the current block, the first predictor subblock in the predictor is the predictor (e.g., a reference block of the current block), the first portion of the current template is the current template, and the first portion of the reference template is the reference template of the reference block. In an example, the updated predictor is determined using Equation 4 or Equation 5.

[0256] In an embodiment, the current block includes a first sub-block and a second sub-block, and at least one of the first parameter μ1 or the second parameter μ2 indicated in the LIC information of one or more blocks is applied to the first sub-block and the second sub-block. i2 and another initial offset β i2 Associated, the other initial scaling factor α i2 The initial scaling factor α of the first sub-block can be i1 The same or different, the other initial offset β i2 The initial offset β of the first sub-block can be i1The same or different. An updated second predictor subblock in the predictor corresponding to the second subblock may be determined using Equation 7. The second subblock may be encoded based on the updated second predictor subblock.

[0257] The embodiments of the present disclosure may be used alone or in combination in any order. In addition, each of the method (or embodiment), encoder, and decoder may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0258] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig. 20 A computer system (2000) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0259] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, linking, etc. to create code comprising instructions that may be executed directly by one or more computer central processing units (CPU), graphics processing units (GPU), etc., or through interpretation, microcode execution, etc.

[0260] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0261] Fig. 20 The components for the computer system (2000) shown in the example are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the exemplary embodiment of the computer system (2000).

[0262] The computer system (2000) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to inputs implemented by one or more human users through, for example, tactile inputs (e.g., keystrokes, swipes, data glove movements), audio inputs (e.g., voice, tapping), visual inputs (e.g., gestures), olfactory inputs (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to intentional human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0263] The input human-machine interface devices may include one or more of the following (only one of each is depicted): keyboard (2001), mouse (2002), trackpad (2003), touch screen (2010), data gloves (not shown), joystick (2005), microphone (2006), scanner (2007) and camera (2008).

[0264] The computer system (2000) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include: tactile output devices (e.g., tactile feedback through a touch screen (2010), a data glove (not shown), or a joystick (2005), but there may also be tactile feedback devices that are not used as input devices); audio output devices (e.g., speakers (2009), headphones (not depicted)); visual output devices (e.g., screens (2010), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be able to output two-dimensional visual output or more than three-dimensional output through methods such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted)); and printers (not depicted).

[0265] The computer system (2000) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (2020) with CD / DVD and other media (2021), thumb drives (2022), removable hard drives or solid-state drives (2023), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), etc.

[0266] Those skilled in the art will also appreciate that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0267] The computer system (2000) may also include an interface (2054) to one or more communication networks (2055). The network may be, for example, wireless, wired, optical. The network may also be a local area network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a delay-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (2049) (e.g., a USB port of the computer system (2000)); other networks are typically integrated into the core of the computer system (2000) by attaching to a system bus as described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smart phone computer system via a cellular network interface). The computer system (2000) can communicate with other entities by using any of these networks. Such communications may be one-way receive-only (e.g., broadcast television), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, such as to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used on each of these networks and network interfaces as described above.

[0268] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core ( 2040 ) of the computer system ( 2000 ).

[0269] The core (2040) may include one or more central processing units (CPUs) (2041), graphics processing units (GPUs) (2042), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (2043), hardware accelerators (2044) for certain tasks, graphics adapters (2050), etc. These devices as well as read-only memory (ROM) (2045), random access memory (2046), internal large-capacity storage devices (2047) such as internal non-user accessible hard drives, SSDs, etc. can be connected through a system bus (2048). In some computer systems, the system bus (2048) can be accessed in the form of one or more physical plugs to achieve expansion through additional CPUs, GPUs, etc. Peripheral devices can be attached to the system bus (2048) of the core directly or through a peripheral bus (2049). In an example, a screen (2010) can be connected to a graphics adapter (2050). The architecture of the peripheral bus includes PCI, USB, etc.

[0270] The CPU (2041), GPU (2042), FPGA (2043) and accelerator (2044) can execute certain instructions, which can be combined to form the computer code mentioned above. The computer code can be stored in ROM (2045) or RAM (2046). Transition data can be stored in RAM (2046), while permanent data can be stored in, for example, an internal mass storage device (2047). Fast storage and retrieval of any storage device in the storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2041), GPUs (2042), mass storage devices (2047), ROMs (2045), RAMs (2046), etc.

[0271] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or the medium and computer code may be of a type well known and available to those skilled in the art of computer software.

[0272] As an example and not limitation, a computer system (2000) having an architecture, in particular a core (2040), can provide functions provided as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, and certain storage devices of the core (2040) having a non-transitory nature, such as a mass storage device (2047) or ROM (2045) inside the core. Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (2040). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the core (2040) - in particular the processor therein (including a CPU, GPU, FPGA, etc.) - to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in the RAM (2046) and modifying such a data structure according to a process defined by the software. Additionally or alternatively, the computer system may provide functionality due to logic hardwired or otherwise embodied in circuits (e.g., accelerator (2044)) that may operate in place of or in conjunction with software to perform specific processing or specific portions of specific processing described herein. Where appropriate, references to software may include logic, and conversely references to logic may include software. Where appropriate, references to computer-readable media may include circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits implementing logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0273] Appendix A: Acronyms

[0274] JEM: Joint Exploration Model

[0275] VVC: Versatile Video Coding

[0276] BMS: Benchmark Set

[0277] MV: Motion Vector

[0278] HEVC: High Efficiency Video Coding

[0279] SEI: Supplemental Enhancement Information

[0280] VUI: Video Availability Information

[0281] GOP: Group of Pictures

[0282] TU: Transform Unit

[0283] PU: prediction unit

[0284] CTU: Coding Tree Unit

[0285] CTB: Coding Tree Block

[0286] PB: prediction block

[0287] HRD: Hypothetical Reference Decoder

[0288] SNR: Signal to Noise Ratio

[0289] CPU: Central Processing Unit

[0290] GPU: Graphics Processing Unit

[0291] CRT: cathode ray tube

[0292] LCD: Liquid Crystal Display

[0293] OLED: Organic Light Emitting Diode

[0294] CD: Compact Disc

[0295] DVD: Digital Video Disc

[0296] ROM: Read Only Memory

[0297] RAM: Random Access Memory

[0298] ASIC: Application-Specific Integrated Circuit

[0299] PLD: Programmable Logic Device

[0300] LAN: Local Area Network

[0301] GSM: Global System for Mobile Communications

[0302] LTE: Long Term Evolution

[0303] CANBus: Controller Area Network Bus

[0304] USB: Universal Serial Bus

[0305] PCI: Peripheral Component Interconnect

[0306] FPGA: Field Programmable Gate Array

[0307] SSD: Solid State Drive

[0308] IC: Integrated Circuit

[0309] CU: Coding Unit

[0310] RD: Rate Distortion

[0311] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. It will therefore be appreciated that, although not explicitly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that implement the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. A video decoding method, characterized in that: The method comprises: decoding prediction information of a current block from an encoded video bitstream, the prediction information indicating that local illumination compensation (LIC) is applied to the current block and indicating an adjustment parameter for adjusting at least one parameter of the LIC; Based on the adjustment parameter and the initial scaling parameter α i , determine the final scaling parameter α of the current block f , where the initial scaling parameter α i is one of the at least one parameter of the LIC; Based on the final scaling parameter α f and a predictor, determining an updated predictor for the current block; and Based on the updated predictor, the current block is reconstructed.

2. The method according to claim 1, characterized in that The adjustment parameters are signaled in the coded video bitstream; and The decoding includes obtaining the adjustment parameters from the encoded video bitstream.

3. The method according to claim 1, characterized in that Determining the updated predictor comprises: Based on the final scaling parameter α f , the offset parameter β of the current block and the predictor, determine the updated predictor, wherein the updated value of the sample in the predictor is equal to α f ×p i +β, where p i is the value of the sample in the predictor.

4. The method according to claim 3, characterized in that Determine the initial scaling parameter α of the current block based on the current template of the block and the reference template of the reference block i and the offset parameter β of the current block, wherein the predictor is based on the reference block.

5. The method according to claim 4, characterized in that The current template of the reference block includes a top template located above the current block and a left template located on the left side of the current block.

6. The method according to claim 4, characterized in that The current template of the reference block is a top template located above the current block.

7. The method according to claim 4, characterized in that The current template of the reference block is a left template located on the left side of the current block.

8. The method according to claim 1, characterized in that: The prediction information of the current block includes an index indicating the adjustment parameter.

9. The method according to claim 1, characterized in that: The adjustment parameter is a floating point value and is quantized using a number of bits, and The prediction information of the current block includes the plurality of bits.

10. The method according to claim 1, characterized in that Determining the final scaling parameter includes: The adjustment parameter is combined with the initial scaling parameter α of the current block i The sum is determined as the final scaling parameter α of the current block f .

11. A video encoding method, characterized in that: The method comprises: Determine the initial scaling parameter α of the current block i and a local illumination compensation (LIC) to be applied to the current block for adjusting the initial scaling parameter α i The adjustment parameters of Based on the adjustment parameter and the initial scaling parameter α i , determine the final scaling parameter α of the current block f ; Based on the final scaling parameter α f and a predictor that determines an updated predictor for the current block; encoding prediction information indicative of the adjustment parameter in a bitstream; and The current block is encoded in the bitstream based on the updated predictor.

12. The method according to claim 11, characterized in that Encoding the prediction information comprises: The adjustment parameters are encoded in the bitstream.

13. The method according to claim 11, characterized in that Determining the updated predictor comprises: Based on the final scaling parameter α f , the offset parameter β of the current block and the predictor, determine the updated predictor, wherein the updated value of the sample in the predictor is equal to α f ×p i +β, where p i is the value of the sample in the predictor.

14. The method according to claim 11, characterized in that The method further comprises: determining the offset parameter β of the LIC to be applied to the current block based on the current template of the block and the reference template of the reference block, wherein the predictor is based on the reference block, Among them, the initial scaling parameter α is determined i The method comprises: determining the initial scaling parameter α of the current block based on the current template of the current block and the reference template of the reference block. i .

15. The method according to claim 11, characterized in that Encoding the prediction information comprises: The code indicates an index of the adjustment parameter.

16. The method according to claim 11, characterized in that Determining the final scaling parameter includes: The adjustment parameter is combined with the initial scaling parameter α of the current block i The sum is determined as the final scaling parameter α of the current block f .

17. A method for processing visual media data, characterized in that: The method comprises: processing a bit stream comprising said visual media data according to format rules, wherein the bitstream includes prediction information of a current block, the prediction information indicating that local illumination compensation (LIC) is applied to the current block and indicating an adjustment parameter for adjusting at least one parameter of the LIC, and The formatting rules state that: Based on the adjustment parameter and the initial scaling parameter α i , determine the final scaling parameter α of the current block f , where the initial scaling parameter α i is one of the at least one parameter of the LIC; Based on the final scaling parameter α f and a predictor, determining an updated predictor for the current block; and Based on the updated predictor, the current block is reconstructed.

18. A method for processing a video code stream, characterized in that: The video code stream is decoded according to the method according to any one of claims 1 to 10, or encoded according to the method according to any one of claims 11 to 16.

19. A computer device comprising a processing circuit system, characterized in that: The processing circuitry is configured to perform the method of any one of claims 1 to 18. 20 . A non-transitory computer-readable storage medium storing a program executable by at least one processor to perform the method of claim 1 .

Images