Adjustment-based local illumination compensation

Abstract

Aspects of the disclosure provide methods and apparatuses for video decoding. The apparatus includes processing circuitry that decodes, from an encoded video bitstream, prediction information of one or more blocks including a current block. The prediction information indicates that local illumination compensation (LIC) is applied to the one or more blocks and includes LIC information of the one or more blocks. The processing circuitry determines, based on the LIC information, a final scaling factor a f1 and a final offset b f1 for a first sub-block in the current block. The processing circuitry determines, based on the final scaling factor a f1 , the final offset b f1 , and the first predictor sub-block, an updated first predictor sub-block in a predictor corresponding to the current block. An updated value of a first sample in the first predictor sub-block is equal to a f1 × p i1 + b f1 . p i1 is a value of the first sample.

Description

[0001] This application is a divisional application. The original application number is 2022800087012, and the application date is September 8, 2022.

[0002] By incorporating via reference

[0003] This application claims the benefit of priority to U.S. Patent Application No. 17 / 903,697, filed September 6, 2022, entitled “Adjuvant Based Local Illumination Compensation,” which also claims the benefit of priority to U.S. Provisional Application No. 63 / 298,788, filed January 12, 2022, entitled “Adjuvant Based Local Illumination Compensation.” The disclosure of the earlier application is incorporated herein by reference in its entirety. Technical Field

[0004] This disclosure describes implementation methods generally related to video coding. Background Technology

[0005] The purpose of the background description provided herein is to provide a general overview of the context of this disclosure. To the extent that the work described in this background section is intended, neither the work of the currently named inventors nor any aspect of the description which at the time of filing may not otherwise be described as prior art is expressly or implicitly acknowledged as prior art to this disclosure.

[0006] Uncompressed digital images and / or videos can comprise a series of pictures, each with a spatial dimension of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of pictures can have, for example, a fixed or variable picture rate (also informally referred to as frame rate) of 60 frames per second or 60Hz. Uncompressed images and / or videos have specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at 60Hz frame rate) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 gigabytes of storage space.

[0007] One objective of image and / or video encoding and decoding can be to reduce redundancy in the input image and / or video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage requirements, in some cases by two orders of magnitude or more. Although the description herein uses video encoding / decoding as an illustrative example, the same techniques can be applied to image encoding / decoding in a similar manner without departing from the spirit of this disclosure. Both lossless and lossy compression, as well as combinations thereof, can be employed. Lossless compression refers to a technique that reconstructs an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can be reflected in the fact that higher allowable / tolerable distortion can result in a higher compression ratio.

[0008] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform processing, quantization, and entropy coding.

[0009] Video codec techniques can include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference images. In some video codecs, an image is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the image can be an intra-frame image. Intra-frame images and their derivatives (e.g., independent decoder refresh images) can be used to reset the decoder state and therefore can be used as the first image in the encoded video bitstream and video session, or as a still image. Samples of an intra-frame block can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0010] For example, traditional intra-frame coding used in MPEG-2 encoding techniques does not employ intra-frame prediction. However, some newer video compression techniques include those that attempt to perform prediction based on metadata and / or surrounding sample data obtained, for example, during the encoding and / or decoding of data blocks. Such techniques are referred to below as "intra-frame prediction" techniques. Note that in at least some cases, intra-frame prediction uses reference data only from the current frame under reconstruction, and not reference data from a reference frame.

[0011] Many different forms of intra-prediction can exist. When more than one such technique can be used in a given video coding technique, the specific technique used can be encoded as a specific intra-prediction mode using that specific technique. In some cases, an intra-prediction mode can have sub-modes and / or parameters, where these sub-modes and / or parameters can be encoded individually or included in a mode codeword, which defines the prediction mode used. Which codeword is used for a given combination of modes, sub-modes, and / or parameters can affect the coding efficiency gain through intra-prediction, and therefore the entropy coding technique used to convert the codeword into a bitstream can also affect the coding efficiency gain through intra-prediction.

[0012] Some intra-frame prediction modes were introduced in H.264, refined in H.265, and further refined in newer coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmarkset (BMS). Predictor blocks can be formed using the values ​​of neighboring samples from available samples. The sample values ​​of neighboring samples are copied into the predictor block according to the direction. The reference for the direction used can be encoded in the bitstream, or the reference for the direction used can be predicted manually.

[0013] Reference Figure 1A The lower right corner depicts a subset of nine predictor directions known from the 33 possible predictor directions defined in H.265 (corresponding to 33 angular modes of 35 intra-frame modes). The point (101) where the arrows intersect represents the sample being predicted. The arrows indicate the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted based on one or more samples at a 45-degree angle to the horizontal in the upper right corner. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more samples at a 22.5-degree angle to the horizontal in the lower left corner.

[0014] Still refer to Figure 1AThe top left depicts a 4×4 square block (104) of samples (indicated by a bold dashed line). This square block (104) comprises 16 samples, each labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top) and the first sample in the X dimension (from left). Similarly, sample S44 is the fourth sample in block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is located in the bottom right. Also shown are reference samples following a similar numbering scheme. Reference samples are labeled with R, their Y position (e.g., row index) and X position (column index) relative to block (104). In both H.264 and H.265, predicted samples are adjacent to the reconstructed block; therefore, negative values ​​are not required.

[0015] Intra-frame image prediction can work by copying reference sample values ​​from neighboring samples indicated by a prediction direction communicated via signaling. For example, suppose the encoded video bitstream includes signaling for that block indicating a prediction direction consistent with arrow (102)—that is, predicting samples based on samples at a 45-degree angle to the upper right of the horizontal. In this case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Then, sample S44 is predicted based on reference sample R08.

[0016] In some cases, for example, the values ​​of multiple reference samples can be combined by interpolation to calculate the reference sample; especially when the orientation cannot be uniformly divided at 45 degrees.

[0017] As video coding technology has developed, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. In H.265 (2013), this increased to 33. Currently, JEM / VVC / BMS can support up to 65 directions. Experiments have been conducted to identify the most probable directions, and certain techniques in entropy coding have been used to represent these possible directions with a small number of bits, at the cost of fewer possible directions. Furthermore, the direction itself can sometimes be predicted based on adjacent directions used in adjacent decoded blocks.

[0018] Figure 1B A schematic diagram (110) is shown, which depicts 65 intra-frame predicted directions based on JEM to show the number of predicted directions increasing over time.

[0019] The mapping of intra-frame predicted direction bits representing direction in a encoded video bitstream can vary depending on the video coding technique. Such mappings can range from simple direct mappings to codewords, to complex adaptive schemes involving the most probable patterns, and similar techniques. However, in most cases, there may be certain directions that are statistically less likely to appear in the video content compared to some other directions. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less probable directions will be represented by a larger number of bits compared to the more probable directions.

[0020] Image and / or video encoding and decoding can be performed using inter-picture prediction with motion compensation. Motion compensation can be a lossy compression technique and can involve techniques in which blocks of sample data from a previously reconstructed picture or a portion thereof (the reference picture), after being spatially shifted in a direction indicated by a motion vector (MV), are used to predict the reconstructed picture or a portion thereof. In some cases, the reference picture may be the same as the picture under the current reconstruction. The MV may have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference picture in use (indirectly, the third dimension may be a temporal dimension).

[0021] In some video compression techniques, an MV applicable to a specific region of the sample data can be predicted based on other MVs, for example, based on another MV that is spatially adjacent to the region of the sample data under reconstruction and precedes that MV in the decoding order. This prediction can significantly reduce the amount of data required to encode the MV, thereby eliminating redundancy and increasing compression. MV prediction works effectively, for example, because when encoding an input video signal (called natural video) from a camera device, there is a statistically likely area larger than the area applicable to a single MV that moves in similar directions, and therefore, in some cases, the larger area can be predicted using similar motion vectors derived from the MVs of adjacent regions. This makes the MV obtained for a given region similar to or the same as the MV predicted based on surrounding MVs, and can again represent the MV with fewer bits after entropy encoding compared to the bits used in the case of directly encoding the MV. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating the predictor based on several surrounding MVs.

[0022] Various video prediction mechanisms are described in H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016). Among the various video prediction mechanisms provided by H.265, refer to... Figure 2 This describes a technique referred to below as "space merging".

[0023] Reference Figure 2 The current block (201) includes samples obtained by the encoder during the motion search process, which can be predicted based on previous blocks of the same size that have been spatially shifted. Instead of directly encoding this MV, the MV can be derived from, for example, the most recent (in decoding order) reference image associated with one or more reference images, using the MV associated with any of five surrounding samples denoted by A0, A1 and B0, B1, B2 (corresponding to 202 to 206, respectively). In H.265, MV prediction can use a predictor from the same reference image being used by adjacent blocks. Summary of the Invention

[0024] This disclosure provides methods and apparatus for video encoding and decoding. In some examples, the apparatus for video decoding includes a processing circuitry system. The processing circuitry system is configured to decode prediction information for one or more blocks from an encoded video bitstream. The prediction information indicates that local illumination compensation (LIC) is applied to one or more blocks and includes LIC information for one or more blocks. The one or more blocks include the current block to be reconstructed. The processing circuitry system determines a final scaling factor α for a first sub-block in the current block based on the LIC information. f1 The final offset β of the first sub-block in the current block f1 The processing circuitry is based on the final scaling factor α. f1 Final offset β f1 The first predictor sub-block in the predictor is used to determine the updated first predictor sub-block in the predictor corresponding to the current block. In the example, the update value of the first sample in the first predictor sub-block in the predictor is equal to α. f1 ×p i1 +β f1 , where p i1 It is the value of the first sample in the first predictor subblock of the predictor. The processing circuitry reconstructs the first subblock in the current block based on the updated first predictor subblock in the predictor.

[0025] In this implementation, the initial scaling factor α of the first sub-block in the current block is determined based on the first part of the current template of the current block and the first part of the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1The predictor can be based on a reference block. LIC information for one or more blocks is signaled in the encoded video bitstream, indicating at least one of the first parameter μ1 or the second parameter μ2 of one or more blocks. The processing circuitry is based on the first parameter μ1 of one or more blocks and the initial scaling factor α of the first sub-block in the current block. i1 To determine the final scaling factor α of the first sub-block in the current block. f1 The processing circuitry is based on the second parameter μ2 of one or more blocks and the initial offset β of the first sub-block in the current block. i1 To determine the final offset β of the first sub-block in the current block. f1 .

[0026] In the example, the LIC information of one or more blocks indicates one of (i) the first parameter μ1 and (ii) the second parameter μ2, and determines the other of (i) the first parameter μ1 and (ii) the second parameter μ2 based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0027] In the example, (i) one of the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0028] In the example, the LIC information for one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0029] In the example, at least one of the first parameter μ1 or the second parameter μ2 is a floating-point value and is quantized with multiple bits, and the LIC information of one or more blocks includes multiple bits.

[0030] In the example, the processing circuitry will use the final scaling factor α of the first sub-block in the current block. f1 The first parameter μ1 is determined as one or more blocks, and the initial scaling factor α is determined as the first sub-block in the current block. i1 The sum of .

[0031] In the example, the processing circuitry determines the parameter T of the first sub-block in the current block based on at least one of the first portion of the current template or the first portion of a reference template. avg，1 And the final offset β of the first sub-block in the current block. f1 Determined to be (β) i1 +μ2×T avg，1 ).

[0032] In the example, LIC information for one or more blocks is communicated via signaling at the coding unit (CU) level.

[0033] In the example, LIC information for one or more blocks is notified via signaling at a level higher than the CU level.

[0034] In the example, the first sub-block includes the current block, the first predictor sub-block includes the predictor, the first part of the current template includes the current template, and the first part of the reference template includes the reference template.

[0035] In the example, the current block includes a first sub-block and a second sub-block, at least one of a first parameter μ1 or a second parameter μ2 indicated in the LIC information of one or more blocks is applied to the first sub-block and the second sub-block, and the second sub-block is associated with another initial scaling factor α. i2 and another initial offset β i2 Related to this, another initial scaling factor α i2 The initial scaling factor α of the first sub-block i1 Unlike the previous one, another initial offset β i2 The initial offset is different from that of the first sub-block.

[0036] This disclosure also provides a non-transitory computer-readable storage medium storing a program executable by at least one processor to perform a method for video decoding. Attached Figure Description

[0037] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0038] Figure 1A This is a schematic illustration of an exemplary subset of intra-frame prediction modes.

[0039] Figure 1B This is an illustration of an exemplary intra-frame prediction direction.

[0040] Figure 2 An example of the current block (201) and surrounding samples is shown.

[0041] Figure 3 This is a schematic illustration of an exemplary block diagram of a communication system (300).

[0042] Figure 4 It is a schematic illustration of an exemplary block diagram of a communication system (400).

[0043] Figure 5 This is a schematic illustration of an exemplary block diagram of a decoder.

[0044] Figure 6 This is a schematic illustration of an exemplary block diagram of an encoder.

[0045] Figure 7 A block diagram of an exemplary encoder is shown.

[0046] Figure 8 A block diagram of an exemplary decoder is shown.

[0047] Figure 9 The locations of spatial merging candidates according to an embodiment of this disclosure are shown.

[0048] Figure 10 Candidate pairs for redundancy checking of spatial merging candidates are shown according to an embodiment of this disclosure.

[0049] Figure 11 An exemplary motion vector scaling of a time merging candidate is shown.

[0050] Figure 12 An exemplary candidate position for the temporal merging candidate of the current coding unit is shown.

[0051] Figures 13A to 13B An example of an affine model is shown.

[0052] Figure 14 An example of affine prediction based on sub-blocks is shown.

[0053] Figure 15 An example of determining control point motion vector (CPMV) candidates in the current affine merging list of the CU is shown.

[0054] Figure 16 Examples of spatial and temporal neighbors of the current block are shown.

[0055] Figure 17 An exemplary LIC with parameter adjustment is shown.

[0056] Figure 18 A flowchart outlining the coding process according to an embodiment of this disclosure is shown.

[0057] Figure 19 A flowchart outlining the decoding process according to an embodiment of this disclosure is shown.

[0058] Figure 20 It is a schematic diagram of a computer system according to an implementation method. Detailed Implementation

[0059] Figure 3 An exemplary block diagram of a communication system (300) is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3In the example, the first pair of terminal devices (310) and (320) perform unidirectional data transmission. For example, terminal device (310) may encode video data (e.g., a video image stream captured by terminal device (310)) for transmission via network (350) to another terminal device (320). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. Terminal device (320) may receive the encoded video data from network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. Unidirectional data transmission can be common in media service applications, etc.

[0060] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) for example, performing bidirectional transmission of encoded video data during a video conference. For bidirectional data transmission, in this example, each of the terminal devices (330) and (340) can encode video data (e.g., a stream of video images captured by the terminal device) for transmission via a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) can also receive encoded video data transmitted by the other terminal device (330) and (340), and can decode the encoded video data to recover the video images, and can display the video images at an accessible display device based on the recovered video data.

[0061] exist Figure 3 In the examples, terminal devices (310), (320), (330), and (340) are shown as servers, personal computers, and smartphones, respectively; however, the principles of this disclosure are not limited thereto. Embodiments of this disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) refers to any number of networks that transmit encoded video data between terminal devices (310), (320), (330), and (340), including, for example, wired (connected) and / or wireless communication networks. Communication networks (350) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (350) may be irrelevant to the operation of this disclosure unless explained herein.

[0062] As an example of the application of the disclosed topic, Figure 4A video encoder and video decoder are illustrated in a streaming environment. The disclosed subject matter can be equally applied to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0063] The streaming system may include a capture subsystem (413) that may include a video source (401), such as a digital camera device, that creates, for example, an uncompressed video picture stream (402). In the example, the video picture stream (402) includes samples captured by the digital camera device. The video picture stream (402) is depicted as a thick line to emphasize the high amount of data when compared to encoded video data (404) (or encoded video bitstream), which may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or carry out aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video bitstream) is depicted as a thin line to emphasize its lower data volume when compared to the video picture stream (402). The encoded video data (404) (or encoded video bitstream) can be stored on a streaming server (405) for future use. One or more streaming client subsystems, for example... Figure 4 Client subsystems (406) and (408) can access a streaming server (405) to retrieve copies (407) and (409) of encoded video data (404). Client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of encoded video data and creates an outgoing video picture stream (411) that can be displayed on a display (412) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., video bitstreams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In this example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0064] Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0065] Figure 5 An exemplary block diagram of a video decoder (510) is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuitry system). The video decoder (510) may replace... Figure 4 The video decoder (410) in the example is used.

[0066] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510). In an embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (531) can receive encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams, which can be forwarded to their respective user entities (not depicted). The receiver (531) can separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other applications, the buffer memory (515) can be external to the video decoder (510) (not depicted). In some other applications, a buffer memory (not depicted) may exist outside the video decoder (510) to prevent network jitter, for example, and another buffer memory (515) may exist inside the video decoder (510) to handle broadcast timing, for example. The buffer memory (515) may not be needed when the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, or the buffer memory (515) may be small. For use on best-effort packet networks such as the Internet, a buffer memory (515) may be required, which may be relatively large and advantageously have an adaptive size, and may be implemented at least partially outside the video decoder (510) in an operating system or similar component (not depicted).

[0067] The video decoder (510) may include a parser (520) to reconstruct symbols (521) based on the encoded video sequence. These symbols may include information for managing the operation of the video decoder (510), and possibly information for controlling a presentation device such as a presentation device (512) (e.g., a display screen), which is not part of the electronic device (530) but may be coupled to it, such as... Figure 5 As shown in the diagram. The control information of the presentation device may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (520) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may conform to video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set for at least one subgroup of pixel subgroups in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. Subgroups may include: Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The parser (520) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0068] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0069] The reconstruction of the symbol (521) may involve multiple different units depending on the type of the encoded video picture or its parts (e.g., inter-frame picture and intra-frame picture, inter-frame block and intra-frame block) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For clarity, such subgroup control information flow between the parser (520) and the following multiple units is not depicted.

[0070] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into several functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0071] The first unit is the scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives quantized transform coefficients and control information (including which transform to use, block size, quantization factor, quantization scaling matrix, etc.) as one or more symbols (521) from the parser (520). The scaler / inverse transform unit (551) can output a block containing sample values, which can be input into the aggregator (555).

[0072] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed image but can use predictive information from a previously reconstructed portion of the current image. Such predictive information can be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates a block of the same size and shape as the block in the reconstruction using surrounding reconstructed information obtained from the current picture buffer (558). For example, the current picture buffer (558) buffers a partially reconstructed current image and / or a fully reconstructed current image. In some cases, the aggregator (555) adds the predictive information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) based on each sample.

[0073] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to inter-frame coded and possibly motion-compensated blocks. In such cases, the motion compensation prediction unit (553) can access the reference image memory (557) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the block, these samples can be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (referred to in this case as residual samples or residual signals) to generate output sample information. The address in the reference image memory (557) from which the motion compensation prediction unit (553) obtains the predicted samples can be controlled by motion vectors, which can be obtained by the motion compensation prediction unit (553) in the form of symbols (521), which may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (557) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0074] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream), which can be obtained by the loop filter unit (556) as symbols (521) from the parser (520). Video compression may also respond to metadata obtained during decoding of previous portions of the encoded picture or encoded video sequence (in decoding order), as well as to previously reconstructed and loop-filtered sample values.

[0075] The output of the loop filter unit (556) can be a sample stream, which can be output to the presentation device (512) and stored in the reference image memory (557) for use in future inter-frame image prediction.

[0076] Once fully reconstructed, certain encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image (by, for example, the parser (520)) is identified as a reference image, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before reconstructing subsequent encoded images begins.

[0077] The video decoder (510) can perform decoding operations according to a predetermined video compression technique or a standard such as ITU-T Recommendation H.265. The encoded video sequence can conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and in the sense that the configuration file recorded in the video compression technique or standard is available. Specifically, the configuration file can select certain tools from all available tools in the video compression technique or standard as tools available only under that configuration file. For compliance, the complexity of the encoded video sequence also needs to be within the limits defined by the level of the video compression technique or standard. In some cases, the level limits the maximum image size, maximum frame rate, maximum reconstruction sample rate (measured, for example, in megapixels per second), maximum reference image size, etc. In some cases, the limitations set by the level can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is notified by signaling in the encoded video sequence.

[0078] In this implementation, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0079] Figure 6 An exemplary block diagram of a video encoder (603) is shown. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit system). The video encoder (603) may replace Figure 4 The video encoder (403) used in the example.

[0080] The video encoder (603) can obtain data from the video source (601) (not...). Figure 6 In one example, the electronic device (620) receives video samples, and the video source (601) can capture one or more video images to be encoded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0081] The video source (601) can provide a sequence of source video samples in the form of a digital video sample stream to be encoded by the video encoder (603). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601YCrCb, RGB…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (601) can be a storage device storing previously prepared video. In a video conferencing system, the video source (601) can be a camera device capturing local image information as a video sequence. The video data can be provided as multiple individual pictures given motion when viewed sequentially. The pictures themselves can be organized as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0082] According to the implementation, the video encoder (603) can encode and compress images of the source video sequence into an encoded video sequence (643) in real time or under any other required time constraints. Implementing an appropriate encoding rate is a function of the controller (650). In some implementations, the controller (650) controls and is functionally coupled to other functional units as described below. Coupling is not depicted for simplicity. Parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions belonging to the video encoder (603) optimized for a specific system design.

[0083] In some implementations, the video encoder (603) is configured to operate within an encoding loop. As a highly simplified description, in this example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and one or more reference images) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder would also create sample data. The reconstructed sample stream (sample data) is input to a reference image memory (634). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (634) are also bit-accurate between the local and remote encoders. In other words, the encoder's prediction portion "treats" the reference image sample as if it were the exact same sample value that the decoder "sees" during prediction as it does during decoding. This fundamental principle of reference image synchronization (and the resulting offsets in cases where synchronization cannot be maintained, for example, due to channel errors) is also used in some related techniques.

[0084] The operation of the "local" decoder (633) can be combined with that of the "remote" decoder, as already mentioned above. Figure 5 The operation of the video decoder (510) described in detail is the same. However, a brief additional reference is provided. Figure 5 Since symbols are available and the encoding of symbols into a encoded video sequence by the entropy encoder (645) and the decoding of symbols by the parser (520) can be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and the parser (520), can be fully implemented in the local decoder (633).

[0085] In the implementation, any decoder technique other than parsing / entropy decoding present in the decoder exists in the corresponding encoder with the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are contrasted with the fully described decoder techniques. In some aspects, a more detailed description is provided below.

[0086] In some examples, during operation, the source encoder (630) may perform motion-compensated predictive coding, which predictively codes the input image with reference to one or more previously encoded images designated as "reference images" from the video sequence. In this way, the encoding engine (632) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images (which may be selected as prediction references for the input image(s)).

[0087] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (630). The operation of the encoding engine (632) can be advantageously for lossy processing. When encoded video data can be decoded by the video decoder (633), Figure 6 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding processing performed on the reference image by the video decoder and can store the reconstructed reference image in the reference image memory (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0088] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search in the reference image memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. The predictor (635) can operate on a sample-by-sample block-by-pixel basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0089] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0090] The outputs of all the functional units mentioned above can be entropy encoded in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a encoded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0091] The transmitter (640) can buffer one or more encoded video sequences created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0092] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoding image type to each encoded image, which may affect the encoding techniques that can be applied to the corresponding image. For example, one of the following image types can typically be assigned to an image:

[0093] An intra-frame picture (I-picture) can be a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand those variations of I-pictures and their corresponding applications and characteristics.

[0094] Predictive images (P-images) can be images that are encoded and decoded using inter-frame or intra-frame prediction that uses at most one motion vector and a reference index to predict the sample values ​​of each block.

[0095] A bidirectional predictive picture (B-picture) can be an image that can be encoded and decoded using inter-frame or intra-frame prediction that uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0096] The source image can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded block by block. These blocks can be predictively encoded with reference to other (encoded) blocks, which are determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of image I can be non-predictively encoded, or blocks of image I can be predictively encoded (spatial prediction or intra-frame prediction) with reference to coded blocks of the same image. Pixel blocks of image P can be predictively encoded with reference to a previously encoded reference image via spatial prediction or via temporal prediction. Blocks of image B can be predictively encoded with reference to one or two previously encoded reference images via spatial prediction or via temporal prediction.

[0097] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0098] In this implementation, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0099] Video can be captured in a time-series manner as multiple source images (video frames). Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, the specific image being encoded / decoded (referred to as the current image) is segmented into blocks. Where a block in the current image resembles a reference block in a previously encoded and buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.

[0100] In some implementations, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image that both precede the current image in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0101] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0102] According to some embodiments of this disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video picture sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively divided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be divided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type for that CU, such as inter-frame prediction or intra-frame prediction. Depending on temporal and / or spatial predictability, a CU is divided into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In implementations, prediction operations in encoding / decoding are performed on a per-prediction-block basis. Using a luminance prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luminance values) such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0103] Figure 7 An exemplary diagram of a video encoder (703) is shown. The video encoder (703) is configured to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In the example, the video encoder (703) is replaced by... Figure 4 The video encoder (403) used in the example.

[0104] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (703) uses, for example, rate-distortion optimization to determine whether to best encode the processing block using intra-frame mode, inter-frame mode, or bidirectional prediction mode. When encoding the processing block in intra-frame mode, the video encoder (703) can encode the processing block into an encoded picture using intra-frame prediction techniques; while when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (703) can encode the processing block into an encoded picture using inter-frame prediction or bidirectional prediction techniques, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, where motion vectors are derived from the predictor without the aid of encoded motion vector components outside of one or more motion vector predictors. In some other video coding techniques, motion vector components applicable to the object block may exist. In this example, the video encoder (703) includes other components, such as a mode decision module (not shown) that determines the mode of the processing block.

[0105] exist Figure 7 In the example, the video encoder (703) includes, for example, Figure 7 The inter-frame encoder (730), intra-frame encoder (722), residual calculator (723), switch (726), residual encoder (724), overall controller (721), and entropy encoder (725) are shown coupled together.

[0106] An inter-frame encoder (730) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and subsequent images), generate inter-frame prediction information (e.g., a description based on redundancy information, motion vectors, and merging mode information of the inter-frame coding technique), and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on the encoded video information.

[0107] The intra encoder (722) is configured to: receive samples of the current block (e.g., the processing block); in some cases compare the block with already encoded blocks in the same image; generate quantization coefficients after transformation; and in some cases also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). In the example, the intra encoder (722) also calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same image.

[0108] The overall controller (721) is configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In an example, the overall controller (721) determines the mode of the block and provides control signals to the switch (726) based on the mode. For example, when the mode is intra-frame mode, the overall controller (721) controls the switch (726) to select intra-frame mode results for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include the intra-frame prediction information in the bitstream; and when the mode is inter-frame mode, the overall controller (721) controls the switch (726) to select inter-frame prediction results for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0109] A residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from an intra encoder (722) or an inter encoder (730). A residual encoder (724) is configured to operate based on the residual data to encode the residual data to generate transform coefficients. In an example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (722) can generate a decoded block based on the decoded residual data and intra-frame prediction information. In some examples, the decoding blocks are properly processed to generate decoded images, and these decoded images can be buffered in memory circuitry (not shown) and used as reference images.

[0110] An entropy encoder (725) is configured to format the bitstream to include coded blocks. The entropy encoder (725) is configured to include various information in the bitstream according to a suitable standard, such as the HEVC standard. In this example, the entropy encoder (725) is configured to include overall control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Note that, according to the disclosed subject matter, residual information is not present when blocks are encoded in inter-frame mode or in a merged sub-mode of bidirectional prediction mode.

[0111] Figure 8An exemplary diagram of a video decoder (810) is shown. The video decoder (810) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In the example, the video decoder (810) replaces... Figure 4 The video decoder (410) used in the example.

[0112] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The entropy decoder (871), inter-frame decoder (880), residual decoder (873), reconstruction module (874), and intra-frame decoder (872) are shown coupled together.

[0113] The entropy decoder (871) can be configured to reconstruct certain symbols from the encoded picture, which represent the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode encoding the block (e.g., intra-mode, inter-mode, bidirectional prediction mode, a combined sub-mode of the latter two, or another sub-mode), and prediction information (e.g., intra-prediction information or inter-prediction information) that can identify certain samples or metadata used by the intra-decoder (872) or inter-decoder (880) for prediction, respectively. Symbols may also include, for example, residual information in the form of quantized transform coefficients. In the example, when the prediction mode is inter-mode or bidirectional prediction mode, inter-prediction information is provided to the inter-decoder (880); and when the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information may be inversely quantized and provided to the residual decoder (873).

[0114] The inter-frame decoder (880) is configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0115] The intra-frame decoder (872) is configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0116] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual information from the frequency domain to the spatial domain. The residual decoder (873) may also require some control information (to include quantizer parameters (QP)), and this information may be provided by the entropy decoder (871) (the data path is not depicted because this may only be a small amount of control information).

[0117] The reconstruction module (874) is configured to combine the residual information output by the residual decoder (873) with the prediction results (output by the inter-frame prediction module or the intra-frame prediction module, depending on the situation) in the spatial domain to form a reconstructed block. The reconstructed block can be a part of a reconstructed image, which in turn can be a part of a reconstructed video. Note that other suitable operations, such as deblocking, can be performed to improve visual quality.

[0118] Note that any suitable technology can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In one implementation, one or more integrated circuits can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In another implementation, one or more processors that execute software instructions can be used to implement the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810).

[0119] Various inter-frame prediction modes can be used in VVC. For a CU in inter-frame prediction, motion parameters may include one or more MVs, one or more reference picture indices, reference picture list usage indices, and additional information for certain encoded features to be used in the generation of inter-frame prediction samples. Motion parameters can be explicitly or implicitly communicated via signaling. When a CU is encoded in skip mode, the CU may be associated with a PU and may not have valid residual coefficients, encoded motion vector increments or MV differences (e.g., MVD), or reference picture indices. A merge mode can be specified, where the motion parameters of the current CU are obtained from neighboring CUs (one or more), including spatial and / or temporal candidates, and optional additional information, such as additional information introduced in VVC. The merge mode can be applied to CUs in inter-frame prediction, not just skip mode. In the example, an alternative to the merge mode is explicit transmission of motion parameters, where one or more MVs, the corresponding reference picture index for each reference picture list, and reference picture list usage flags and other information are explicitly communicated via signaling according to the CU.

[0120] In implementations, such as in VVC, the VVC Test model (VTM) reference software includes one or more refined inter-frame predictive coding tools, which include: extended merge prediction, merge motion vector difference (MMVD) mode, adaptive motion vector prediction (AMVP) mode with symmetric MVD signaling, affine motion compensation prediction, subblock-based temporal motion vector prediction (SbTMVP), adaptive motion vector resolution (AMVR), motion field storage (1 / 16th luminance sample MV storage and 8×8 motion field compression), bi-prediction with CU-level weight (BCW), bi-directional optical flow (BDOF), prediction refinement using optical flow (PROF), and decoder-side motion vector refinement. Inter-frame prediction (DMVR), combined inter-frame and intra-frame prediction (CIIP), geometric partitioning mode (GPM), etc. Inter-frame prediction and related methods are described in detail below.

[0121] In some examples, extended merge predictions can be used. In examples, such as VTM4, the merge candidate list is constructed by sequentially including the following five types of candidates: spatial motion vector predictor (MVP) from spatially adjacent CUs, temporal MVP from co-located CUs, history-based MVP from a first-in-first-out (FIFO) table, pairwise average MVP, and zero MV.

[0122] The size of the merge candidate list can be notified via signaling in the slice header. In the example, the maximum allowed size of the merge candidate list in VTM4 is 6. For each CU encoded in merge mode, the index of the best merge candidate (e.g., the merge index) can be encoded using truncated unary binary (TU). The first CU of the merge index can be encoded using context (e.g., context-adaptive binary arithmetic coding (CABAC)), and bypass coding can be used for other CUs.

[0123] Below are some examples of the generation process for different categories of merge candidates. In the implementation, spatial candidates are derived as follows. For example, the derivation of spatial merge candidates in VVC can be the same as the derivation of spatial merge candidates in HEVC. In the example, in the location... Figure 9 Choose up to four merge candidates from the candidates depicted in the diagram. Figure 9 The locations of spatial merging candidates according to an embodiment of this disclosure are shown. (Refer to...) Figure 9 The resulting order is B1, A1, B0, A0, and B2. Position B2 is considered only if any CU at positions A0, B0, B1, and A1 is unavailable (e.g., because the CU belongs to another slice or another tile) or is intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates undergoes redundancy checking, which ensures that candidates with the same motion information are excluded from the candidate list, thereby improving coding efficiency.

[0124] To reduce computational complexity, in some implementations, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only certain pairs are considered, for example... Figure 10 The pairs are linked by arrows, and the candidate is added to the candidate list only if the corresponding candidate used for redundancy check does not have the same motion information. Figure 10 Candidate pairs considered for redundancy checking in spatial merging candidates according to an embodiment of this disclosure are shown. (Refer to...) Figure 10 The pairs linked by the corresponding arrows include A1 and B1, A1 and A0, A1 and B2, B1 and B0, and B1 and B2. Therefore, candidates at positions B1, A0, and / or B2 can be compared with candidates at position A1, and candidates at positions B0 and / or B2 can be compared with candidates at position B1.

[0125] In this implementation, time candidates are derived as follows. In the example, only one time merging candidate is added to the candidate list. Figure 11An example of motion vector scaling of temporal merging candidates is shown. To derive temporal merging candidates for the current CU (1111) in the current image (1101), the scaled MV (1121) can be derived based on the co-located CU (1112) belonging to the co-located reference image (1104) (e.g., by...). Figure 11 (As shown by the dashed line in the image). The list of reference images used to derive the corresponding CU (1112) can be explicitly notified in the slice header using signaling. For example, by... Figure 11 As shown by the dashed line, the scaled MV (1121) of the time merging candidate can be obtained. The scaled MV (1121) can be scaled from the MV of the co-located CU (1112) using the picture order count (POC) distances tb and td. The POC distance tb can be defined as the POC difference between the current reference picture (1102) and the current picture (1101). The POC distance td can be defined as the POC difference between the co-located reference picture (1104) and the co-located picture (1103) of the co-located picture (1103). The reference picture index of the time merging candidate can be set to zero.

[0126] Figure 12 Exemplary candidate positions (e.g., C0 and C1) for the current CU's time-merging candidate are shown. The position of the time-merging candidate can be selected between candidate positions C0 and C1. Candidate position C0 is located at the lower right corner of the current CU's sibling CU (1210). Candidate position C1 is located at the center of the current CU's sibling CU (1210). If the CU at candidate position C0 is unavailable, intra-coded, or outside the current line of the CTU, candidate position C1 is used to derive the time-merging candidate. Otherwise, for example, if the CU at candidate position C0 is available, intra-coded, and in the current line of the CTU, candidate position C0 is used to derive the time-merging candidate.

[0127] In some examples, translational motion models are applied to motion compensation prediction (MCP). However, translational motion models may not be suitable for modeling other types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In some implementations, block-based affine transformation motion compensation prediction is applied. Figure 13A In this context, when using a 4-parameter affine model, the affine motion field of the block is described by two control points (CP) CP0 and CP1, and their corresponding control point motion vectors CPMV0 and CPMV1. Figure 13BIn the 6-parameter affine model, the affine motion field of the block is described by the three CPMVs (CPMV0, CPMV1, and CPMV3) of CP (CP0, CP1, and CP2).

[0128] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is obtained as:

[0129]

[0130] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is obtained as follows:

[0131]

[0132] In equations 1 and 2, (mv 0x ,mv 0y (mv) is the motion vector of the top-left control point. 1x ,mv 1y ) is the motion vector of the upper right control point, and (mv 2x ,mv 2y ) is the motion vector of the lower left control point. Additionally, the coordinates (x, y) are relative to the upper left corner of the corresponding block, and W and H represent the width and height of the corresponding block.

[0133] To simplify motion compensation prediction, some implementations employ sub-block-based affine transformation prediction. For example, in Figure 14 In this study, a 4-parameter affine motion model was used, and two CPMVs were determined. and To derive the motion vector for each 4×4 (sample) luma sub-block (1402) segmented from the current block (1410), the motion vector (1401) of the center sample of each sub-block (1402) is calculated according to Equation 1 and rounded to 1 / 16 fractional precision. A motion-compensated interpolation filter is then applied to generate a prediction for each sub-block (1402) using the derived motion vector (1401). The sub-block size for the chroma component is set to 4×4. The MV of the 4×4 chroma sub-block is calculated as the average of the MVs of the four corresponding 4×4 luma sub-blocks.

[0134] Similar to translational motion inter-frame prediction, some implementations employ two affine motion inter-frame prediction modes: affine merging mode and affine AMVP mode.

[0135] In some implementations, an affine merging pattern can be applied to CUs with a width and height greater than or equal to 8. Affine merging candidates for the current CU can be generated based on motion information of spatially adjacent CUs. Up to five affine merging candidates can exist, and a signaling notification index indicates which affine merging candidate to use for the current CU. For example, the following three types of affine merging candidates can be used to form the affine merging candidate list:

[0136] (i) Inheritance affine merge candidates inferred from the CPMV of adjacent CUs;

[0137] (ii) Constructing affine merging candidates using the translation MV of adjacent CUs; ​​and

[0138] (iii) Zero MV.

[0139] In some implementations, there may be at most two inherited affine candidates derived from the affine motion model of adjacent blocks, one affine candidate from the left adjacent CU and one affine candidate from the upper adjacent CU. For example, the candidate block can be located at... Figure 9 The positions are shown in the diagram. For the left predictor, the scan order is A0>A1, while for the top predictor, the scan order is B0>B1>B2. Only the first inheritance candidate from each side is selected. No pruning check is performed between the two inheritance candidates.

[0140] When adjacent affine CUs are identified, the CPMV of the identified adjacent affine CUs is used to derive CPMV candidates from the affine merging list of the current CU. For example... Figure 15 As shown, the adjacent lower-left block A of the current CU (1510) is encoded in affine mode. The motion vectors of the upper-left, upper-right, and lower-left corners of the CU (1520) containing block A are also shown. and It was obtained. When block A is encoded using a 4-parameter affine model, according to and Calculate the two CPMVs of the current CU(1510). and When encoding block A using a 6-parameter affine model, according to and Calculate the three CPMVs (not shown) of the current CU.

[0141] Affine candidates are constructed by combining the translational motion information of adjacent control points. The motion information of the control points comes from... Figure 16The spatial and temporal neighbors specified in the diagram are used to derive the values. CPMVk (k = 1, 2, 3, 4) represents the k-th control point. For CPMV1, blocks B2>B3>A2 are checked in sequence, and the MV of the first available block is used. For CPMV2, blocks B1>B0 are checked, and for CPMV3, blocks A1>A0 are checked. If available, the TMVP at block T is used as CPMV4.

[0142] After obtaining the motion MVs of the four control points, affine merging candidates are constructed based on this motion information. The following combinations of control point MVs are constructed in sequence: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}.

[0143] Combining three CPMVs constructs a 6-parameter affine merge candidate, while combining two CPMVs constructs a 4-parameter affine merge candidate. To avoid motion scaling, combinations of control point MVs are discarded if the reference indices of the control points are different.

[0144] After checking the inheritance affine merge candidates and the construction affine merge candidates, if the list is still not full, zero MV is inserted at the end of the merge candidate list.

[0145] In some implementations, an affine AMVP mode can be applied to CUs with a width and height greater than or equal to 16. A signaling flag at the CU level is used in the bitstream to indicate whether the affine AMVP mode is used, followed by a signaling flag indicating whether a 4-parameter or 6-parameter affine mode is used. The bitstream is also signaled to inform the CPVM differences of the current CUs and their predictors. The affine AVMP candidate list is 2^n in size and can be generated by sequentially using the following four types of CPVM candidates:

[0146] (i) Inherited affine AMVP candidates inferred from the CPMV of adjacent CUs;

[0147] (ii) Constructing affine AMVP candidates using translational MVs of adjacent CUs; ​​and

[0148] (iii) Translation MV from adjacent CU; and

[0149] (iv) Zero MV.

[0150] The checking order for inherited affine AMVP candidates is similar to that of the inherited affine merge candidates in the example. The difference is that for AVMP candidates, affine CUs with the same reference image as those in the current block are considered. No pruning is applied when an inherited affine motion predictor is inserted into the candidate list.

[0151] from Figure 16 The specified spatial neighbors shown are used to derive AMVP candidates. The same checking order as performed in affine merging candidate construction is used. Additionally, the reference picture indices of adjacent blocks are checked. The first block in the checking order that is inter-coded and has the same reference picture as the current CU is used. When the current CU is encoded with a 4-parameter affine model, and both CPMV0 and CPMV1 are available, the available CPMV is added as a candidate in the affine AMVP list. When the current CU is encoded with a 6-parameter affine model, and all three CPMVs (CPMV0, CPMV1, and CPMV2) are available, the available CPMV is added as a candidate in the affine AMVP list. Otherwise, the AMVP candidate is set to unavailable.

[0152] If the affine AMVP list still contains fewer than two candidates after checking the inherited and constructed AMVP candidates, translational motion vectors adjacent to the control points (if available) are added to predict the MV of all control points in the current CU. Finally, if the affine AMVP list is still not full, zero MV is used to fill the affine AMVP list.

[0153] In this implementation, an affine merging (affine MMVD) mode with motion vector difference can be used. Available affine merging candidates can be selected as the base predictor from a sub-block-based merging list. The motion vector value for each control point can be offset from the motion vector of the base predictor. If no affine merging candidate is available, affine MMVD is not used. If affine MMVD is used, the distance index and offset direction index can subsequently be signaled.

[0154] The distance index (IDX) can be signaled to indicate which distance offset from the offset table shown in Table 1 should be used.

[0155] Table 1. Default Distance Offset Table

[0156] Distance from IDX 0 1 2 3 4 Distance offset 1 / 2 pixel 1 pixel 2 pixels 4 pixels 8 pixels

[0157] The direction index can represent four directions as shown in Table 2, where only the x-direction or y-direction can have an MV difference, rather than having an MV difference in both directions.

[0158] Table 2. Default Offset Direction Table

[0159] Offset direction IDX 00 01 10 11 x-direction factor +1 –1 0 0 y-direction factor 0 0 +1 –1

[0160] If inter-frame prediction is unidirectional, a distance offset, signaled by a signaling protocol, can be applied in the offset direction of each control point predictor. The result can be the MV value for each control point.

[0161] If the inter-frame prediction is bidirectional, a signaled distance offset can be applied in the offset direction of the L0 motion vector of the control point predictor, and an offset to be applied to L1 MV can be applied on a mirror or scale basis, as described below.

[0162] If inter-frame prediction is bidirectional, a signaled distance offset is applied in the offset direction of the L0 motion vector of the control point predictor. For L1 CPMV, the offset can be applied on a mirror basis, where the same amount of distance offset is applied in the opposite direction.

[0163] In the implementation, the offset mirroring method based on POC distance can be used for bidirectional prediction. When the basic candidate is predicted bidirectionally, the offset applied to L0 is notified by signaling, while the offset on L1 can depend on the temporal position of the reference images on list 0 and list 1. If both reference images are on the same temporal side of the current image, the same distance offset and the same offset direction can be applied to the CPMV of L0 and L1. When the two reference images are on different sides of the current image, the CPMV of L1 can have a distance offset applied in opposite offset directions.

[0164] In this implementation, an offset scaling method based on POC distance is used for bidirectional prediction. When the basic candidate is predicted bidirectionally, the offset applied to L0 is signaled, while the offset applied to L1 can be scaled based on the temporal distance of the reference images on list 0 and list 1.

[0165] In this implementation, the range of distance offset values ​​is expanded into three offset tables, such as the three different entries shown in Table 3. Each offset table is adaptively selected based on the image resolution (e.g., each row in Table 3). In this example, the offset table is selected based on the image resolution. Referring to Table 3, when the image height is greater than or equal to 1080, the first offset table (e.g., "Distance Offset 1") is selected. When the image height is less than 1080 but greater than or equal to 720, the second offset table (e.g., "Distance Offset 2") is selected. When the image height is less than 720, the third offset table (e.g., "Distance Offset 3") is selected.

[0166] Table 3. Extended Distance Offset Table

[0167]

[0168] Local Illumination Compensation (LIC) is a prediction technique, such as inter-frame prediction, that models local illumination variations between the current block and its predictor (also called the predictor block) using a linear function. LIC can be applied to, but is not limited to, unidirectional prediction modes. The predictor can be determined based on a reference block of the current block. The reference block is located in a reference image. In the example, for instance in unidirectional prediction mode, the MV of the current block points from the current block to the reference block, and the predictor is the reference block of the current block. In the example, the value p of a sample (e.g., a reference sample) in the predictor (e.g., the reference block) is modified using a function such as the linear function shown in Equation 3. i [x,y], to determine the updated value p of the sample in the updated predictor. f [x,y].

[0169] p f [x, y] = α × p i Equation 3: [x, y] + β

[0170] The parameters of a linear function can be represented by a scaling factor (also called a scale) α and an offset β to compensate for changes in illumination. i [x,y] can be a reference sample at a location [x,y] on the reference image, and the reference sample can be pointed to by the MV of the current block. The scaling factor α and offset β can be derived using any suitable method (e.g., least squares) based on the current template of the current block (also called the current block template) and the reference template of the reference block (also called the reference block template). In various implementations, the scaling factor α and offset β are not notified by signaling, thus eliminating signaling overhead. In the example, the LIC flag is notified by signaling to indicate the use of LIC. LIC can be used under any suitable standard, such as under VVC and under standards other than VVC.

[0171] The current block can be encoded based on an updated predictor, for example, using merge mode or skip mode. The current block can also be encoded using AMVP mode based on an updated predictor and additional information (e.g., residual data).

[0172] In related technologies, LIC only uses a flag (e.g., LIC flag) to indicate whether LIC is enabled, and when LIC is enabled, there is no parameter adjustment for improving the accuracy of illumination compensation.

[0173] According to aspects of this disclosure, one or more parameters of the LIC's parameter set (e.g., including scaling factor α and offset β) can be adjusted. A single index or multiple indices can be used to adjust one or more parameters. In the example, scaling factor α and / or offset β are adjusted using one or more indices.

[0174] Figure 17 An exemplary LIC with parameter adjustments is shown. The current block (1701) in the current image has a current template (also called a neighboring reconstruction template) (1721). The current template (1721) can have any suitable shape and any suitable size. The current template (1721) can include samples (e.g., reconstruction samples) from neighboring reconstruction blocks of the current block (1701).

[0175] exist Figure 17 In the example shown, the current template (1721) of the current block (1701) includes a top template (1722) and a left template (1723). Each of the top template (1722) and the left template (1723) can have any suitable shape and any suitable size. The top template (1722) can include samples from one or more top adjacent blocks of the current block (1701). In the example, the top template (1722) includes four rows of samples from one or more top adjacent blocks of the current block (1701). The left template (1723) can include samples from one or more left adjacent blocks of the current block (1701). In the example, the left template (1723) includes four columns of samples from one or more left adjacent blocks of the current block (1701).

[0176] In the example, the current template of the current block (1701) includes either a top-only template (1722) or a left-only template (1723). In the example, the current template of the current block (1701) includes a top template (1722), a left template (1723), and a top-left template (1731).

[0177] The MV (1702) of the current block (1701) can point to the reference block (1703) in the reference image. The reference block (1703) can have a reference template corresponding to the current template (1721), such as reference template (1725).

[0178] The reference template (1725) can have the same shape and size as the current template, respectively. Figure 17 In the example, the reference template (1725) of the reference block (1703) includes a top template (1726) and a left template (1727). The top template (1726) corresponding to the top template (1722) may include samples from one or more top adjacent blocks of the reference block (1703). The left template (1727) corresponding to the left template (1723) may include samples from one or more left adjacent blocks of the reference block (1703).

[0179] The predictor for the current block (1701) can be determined based on the reference block (1703). In the example, for instance in one-way prediction mode, the predictor is the reference block (1703), for example, the sample value of the predictor is equal to the corresponding sample value of the reference block (1703).

[0180] The sample values ​​in the predictor of the current block (1701) can be modified using a LIC for compensating for local illumination variations. According to an embodiment of this disclosure, the updated value p of the sample (e.g., the reference sample) in the predictor... f [x,y] can be a linear function of the values ​​pi[x,y] of the samples in the predictor, as shown in Equation 4, for example.

[0181] p f [,y]=(α+μ1)×p i [x, y]+(β+μ2|×T) avg Equation 4

[0182] Equation 4 can be written as Equation 5 below.

[0183] p f [x, y] = α f ×p i [x, y] + β f Equation 5

[0184] In Equation 5, the final scaling factor α f It equals the sum of the scaling factor α and the first parameter μ1, and the final offset β. f Based on the offset β and the second parameter μ2. In the example, the final offset β f Equals (β + μ2 × T) avg Equation 5 shows that it can be based on the final scaling factor α. f Final offset β f The predictor (e.g., reference block (1703)) is used to determine the updated predictor.

[0185] When the first parameter μ1 and the second parameter μ2 in Equation 4 are zero, parameter adjustment should not be applied to LIC.

[0186] As described above, the scaling factor α and offset β can be determined by using any suitable method (e.g., least squares) based on the current template of the current block (1701) (e.g., the current template (1721)) and the reference template of the reference block (1703) (e.g., the reference template (1725)).

[0187] Parameter T avgIt can be determined based on the current template (e.g., current template (1721)) and / or reference template (e.g., reference template (1725)) of the current block (1701). In the example, parameter T avg It is the average value of the reference template (e.g., reference template (1725)). For example, parameter T avg It is the average of the sample values ​​of the entire reference template (e.g., reference template (1725)) or a subset thereof. In the example, parameter T avg It is the average value of the current template (e.g., the current template (1721)). For example, parameter T avg It is the average of the sample values ​​of the entire current template or its subset.

[0188] In the example, the parameter T is used to determine avg The reference template used is different from the reference template used to determine the scaling factor α and the offset β. In the example, the template used to determine the parameter T is... avg The current template is different from the current template used to determine the scaling factor α and the offset β.

[0189] In the example, parameter T avg It is 1, and therefore ultimately offset by β. f It equals (β+μ2). Therefore, the updated value p f [x,y] equals (α+μ1)×p i [x, y]+(β+μ2).

[0190] The first parameter μ1 and / or the second parameter μ2 can be notified by signaling, for example, by LIC information of one or more blocks including the current block (1701). In an implementation, the LIC information of one or more blocks is notified by signaling in the video bitstream.

[0191] The first parameter μ1 and / or the second parameter μ2 can be notified by signaling within a finite range of values ​​(e.g., from -5 to +5). In the example, the first parameter μ1 is notified by signaling within a finite range of positive values ​​or a finite range of negative values. In the example, the second parameter μ2 is notified by signaling within a finite range of positive values ​​or a finite range of negative values.

[0192] In an implementation, the LIC information of one or more blocks indicates one of the first parameter μ1 and the second parameter μ2, and the other of the first parameter μ2 and the second parameter μ2 can be determined based on one of the first parameter μ1 and the second parameter μ2.

[0193] In the example, μ2 = -μ1. The LIC information of one or more blocks indicates the first parameter μ1, while the second parameter μ2 is determined to be -μ1 based on the first parameter μ1. In the example, the LIC information of one or more blocks indicates the second parameter μ2, while the first parameter μ1 is determined to be -μ2 based on the second parameter μ2.

[0194] When the relationship between the first parameter μ1 and the second parameter μ2 (e.g., μ2 = -μ1) is known, for example, for a decoder, information indicating only one of the first parameter μ1 and the second parameter μ2 (e.g., LIC information) is signaled in the bitstream. For example, if μ2 = -μ1, the LIC information indicating only the first parameter μ1 is signaled in the bitstream. In the example, only the first parameter μ1 is signaled in the bitstream.

[0195] In this implementation, (i) one of the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and neither (i) the first parameter μ1 nor (ii) the second parameter μ2 is not notified by signaling. In this implementation, the LIC information of one or more blocks does not indicate either (i) the first parameter μ1 or (ii) the second parameter μ2. For example, the first parameter μ1 is equal to zero, and the LIC information of one or more blocks does not indicate the first parameter μ1. In this example, the first parameter μ1 is equal to zero and is not notified by signaling in the bitstream. In this example, the second parameter μ2 is equal to zero and is not notified by signaling in the bitstream.

[0196] The LIC information for one or more blocks may include (i) a first parameter μ1 and / or a second parameter μ2 or (ii) one or more indices indicating the first parameter μ1 and / or the second parameter μ2.

[0197] The first parameter μ1 can be a floating-point value and can be quantized using a first set of bits. The LIC information for one or more blocks may include the first set of bits. The second parameter μ2 can be a floating-point value and can be quantized using a second set of bits. The LIC information for one or more blocks may include the second set of bits.

[0198] The LIC information for one or more blocks may include at least one index indicating at least one of the first parameter μ1 or the second parameter μ2. The first parameter μ1 and / or the second parameter μ2 may be signaled by pointing to one or more indexes of a lookup table.

[0199] In the example, the lookup table indicates the relationship between the index and the corresponding parameter pair (e.g., the first parameter μ1 and the second parameter μ2), and the LIC information includes a single index indicating the first parameter μ1 and the second parameter μ2.

[0200] In the example, the first lookup table indicates a first relationship between a first index and a first parameter μ1, and the second lookup table indicates a second relationship between a second index and a second parameter μ2. Therefore, the LIC information includes a first index indicating the first parameter μ1 and a second index indicating the second parameter μ2. When either the first parameter μ1 or the second parameter μ2 is zero, a single index is used to indicate either the first parameter μ1 or the second parameter μ2 if it is not zero. When the relationship between the first parameter μ1 and the second parameter μ2 (e.g., μ2 = -μ1) is known, a single index can be used to indicate either the first parameter μ1 or the second parameter μ2.

[0201] The LIC information for one or more blocks can be signaled at any suitable level, such as the coding unit (CU) level or a higher level than the CU level (e.g., CTU level, slice level, etc.).

[0202] In the example, LIC information for one or more blocks is notified at the CU level via signaling. These blocks reside in the same CU, and the first parameter μ1 and the second parameter μ2 apply to the CU. Different CUs may have different first parameters and / or different second parameters. For example, the first parameter μ1 and the second parameter μ2 of the first CU may differ from those of the second CU.

[0203] In the example, LIC information for one or more blocks is notified via signaling at a high level (e.g., slice level). One or more blocks comprise blocks within multiple CUs in a slice. The first parameter μ1 and the second parameter μ2 apply to multiple CUs in the slice. Multiple CUs in a slice can have the same first parameter μ1. Multiple CUs in a slice can have the same second parameter μ2.

[0204] In the example, one or more indices of the first parameter μ1 and / or the second parameter μ2 are indicated by signaling at different levels (e.g., CU level, CTU level, slice level, etc.).

[0205] The above implementation can be appropriately adjusted when the LIC operates at the sub-block level within a CU, for example, when the LIC is executed at the sub-block level within the current block. The current block can include multiple sub-blocks. Each sub-block can include one or more samples from the current block. For example, each of the multiple sub-blocks is associated with a MV of the corresponding sub-block. The MVs associated with the corresponding multiple sub-blocks can be different. Each MV can point to a corresponding reference sub-block in a reference image. Multiple sub-blocks can be predicted based on the multiple reference sub-blocks associated with the corresponding MVs. In the example, affine merging mode, affine AMVP mode, affine MMVD mode, etc., are used to predict multiple sub-blocks.

[0206] Multiple sub-blocks may include a first sub-block and a second sub-block. LIC information indicating the first parameter μ1 and / or the second parameter μ2 of one or more blocks can be applied to one or more sub-blocks within the current block. The first parameter μ1 and the second parameter μ2 can be applied to multiple sub-blocks. For example, the first sub-block and the second sub-block may have the same first parameter μ1, and the first sub-block and the second sub-block may have the same second parameter μ2.

[0207] In the implementation, each sub-block has a corresponding derived set of parameters used in the LIC.

[0208] For the first sub-block in the current block, Equation 4 can be adapted to Equation 6. The update value p of the first sample in the first predictor sub-block (e.g., the first reference sub-block) f1 [x1, y1] can be the value p of the first sample. i1 A linear function of [x1, y1].

[0209] p f1 [x1, y1] = (α i1 +μ1)×p i1 [x1, y1]+(β) i1 +μ2×T avg，1 Equation 6

[0210] The first MV of the first sub-block points to the first reference sub-block. The first initial scaling factor α i1 and the first initial offset β i1 Associated with the first sub-block to compensate for local lighting variations in the first sub-block. Parameter T avg，1 It can be associated with the first sub-block to compensate for local lighting variations in the first sub-block. The first initial scaling factor α can be determined based on the first part of the current template and the first part of the reference template. i1 and the first initial offset β i1 For example, the first part of the current template includes samples of the reconstructed spatially adjacent blocks of the first sub-block. The first part of the reference template includes samples of the reconstructed spatially adjacent blocks of the first reference sub-block. Parameter T can be determined based on the first part of the current template and / or the first part of the reference template. avg，1 .

[0211] For the second sub-block in the current block, Equation 4 can be adapted to Equation 7. The update value p of the second sample in the second predictor sub-block (e.g., the second reference sub-block) within the predictor. f2 [x2, y2] can be the values ​​p of the second sample. i2 A linear function of [x2, y2].

[0212] p f2 [x2, y2] = (α) i2 +μ1)×pi2 [x2, y2]+(β) i2 +μ2×T avg，2 Equation 7

[0213] The second MV of the second sub-block points to the second reference sub-block. Second initial scaling factor α i2 Second initial offset β i2 Associated with the second sub-block to compensate for local lighting variations in the second sub-block. Parameter T avg，2 It can be associated with a second sub-block to compensate for local lighting variations in the second sub-block. The second initial scaling factor α can be determined based on the second part of the current template and the second part of the reference template. i2 Second initial offset β i2 For example, the second part of the current template includes samples of the reconstructed spatially adjacent blocks of the second sub-block. The second part of the reference template includes samples of the reconstructed spatially adjacent blocks of the second reference sub-block. Parameter T can be determined based on the second part of the current template and / or the second part of the reference template. avg，2 .

[0214] In this implementation, multiple sub-blocks share the same derived set of parameters (e.g., including a first initial scaling factor α). i1 First initial offset β i1 and / or parameter T avg，1 For example, the first initial scaling factor α i1 Equal to the second initial scaling factor α i2 and the first initial offset β i1 Equal to the second initial offset β i2 In the example, parameter T avg，1 Equal to parameter T avg，2 A shared set of parameters for multiple sub-blocks can be determined based on a current template and a reference template. In the example, the reference template can be determined based on the boundary sub-blocks of the current block, which are neighbors of other blocks outside the current block. For example, the MV associated with the boundary sub-blocks can point to multiple sub-blocks within the reference block, and the reference template includes samples of reconstructed spatially adjacent blocks of the multiple sub-blocks within the reference block. In the example, the reference template includes a first portion that can be determined based on a first MV and a second portion that can be determined based on a second MV.

[0215] Figure 18A flowchart outlining an encoding process (1800) according to an embodiment of this disclosure is shown. In various embodiments, the process (1800) is executed by a processing circuitry system, such as the processing circuitry system in terminal devices (310), (320), (330), and (340), a processing circuitry system performing the functions of a video encoder (e.g., (403), (603), (703)), etc. In some embodiments, the process (1800) is implemented as software instructions, so that the processing circuitry system executes the process (1800) when the software instructions are executed. The process begins at (S1801) and proceeds to (S1810).

[0216] At (S1810), at least one of the first parameter μ1 and the second parameter μ2 of one or more blocks, and the initial scaling factor α of the first sub-block in the current block can be determined. i1 and the initial offset β of the first sub-block i1 One or more blocks may include the current block to be encoded using LIC. The first MV of the first sub-block points to the first reference block of the reference block in the reference image. The reference block corresponds to the current block.

[0217] The initial scaling factor α of the first sub-block in the current block can be determined based on the first part of the current template of the current block and the first part of the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0218] In the example, the initial scaling factor α of the first sub-block in the current block can be determined based on the current template of the current block and the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0219] The first parameter μ1 and / or the second parameter μ2 of one or more blocks can be determined based on one or more blocks.

[0220] In the example, the higher levels above the CU (e.g., CTU or slice) include a first block encoded with LIC and a second block not encoded with LIC. The first block may have the same first parameter μ1 and the same second parameter μ2, determined, for example, based on one or more of the parameters in the first block. Alternatively, the first block may have different first parameters μ1 and different second parameters μ2.

[0221] At (S1820), in the example, based on the first parameter μ1 and the initial scaling factor α i1 Determine the final scaling factor α for the first sub-block. f1 For example, the final scaling factor α of the first sub-block in the current block. f1The first parameter μ1, which is determined to be one or more blocks, and the initial scaling factor α of the first sub-block in the current block. i1 The sum of .

[0222] In the example, based on the second parameter μ2 and the initial offset β i1 Determine the final offset β of the first sub-block. f1 For example, the parameter T of the first sub-block in the current block is determined based on at least one of the first part of the current template or the first part of the reference template. avg，1 The final offset β of the first sub-block in the current block. f1 Determined to be (β) i1 +μ2×T avg，1 ).

[0223] In the example, parameter T can be determined based on the current template of the current block and the reference template of the reference block. avg，1 .

[0224] At (S1830), the final scaling factor α can be used as a basis. f1 Final offset β f1 The first predictor sub-block (e.g., the first reference sub-block) in the predictor (e.g., the updated first predictor sub-block) in the predictor (e.g., the reference block) corresponding to the current block is determined by the first predictor sub-block (e.g., the updated first reference sub-block) in the predictor (e.g., the reference block).

[0225] At (S1840), the first sub-block in the current block can be encoded based on the updated first predictor sub-block. LIC information indicating the first parameter μ1 and / or the second parameter μ2 of one or more blocks can be encoded.

[0226] The LIC information for one or more blocks can be signaled in the video bitstream. The LIC information for one or more blocks can be signaled at the Code Unit (CU) level or higher.

[0227] In the example, (i) one of the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0228] In the example, the LIC information for one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0229] In the example, at least one of the first parameter μ1 or the second parameter μ2 is a floating-point value and is quantized with multiple bits, and the LIC information of one or more blocks includes multiple bits.

[0230] Then, process (1800) proceeds to (S1899) and ends.

[0231] Process (1800) can be appropriately adapted to various scenarios, and the steps in process (1800) can be adjusted accordingly. One or more steps in process (1800) can be adjusted, omitted, repeated, and / or combined. Process (1800) can be implemented using any suitable order. Additional steps can be added.

[0232] In the example, the LIC information of one or more blocks indicates one of (i) the first parameter μ1 and (ii) the second parameter μ2, and the other of (i) the first parameter μ1 and (ii) the second parameter μ2 is determined based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0233] In this implementation, the first sub-block in the current block is the current block, the first predictor sub-block in the predictor is the predictor (e.g., the reference block), the first portion of the current template is the current template, and the first portion of the reference template is the reference template. In the example, Equation 4 or Equation 5 is used to determine the updated predictor.

[0234] In this implementation, the current block includes a first sub-block and a second sub-block. At least one of a first parameter μ1 or a second parameter μ2 indicated in the LIC information of one or more blocks is applied to both the first and second sub-blocks. The second sub-block is associated with another initial scaling factor α. i2 and another initial offset β i2 Relatedly, the other initial scaling factor α i2 It can be compared with the initial scaling factor α of the first sub-block. i1 Whether the other initial offset β is the same or different i2 It can be compared with the initial offset β of the first sub-block i1 Whether they are the same or different, the updated second predictor sub-block (e.g., the updated second reference sub-block) corresponding to the second sub-block in the predictor can be determined using Equation 7. The second sub-block can be encoded based on the updated second predictor sub-block.

[0235] Figure 19A flowchart outlining the decoding process (1900) according to embodiments of this disclosure is shown. In various embodiments, the process (1900) is executed by a processing circuitry system, such as the processing circuitry system in terminal devices (310), (320), (330), and (340), a processing circuitry system performing the function of a video encoder (403), a processing circuitry system performing the function of a video decoder (410), a processing circuitry system performing the function of a video decoder (510), a processing circuitry system performing the function of a video encoder (603), etc. In some embodiments, the process (1900) is implemented as software instructions, so the processing circuitry system executes the process (1900) when the processing circuitry system executes the software instructions. The process begins at (S1901) and proceeds to (S1910).

[0236] At (S1910), prediction information for one or more blocks can be decoded from the encoded video bitstream. The prediction information may indicate that local illumination compensation (LIC) is applied to one or more blocks, and the prediction information includes LIC information for one or more blocks, which includes the current block to be reconstructed.

[0237] In the example, LIC information for one or more blocks is communicated via signaling at the coding unit (CU) level.

[0238] In the example, LIC information for one or more blocks is notified via signaling at a level higher than the CU level.

[0239] At (S1920), the final scaling factor α of the first sub-block in the current block can be determined based on the LIC information. f1 The final offset β of the first sub-block in the current block f1 .

[0240] In this implementation, the initial scaling factor α of the first sub-block in the current block is determined based on the first part of the current template of the current block and the first part of the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0241] In the example, the initial scaling factor α of the first sub-block in the current block can be determined based on the current template of the current block and the reference template of the reference block. i1 and the initial offset β of the first sub-block in the current block i1 .

[0242] The LIC information for one or more blocks can be signaled in the encoded video bitstream, and can indicate at least one of the first parameter μ1 or the second parameter μ2 of one or more blocks. It can be based on the first parameter μ1 of one or more blocks and the initial scaling factor α of the first sub-block in the current block. i1 To determine the final scaling factor α of the first sub-block in the current block. f1 It can be based on the second parameter μ2 of one or more blocks and the initial offset β of the first sub-block in the current block. i1 To determine the final offset β of the first sub-block in the current block. f1 .

[0243] In the example, the final scaling factor α of the first sub-block in the current block. f1 The first parameter μ1, which is determined to be one or more blocks, and the initial scaling factor α of the first sub-block in the current block. i1 The sum of .

[0244] In the example, the parameter T of the first sub-block in the current block is determined based on at least one of the first part of the current template or the first part of the reference template. avg，1 And the final offset β of the first sub-block in the current block. f1 Determined to be (β) i1 +μ2×T avg，1 ).

[0245] In the example, parameter T can be determined based on the current template of the current block and the reference template of the reference block. avg，1 .

[0246] In the example, the LIC information of one or more blocks indicates one of (i) the first parameter μ1 and (ii) the second parameter μ2, and the other of (i) the first parameter μ1 and (ii) the second parameter μ2 is determined based on one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0247] In the example, (i) one of the first parameter μ1 and (ii) the second parameter μ2 is equal to zero, and the LIC information of one or more blocks does not indicate one of (i) the first parameter μ1 and (ii) the second parameter μ2.

[0248] In the example, the LIC information for one or more blocks includes at least one index indicating at least one of the first parameter μ1 or the second parameter μ2.

[0249] In the example, at least one of the first parameter μ1 or the second parameter μ2 is a floating-point value and is quantized with multiple bits, and the LIC information of one or more blocks includes multiple bits.

[0250] At (S1930), it can be based on the final scaling factor α. f1 Final offset β f1 The updated first predictor sub-block in the predictor is determined by the first predictor sub-block in the predictor corresponding to the current block. As mentioned above, the predictor can be determined based on a reference block. In the example, the predictor is the reference block; for example, the sample values ​​of the predictor are equal to the corresponding sample values ​​of the reference block. The updated value of the first sample in the first predictor sub-block in the predictor can be equal to α. f1 ×p i1 +β f1 p i1 It can be the value of the first sample in the first predictor sub-block of the predictor.

[0251] At (S1940), the first sub-block in the current block can be reconstructed based on the updated first predictor sub-block in the predictor.

[0252] In the example, the sample value in the first sub-block of the current block is equal to the corresponding sample value in the updated first predictor sub-block of the predictor.

[0253] In the example, the first sub-block in the current block is reconstructed based on the updated first predictor sub-block in the predictor and additional information (e.g., residual data).

[0254] Process (1900) can be appropriately adapted to various scenarios, and the steps in process (1900) can be adjusted accordingly. One or more steps in process (1900) can be adjusted, omitted, repeated, and / or combined. Process (1900) can be implemented using any suitable order. Additional steps can be added.

[0255] In this implementation, the first sub-block in the current block is the current block, the first predictor sub-block in the predictor is the predictor (e.g., the reference block of the current block), the first portion of the current template is the current template, and the first portion of the reference template is the reference template of the reference block. In the example, Equation 4 or Equation 5 is used to determine the updated predictor.

[0256] In this implementation, the current block includes a first sub-block and a second sub-block. At least one of a first parameter μ1 or a second parameter μ2 indicated in the LIC information of one or more blocks is applied to both the first and second sub-blocks. The second sub-block is associated with another initial scaling factor α. i2 and another initial offset β i2 Relatedly, the other initial scaling factor α i2 It can be compared with the initial scaling factor α of the first sub-block. i1 Whether the other initial offset β is the same or different i2 It can be compared with the initial offset β of the first sub-block i1Whether they are the same or different, the updated second predictor sub-block corresponding to the second sub-block in the predictor can be determined using Equation 7. The second sub-block can then be encoded based on the updated second predictor sub-block.

[0257] The embodiments described in this disclosure can be used individually or in any combination in any order. Furthermore, each of the method (or embodiment), encoder, and decoder can be implemented using a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium.

[0258] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 20 A computer system (2000) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0259] Computer software can be coded using any suitable machine code or computer language, which can be subjected to mechanisms such as assembly, compilation, and linking to create code including instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpretation, microcode execution, etc.

[0260] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0261] Figure 20 The components shown for the computer system (2000) are exemplary in nature and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in the exemplary embodiments of the computer system (2000).

[0262] Computer systems (2000) may include certain human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., gestures), and olfactory input (not depicted). Human-machine interface devices can also be used to capture certain media that are not necessarily directly related to intentional human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0263] The input human-machine interface device may include one or more of the following (only one of each is depicted): keyboard (2001), mouse (2002), trackpad (2003), touch screen (2010), data glove (not shown), joystick (2005), microphone (2006), scanner (2007), and camera device (2008).

[0264] The computer system (2000) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touchscreen (2010), data gloves (not shown), or joystick (2005), but tactile feedback devices that are not used as input devices may also exist); audio output devices (e.g., speakers (2009), headphones (not depicted)); visual output devices (e.g., screens (2010), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability—some of which may be able to output two-dimensional or more than three-dimensional visual output in a manner such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and ashtrays (not depicted)); and printers (not depicted).

[0265] The computer system (2000) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (2020) having media such as CD / DVD (2021), thumb drives (2022), removable hard disk drives or solid-state drives (2023), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted).

[0266] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0267] The computer system (2000) may also include interfaces (2054) to one or more communication networks (2055). Networks may be, for example, wireless, wired, or optical. Networks may also be local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), vehicle and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of networks include LANs such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require external network interface adapters attached to certain general-purpose data ports or peripheral buses (2049) (e.g., USB ports of the computer system (2000)); other networks are typically integrated into the core of the computer system (2000) via system buses as described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). The computer system (2000) can communicate with other entities using any of these networks. Such communication can be one-way (receive only, e.g., broadcast television), one-way (transmit only, e.g., to a CANbus device), or bidirectional, e.g., to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0268] The human-machine interface devices, human-accessible storage devices and network interfaces mentioned above can be attached to the core (2040) of a computer system (2000).

[0269] The core (2040) may include one or more central processing units (CPUs) (2041), graphics processing units (GPUs) (2042), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (2043), hardware accelerators (2044) for certain tasks, graphics adapters (2050), etc. These devices, along with read-only memory (ROMs) (2045), random access memory (RAMs) (2046), and internal mass storage devices (2047) such as internal non-user-accessible hard disk drives (HDDs), SSDs, etc., can be connected via the system bus (2048). In some computer systems, the system bus (2048) can be accessed via one or more physical connectors to allow for expansion through additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (2049) to the core's system bus (2048). In the example, a screen (2010) may be connected to the graphics adapter (2050). Peripheral bus architectures include PCI, USB, etc.

[0270] The CPU (2041), GPU (2042), FPGA (2043), and accelerator (2044) can execute certain instructions, which can be combined to form the computer code mentioned above. This computer code can be stored in ROM (2045) or RAM (2046). Transient data can be stored in RAM (2046), while permanent data can be stored, for example, in an internal mass storage device (2047). Fast storage and retrieval of any storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2041), GPUs (2042), mass storage devices (2047), ROMs (2045), RAMs (2046), etc.

[0271] Computer-readable media may contain computer code for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0272] By way of example and not limitation, a computer system (2000) with an architecture, particularly a core (2040), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices as described above, as well as certain storage devices of the core (2040) having non-transitory characteristics, such as a mass storage device (2047) or ROM (2045) within the core. Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core (2040). Depending on specific needs, the computer-readable media may include one or more memory devices or chips. The software can cause the core (2040)—particularly the processor therein (including a CPU, GPU, FPGA, etc.)—to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (2046) and modifying such data structures according to the processes defined by the software. Alternatively or as an alternative, the computer system may be functionalized by logic hardwired or otherwise embodied in circuitry (e.g., accelerator (2044)), which may operate in place of or with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and conversely, references to logic may include software. Where appropriate, references to a computer-readable medium may include circuitry (e.g., an integrated circuit, IC) storing software for execution, circuitry implementing logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0273] Appendix A: Acronyms

[0274] JEM: Joint Exploration Model

[0275] VVC: Universal Video Coding

[0276] BMS: Benchmark Set

[0277] MV: Motion Vector

[0278] HEVC: High-Efficiency Video Coding

[0279] SEI: Supplemental Enhancement Information

[0280] VUI: Video Availability Information

[0281] GOP: Image Group

[0282] TU: Transformation Unit

[0283] PU: Prediction Unit

[0284] CTU: Coding Tree Unit

[0285] CTB: Coded Tree Block

[0286] PB: Prediction Block

[0287] HRD: Hypothetical Reference Decoder

[0288] SNR: Signal-to-noise ratio

[0289] CPU: Central Processing Unit

[0290] GPU: Graphics Processing Unit

[0291] CRT: Cathode Ray Tube

[0292] LCD: Liquid Crystal Display

[0293] OLED: Organic Light Emitting Diode

[0294] CD: Compact Disc

[0295] DVD: Digital Video Disc

[0296] ROM: Read-Only Memory

[0297] RAM: Random Access Memory

[0298] ASIC: Application-Specific Integrated Circuit

[0299] PLD: Programmable Logic Device

[0300] LAN: Local Area Network

[0301] GSM: Global System for Mobile Communications

[0302] LTE: Long Term Evolution

[0303] CANBus: Controller Area Network Bus

[0304] USB: Universal Serial Bus

[0305] PCI: Peripheral Component Interconnect

[0306] FPGA: Field Programmable Gate Domain

[0307] SSD: Solid State Drive

[0308] IC: Integrated Circuit

[0309] CU: Encoding Unit

[0310] RD: Rate Distortion

[0311] While this disclosure has described several exemplary embodiments, variations, substitutions, and various alternative equivalents fall within the scope of this disclosure. It will therefore be appreciated that, although not expressly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that implement the principles of this disclosure and thus within its spirit and scope.

Claims

1. A method for video decoding, characterized in that, The method includes: Decode prediction information for the current block from the encoded video bitstream, the prediction information indicating that local illumination compensation (LIC) is applied to the current block and indicating adjustment parameters for adjusting at least one parameter of the LIC; Based on the adjustment parameters and initial scaling parameters α i Determine the final scaling parameters of the current block. α f The initial scaling parameter α i One of the at least one parameters of the LIC; Based on the adjustment parameters and initial offset parameters β i Determine the final offset parameter of the current block. β f , wherein the initial offset parameter β i One of the at least one parameters of the LIC; Based on the final scaling parameters α f and the final offset parameter β f and a predictor, a predictor that determines the update of the current block; and Based on the updated predictor, reconstruct the current block; Determining the updated predictor includes: Based on the final scaling parameters α f 、 The final offset parameter of the current block β f and the predictor, determining the updated predictor, wherein the updated value of the sample in the predictor is equal to α f × p i + β f ,in, p i The value of the sample in the predictor; Determining the final scaling parameters includes: The adjustment parameter is compared with the initial scaling parameter of the current block. α i The sum is determined as the final scaling parameter of the current block. α f .

2. The method according to claim 1, characterized in that, The adjustment parameters are communicated via signaling in the encoded video bitstream; and The decoding includes obtaining the adjustment parameters from the encoded video bitstream.

3. The method according to claim 1, characterized in that, Based on the current template of the current block and the reference template of the reference block, the initial scaling parameters of the current block are determined. α i and the initial offset parameter of the current block β i The predictor is determined based on the reference block, wherein the current template includes samples from the adjacent reconstructed blocks of the current block, and the reference template includes samples from the reconstructed spatially adjacent blocks of multiple sub-blocks in the reference block.

4. The method according to claim 3, characterized in that, The current template of the current block includes a top template located above the current block and a left template located to the left of the current block.

5. The method according to claim 3, characterized in that, The current template of the current block is the top template located above the current block.

6. The method according to claim 3, characterized in that, The current template of the current block is the left template located to the left of the current block.

7. The method according to claim 1, characterized in that, The prediction information for the current block includes an index indicating the adjustment parameters.

8. The method according to claim 1, characterized in that, The adjustment parameter is a floating-point value and is quantized using multiple bits. The prediction information for the current block includes the plurality of bits.

9. A video encoding method, characterized in that, The method includes: Determine the initial scaling parameters for the current block. α i And the initial scaling parameters to be adjusted in the local illumination compensation (LIC) to be applied to the current block. α i Adjustment parameters; Determine the initial offset parameters of the current block. β i And the initial offset parameter to be adjusted in the local illumination compensation (LIC) to be applied to the current block. β i Adjustment parameters; Based on the adjustment parameters and the initial scaling parameters α i Determine the final scaling parameters of the current block. α f The initial scaling parameter α i It is one of at least one parameters of the LIC; Based on the adjustment parameters and initial offset parameters β i Determine the final offset parameter of the current block. β f , wherein the initial offset parameter β i One of the at least one parameters of the LIC; Based on the final scaling parameters α f and the final offset parameter β f And a predictor, which determines the update of the current block; The prediction information indicating the adjustment parameters is encoded in a bitstream; and Based on the updated predictor, the current block is encoded in the bitstream; Determining the updated predictor includes: Based on the final scaling parameters α f 、 The final offset parameter of the current block β f and the predictor, determining the updated predictor, wherein the updated value of the sample in the predictor is equal to α f × p i + β f ,in, p i The value of the sample in the predictor; Determining the final scaling parameters includes: The adjustment parameter is compared with the initial scaling parameter of the current block. α i The sum is determined as the final scaling parameter of the current block. α f .

10. The method according to claim 9, characterized in that, Encoding the prediction information includes: The adjustment parameters are encoded in the bitstream.

11. The method according to claim 9, characterized in that, The method further includes: Based on the current template of the current block and the reference template of the reference block, determine the initial offset parameter of the LIC to be applied to the current block. β i The predictor is determined based on the reference block, the current template includes samples from the adjacent reconstructed blocks of the current block, and the reference template includes samples from the reconstructed spatially adjacent blocks of multiple sub-blocks in the reference block; Wherein, the initial scaling parameter is determined. α i This includes: determining the initial scaling parameters of the current block based on the current template of the current block and the reference template of the reference block. α i .

12. The method according to claim 9, characterized in that, Encoding the prediction information includes: The encoding indicates the index of the adjustment parameter.

13. A method for processing visual media data, characterized in that, The method includes: The bitstream containing the visual media data is processed according to the format rules. The bitstream includes prediction information for the current block, which indicates that a local illumination compensation (LIC) is applied to the current block and indicates adjustment parameters for adjusting at least one parameter of the LIC. The format rules stipulate that: Based on the adjustment parameters and initial scaling parameters α i Determine the final scaling parameters of the current block. α f Wherein, the initial scaling parameter α i One of the at least one parameters of the LIC; Based on the adjustment parameters and initial offset parameters β i Determine the final offset parameter of the current block. β f , wherein the initial offset parameter β i One of the at least one parameters of the LIC; Based on the final scaling parameters α f The final offset parameter β f and predictor, determine the update predictor of the current block; and Based on the updated predictor, reconstruct the current block; Determining the updated predictor includes: Based on the final scaling parameters α f 、 The final offset parameter of the current block β f and the predictor, determining the updated predictor, wherein the updated value of the sample in the predictor is equal to α f × p i + β f ,in, p i The value of the sample in the predictor; Determining the final scaling parameters includes: The adjustment parameter is compared with the initial scaling parameter of the current block. α i The sum is determined as the final scaling parameter of the current block. α f .

14. A video decoding apparatus, characterized in that, The device includes: Memory, which stores instructions; and A processor that communicates with the memory, wherein, when the processor executes the instructions, the processor is configured to cause the device to perform the method according to any one of claims 1 to 8.

15. A video encoding apparatus, characterized in that, The device includes: Memory, which stores instructions; and A processor that communicates with the memory, wherein, when the processor executes the instructions, the processor is configured to cause the device to perform the method according to any one of claims 9 to 12.

16. An apparatus for processing visual media data, characterized in that, The device includes: Memory, which stores instructions; and A processor that communicates with the memory, wherein, when the processor executes the instructions, the processor is configured to cause the device to perform the method according to claim 13.

17. A method for storing a bit stream, characterized in that, Generate a bitstream by performing the method of any one of claims 9 to 12; and store the bitstream.

18. A method for transmitting a bit stream, characterized in that, Generate a bit stream by performing the method of any one of claims 9 to 12; and transmit the bit stream.

19. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by a processor, they implement the steps of the method according to any one of claims 9 to 12 to generate the bit stream.

20. A computer device, the computer device comprising a processing circuit system, characterized in that, The processing circuitry is configured to perform the method according to any one of claims 1 to 13, 17-18.

21. A non-transitory computer-readable storage medium storing a program that can be executed by at least one processor to perform the method of any one of claims 1 to 13, 17-18.

Images