Image encoder, image decoder, and non-transient computer-readable medium.
The image encoder and decoder system addresses inefficiencies in video coding by using weighted motion vectors and advanced prediction methods, enhancing encoding/decoding efficiency and speed through optimized video coding processes.
Patent Information
- Application Number
- BR122026025423
- Authority / Receiving Office
- BR · BR
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2018-09-07
- Filing Date
- 2019-09-03
- Publication Date
- 2026-11-10
AI Technical Summary
Existing video coding technologies face challenges in efficiently processing increasing amounts of digital video data, particularly in constructing accurate predictions of current frames based on reference frames.
An image encoder and decoder system that utilizes weighted motion vectors for partitions of current images, incorporating interprediction and intraprediction functions to enhance encoding and decoding efficiency, and includes components like dividers, adders, transformers, quantizers, and predictors to optimize video coding.
Improves encoding efficiency and accelerates the encoding/decoding process by effectively utilizing weighted samples and motion vectors, enabling efficient selection of components and operations for improved video coding performance.
Description
1 / 121 Image encoder, image decoder, and non-transient computer-readable media. Separated from BR112021001156-0 filed on September 3, 2019. Technical Field
[001] This description refers to video encoding and, in particular, to video encoding and decoding systems, components and methods for performing an interprediction function in order to construct a prediction of a current frame based on a reference frame. Fundamental Technique
[002] With the advancement in video coding technology, from H.261 and MPEG-1 to H.264 / AVC (Advanced Video Coding), MPEG-LA, H.265 / HEVC (High Efficiency Video Coding), and H.266 / VVC (Versatile Video Codec), there remains a constant need for improvements and optimizations to video coding technology to process an ever-increasing amount of digital video data in various applications. This description refers to further advancements, improvements, and optimizations in video coding, particularly in an interprediction function to construct a prediction of a current frame based on a reference frame. Summary of the Invention
[003] In one aspect, an image encoder includes a set of circuits and a memory coupled to the set of circuits. The set of circuits, in operation, provides a first set of samples for a first partition of a current image with one or more motion vectors including a first motion vector and provides a second set of samples for a first part of the first partition with one or more motion vectors from a second partition, different from the first partition. The samples from the first set of samples of the first part of the first partition, and from the second set of samples of the first part of the first partition, are weighted. A motion vector for the first part of the first partition is stored and based on one or both of the first motion vector and the second motion vector. The first partition is encoded using the weighted samples from the first part of the first partition. Petition 870260103461, dated 07 / 10 / 2026, p. 11 / 376 2 / 121
[004] In one aspect, an image encoder comprises a divider that, in operation, receives and divides an original image into blocks, a first adder that, in operation, receives the blocks from the divider and predictions from a prediction controller, and subtracts each prediction from its corresponding block to send a residual, a transformer that, in operation, performs a transformation on the residuals sent from the adder to send the transformation coefficients, a quantizer that, in operation, quantizes the transformation coefficients to generate quantized transformation coefficients, an entropy encoder that, in operation, encodes the quantized transformation coefficients to generate a bit sequence, an inverse quantizer and transformer that, in operation, inversely quantizes the quantized transformation coefficients to obtain transformation coefficients and inversely transforms the transformation coefficients to obtain residuals,A second adder, in operation, adds the residuals sent from the inverse quantizer and transformer and the predictions sent from the prediction controller to reconstruct the blocks, and the prediction controller coupled to an interpredictor, an intrapredictor, and a memory. The interpredictor, in operation, generates a prediction of a current block based on a reference block in an encoded reference image. The intrapredictor, in operation, generates a prediction of a current block based on a reference block encoded in a current image. The interpredictor, in operation, predicts a first set of samples for a first partition of a current image with one or more motion vectors, including a first motion vector, and predicts a second set of samples for a first part of the first partition with one or more motion vectors from a second partition.Unlike the first partition, the samples from the first set of samples from the first part of the first partition, and from the second set of samples from the first part of the first partition, are weighted. A motion vector for the first part of the first partition is stored and based on one or both of the first motion vector and the second motion vector. The first partition is encoded using the weighted samples from the first part of the first partition.
[005] In one aspect, an encoding method comprises predicting a first set of samples for a first partition of a current image with one or more motion vectors including a first motion vector and predicting Petition 870260103461, dated 07 / 10 / 2026, page 12 / 376 3 / 121 A second set of samples for a first part of the first partition with one or more motion vectors from a second partition, different from the first partition. The samples from the first set of samples from the first part of the first partition, and the second set of samples from the first part of the first partition, are weighted. A motion vector for the first part of the first partition is stored and based on one or both of the first motion vector and the second motion vector. The first partition is encoded using at least the weighted samples from the first part of the first partition.
[006] In one aspect, an image decoder includes the circuitry and a memory coupled to the circuitry. The circuitry, in operation, provides a first set of samples for a first partition of a current image with one or more motion vectors including a first motion vector and provides a second set of samples for a first part of the first partition with one or more motion vectors from a second partition, different from the first partition. The samples from the first set of samples from the first part of the first partition, and from the second set of samples from the first part of the first partition, have been weighted. A motion vector for the first part of the first partition has been stored and is based on one or both of the first motion vector and the second motion vector. The first partition is decoded using at least the weighted samples from the first part of the first partition.
[007] In one aspect, an image decoder comprises an entropy encoder that, in operation, receives and decodes a sequence of encoded bits to obtain quantized transformation coefficients, an inverse quantizer and transformer that, in operation, inversely quantizes the quantized transformation coefficients to obtain transformation coefficients, and inversely transforms the transformation coefficients to obtain residuals, an adder that, in operation, adds the residuals sent from the inverse quantizer and transformer and predictions sent from a prediction controller to reconstruct blocks, and the prediction controller coupled to an interpredictor, an intrapredictor, and a memory. The interpredictor, in operation, generates a prediction of a current block based on a reference block in a decoded reference image. The intrapredictor, in operation, generates a prediction Petition 870260103461, dated 07 / 10 / 2026, page 13 / 376 4 / 121 of a current block based on a reference block encoded in a current image. The interpredictor, in operation, predicts a first set of samples for a first partition of a current image with one or more motion vectors including a first motion vector and predicts a second set of samples for a first part of the first partition with one or more motion vectors from a second partition, different from the first partition. The samples from the first set of samples of the first part of the first partition, and from the second set of samples of the first part of the first partition, are weighted. A motion vector for the first part of the first partition is stored and is based on one or both of the first motion vector and the second motion vector. The first partition is decoded using at least the weighted samples from the first part of the first partition.
[008] In one aspect, a decoding method comprises predicting a first set of samples for a first partition of a current image with one or more motion vectors including a first motion vector and predicting a second set of samples for a first part of the first partition with one or more motion vectors from a second partition, different from the first partition. The samples from the first set of samples of the first part of the first partition, and from the second set of samples of the first part of the first partition, are weighted. A motion vector for the first part of the first partition is stored and is based on one or both of the first motion vector and the second motion vector. The first partition is decoded using at least the weighted samples from the first part of the first partition.
[009] Some implementations of the embodiments in this description may improve encoding efficiency, may simply be an encoding / decoding process, may accelerate the speed of an encoding / decoding process; may efficiently select suitable components / operations used in encoding and decoding, such as suitable filter, block size, motion vector, reference image, reference block, etc.
[0010] Additional benefits and advantages of the described modalities will become apparent from the specification and drawings. The benefits and / or advantages can be obtained individually through the various modalities and characteristics of Petition 870260103461, dated 07 / 10 / 2026, page 14 / 376 5 / 121 specification and drawings, not all of which are foreseen to be provided in order to obtain one or more of these benefits and / or advantages.
[0011] It should be noted that general or specific embodiments may be implemented as a system, a method, an integrated circuit, a computer program, a storage medium, or any selective combination thereof. Brief Description of the Drawings
[0012] Figure 1 is a block diagram illustrating a functional configuration of an encoder, according to one modality.
[0013] Figure 2 is a flowchart showing an example of a general coding process performed by the coder.
[0014] Figure 3 is a conceptual diagram illustrating an example of block division.
[0015] Figure 4A is a conceptual diagram illustrating an example of a slice configuration.
[0016] Figure 4B is a conceptual diagram illustrating an example of a "tile" configuration.
[0017] Figure 5A is a graph showing basic transformation functions for various types of transformation.
[0018] Figure 5B is a conceptual diagram illustrating spatially variable illustrative transformations (SVT).
[0019] Figure 6A is a conceptual diagram illustrating an example of a filter format used in an adaptive circuit filter (ALF).
[0020] Figure 6B is a conceptual diagram illustrating another example of a filter format used in an ALF.
[0021] Figure 6C is a conceptual diagram illustrating another example of a filter format used in an ALF.
[0022] Figure 7 is a block diagram showing an example of a specific configuration of a circuit filter that functions as a block filter (DBF). Petition 870260103461, dated 07 / 10 / 2026, page 15 / 376 6 / 121
[0023] Figure 8 is a conceptual diagram showing an example of a deblocking filter having a symmetrical filtering characteristic with respect to a block boundary.
[0024] Figure 9 is a conceptual diagram to illustrate a block boundary in which an unblocking filter process is performed.
[0025] Figure 10 is a conceptual diagram showing examples of B values.
[0026] Figure 11 is a flowchart illustrating an example of a process performed by an encoder prediction processor.
[0027] Figure 12 is a flowchart illustrating another example of a process performed by the encoder's prediction processor.
[0028] Figure 13 is a flowchart illustrating another example of a process performed by the encoder's prediction processor.
[0029] Figure 14 is a conceptual diagram illustrating sixty-seven intraprediction modes used in intraprediction in a modality.
[0030] Figure 15 is a flowchart illustrating a basic processing flow illustrative of interprediction.
[0031] Figure 16 is a flowchart illustrating an example of deriving motion vectors.
[0032] Figure 17 is a flowchart illustrating another example of deriving motion vectors.
[0033] Figure 18 is a flowchart illustrating another example of deriving motion vectors.
[0034] Figure 19 is a flowchart illustrating an example of interprediction in normal intermode.
[0035] Figure 20 is a flowchart illustrating an example of interprediction in mixing mode.
[0036] Figure 21 is a conceptual diagram to illustrate an example of a motion vector derivation process in mixing mode. Petition 870260103461, dated 07 / 10 / 2026, page 16 / 376 7 / 121
[0037] Figure 22 is a flowchart illustrating an example of the frame rate upconversion (FRUC) process.
[0038] Figure 23 is a conceptual diagram to illustrate an example of pattern matching (bilateral matching) between two blocks along a motion path.
[0039] Figure 24 is a conceptual diagram to illustrate an example of pattern matching (template matching) between a template in a current image and a block in a reference image.
[0040] Figure 25A is a conceptual diagram to illustrate an example of deriving a motion vector for each sub-block based on the motion vectors among a plurality of neighboring blocks.
[0041] Figure 25B is a conceptual diagram to illustrate an example of deriving a motion vector for each sub-block in affine mode in which three control points are used.
[0042] Figure 26A is a conceptual diagram to illustrate an affine mixing mode.
[0043] Figure 26B is a conceptual diagram to illustrate an affine mixing mode in which two control points are used.
[0044] Figure 26C is a conceptual diagram to illustrate an affine mixing mode in which three control points are used.
[0045] Figure 27 is a flowchart illustrating an example of a process in affine mixing mode.
[0046] Figure 28A is a conceptual diagram to illustrate an affine intermode in which two control points are used.
[0047] Figure 28B is a conceptual diagram to illustrate an affine intermode in which three control points are used.
[0048] Figure 29 is a flowchart illustrating an example of a process in the affine intermode.
[0049] Figure 30A is a conceptual diagram to illustrate an affine intermode in which a current block has three control points and a neighboring block has two. Petition 870260103461, dated 07 / 10 / 2026, page 17 / 376 8 / 121 control points.
[0050] Figure 30B is a conceptual diagram to illustrate an affine intermode in which a current block has two control points and a neighboring block has three control points.
[0051] Figure 31A is a flowchart illustrating a mixing mode process including decoder motion vector refinement (DMVR).
[0052] Figure 31B is a conceptual diagram to illustrate an example of a DMVR process.
[0053] Figure 32 is a flowchart illustrating an example of generating a prediction image.
[0054] Figure 33 is a flowchart illustrating another example of generating a prediction image.
[0055] Figure 34 is a flowchart illustrating another example of generating a prediction image.
[0056] Figure 35 is a flowchart illustrating an example of a prediction image correction process performed by an overlapping block motion compensation (OBMC) process.
[0057] Figure 36 is a conceptual diagram to illustrate an example of a prediction image correction process performed by an OBMC process.
[0058] Figure 37 is a conceptual diagram to illustrate the generation of two rectangular prediction images.
[0059] Figure 38 is a conceptual diagram to illustrate a model assuming uniform linear motion.
[0060] Figure 39 is a conceptual diagram to illustrate an example of a predictive image generation method using a luminescence correction process performed by a local illumination compensation (LIC) process.
[0061] Figure 40 is a block diagram illustrating an example of encoder assembly.
[0062] Figure 41 is a block diagram illustrating a functional configuration of a decoder according to a mode. Petition 870260103461, dated 07 / 10 / 2026, page 18 / 376 9 / 121
[0063] Figure 42 is a flowchart illustrating an example of a general decoding process performed by the decoder.
[0064] Figure 43 is a flowchart illustrating an example of a process performed by a decoder prediction processor.
[0065] Figure 44 is a flowchart illustrating another example of a process performed by the decoder's prediction processor.
[0066] Figure 45 is a flowchart illustrating an example of interprediction in normal intermode in the decoder.
[0067] Figure 46 is a block diagram illustrating an example of decoder assembly.
[0068] Figure 47 is a flowchart illustrating a general process flow for dividing an image block into a plurality of partitions including at least one first partition, which has a non-rectangular shape (e.g., a triangle), and a second partition, and performing further processing according to a modality.
[0069] Figure 48 illustrates two illustrative methods of dividing an image block into a first partition that has a non-rectangular shape (for example, a triangle) and a second partition (also having a non-rectangular shape in the illustrated examples).
[0070] Figure 49 illustrates an example of a boundary smoothing process involving weighting the first values of the predicted boundary pixels based on the first partition and the second values of the predicted boundary pixels based on the second partition.
[0071] Figure 50 illustrates three additional samples of a boundary smoothing process involving weighting the first predicted boundary pixel values based on the first partition and the second predicted boundary pixel values based on the second partition.
[0072] Figure 51 is a table of sample parameters (first index values) and sets of information coded respectively by the parameters.
[0073] Figure 52 is a table illustrating the trivialization of parameters (index values). Petition 870260103461, dated 07 / 10 / 2026, page 19 / 376 10 / 121
[0074] Figure 53 is a flowchart illustrating a process of dividing an image block into a plurality of partitions, including a first partition having a non-rectangular shape and a second partition.
[0075] Figure 54 illustrates examples of dividing an image block into a plurality of partitions that include one partition that has a non-rectangular shape, which is a triangle in the examples shown, and a second partition.
[0076] Figure 55 illustrates additional examples of dividing an image block into a plurality of partitions that include a first partition having a non-rectangular shape, which is a polygon with at least five sides and angles in the examples illustrated, and a second partition.
[0077] Figure 56 is a flowchart illustrating a boundary smoothing process involving weighting the first predicted boundary pixel values based on the first partition and the second predicted boundary pixel values based on the second partition.
[0078] Figure 57A illustrates an example of a boundary smoothing process where the boundary pixels for which the first values to be weighted are predicted based on the first partition and the second values to be weighted are predicted based on the second partition.
[0079] Figure 57B illustrates an example of a boundary smoothing process where the boundary pixels, for which the first values to be weighted are predicted based on the first partition, and the second values to be weighted are predicted based on the second partition.
[0080] Figure 57C illustrates an example of a boundary smoothing process where the boundary pixels, for which the first values to be weighted are predicted based on the first partition, and the second values to be weighted are predicted based on the second partition.
[0081] Figure 57D illustrates an example of a boundary smoothing process where the boundary pixels for which the first values to be weighted are predicted based on the first partition and the second values to be weighted are predicted based on the second partition.
[0082] Figure 58 is a flowchart illustrating a method performed on the side of Petition 870260103461, dated 07 / 10 / 2026, page 20 / 376 11 / 121 encoder for dividing an image block into a plurality of partitions including a first partition that has a non-rectangular shape and a second partition, based on a partition parameter indicative of the division, and writing one or more parameters, including the partition parameter, in a bit sequence in entropy encoding.
[0083] Figure 59 is a flowchart illustrating a method performed on the decoder side of parsing one or more parameters from a bit sequence, which includes a partition parameter indicating the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition, and dividing the image block into the plurality of partitions based on the partition parameter, and decoding the first partition and the second partition.
[0084] Figure 60 is a sample partition parameter table (first index values) that indicate, respectively, the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape, and a second partition, and configures the information that can be encoded together by the partition parameters, respectively.
[0085] Figure 61 is a sample combination table of a first parameter and a second parameter, one of which is a partition parameter, indicating the division of an image block into a plurality of partitions including a first partition that has a non-rectangular shape, and a second partition.
[0086] Figure 62 is a flowchart illustrating an example of a process flow for predicting a first set of samples for a first partition of a current image with a first motion vector, predicting a second set of samples for a first part included in the first partition with a second motion vector, weighting the first and second sets of samples, storing at least one of the first and second motion vectors of the first partition, and encoding or decoding the first partition using weighted samples, and performing further processing according to a modality.
[0087] Figure 63 is a conceptual diagram to illustrate the illustrative methods. Petition 870260103461, dated 07 / 10 / 2026, page 21 / 376 12 / 121 splitting of an image block into a first partition and a second partition.
[0088] Figure 64 is a conceptual diagram to illustrate spatially neighboring adjacent and non-adjacent partitions of a first partition.
[0089] Figure 65 is a conceptual diagram to illustrate uniprediction and biprediction motion vector candidates for an image block.
[0090] Figure 66 is a conceptual diagram to illustrate examples of a first part of a first partition and first and second sample sets.
[0091] Figure 67 is a conceptual diagram to illustrate a first part of a first partition, which is a part of the first partition that overlaps a part of an adjacent partition.
[0092] Figure 68 is a conceptual diagram to illustrate an illustrative case in which a first motion vector and a second motion vector are uniprediction motion vectors pointing to different images in different reference image lists.
[0093] Figure 69 is a conceptual diagram to illustrate an illustrative case in which the first motion vector and the second motion vector are single-prediction motion vectors pointing to images in a single reference image list.
[0094] Figure 70 is a conceptual diagram to illustrate illustrative cases in which the first motion vector and the second motion vector are single-prediction motion vectors pointing to the same reference image in the same list of reference images.
[0095] Figure 71 is a conceptual diagram to illustrate illustrative cases in which the first motion vector and the second motion vector are single-prediction motion vectors pointing to the same reference image in the same list of reference images.
[0096] Figure 72 is a conceptual diagram to illustrate illustrative cases in which the first motion vector and the second motion vector are single-prediction motion vectors pointing to the same reference image in the same list of reference images.
[0097] Figure 73 is a block diagram illustrating a general configuration. Petition 870260103461, dated 07 / 10 / 2026, page 22 / 376 13 / 121 of a content delivery system to implement a content distribution service.
[0098] Figure 74 is a conceptual diagram illustrating an example of a coding structure in scalable coding.
[0099] Figure 75 is a conceptual diagram illustrating an example of a coding structure in scalable coding.
[00100] Figure 76 is a conceptual diagram illustrating an example of a web page display screen.
[00101] Figure 77 is a conceptual diagram illustrating an example of a web page display screen.
[00102] Figure 78 is a block diagram illustrating an example of a smartphone.
[00103] Figure 79 is a block diagram illustrating an example of a smartphone configuration. Description of the Modalities
[00104] Hereafter, embodiments will be described by reference to the drawings. Note that the embodiments described below each illustrate a general or specific example. The numerical values, formats, materials, components, the arrangement and connection of components, steps, the relationship and order of steps, etc., indicated in the embodiments below are merely examples and should not limit the scope of embodiments.
[00105] The embodiments of an encoder and a decoder will now be described. The embodiments are examples of an encoder and a decoder to which the processes and / or configurations, presented in the description of the aspects of this description, are applicable. The processes and / or configurations may also be implemented in an encoder and a decoder different from those according to the embodiments. For example, with reference to the processes and / or configurations, as applied to the embodiments, any of the following may be applied:
[00106] (1) Any of the components of the encoder or decoder, according to the embodiments presented in the description of aspects of this description, may be replaced or combined with another component. Petition 870260103461, dated 07 / 10 / 2026, p. 23 / 376 14 / 121 shown elsewhere in the description of the aspects of the present description.
[00107] (2) In the encoder or decoder, according to the embodiments, discretionary changes may be made to the functions or processes performed by one or more components of the encoder or decoder, such as adding, replacing, removing, etc., functions or processes. For example, any function or process may be replaced or combined with another function or process presented elsewhere in the description of the aspects of the present description.
[00108] (3) In the methods implemented by the encoder or decoder, according to the embodiments, discretionary changes may be made such as by adding, replacing and removing one or more of the processes included in the method. For example, any process in the method may be replaced or combined with another process presented elsewhere in the description of the aspects of the present description.
[00109] (4) One or more components included in the encoder or decoder, according to the embodiments, may be combined with a component presented elsewhere in the description of aspects of this description, may be combined with a component including one or more functions presented elsewhere in the description of aspects of this description, and may be combined with a component that implements one or more processes implemented by a component presented in the description of aspects of this description.
[00110] (5) A component including one or more encoder or decoder functions, according to the embodiments, or a component that implements one or more encoder or decoder processes, according to the embodiments, may be combined or replaced by a component presented anywhere in the description of aspects of this description, with a component including one or more functions presented elsewhere in the description of aspects of this description, or with a component that implements one or more processes presented elsewhere in the description of aspects of this description.
[00111] (6) In the methods implemented by the encoder or decoder, Petition 870260103461, dated 07 / 10 / 2026, p. 24 / 376 15 / 121 according to the modalities, any of the processes included in the method may be substituted or combined with a process presented elsewhere in the description of the aspects of this description or with any corresponding or equivalent process.
[00112] (7) One or more processes included in the methods implemented by the encoder or decoder, according to the embodiments, may be combined with a process presented elsewhere in the description of aspects of the present description.
[00113] (8) The implementation of the processes and / or configurations presented in the description of the aspects of this description is not limited to the encoder or decoder, according to the embodiments. For example, the processes and / or configurations may be implemented in a device used for a purpose other than the motion picture encoder or motion picture decoder described in the embodiments. Encoder
[00114] First, a modality-based encoder will be described. Figure 1 is a block diagram illustrating a functional configuration of modality-based encoder 100. Encoder 100 is a video encoder that encodes a video into one-block units.
[00115] As illustrated in figure 1, the encoder 100 is a device that encodes an image into one-block units, and includes the divider 102, the subtractor 104, the transformer 106, the quantizer 108, the entropy encoder 110, the inverse quantizer 112, the inverse transformer 114, the adder 116, the block memory 118, the circuit filter 120, the frame memory 122, the intrapredictor 124, the interpredictor 126 and the prediction controller 128.
[00116] Encoder 100 is implemented as, for example, a generic processor and memory. In this case, when a software program stored in memory is executed by the processor, the processor functions as a divider 102, subtractor 104, transformer 106, quantizer 108, entropy encoder 110, inverse quantizer 112, inverse transformer 114, adder 116, circuit filter 120, intrapredictor 124, interpredictor 126, and prediction controller 128. Alternatively, encoder 100 can be implemented as one or more Petition 870260103461, dated 07 / 10 / 2026, page 25 / 376 16 / 121 dedicated electronic circuits corresponding to the divider 102, subtractor 104, transformer 106, quantizer 108, entropy encoder 110, inverse quantizer 112, inverse transformer 114, adder 116, circuit filter 120, intrapredictor 124, interpredictor 126 and prediction controller 128.
[00117] Hereafter, a general flow of processes performed by encoder 100 will be described, and then each of the constituent elements included in encoder 100 will be described. General Flow of the Coding Process
[00118] Figure 2 is a flowchart showing an example of a general coding process performed by encoder 100.
[00119] First, the 102 splitter of the 100 encoder divides each of the images included in the input image, which is a video, into a plurality of blocks having a fixed size (e.g., 128 x 128 pixels) (Step Sa_1). The 102 splitter then selects a splitting pattern for the fixed-size block (also referred to as a block format) (Step Sa_2). In other words, the 102 splitter further divides the fixed-size block into a plurality of blocks that form the selected splitting pattern. The 100 encoder performs, for each of the plurality of blocks, Steps Sa_3 to Sa_9 for the block (which is the current block to be encoded).
[00120] In other words, a forecast processor, which includes all or part of intrapredictor 124, interpredictor 126 and forecast controller 128, generates a forecast signal (also referred to as a forecast block) from the current block to be encoded (also referred to as a current block) (Step Sa_3).
[00121] Next, subtractor 104 generates a difference between the current block and a forecast block as a forecast residual (also referred to as a difference block) (Step Sa_4).
[00122] Next, transformer 106 transforms the difference block and quantizer 108 quantizes the result, to generate a plurality of quantized coefficients (Step Sa_5). It should be noted that the block possessing the plurality of quantized coefficients is also referred to as a coefficient block.
[00123] Next, the entropy encoder 110 encodes (specifically, encodes Petition 870260103461, dated 07 / 10 / 2026, page 26 / 376 17 / 121 by entropy) the coefficient block and a prediction parameter related to generating a prediction signal to generate an encoded signal (Step Sa_6). It should be noted that the encoded signal is also referred to as an encoded bit sequence, a compressed bit sequence, or a sequence.
[00124] Next, the inverse quantizer 112 performs the inverse quantization of the coefficient block and the inverse transformer 114 performs the inverse transformation of the result, to restore a plurality of prediction residuals (i.e., a difference block) (Step Sa_7).
[00125] Next, adder 116 adds the prediction block to the restored difference block to reconstruct the current block as a reconstructed image (also referred to as a reconstructed block or a decoded image block). (Step Sa_8). In this way, the reconstructed image is generated.
[00126] When the reconstructed image is generated, circuit filter 120 performs filtering of the reconstructed image as needed (Step Sa_9).
[00127] Encoder 100 then determines whether the encoding of the entire image has been completed (Step Sa_10). When it determines that the encoding has not yet been completed (Not in Step Sa_10), the processes of Step Sa_2 are executed repeatedly.
[00128] Although encoder 100 selects a division pattern for a fixed-size block, and encodes each block according to the division pattern in the example described above, it should be noted that each block can be encoded according to a corresponding pattern from among a plurality of division patterns. In this case, encoder 100 can evaluate a cost for each of the plurality of division patterns and, for example, can select the encoded signal obtainable by encoding according to the division pattern that results in the lowest cost as an encoded signal that is sent.
[00129] As illustrated, the processes in Steps Sa_1 to Sa_10 are performed sequentially by encoder 100. Alternatively, two or more of the processes can be performed in parallel, the processes can be reordered, etc. Divider
[00130] The divisor 102 divides each of the images included in a video by Petition 870260103461, dated 07 / 10 / 2026, p. 27 / 376 18 / 121 input into a plurality of blocks and sends each block to the 104 subtractor. For example, the 102 divider first divides an image into blocks of a fixed size (e.g., 128 x 128). Other fixed block sizes may be employed. The fixed-size block is also referred to as a coding tree unit (CTU). The 102 divider then divides each fixed-size block into blocks of variable sizes (e.g., 64 x 64 or smaller), based on recursive quadtree and / or binary tree block division. In other words, the 102 divider selects a division pattern. The variable-size block is also referred to as a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). It should be noted that in many types of processing examples, there is no need to differentiate between CU, PU, and TU; All or some of the blocks in an image can be processed in units of one CU, one PU, or one TU.
[00131] Figure 3 is a conceptual diagram illustrating an example of block splitting according to an embodiment. In Figure 3, the solid lines represent the block boundaries of the blocks split by quadtree block splitting, and the dashed lines represent the block boundaries of the blocks split by binary tree block splitting.
[00132] Here, block 10 is a square block with dimensions of 128 x 128 pixels (128 x 128 block). This 128 x 128 block is first divided into four 64 x 64 square blocks (quadtree block division).
[00133] The top left 64 x 64 block is further vertically split into two rectangular 32 x 64 blocks, and the left 32 x 64 block is further vertically split into two rectangular 16 x 64 blocks (binary tree block splitting). As a result, the top left 64 x 64 block is split into two 16 x 64 blocks 11 and 12 and one 32 x 64 block 13.
[00134] The upper right 64 x 64 block is horizontally divided into two rectangular 64 x 32 blocks 14 and 15 (binary tree block division).
[00135] The lower left 64 x 64 block is first divided into four 32 x 32 square blocks (quadtree block division). The upper left and lower right blocks, among the four 32 x 32 blocks, are further divided. The upper left 32 x 32 block is vertically divided into two Petition 870260103461, dated 07 / 10 / 2026, page 28 / 376 19 / 121 rectangular blocks of 16 x 32, and the rightmost 16 x 32 block is further horizontally divided into two 16 x 16 blocks (binary tree block division). The bottom rightmost 32 x 32 block is horizontally divided into two 32 x 16 blocks (binary tree block division). As a result, the bottom leftmost 64 x 64 block is divided into one 16 x 32 block (16), two 16 x 16 blocks (17 and 18), two 32 x 32 blocks (19 and 20), and two 32 x 16 blocks (21 and 22).
[00136] The bottom right block of 64 x 64 23 is not divided.
[00137] As described above in Figure 3, block 10 is divided into thirteen blocks of varying sizes from 11 to 23 based on recursive quadtree and binary tree block splitting. This type of splitting is also referred to as quadtree plus binary tree splitting (QTBT).
[00138] It should be noted that in Figure 3, a block is split into four or two blocks (quadtree or binary tree block splitting), but splitting is not limited to these examples. For example, a block can be split into three blocks (ternary block splitting). Splitting including such ternary block splitting is also referred to as multi-type tree splitting (MBT). Image Structure: Slice / “tile”
[00139] An image can be configured in units of one or more slices or “tiles” in order to decode the image in parallel. The image configured in units of one or more slices or “tiles” can be configured by the divider 102.
[00140] Slices are basic encoding units included in an image. An image may include, for example, one or more slices. Additionally, a slice includes one or more successive encoding tree units (CTUs).
[00141] Figure 4A is a conceptual diagram illustrating an example of a slice configuration. For example, an image includes 11 x 8 CTUs and is divided into four slices (slices 1 to 4). Slice 1 includes sixteen CTUs, slice 2 includes twenty-one CTUs, slice 3 includes twenty-nine CTUs, and slice 4 includes twenty-two CTUs. Here, each CTU in the image belongs to one of the slices. The shape of each slice is a shape obtainable by dividing the image horizontally. A boundary of each slice does not need to coincide with an end of the image, and can coincide with any of the boundaries between the CTUs in the image. The processing order of the CTUs in a slice (an encoding order or a decoding order) is, for example, Petition 870260103461, dated 07 / 10 / 2026, page 29 / 376 20 / 121 is a "raster" scan order. A slice includes header information and encoded data. The characteristics of the slice can be described in the header information. Characteristics include a CTU address, a higher CTU address in the slice, a slice type, etc.
[00142] A “tile” is a unit of a rectangular region included in an image. Each of the “tiles” can receive a number referred to as TileId in the raster scanning order.
[00143] Figure 4B is a conceptual diagram showing an example of a "tile" configuration. For example, an image includes 11 x 8 CTUs and is divided into four "tiles" of rectangular regions ("tiles" 1 to 4). When "tiles" are used, the processing order of the CTUs is changed from the processing order in the case where no "tile" is used. When no "tile" is used, the CTUs in an image are processed in raster scanning order. When "tiles" are used, at least one CTU in each of the "tiles" is processed in raster scanning order. For example, as illustrated in Figure 4B, the processing order of the CTUs included in tile 1 is the order that starts from the left edge of the first row of tile 1 towards the right edge of the first row of tile 1, and then starts from the left edge of the second row of tile 1 towards the right edge of the second row of tile 1.
[00144] It should be noted that a “tile” can include one or more slices, and a slice can include one or more “tiles”. Subtractor
[00145] Subtractor 104 subtracts a forecast signal (forecast sample that is recorded from forecast controller 128 indicated below) from an original signal (original sample) in block units recorded from divisor 102 and divided by divisor 102. In other words, subtractor 104 calculates the forecast errors (also referred to as residuals) of a block to be encoded (hereinafter also referred to as a current block). Subtractor 104 then sends the calculated forecast errors (residuals) to transformer 106.
[00146] The original signal is a signal that was recorded in encoder 100 and represents one image of each image included in a video (for example, a Petition 870260103461, dated 07 / 10 / 2026, page 30 / 376 21 / 121 luminescence signal and two chrominance signals). Hereafter, a signal representing an image is also referred to as a sample. Transformer
[00147] Transformer 106 transforms spatial domain forecast errors into frequency domain transformation coefficients, and sends the transformation coefficients to quantizer 108. More specifically, transformer 106 applies, for example, a defined discrete cosine transform (DCT) or discrete sine transform (DST) to spatial domain forecast errors. The defined DCT or DST can be predefined.
[00148] It should be noted that transformer 106 can adaptively select a transformation type from a plurality of transformation types, and the transformation prediction errors in transformation coefficients by using a basic transformation function corresponding to the selected transformation type. This type of transformation is also referred to as explicit multi-core transformation (EMT) or adaptive multiple transformations (AMT).
[00149] Transformation types include, for example, DCT-II, DCT-V, DCTVIII, DST-I, and DST-VII. Figure 5A is a graph indicating basic transformation functions for the illustrative transformation types. In Figure 5A, N indicates the number of input pixels. For example, the selection of a transformation type, from among the plurality of transformation types, may depend on a forecast type (one among intraforecast and interforecast), and may depend on an intraforecast mode.
[00150] Information indicating whether such EMT or AMT applies (referred to as, for example, EMT indicator or AMT indicator) and information indicating the type of transformation selected are normally signaled at the CU level. It should be noted that signaling of such information does not necessarily have to be done at the CU level, and can be done at another level (for example, at the bit sequence level, image level, slice level, tile level or CTU level).
[00151] Additionally, transformer 106 can perform a new transformation of the transformation coefficients (transformation result). This new transformation is also referred to as adaptive secondary transformation. Petition 870260103461, dated 07 / 10 / 2026, p. 31 / 376 22 / 121 (AST) or non-separable secondary transform (NSST). For example, the 106 transformer performs the new transformation in units of a sub-block (e.g., the 4 x 4 sub-block) included in a block of transformation coefficients that corresponds to an intraprediction error. Information indicating whether NSST applies and information related to a transformation matrix for use in NSST is normally signaled at the CU level. It should be noted that signaling of such information does not necessarily need to be performed at the CU level and may be performed at another level (e.g., sequence level, image level, slice level, tile level, or CTU level).
[00152] The 106 transformer can employ a separable transformation or a non-separable transformation. A separable transformation is a method in which a transformation is performed multiple times by separately performing a transformation for each of the various directions, according to the number of input dimensions. A non-separable transformation is a method of performing a collective transformation in which two or more dimensions in the multidimensional inputs are collectively considered a single dimension.
[00153] In an example of a non-separable transformation, when an input is a 4 x 4 block, the 4 x 4 block is considered a single set including sixteen elements, and the transformation applies a 16 x 16 transformation matrix to the set.
[00154] In another example of a non-separable transformation, a 4 x 4 input block is considered a single set including sixteen elements, and then a transformation (Givens hypercube transformation), in which the Givens revolution is performed on the set multiple times, can be carried out.
[00155] In the transformation in transformer 106, the types of basis sets to be transformed in the frequency domain, according to the regions in a CU, can be switched. Examples include spatially variable transformations (SVT). In SVT, as illustrated in Figure 5B, the CUs are divided into two equal regions, horizontally or vertically, and only one of the regions is transformed in the frequency domain. A type of basis set can be determined for each region. For example, DST7 and DST8 are used. In this example, only one of these two regions in the CU is transformed, and the other is not transformed. In Petition 870260103461, dated 07 / 10 / 2026, page 32 / 376 23 / 121 However, both of these regions can be transformed. Additionally, the splitting method is not limited to splitting into two equal regions and can be more flexible. For example, the CU can be split into four equal regions, or the information indicating the split can be encoded separately and signaled in the same way as the CU split. It should be noted that SVT is also referred to as a sub-block transformation (SBT). Quantizer
[00156] Quantizer 108 quantizes the transformation coefficients sent from transformer 106. More specifically, quantizer 108 digitizes, in a determined digitization order, the transformation coefficients of the current block and quantizes the digitized transformation coefficients based on the quantization parameters (QP) corresponding to the transformation coefficients. Quantizer 108 then sends the quantized transformation coefficients (hereinafter also referred to as quantized coefficients) of the current block to the entropy encoder 110 and the inverse quantizer 112. The determined digitization order can be predetermined.
[00157] A determined digitization order is an order for quantization / inverse quantization transformation coefficients. For example, a determined digitization order can be defined as the ascending frequency order (from low to high frequency) or the descending frequency order (from high to low frequency).
[00158] A quantization parameter (QP) is a parameter that defines a quantization step (quantization width). For example, when the value of the quantization parameter increases, the quantization step also increases. In other words, when the value of the quantization parameter increases, the quantization error increases.
[00159] Additionally, a quantization matrix can be used for quantization. For example, various types of quantization matrices can be used corresponding to frequency transformation sizes, such as 4 x 4 and 8 x 8, prediction modes, such as intraprediction and interprediction, and pixel components, such as luminance and chrominance pixel components. It should be noted that quantization means the sweeping of values Petition 870260103461, dated 07 / 10 / 2026, p. 33 / 376 24 / 121 samples are sampled at predetermined intervals corresponding to the determined levels. In this technical field, quantization can be referred to using other expressions, such as rounding and scaling, and rounding and scaling can be employed. The determined intervals and levels can be predetermined.
[00160] Methods using quantization matrices include a method using a quantization matrix that has been configured directly on the encoder side and a method using a quantization matrix that has been configured as a default (default matrix). On the encoder side, a quantization matrix suitable for the characteristics of an image can be configured by directly configuring a quantization matrix. This case, however, presents a disadvantage of increasing an amount of encoding for encoding the quantization matrix.
[00161] There is a method for quantizing a high-frequency coefficient and a low-frequency coefficient without using a quantization matrix. It should be noted that this method is equivalent to a method using a quantization matrix (planar matrix) whose coefficients have the same value.
[00162] The quantization matrix can be specified using, for example, a sequence parameter set (SPS) or an image parameter set (PPS). SPS includes a parameter that is used for a sequence, and PPS includes a parameter that is used for an image. Each of the SPS and PPS can simply be referred to as a parameter set. Entropy Encoder
[00163] The entropy encoder 110 generates an encoded signal (a sequence of encoded bits) based on quantized coefficients that were recorded from the quantizer 108. More specifically, the entropy encoder 110, for example, binarizes the quantized coefficients, and arithmetically encodes the binary signal, and sends a compressed bit sequence. Inverse Quantizer
[00164] The inverse quantizer 112 inversely quantizes the quantized coefficients that were recorded from the quantizer 108. More specifically, the inverse quantizer 112 inversely quantizes, in one digit order. Petition 870260103461, dated 07 / 10 / 2026, page 34 / 376 25 / 121 determined, the quantized coefficients of the current block. The inverse quantizer 112 then sends the inversely quantized transformation coefficients of the current block to the inverse transformer 114. The determined digitization order can be predetermined. Reversing Transformer
[00165] The inverse transformer 114 restores the (residual) prediction errors by inversely transforming the transformation coefficients that were recorded from the inverse quantizer 112. More specifically, the inverse transformer 114 restores the prediction errors of the current block by applying an inverse transformation corresponding to the transformation applied by transformer 106 to the transformation coefficients. The inverse transformer 114 then sends the restored prediction errors to the adder 116.
[00166] It should be noted that since information is lost in quantization, the restored forecast errors do not match the forecast errors calculated by the subtractor 104. In other words, the restored forecast errors typically include quantization errors. Adder
[00167] Adder 116 reconstructs the current block by adding prediction errors that were recorded from the inverse transformer 114 and prediction samples that were recorded from the prediction controller 128. Adder 116 then sends the reconstructed block to the block memory 118 and circuit filter 120. A reconstructed block is also referred to as a local decoded block. Block Memory
[00168] Block memory 118 is, for example, the storage for storing the blocks in an image to be encoded (hereinafter referred to as a current image), which is referred to in intraprediction. More specifically, block memory 118 stores the reconstructed blocks sent from adder 116. Frame Memory
[00169] Frame memory 112 is, for example, the storage for storing reference images for use in interprediction, and is also referred to as a frame store. More specifically, frame memory 122 Petition 870260103461, dated 07 / 10 / 2026, p. 35 / 376 26 / 121 stores the reconstructed blocks filtered by the 120 circuit filter. Circuit Filter
[00170] Circuit filter 120 applies a circuit filter to the blocks reconstructed by the adder 116, and sends the filtered reconstructed blocks to the frame memory 122. A circuit filter is a filter used in an encoding circuit (in-circuit filter) and includes, for example, a deblock filter (DF or DBF), an adaptive sample shift (SAO), and an adaptive circuit filter (ALF).
[00171] In an ALF, a mean square error filter is applied to remove compression artifacts. For example, a filter selected from a plurality of filters, based on the direction and activity of local gradients, is applied to each of the 2 x 2 sub-blocks in the current block.
[00172] More specifically, first, each sub-block (e.g., each 2 x 2 subblock) is categorized into one of a plurality of classes (e.g., fifteen or twenty-five classes). The sub-block classification is based on gradient directionality and activity. For example, the classification index C (e.g., C = 5D + A) is derived based on gradient directionality D (e.g., 0 to 2 or 0 to 4) and gradient activity A (e.g., 0 to 4). Then, based on the classification index C, each sub-block is categorized into one of a plurality of classes.
[00173] For example, gradient directionality D is calculated by comparing the gradients of a plurality of directions (e.g., the horizontal direction, the vertical direction, and two diagonal directions). Furthermore, for example, gradient activity A is calculated by adding gradients from a plurality of directions and quantizing the result of the addition.
[00174] The filter to be used for each sub-block is determined from among a plurality of filters based on the result of such categorization.
[00175] The filter format to be used in an ALF is, for example, a circular symmetrical filter format. Figures 6A to 6C illustrate examples of filter formats used in ALFs. Figure 6A illustrates a 5 x 5 diamond-shaped filter, Figure 6B illustrates a 7 x 7 diamond-shaped filter, and Figure 6C illustrates a 9 x 9 diamond-shaped filter. Information indicating the filter format is normally indicated at the image level. It should be noted that such indication Petition 870260103461, dated 07 / 10 / 2026, p. 36 / 376 27 / 121 Information indicating the filter format does not necessarily need to be performed at the image level and can be performed at another level (for example, at the sequence level, slice level, tile level, CTU level, or CU level).
[00176] The ON or OFF of ALF is determined, for example, at the image level or CU level. For example, the decision to apply ALF to luminescence can be made at the CU level, and the decision to apply ALF to chrominance can be made at the image level. The information indicating whether ALF is ON or OFF is normally signaled at the image level or CU level. It should be noted that the signaling of the information indicating whether ALF is ON or OFF does not necessarily have to be done at the image level or CU level, and can be done at another level (for example, at the sequence level, slice level, tile level or CTU level).
[00177] The coefficient determined for the plurality of selectable filters (e.g., fifteen or even twenty-five filters) is normally signaled at the image level. It should be noted that signaling of the determined coefficient does not necessarily have to be done at the image level, and can be done at another level (e.g., at the sequence level, slice level, tile level, CTU level, CU level, or sub-block level). Circuit Filter > Unblocking Filter
[00178] In a block filter, the 120 circuit filter performs a filtering process at a block boundary in a reconstructed image, so as to reduce the distortion that occurs at the block boundary.
[00179] Figure 7 is a block diagram illustrating an example of a specific configuration of the 120 circuit filter that functions as an unblocking filter.
[00180] The 120 circuit filter includes the 1201 limit determiner; the 1203 filter determiner; the 1206 filter executor; the 1208 processor determiner; the 1207 filter characteristic determiner; and the 1202, 1204 and 1206 switches.
[00181] Boundary determinant 1201 determines whether a pixel to be filtered by unlocking (i.e., a current pixel) is present around a block boundary. Boundary determinant 1201 then sends the result of the determination to switch 1202 and processing determinant 1208. Petition 870260103461, dated 07 / 10 / 2026, page 37 / 376 28 / 121
[00182] In the case where boundary determiner 1201 has determined that a current pixel is present around a block boundary, switcher 1202 sends an unfiltered image to switcher 1204. In the opposite case where boundary determiner 1201 has determined that no current pixel is present around a block boundary, switcher 1202 sends an unfiltered image to switcher 1206.
[00183] Filter determinant 1203 determines whether to perform unlock filtering of the current pixel, based on the pixel value of at least one surrounding pixel located around the current pixel. Filter determinant 1203 then sends the determination result to switch 1204 and processing determinant 1208.
[00184] In the case where filter determiner 1203 determined that the current pixel unlock filtering should be performed, switch 1204 sends the unfiltered image obtained through switch 1202 to filter executor 1205. In the opposite case, where filter determiner 1203 determined that the current pixel unlock filtering should not be performed, switch 1204 sends the unfiltered image obtained through switch 1202 to switch 1206.
[00185] When the unfiltered image is obtained through switches 1202 and 1204, filter executor 1205 performs, for the current pixel, the unlock filtering with the filter characteristic determined by the filter characteristic determiner 1207. Filter executor 1205 then sends the filtered pixel to switch 1206.
[00186] Under the control of processing determinant 1208, switch 1206 selectively sends a pixel that was not filtered by unlocking and a pixel that was filtered by unlocking by filtering executor 1205.
[00187] Processing determinant 1208 controls switch 1206 based on the results of the determination made by boundary determinant 1201 and filter determinant 1203. In other words, processing determinant 1208 causes switch 1206 to send the pixel that was filtered by unlocking when boundary determinant 1201 determines that the current pixel is present around the block boundary and filter determinant 1203 has determined that unlock filtering of the current pixel has been performed. Additionally, besides the above case, processing determinant 1208 causes switch 1206 to send the pixel Petition 870260103461, dated 07 / 10 / 2026, page 38 / 376 29 / 121 which was not filtered by unlocking. A filtered image is sent from switch 1206 by repeatedly sending a pixel in this way.
[00188] Figure 8 is a conceptual diagram showing an example of a deblocking filter having a symmetrical filtering characteristic with respect to a block boundary.
[00189] In an unlocking filter process, one of two unlocking filters with different characteristics, i.e., a strong filter and a weak filter, is selected using pixel values and quantization parameters. In the case of a strong filter, pixels p0 to p2 and pixels q0 to q2 are present across a block boundary as illustrated in Figure 8; the pixel values of the respective pixel q0 to q2 are changed to pixel values q'0 to q'2 by performing, for example, computations according to the expressions below. q'0 = (p1 + 2 x p0 + 2 x q0 + 2 q1 + q2 + 4) / 8 q'1 = (p0 + q0 + q1 + q2 +2) / 4 q'2 = (p0 + q0 + q1 + 3 x q2 + 2 x q3 + 4) / 8
[00190] It should be noted that, in the expressions above, p0 to p2 and q0 to q2 are the pixel values of the respective pixels p0 to p2 and q0 to q2. Additionally, q3 is the pixel value of the neighboring pixel q3 located on the opposite side of pixel q2 with respect to the block boundary. Additionally, on the right-hand side of each of the expressions, the coefficients that are multiplied by the respective pixel values of the pixels to be used for unlock filtering are filter coefficients.
[00191] Additionally, in unlock filtering, fixing can be performed so that the calculated pixel values are not determined above a threshold value. In the fixing process, the pixel values calculated according to the expressions above are fixed to a value obtained according to a computation pixel value + / - 2 x a threshold value using the threshold value determined based on a quantization parameter. In this way, it is possible to prevent excessive smoothing.
[00192] Figure 9 is a conceptual diagram to illustrate a block boundary in which an unblocking filter process is performed. Figure 10 is a diagram Petition 870260103461, dated 07 / 10 / 2026, page 39 / 376 30 / 121 conceptual indicating examples of B values.
[00193] The block boundary on which the unlocking filter process is performed is, for example, a boundary between prediction units (PU) having 8 x 8 pixel blocks, as illustrated in Figure 9, or a boundary between transformation units (TU). The unlocking filter process can be performed on units of four rows or four columns. First, the threshold intensity values (Bs) are determined, as indicated in Figure 10, for block P and block Q, illustrated in Figure 9.
[00194] According to the Bs values in Figure 10, the unlocking filter processes for the block boundaries belonging to the same image using different intensities are determined. The unlocking filter process for a chrominance signal is performed when a Bs value is equal to 2. The unlocking filter process for a luminescence signal is performed when a Bs value is equal to 1 or more and a specific condition is met. The specific condition can be predetermined. It should be noted that the conditions for determining the Bs values are not limited to those indicated in Figure 10, and a Bs value can be determined based on another parameter. Forecast Processor (Intrapredictor, Interpredictor, Forecast Controller)
[00195] Figure 11 is a flowchart illustrating an example of a process performed by the encoder 100's prediction processor. It should be noted that the prediction processor includes all or part of the following constituent elements: intrapredictor 124; interpredictor 126; and prediction controller 128.
[00196] The forecast processor generates a forecast image of a current block (Step Sb_1). This forecast image is also referred to as a forecast signal or a forecast block. Note that the forecast signal, for example, is either an intra-forecast signal or an inter-forecast signal. Specifically, the forecast processor generates the forecast image of the current block using a reconstructed image that has already been obtained through forecast block generation, difference block generation, coefficient block generation, difference block restoration, and decoded image block generation.
[00197] The reconstructed image can be, for example, an image within a reference image, or an image of a block encoded within an actual image. Petition 870260103461, dated 07 / 10 / 2026, page 40 / 376 31 / 121 is the image that includes the current block. The block encoded in the current image, for example, is a neighboring block of the current block.
[00198] Figure 12 is a flowchart illustrating another example of a process performed by the encoder 100 prediction processor.
[00199] The forecast processor generates a forecast image using a first method (Step Sc_1a), generates a forecast image using a second method (Step Sc_1b), and generates a forecast image using a third method (Step Sc_1c). The first method, the second method, and the third method may be mutually different methods for generating a forecast image. Each of the first to third methods may be an interforecast method, an intraforecast method, or another forecast method. The reconstructed image described above may be used in these forecast methods.
[00200] Next, the forecast processor selects any one of a plurality of forecast methods generated in Steps Sc_1a, Sc_1b, and Sc_1c (Step Sc_2). The selection of the forecast image, which is the selection of a method or mode to obtain a final forecast image, can be performed by calculating a cost for each of the generated forecast images and based on the cost. Alternatively, the selection of the forecast image can be performed based on a parameter that is used in an encoding process. Encoder 100 can transform the information to identify a selected forecast image, a method, or a mode into an encoded signal (also referred to as a sequence of encoded bits). The information can be, for example, an indicator or similar. In this way, the decoder is able to generate a forecast image according to the method or mode selected based on the information in encoder 100.It should be noted that, in the example illustrated in Figure 12, the forecast processor selects any of the forecast images after the forecast images have been generated using the respective methods. However, the forecast processor can select a method or mode based on a parameter for use in the coding process described above, before the generation of forecast images, and can generate a forecast image according to the selected method or mode.
[00201] For example, the first method and the second method can be Petition 870260103461, dated 07 / 10 / 2026, page 41 / 376 32 / 121 intraprediction and interprediction, respectively, and the forecast processor can select a final forecast image for a current block from the forecast images generated according to the forecasting methods.
[00202] Figure 13 is a flowchart illustrating another example of a process performed by the encoder 100 prediction processor.
[00203] First, the forecast processor generates a forecast image using intraforecasting (Step Sd_1a), and generates a forecast image using interforecasting (Step Sd_1b). It should be noted that the forecast image generated by intraforecasting is also referred to as an intraforecast image, and the forecast image generated by interforecasting is also referred to as an interforecast image.
[00204] Next, the forecast processor evaluates each intraprediction image and the interprediction image (Step Sd_2). A cost can be used in the evaluation. In other words, the forecast processor calculates the cost C for each of the intraprediction and interprediction images. The cost C can be calculated according to an expression from an RD optimization model, for example, C = D + λ x R. In this expression, D indicates a coding distortion of a forecast image, and is represented as, for example, a sum of the absolute differences between the pixel value of a current block and the pixel value of a forecast image. Additionally, R indicates a predicted coding amount of a forecast image, specifically, the amount of coding needed to encode the motion information to generate a forecast image, etc. Additionally, λ indicates, for example, a multiplier, according to the Lagrange multiplier method.
[00205] The forecast processor then selects the forecast image for which the lowest cost C was calculated among the intra-forecast image and the inter-forecast image, as the final forecast image for the current block (Step Sd_3). In other words, the forecast method or mode for generating the forecast image for the current block is selected. Intrapredictor
[00206] Intrapredictor 124 generates a forecast signal (intraprediction signal) by performing the intraprediction (also referred to as intraframe forecast) of the block. Petition 870260103461, dated 07 / 10 / 2026, page 42 / 376 33 / 121 current by reference to a block or blocks in the current image and stored in block memory 118. More specifically, the intrapredictor 124 generates an intraprediction signal by performing intraprediction by reference to samples (e.g., luminescence and / or chrominance values) of a block or blocks neighboring the current block, and then sends the intraprediction signal to the prediction controller 128.
[00207] For example, the intrapredictor 124 performs intraprediction by using one mode from a plurality of intraprediction modes that have been defined. The intraprediction modes include one or more non-directional prediction modes and a plurality of directional prediction modes. The defined modes can be predefined.
[00208] One or more non-directional prediction modes include, for example, flat prediction mode and DC prediction mode defined in the H.265 / High Efficiency Video Coding (HEVC) standard.
[00209] The plurality of directional forecasting modes includes, for example, thirty-three directional forecasting modes defined in the H.265 / HEVC standard. It should be noted that the plurality of directional forecasting modes may additionally include thirty-two directional forecasting modes in addition to the thirty-three directional forecasting modes (for a total of sixty-five directional forecasting modes). Figure 14 is a conceptual diagram illustrating sixty-seven intraforecasting modes in total that can be used in intraforecasting (two non-directional forecasting modes and sixty-five directional forecasting modes). The solid arrows represent the thirty-three directions defined in the H.265 / HEVC standard, and the dashed arrows represent the thirty-two additional directions (the two non-directional forecasting modes are not illustrated in Figure 14).
[00210] In several types of processing examples, a luminescence block can be referenced in the intraprediction of a chrominance block. In other words, a chrominance component of the current block can be predicted based on a luminescence component of the current block. Such intraprediction is also referred to as a cross-component linear model (CCLM) prediction. The intraprediction mode for a chrominance block in which such a luminescence block is referenced (also referred to, for example, as a CCLM mode) can be added as one of the intraprediction modes for chrominance blocks. Petition 870260103461, dated 07 / 10 / 2026, p. 43 / 376 34 / 121
[00211] The 124 intrapredictor can correct intraprediction pixel values based on horizontal / vertical reference pixel gradients. Intraprediction accompanied by this type of correction is also referred to as a combination of position-dependent intraprediction (PDPC). Information indicating whether you apply PDPC (referred to as, for example, a PDPC indicator) is normally signaled at the CU level. It should be noted that signaling of such information does not necessarily have to be done at the CU level, and can be done at another level (e.g., sequence level, image level, slice level, tile level, or CTU level). Interprevisor
[00212] Interpredictor 126 generates a prediction signal (interprediction signal) by performing interprediction (also referred to as interframe prediction) of the current block by reference to a block or blocks in a reference image, which is different from the current image and is stored in frame memory 122. Interprediction is performed in units of a current block or a current sub-block (e.g., a 4 x 4 block) within the current block. For example, interpredictor 126 performs motion estimation on a reference image for the current block or current sub-block, and discovers a reference block or sub-block that best matches the current block or current sub-block. Interpredictor 126 then obtains motion information (e.g., a motion vector) that compensates for a motion or change from the reference block or sub-block to the current block or sub-block.Interpredictor 126 generates an interprediction signal for the current block or sub-block by performing motion compensation (or motion prediction) based on motion information. Interpredictor 126 sends the generated interprediction signal to forecast controller 128.
[00213] The motion information used in motion compensation can be signaled as interprediction signals in various forms. For example, a motion vector can be signaled. As another example, the difference between a motion vector and a motion vector predictor can be signaled. Basic Interprediction Flow
[00214] Figure 15 is a flowchart illustrating a basic processing flow illustrative of interprediction. Petition 870260103461, dated 07 / 10 / 2026, page 44 / 376 35 / 121
[00215] First, the interpredictor 126 generates a prediction signal (Steps Se_1 to Se_3). Next, the subtractor 104 generates the difference between a current block and a prediction image as a prediction residual (Step Se_4).
[00216] Here, in the generation of the prediction image, the 126 interpredictor generates the prediction image by determining a motion vector (MV) of the current block (Steps Se_1 and Se_2) and motion compensation (Step Se_3). Additionally, in determining an MV, the 126 interpredictor determines the MV by selecting a motion vector candidate (MV candidate) (Step Se_1) and deriving an MV (Step Se_2). The selection of the MV candidate is done, for example, by selecting at least one MV candidate from a list of MV candidates. Alternatively, in deriving an MV, the 126 interpredictor can additionally select at least one MV candidate from at least one MV candidate, and determine the selected at least one MV candidate as the MV for the current block.Alternatively, the 126 interpredictor can determine the MV for the current block by performing the estimate on a reference image region specified by each of at least one selected MV candidate. It should be noted that the estimate on a reference image region can be referred to as motion estimation.
[00217] Additionally, although Steps Se_1 to Se_3 are performed by interprevisor 126 in the example described above, a process that is, for example, Step Se_1, Step Se_2 or similar, may be performed by another constituent element included in encoder 100. Vector Derivation Flow
[00218] Figure 16 is a flowchart illustrating an example of deriving motion vectors.
[00219] The 126 interpredictor derives a MV from a current block in a mode to encode motion information (e.g., a MV). In this case, for example, the motion information is encoded as a prediction parameter, and is signaled. In other words, the encoded motion information is included in an encoded signal (also referred to as an encoded bit sequence).
[00220] Alternatively, the 126 interpredictor derives a MV in a mode in which motion information is not encoded. In this case, no information of Petition 870260103461, dated 07 / 10 / 2026, page 45 / 376 36 / 121 movement is included in an encoded signal.
[00221] Here, MV derivation modes may include a normal intermode, a blending mode, a FRUC mode, an affine mode, etc., which will be described later. Modes in which motion information is encoded between modes include the normal intermode, the blending mode, the affine mode (specifically, an affine intermode and an affine blending mode), etc. It should be noted that motion information may include not only an MV but also motion vector predictor selection information, which will be described later. Modes in which no motion information is encoded include the FRUC mode, etc. Interpredictor 126 selects a mode to derive a MV from the current block from modes, and derives the MV from the current block using the selected mode.
[00222] Figure 17 is a flowchart illustrating another example of deriving motion vectors.
[00223] The 126 interpredictor derives a MV from a current block in a mode in which an MV difference is encoded. In this case, for example, the MV difference is encoded as a prediction parameter, and is signaled. In other words, the encoded MV difference is included in an encoded signal. The MV difference is the difference between the MV of the current block and the MV predictor.
[00224] Alternatively, the 126 interpredictor derives an MV in a mode in which no MV difference is encoded. In this case, no encoded MV difference will be included in an encoded signal.
[00225] Here, as described above, the MV derivation modes include the normal intermode, the mixing mode, the FRUC mode, the affine mode, etc., which will be described later. The modes in which an MV difference is encoded, among the modes, include the normal intermode, the affine mode (specifically, the affine intermode), etc. The modes in which no MV difference is encoded include the FRUC mode, the mixing mode, the affine mode (specifically, the affine mixing mode), etc. The 126 interpredictor selects a mode to derive a MV from the current block from the plurality of modes, and derives the MV from the current block using the selected mode. Vector Derivation Flow Petition 870260103461, dated 07 / 10 / 2026, page 46 / 376 37 / 121
[00226] Figure 18 is a flowchart illustrating another example of motion vector derivation. The modes of MV derivation, which are the interprediction modes, include a plurality of modes and are roughly divided into modes in which a MV difference is encoded and modes in which no motion vector difference is encoded. The modes in which no MV difference is encoded include the blending mode, the FRUC mode, the affine mode (especially the affine blending mode), etc. These modes are described in detail later. Simply put, the blending mode is a mode for deriving a MV from a current block by selecting a motion vector from a surrounding encoded block, and the FRUC mode is a mode for deriving a MV from a current block by performing the estimate between the encoded regions.The affine mode is a way to derive, as a motion vector of a current block, a motion vector of each of a plurality of sub-blocks included in the current block, assuming the affine transformation.
[00227] More specifically, as illustrated when the interprediction mode information indicates 0 (0 in Sf_1), the interpredictor 126 derives a motion vector using the blending mode (Sf_2). When the interprediction mode information indicates 1 (1 in Sf_1), the interpredictor 126 derives a motion vector using the FRUC mode (Sf_3). When the interprediction mode information indicates 2 (2 in Sf_1), the interpredictor 126 derives a motion vector using an affine mode (specifically, the affine blending mode) (Sf_4). When the interprediction mode information indicates 3 (3 in Sf_1), the interpredictor 126 derives a motion vector using a mode in which a MV difference is encoded (e.g., a normal intermode (Sf_5)). MV Derivation > Normal Intermode
[00228] Normal intermode is an interprediction mode for deriving a MV from a current block based on a block similar to the current block image from a reference image region specified by a candidate MV. In this normal intermode, a MV difference is encoded.
[00229] Figure 19 is a flowchart illustrating an example of interprediction in normal intermode.
[00230] First, the interpredictor 126 obtains a plurality of MV candidates Petition 870260103461, dated 07 / 10 / 2026, page 47 / 376 38 / 121 for a current block based on information, such as MVs among a plurality of temporally or spatially encoded blocks surrounding the current block (Step Sg_1). In other words, the 126 interpredictor generates a list of MV candidates.
[00231] Next, the interpredictor 126 extracts N (an integer equal to or greater than 2) candidates from the plurality of MV candidates obtained in Step Sg_1, as motion vector predictor candidates (also referred to as MV predictor candidates), according to a determined priority order (Step Sg_2). It should be noted that the priority order can be determined in advance for each of the N MV candidates.
[00232] Next, the 126 interpredictor selects a motion vector predictor candidate from the N motion vector predictor candidates as the motion vector predictor (also referred to as a MV predictor) of the current block (Step Sg_3). At this point, the 126 interpredictor encodes the motion vector predictor selection information into a sequence to identify the selected motion vector predictor. It should be noted that the sequence is an encoded signal or a sequence of encoded bits as described above.
[00233] Next, the 126 interpredictor derives a MV from a current block by reference to an encoded reference image (Step Sg_4). At this point, the 126 interpredictor further encodes, in sequence, the difference value between the derived MV and the motion vector predictor as a MV difference. It should be noted that the encoded reference image is an image that includes a plurality of blocks that have been reconstructed after being encoded.
[00234] Finally, the interpredictor 126 generates a prediction image for the current block by performing motion compensation of the current block using the derived MV and the encoded reference image (Step Sg_5). It should be noted that the prediction image is an interprediction signal as described above.
[00235] Additionally, the information indicating the interprediction mode (the normal intermode in the example above) used to generate the prediction image is, for example, encoded as a prediction parameter.
[00236] It should be noted that the VM candidate list can also be used as a list for use in another mode. Additionally, the related processes Petition 870260103461, dated 07 / 10 / 2026, p. 48 / 376 39 / 121 with the list of MV candidates can be applied to processes related to the list for use in another mode. Processes related to the list of MV candidates include, for example, extracting or selecting an MV candidate from the list of MV candidates, reordering the MV candidates, or deleting an MV candidate. MV Derivation > Mixing Mode
[00237] The blending mode is an interprediction mode for selecting a MV candidate from a list of MV candidates as an MV of a current block, thus deriving the MV.
[00238] Figure 20 is a flowchart illustrating an example of interprediction in mixing mode.
[00239] First, the 126 interpredictor obtains a plurality of MV candidates for a current block based on information such as MVs from a plurality of encoded blocks that temporally or spatially surround the current block (Step Sh_1). In other words, the 126 interpredictor generates a list of MV candidates.
[00240] Next, the 126 interpredictor selects a MV candidate from the plurality of MV candidates obtained in Step Sh_1, thus deriving an MV from the current block (Step Sh_2). At this point, the 126 interpredictor encodes the MV selection information in a sequence to identify the selected MV candidate.
[00241] Finally, the interpredictor 126 generates a prediction image for the current block by performing motion compensation of the current block using the derived MV and the encoded reference image (Step Sh_3).
[00242] Additionally, the information indicating the interprediction mode (mixing mode in the example above) used to generate the prediction image and included in the encoded signal is, for example, encoded as a prediction parameter.
[00243] Figure 21 is a conceptual diagram to illustrate an example of a motion vector derivation process from a current image in blending mode.
[00244] First, a list of MV candidates, in which the forecaster candidates Petition 870260103461, dated 07 / 10 / 2026, page 49 / 376 40 / 121 of MV are recorded, and it is generated. Examples of MV predictor candidates include: spatially neighboring MV predictors, which are MVs of a plurality of encoded blocks spatially located around a current block; temporally neighboring MV predictors, which are MVs of surrounding blocks onto which the position of a current block, in an encoded reference image, is projected; combined MV predictors, which are MVs generated by combining the MV value of a spatially neighboring MV predictor and the MV of a temporally neighboring MV predictor; and a zero MV predictor, which is an MV having a value equal to zero.
[00245] Next, an MV predictor is selected from a plurality of MV predictors registered in an MV predictor list, and the selected MV predictor is determined as the MV for a current block.
[00246] Additionally, the variable-length encoder describes and encodes, in a sequence, merge_idx, which is a signal indicating that the MV predictor was selected.
[00247] It should be noted that the MV forecasters listed in the MV forecaster list described in Figure 21 are examples. The number of MV forecasters may differ from the number of MV forecasters in the diagram, the MV forecaster list may be configured in such a way that some of the MV forecaster types in the diagram may not be included, or that one or more MV forecasters, in addition to the MV forecaster types in the diagram, are included.
[00248] A final MV can be determined by performing a decoder motion vector refinement (DVMR) process to be described later using the MV of the current block derived in mix mode.
[00249] It should be noted that the VM predictor candidates are the VM candidates described above, and the VM predictor list is the list of VM candidates described above. It should be noted that the VM candidate list may be referred to as a candidate list. Additionally, merge_idx is the VM selection information. MV Derivation > FRUC Mode
[00250] Motion information can be derived on the decoder side without being signaled on the encoder side. It should be noted that, as described above, the mixing mode defined in the H.265 / HEVC standard can be used. Additionally, for example, motion information can be derived by Petition 870260103461, dated 07 / 10 / 2026, p. 50 / 376 41 / 121 Decoder-side motion estimation. In one mode, on the decoder side, motion estimation is performed without using any pixel value in a current block.
[00251] Here, a mode of performing motion estimation on the decoder side is described. The mode of performing motion estimation on the decoder side may be referred to as a combined pattern motion vector derivation (PMMVD) mode, or a frame rate upconversion (FRUC) mode.
[00252] An example of a FRUC process in the form of a flowchart is illustrated in Figure 22. First, a list from a plurality of candidates, each possessing a motion vector (MV) predictor (i.e., a list of MV candidates that can also be used as a mix list) is generated by reference to a motion vector in a coded block that is spatially or temporally neighboring a current block (Step Si_1). Next, a best MV candidate is selected from the plurality of MV candidates registered in the MV candidate list (Step Si_2). For example, the evaluation values of the respective MV candidates included in the MV candidate list are calculated, and an MV candidate is selected based on the evaluation values. Based on the selected motion vector candidates, a motion vector for the current block is then derived (Step Si_4).More specifically, for example, the selected motion vector candidate (best MV candidate) is directly derived as the motion vector for the current block. Additionally, for example, the motion vector for the current block can be derived using pattern matching in a surrounding region of a position in a reference image, where the position in the reference image corresponds to the selected motion vector candidate. In other words, estimation using pattern matching and evaluation values can be performed in the surrounding region of the best MV candidate, and when there is an MV that results in a better evaluation value, the best MV candidate can be updated to the MV that results in the best evaluation value, and the updated MV can be determined as the final MV for the current block.A configuration in which none of these processes, to update the best MV candidate to the MV possessing a better valuation, is performed, is also possible. Petition 870260103461, dated 07 / 10 / 2026, page 51 / 376 42 / 121
[00253] Finally, the interpredictor 126 generates a prediction image for the current block by performing motion compensation of the current block using the derived MV and the encoded reference image (Step Si_5).
[00254] A similar process can be performed on units of a sub-block.
[00255] Evaluation values can be calculated according to various types of methods. For example, a comparison is made between an image reconstructed in a region in a reference image that corresponds to a motion vector, and an image reconstructed in a specified region (the region can be, for example, a region in another reference image or a region in a neighboring block of a current image, as indicated below). The specified region can be predetermined.
[00256] The difference between the pixel values of two reconstructed images can be used for a motion vector evaluation value. It should be noted that an evaluation value can be calculated using information beyond the difference value.
[00257] Next, an example of pattern matching is described in detail. First, a candidate MV included in a list of MV candidates (e.g., a mix list) is selected as an estimation starting point by pattern matching. For example, as pattern matching, a first pattern matching or a second pattern matching can be used. The first pattern matching and the second pattern matching are also referred to as bilateral matching and template matching, respectively. Derivation of MV > FRUC > Bilateral Combination
[00258] In the first pattern matching, the pattern matching is performed between two blocks along a movement path of a current block, which are two blocks in two different reference images. Accordingly, in the first pattern matching, a region in another reference image, along the movement path of the current block, is used as a determined region to calculate the candidate evaluation value described above. The determined region can be predetermined.
[00259] Figure 23 is a conceptual diagram to illustrate an example of the first pattern combination (bilateral combination) between the two blocks in the two images. Petition 870260103461, dated 07 / 10 / 2026, page 52 / 376 43 / 121 reference along the motion path. As illustrated in Figure 23, in the first pattern matching, two motion vectors (MV0, MV1) are derived by estimating a best-matching pair between the pairs in the two blocks in the two different reference images (Ref0, Ref1), which are two blocks along the motion path of the current block (Cur block). More specifically, a difference between the reconstructed image at a specified location in the first encoded reference image (Ref0), specified by a candidate MV, and the reconstructed image at a specified location in the second encoded reference image (Ref1), specified by a symmetrical MV, obtained by scaling the candidate MV over a display time interval, is derived for the current block, and an evaluation value is calculated using the obtained difference value.It is possible to select, as the final MV, the MV candidate who results in the best evaluation value, among the plurality of MV candidates, and who has the best chance of producing good results.
[00260] Considering a continuous motion trajectory, the motion vectors (MV0, MV1), which specify the two reference blocks, are proportional to the temporal distances (TD0, TD1) between the current image (Cr Pic) and two reference images (Ref0, Ref1). For example, when the current image is temporally located between the two reference images and the temporal distances from the current image to the two respective reference images are equal to each other, the symmetrical and mirrored bidirectional motion vectors are derived in the first pattern combination. Derivation of MV > FRUC > Template Combination
[00261] In the second pattern combination (template combination), the pattern combination is performed between a block in a reference image and a template in the current image (the template is a block adjacent to the current block in the current image (the adjacent block is, for example, a top and / or left adjacent block)). Accordingly, in the second pattern combination, the block adjacent to the current block in the current image is used as the determined region to calculate the candidate's evaluation value described above.
[00262] Figure 24 is a conceptual diagram to illustrate an example of pattern matching (template matching) between a template and an image. Petition 870260103461, dated 07 / 10 / 2026, page 53 / 376 44 / 121 current and a block in a reference image. As illustrated in Figure 24, in the second pattern combination, the motion vector of the current block (Cur block) is derived by estimation, in the reference image (Ref0), the block that best matches the block adjacent to the current block in the current image (Cur Pic). More specifically, it is possible that the difference between a reconstructed image in a coded region, which is adjacent on the left and above, or on the left or above, and a reconstructed image, which is in a corresponding region in the coded reference image (Ref0) and is specified by a MV candidate, is derived, an evaluation value is calculated using the value of the difference obtained, and the MV candidate, which results in the best evaluation value among a plurality of MV candidates, is selected as the best MV candidate.
[00263] Such information indicates whether the FRUC mode (referred to as, for example, a FRUC indicator) can be signaled at the CU level. Additionally, when the FRUC mode is applied (for example, when a FRUC indicator is true), information indicating an applicable pattern matching method (the first pattern matching or the second pattern matching) can be signaled at the CU level. It should be noted that signaling of such information does not necessarily have to be done at the CU level, and can be done at another level (for example, at the sequence level, image level, slice level, tile level, CTU level, or sub-block level). Derivation of MV > Affine Mode
[00264] Next, the affine method for deriving a motion vector in units of a sub-block, based on the motion vectors among a plurality of motion vectors, is described. This method is also referred to as an affine motion compensation prediction method.
[00265] Figure 25A is a conceptual diagram to illustrate an example of deriving a motion vector from each sub-block based on the motion vectors among a plurality of neighboring blocks. In Figure 25A, the current block includes sixteen 4 x 4 sub-blocks. Here, the motion vector Vo at a control point in the upper left corner of the current block is derived based on a motion vector from a neighboring block, and similarly, the motion vector Vi at a control point in the upper right corner of the current block is derived based on Petition 870260103461, dated 07 / 10 / 2026, page 54 / 376 45 / 121 a motion vector of a neighboring sub-block. Two motion vectors vo and vi can be projected according to an expression (1A) shown below, and the motion vectors (vx, vy) for the respective sub-blocks in the current block can be derived. Mathematics 1Vx = + Vox(Wx .w. (IA)=(¼ Vy)x_ (Vtx-Vox)+yw w7
[00266] Here, x and y indicate the horizontal and vertical positions of the sub-block, respectively, and ew indicates a determined weighting coefficient. The determined weighting coefficient can be predetermined.
[00267] Such information indicating the affine mode (e.g., referred to as an affine indicator) can be signaled at the CU level. It is noted that signaling of information indicating the affine mode does not necessarily have to be done at the CU level, and can be done at another level (e.g., at the sequence level, at the image level, at the slice level, at the tile level, at the CTU level, or at the sub-block level).
[00268] Additionally, the affine mode can include several modes for different methods of deriving motion vectors at the upper left and upper right corner control points. For example, the affine mode includes two modes which are the affine intermode (also referred to as an affine normal intermode) and the affine blending mode. Derivation of MV > Affine Mode
[00269] Figure 25B is a conceptual diagram to illustrate an example of deriving a motion vector for each sub-block in affine mode using three control points. In Figure 25B, the current block includes sixteen 4x4 blocks. Here, the motion vector Vo at the upper left corner control point for the current block is derived based on a motion vector of a neighboring block, and similarly, the motion vector Vi at the upper right corner control point for the current block is derived based on a motion vector of a neighboring block, and the motion vector V2 at the control point of Petition 870260103461, dated 07 / 10 / 2026, page 55 / 376 46 / 121 bottom left corner for the current block is derived based on a motion vector of a neighboring block. Three motion vectors vo, vi and V2 can be projected according to an expression (1B) shown below, and the motion vectors (vx, vy) for the respective sub-blocks in the current block can be derived. Mathematics 2 _ (v>y -v^) (vy ~voy)yw hy Oy
[00270] Here, x and y indicate the horizontal and vertical positions of the sub-block center, respectively, w indicates the width of the current block, and eh indicates the height of the current block.
[00271] Affine modes, in which different numbers of control points (e.g., two and three control points) are used, can be switched and signaled at the CU level. It should be noted that the information indicating the number of control points in affine mode, used at the CU level, can be signaled at another level (e.g., sequence level, image level, slice level, tile level, CTU level, or sub-block level).
[00272] Additionally, such an affine mode, in which three control points are used, may include different methods of deriving motion vectors at the upper left, upper right, and lower left corner control points. For example, affine modes include two modes which are the affine intermode (also referred to as the affine normal intermode) and the affine mixing mode. MV Derivation > Affine Mixing Mode
[00273] Figures 26A, 26B and 26C are conceptual diagrams to illustrate the affine mixing mode.
[00274] As illustrated in Figure 26A, in affine mixing mode, for example, the motion vector predictors at the respective control points of a current block are calculated based on a plurality of motion vectors that correspond to the blocks coded according to the affine mode, among the coded block A (left), block B (top), block C (top right), block D (bottom) Petition 870260103461, dated 07 / 10 / 2026, page 56 / 376 47 / 121 left), and block E (upper left), which is adjacent to the current block. More specifically, the encoded block A (left), block B (upper), block C (upper right), block D (lower left), and block E (upper left) are checked in the listed order, and the first effective block encoded according to the affine mode is identified. The motion vector predictors at the control points of the current block are calculated based on a plurality of motion vectors that correspond to the identified block.
[00275] For example, as illustrated in Figure 26A, when block A, which is adjacent to the left of the current block, has been encoded in an affine mode, in which two control points are used, the motion vectors v3 and v4 projected onto the upper-left corner position and upper-right corner position of the encoded block, including block A, are derived. The motion vector predictor vo, at the upper-left corner control point of the current block, and the motion vector predictor vi at the upper-right corner control point of the current block are then calculated from the derived motion vectors v3 and v4.
[00276] For example, as illustrated in Figure 26C, when block A, which is adjacent to the left of the current block, has been encoded in an affine mode, in which three control points are used, the motion vectors v3, v4, and v5 projected onto the upper-left corner position, upper-right corner position, and lower-left corner position of the encoded block, including block A, are derived. The motion vector predictor vo at the upper-left corner control point of the current block, the motion vector predictor vi at the upper-right corner control point of the current block, and the motion vector predictor v2 at the lower-left corner control point of the current block are then calculated from the derived motion vectors v3, v4, and v5.
[00277] It should be noted that this method of deriving motion vector predictors can be used to derive the motion vector predictors of the respective control points of the current block in Step Sj_1 in Figure 29 described later.
[00278] Figure 27 is a flowchart illustrating an example of the affine mixing mode. Petition 870260103461, dated 07 / 10 / 2026, page 57 / 376 48 / 121
[00279] In affine mixing mode, as illustrated, first, the interpredictor 126 derives the MV predictors from the respective control points of a current block (Step Sk_1). The control points are an upper left corner point of the current block and an upper right corner point of the current block, as illustrated in Figure 25A, or an upper left corner point of the current block, an upper right corner point of the current block, and a lower left corner point of the current block, as illustrated in Figure 25B.
[00280] In other words, as illustrated in figure 26A, the interpredictor 16 checks the encoded block (upper left) in the listed order, and identifies the first effective encoded block according to the affine mode.
[00281] When block A is identified and block A has two control points, as illustrated in figure 26B, the interpredictor 126 calculates the motion vector vo at the control point in the upper left corner of the current block and the motion vector vi at the control point in the upper right corner of the current block from the motion vectors v3 and v4 in the upper left and upper right corners of the encoded block, including block A. For example, the interpredictor 126 calculates the motion vector vo at the control point in the upper left corner of the current block and the motion vector vi at the control point in the upper right corner of the current block by projecting the motion vectors v3 and v4 in the upper left and upper right corners of the encoded block onto the current block.
[00282] Alternatively, when block A is identified and block A has three control points, as illustrated in figure 26C, the interpredictor 126 calculates the motion vector vo at the control point in the upper left corner of the current block, the motion vector vi at the control point in the upper right corner of the current block, and the motion vector v2 at the control point in the lower left corner of the current block from the motion vectors v3, v4, and v5 in the upper left, upper right, and lower left corners of the encoded block including block A.For example, the 126 interpredictor calculates the motion vector vo at the control point in the upper left corner of the current block, the motion vector vi at the control point in the upper right corner of the current block, and the motion vector v2 at the control point in the lower left corner of the current block by projecting motion vectors v3, v4, and v5, as the upper left corner, upper right corner, and lower left corner of the block encoded in the current block. Petition 870260103461, dated 07 / 10 / 2026, page 58 / 376 49 / 121
[00283] Next, the 126 interpredictor performs motion compensation for each of a plurality of sub-blocks included in the current block. In other words, the 126 interpredictor calculates, for each of the plurality of sub-blocks, a sub-block motion vector as an affine MV, using (i) two motion vector predictors vo and vi and expression (1A) described above or (ii) three motion vector predictors vo, vi and v2 and expression (1B) described above (Step Sk_2). The 126 interpredictor then performs motion compensation for the sub-blocks using these affine MVs and the encoded reference images (Step Sk_3). As a result, motion compensation for the current block is performed to generate a prediction image of the current block. Derivation of MV > Affine Intermode
[00284] Figure 28A is a conceptual diagram to illustrate an affine intermode in which two control points are used.
[00285] In the affine intermode, as illustrated in Figure 28A, a motion vector selected from the motion vectors of encoded block A, block B, and block C, which are neighbors of the current block, is used as the motion vector predictor vo at the control point in the upper left corner of the current block. Similarly, a motion vector, selected from the motion vectors of encoded block D and block E, which are neighbors of the current block, is used as the motion vector predictor vi at the control point in the upper right corner of the current block.
[00286] Figure 28B is a conceptual diagram to illustrate an affine intermode in which three control points are used.
[00287] In the affine intermode, as illustrated in Figure 28B, a motion vector, selected from the encoded motion vectors of block A, block B, and block C, which are neighbors of the current block, is used as the motion vector predictor vo at the control point in the upper-left corner of the current block. Similarly, a motion vector selected from the encoded motion vectors of block D and block E, which are neighbors of the current block, is used as the motion vector predictor vi at the control point in the upper-right corner of the current block. Additionally, a motion vector selected from the encoded motion vectors of block F and block G, which are neighbors of the current block, is used as the motion vector predictor v2 at the corner control point. Petition 870260103461, dated 07 / 10 / 2026, page 59 / 376 50 / 121 bottom left of the current block.
[00288] Figure 29 is a flowchart illustrating an example of an affine intermode.
[00289] In the affine intermode, as illustrated, first, the interpredictor 126 derives the MV predictors (vo, vi) or (vo, vi, V2) from the respective two or three control points of a current block (Step Sj_1). The control points are an upper left corner point of the current block and an upper right corner point of the current block, as illustrated in figure 25A, or an upper left corner point of the current block, an upper right corner point of the current block, and a lower left corner point of the current block, as illustrated in figure 25B.
[00290] In other words, the 126 interpredictor derives the motion vector predictors (vo, vi) or (vo, vi, V2) from the respective two or three control points of the current block by selecting motion vectors from any of the blocks encoded in the vicinity of the respective control points of the current block illustrated in Figure 28A or Figure 28B. At this point, the 126 interpredictor encodes, in a sequence, the motion vector predictor selection information to identify the two selected motion vectors.
[00291] For example, the 126 interpredictor can determine, using a cost evaluation or similar, the block from which a motion vector, such as a motion vector predictor at a control point, is selected from among the coded blocks, neighbors of the current block, and can describe, in a bit sequence, an indicator showing which motion vector predictor was selected.
[00292] Next, the 126 interpredictor performs motion estimation (Steps Sj_3 and Sj_4) while updating a motion vector predictor selected or derived in Step Sj_1 (Step Sj_2). In other words, the 126 interpredictor calculates, as an affine MV, a motion vector for each of the sub-blocks, which corresponds to an updated motion vector predictor, using expression (1A) or expression (1B) described above (Step Sj_3). The 126 interpredictor then performs motion compensation of the sub-blocks using these affine MVs and encoded reference images (Step Sj_4). As a result, for example, the 126 interpredictor determines the motion vector predictor that results in the lowest cost as a motion vector at a control point in a motion estimation loop (Step Sj_5). At this moment, the interpredictor 126 encodes Petition 870260103461, dated 07 / 10 / 2026, page 60 / 376 51 / 121 additionally, in sequence, the difference value between the determined MV and the predicted motion vector is given as a difference in MV.
[00293] Finally, the interpredictor 126 generates a prediction image for the current block by performing motion compensation of the current block using the determined MV and the encoded reference image (Step Sj_6). Derivation of MV > Affine Intermode
[00294] When affine modes, in which different numbers of control points (e.g., two and three control points) are used, can be switched and signaled at the CU level, the number of control points in a coded block and the number of control points in an actual block may differ from each other. Figure 30A and Figure 30B are conceptual diagrams to illustrate the methods for deriving motion vector predictors at the control points when the number of control points in a coded block and the number of control points in an actual block differ from each other.
[00295] For example, as illustrated in Figure 30A, when a current block has three control points at the upper left corner, upper right corner, and lower left corner, and block A, which is adjacent to the left side of the current block, has been coded according to an affine mode, in which two control points are used, the motion vectors v3 and v4 projected onto the upper left corner and upper right corner positions in the coded block, including block A, are derived. The motion vector predictor vo at the upper left corner control point of the current block and the motion vector predictor vi at the upper right corner control point of the current block are then calculated from the derived motion vectors v3 and v4. Additionally, the motion vector predictor v2 at the lower left corner control point is calculated from the derived motion vectors vo and vi.
[00296] For example, as illustrated in Figure 30B, when a current block has two control points at the upper left and upper right corners, and block A, which is adjacent to the left of the current block, has been encoded in affine mode, in which three control points are used, the motion vectors v3, v4, and v5 are projected onto the upper left, upper right, and lower left corner positions on the encoded block. Petition 870260103461, dated 07 / 10 / 2026, page 61 / 376 52 / 121, including block A, are derived. The predicted motion vector vo at the control point in the upper left corner of the current block, and the predicted motion vector vi at the control point in the upper right corner of the current block are then calculated from the derived motion vectors v3, v4, and v5.
[00297] It should be noted that this method of deriving motion vector predictors can be used to derive the motion vector predictors of the respective control points of the current block in Step Sj_1 in Figure 29. Derivation of MV > DMVR
[00298] Figure 31A is a flowchart illustrating a relationship between mixing mode and DMVR.
[00299] Interpredictor 126 derives a motion vector of a current block according to the mixing mode (Step Sl_1). Next, interpredictor 126 determines whether to perform the motion vector estimate, that is, the motion estimate (Step Sl_2). Here, when determining whether the motion estimate will not be performed (Not in Step Sl_2), interpredictor 126 determines the motion vector derived in Step Sl_1 as the final motion vector for the current block (Step Sl_4). In other words, in this case, the motion vector of the current block is determined according to the mixing mode.
[00300] When determining whether to perform the motion estimate in Step Sl_1 (Yes in Step Sl_2), the 126 interpredictor derives the final motion vector for the current block by estimating a surrounding region of the reference image specified by the motion vector derived in Step Sl_1 (Step Sl_3). In other words, in this case, the motion vector of the current block is determined according to DMVR.
[00301] Figure 31B is a conceptual diagram to illustrate an example of a DMVR process for determining a MV.
[00302] First, (for example, in blend mode), the best MVP, which has been configured for the current block, is determined to be a candidate MV. A reference pixel is identified from a first reference image (L0) which is an image encoded in the LO direction according to a candidate MV (LO). Similarly, a reference pixel is identified from a second reference image (L1) which is an image encoded in the L1 direction, according Petition 870260103461, dated 07 / 10 / 2026, page 62 / 376 53 / 121 with a candidate for MV (L1). A template is generated by calculating an average of these reference pixels.
[00303] Next, each of the surrounding regions of the MV candidates from the first reference image (L0) and the second reference image (L1) is estimated, and the MV that results in the lowest cost is determined as the final MV. It should be noted that the cost value can be calculated, for example, using a difference value between each of the pixel values in the template and a corresponding value among the pixel values in the estimation region, the MV candidate values, etc.
[00304] It should be noted that the processes, configurations and operations described above are typically and basically common between the encoder and a decoder to be described later.
[00305] Exactly the same illustrative processes described here do not always need to be performed. Any process to allow the derivation of the final MV by estimation in the surrounding regions of the MV candidates can be used. Motion Compensation > BIO / OBMC
[00306] Motion compensation involves a method of generating a prediction image, and correcting the prediction image. The method is, for example, BIO and OBMC, which will be described later.
[00307] Figure 32 is a flowchart illustrating an example of generating a prediction image.
[00308] The 126 interpredictor generates a prediction image (Step Sm_1), and corrects the prediction image, for example, according to any of the modes described above (Step Sm_2).
[00309] Figure 33 is a flowchart illustrating another example of generating a prediction image.
[00310] Interpredictor 126 determines a motion vector of a current block (Step Sn_1). Next, interpredictor 126 generates a prediction image (Step Sn_2), and determines whether to perform a correction process (Step Sn_3). Here, when determining whether to perform a correction process (Yes in Step Sn_3), interpredictor 126 generates the final prediction image by correcting the image of Petition 870260103461, dated 07 / 10 / 2026, page 63 / 376 54 / 121 forecast (Step Sn_4). When it is determined that a correction process is not to be carried out (Not in Step Sn_3), the interpredictor 126 sends the forecast image as the final forecast image without correcting the forecast image (Step Sn_5).
[00311] Additionally, motion compensation involves a mode of correcting the luminance of a prediction image when the prediction image is generated. The mode is, for example, LIC, which will be described later.
[00312] Figure 34 is a flowchart illustrating another example of generating a prediction image.
[00313] Interpredictor 126 derives a motion vector from a current block (Step So_1). Next, interpredictor 126 determines whether to perform a luminescence correction process (Step So_2). Here, when determining whether to perform a luminescence correction process (Yes in Step So_2), interpredictor 126 generates the prediction image while performing a luminescence correction process (Step So_3). In other words, the prediction image is generated using LIC. When determining whether not to perform a luminescence correction process (No in Step So_2), interpredictor 126 generates a prediction image by performing normal motion compensation without performing a luminescence correction process (Step So_4). Motion Compensation > OBMC
[00314] It should be noted that an interprediction signal can be generated using motion information for a neighboring block in addition to motion information for the current block obtained from motion estimation. More specifically, the interprediction signal can be generated in units of a subblock in the current block by performing a weighted addition of a prediction signal based on motion information obtained from motion estimation (in the reference image) and a prediction signal based on motion information for a neighboring block (in the current image). Such interprediction (motion compensation) is also referred to as overlapping block motion compensation (OBMC).
[00315] In OBMC mode, information indicating a sub-block size for OBMC (referred to as, for example, an OBMC block size) can be signaled at the sequence level. Furthermore, information indicating whether the mode applies... Petition 870260103461, dated 07 / 10 / 2026, p. 64 / 376 55 / 121 OBMC (referred to as, for example, an OBMC indicator) can be signaled at the CU level. It should be noted that signaling of such information does not necessarily need to be performed at the sequence level and CU level, and can be performed at another level (e.g., at the image level, slice level, tile level, CTU level, or sub-block level).
[00316] Examples of the OBMC mode will be described in more detail. Figures 35 and 36 are a flowchart and a conceptual diagram to illustrate a general outline of a prediction image correction process performed by an OBMC process.
[00317] First, as illustrated in Figure 36, a prediction image (Pred) is obtained through normal motion compensation using a motion vector (MV) assigned to the target processing block (current). In Figure 36, the MV arrow points to a reference image, and indicates what the current block of the current image refers to in order to obtain a prediction image.
[00318] Next, a prediction image (Pred_L) is obtained by applying a motion vector (MV_L), which has already been derived for the neighboring encoded block on the left side of the current block, to the current block (reusing the motion vector for the current block). The motion vector (MV_L) is indicated by an arrow MV_L pointing to a reference image from a current block. A first correction of a prediction image is performed by overlaying two prediction images, Pred and Pred_L. This provides a blending effect of the boundary between neighboring blocks.
[00319] Similarly, a prediction image (Pred_U) is obtained by applying a motion vector (MV_U), which has already been derived for the neighboring encoded block above the current block, to the current block (reusing the motion vector for the current block). The motion vector (MV_U) is indicated by an arrow MV_U, indicating a reference image from a current block. A second correction of a prediction image is performed by overlaying the prediction image Pred_U onto the prediction images (e.g., Pred and Pred_L) where the first correction was performed. This provides a blending effect of the boundary between neighboring blocks. The prediction image obtained by the second correction is one where the boundary between neighboring blocks has been blended (smoothed), and thus is the Petition 870260103461, dated 07 / 10 / 2026, page 65 / 376 56 / 121 final forecast image for the current block.
[00320] Although the example above is a two-path correction method using the left and top neighboring blocks, it should be noted that the correction method can be a three- or more-path correction method also using the right neighboring block and / or the bottom neighboring block.
[00321] It should be noted that the region in which such overlapping is performed may be only part of a region close to a block boundary rather than the pixel region of the entire block.
[00322] It should be noted that the process of correcting a prediction image, according to OBMC, to obtain a Pred prediction image from a reference image, by overlaying the additional Pred_L and Pred_U prediction images, was described above. However, when a prediction image is corrected based on a plurality of reference images, a similar process can be applied to each of the plurality of reference images. In such a case, after the corrected prediction images are obtained from the respective reference images by performing the OBMC image correction based on the plurality of reference images, the corrected prediction images obtained are additionally overlaid to obtain the final prediction image.
[00323] Note that in OBMC, the unit of a current block can be the unit of a forecast block or the unit of a sub-block obtained by further splitting the forecast block.
[00324] An example of a method for determining whether or not to apply an OBMC process is a method for using an obmc_flag, which is a signal indicating whether or not to apply an OBMC process. As a specific example, an encoder determines whether the current block belongs to a region with complicated motion. The encoder sets obmc_flag to a value of 1 when the block belongs to a region with complicated motion and applies an OBMC process during encoding, and sets obmc_flag to a value of 0 when the block does not belong to a region with complicated motion, and encodes the block without applying an OBMC process. The decoder switches between applying and not applying an OBMC process by decoding the obmc_flag written in the sequence (for example, a compressed sequence), and decodes the block by Petition 870260103461, dated 07 / 10 / 2026, page 66 / 376 57 / 121 switching between application and non-application of the OBMC process according to the indicator value.
[00325] The 126 interpredictor generates a rectangular forecast image for a rectangular current block in the example above. However, the 126 interpredictor can generate a plurality of forecast images, each having a different shape from a rectangle for the rectangular current block, and can combine the plurality of forecast images to generate the final rectangular forecast image. The shape other than a rectangle could be, for example, a triangle.
[00326] Figure 37 is a conceptual diagram to illustrate the generation of two triangular forecast images.
[00327] The 126 interpredictor generates a triangular forecast image by performing motion compensation on a first partition, which has a triangular shape in the current block, using a first MV from the first partition to generate a triangular forecast image. Similarly, the 126 interpredictor generates a triangular forecast image by performing motion compensation on a second partition, which has a triangular shape in the current block, using a second MV from the second partition to generate a triangular forecast image. The 126 interpredictor then generates a forecast image with the same rectangular shape as the rectangular shape of the current block by combining these forecast images.
[00328] It is noted that, although the first partition and the second partition are triangular in the example illustrated in Figure 37, the first partition and the second partition may be trapezoidal, or other shapes different from each other. Additionally, although the current block includes two partitions in the example illustrated in Figure 37, the current block may include three or more partitions.
[00329] Additionally, the first partition and the second partition can overlap each other. In other words, the first partition and the second partition can include the same pixel region. In this case, a prediction image for a current block can be generated using a prediction image in the first partition and a prediction image in the second partition.
[00330] Additionally, despite an example in which a prediction image is generated for each of the two partitions using interprediction, an image Petition 870260103461, dated 07 / 10 / 2026, page 67 / 376 A 58 / 121 forecast can be generated for at least one partition using intraforecasting. Motion Compensation > BIO
[00331] Next, a method for deriving a motion vector is described. First, a mode for deriving a motion vector, based on a model that assumes uniform linear motion, will be described. This mode is also referred to as a bidirectional optical flow (BIO) mode.
[00332] Figure 38 is a conceptual diagram to illustrate a model assuming uniform linear motion. In Figure 38, (vx, vy) indicates a velocity vector and τ0 and τ1 indicate the time distances between a current image (Cur Pic) and two reference images (Ref0, Ref1). (MVx0, MVy0) indicate the motion vectors corresponding to the reference image Ref0, and (MVx1, MVy1) indicate the motion vectors corresponding to the reference image Ref1.
[00333] Here, considering the uniform linear motion exhibited by the velocity vectors (vx, vy), (MVxo, MVyo) and (MVxi, MVy1) are represented as (Vχτo, Vyτo) and (-Vχτ1, -vy^), respectively, and the following optical flux equation (2) can be employed. Mathematics 3 SI(k) / at + vxõI(k) / õx + vyõI(k) / õy = 0 (2)
[00334] Here, I(k) indicates a motion-compensated luminescence value of the reference image k (k = 0, 1). This optical flow equation illustrates that the sum of (i) the time derivative of the luminescence value, (ii) the product of the horizontal velocity and the horizontal component of the spatial gradient of a reference image, and (iii) the product of the vertical velocity and the vertical component of the spatial gradient of a reference image, is equal to zero. A motion vector for each block obtained from, for example, a mix list, can be corrected to one-pixel units, based on a combination of the optical flow equation and Hermite interpolation.
[00335] It should be noted that a motion vector can be derived on the decoder side using a method beyond the derivation of a motion vector, based on a model that assumes uniform linear motion. For example, a motion vector can be derived in units of a sub-block based on Petition 870260103461, dated 07 / 10 / 2026, page 68 / 376 59 / 121 motion vectors of neighboring blocks. Motion Compensation > LIC
[00336] Next, an example of a mode in which a prediction image is generated using a local illumination compensation (LIC) process will be described.
[00337] Figure 39 is a conceptual diagram to illustrate an example of a predictive image generation method using a luminescence correction process performed by a LIC process.
[00338] First, a MV is derived from an encoded reference image, and a reference image corresponding to the current block is obtained.
[00339] Next, information indicating how the luminescence value changed between the reference image and the current image is extracted for the current block. This extraction is performed based on the luminescence pixel values for the encoded left-side neighboring reference region (surrounding reference region) and the encoded top-side neighboring reference region (surrounding reference region), and the luminescence pixel value at the corresponding position in the reference image specified by the derived MV. A luminescence correction parameter is calculated using the information indicating how the luminescence value changed.
[00340] The prediction image for the current block is generated by performing a luminescence correction process in which the luminescence correction parameter is applied to the reference image in the reference image specified by the MV.
[00341] It should be noted that the shape of the surrounding reference region illustrated in Figure 39 is only an example; the surrounding reference region may have a different shape.
[00342] Furthermore, although the process in which a prediction image is generated from a single reference image has been described here, cases in which a prediction image is generated from a plurality of reference images can be described in the same way. The prediction image can be generated after performing a luminescence correction process on the Petition 870260103461, dated 07 / 10 / 2026, p. 69 / 376 60 / 121 reference images taken from the reference images in the same way as described above.
[00343] An example of a method for determining whether or not to apply a LIC process is a method for using a lic_flag, which is a signal indicating whether or not to apply the LIC process. As a specific example, the encoder determines whether the current block belongs to a region possessing a change in luminescence. The encoder sets lic_flag to a value equal to 1 when the block belongs to a region possessing a change in luminescence and applies a LIC process during decoding, and sets lic_flag to a value equal to 0 when the block does not belong to a region possessing a change in luminescence and encodes the current block without applying a LIC process. The decoder can decode the lic_flag written in the sequence and decode the current block by switching between applying and not applying a LIC process, according to the value of the indicator.
[00344] An example of a different method of determining whether a LIC process applies or not is a method of determination based on whether a LIC process has been applied to a surrounding block. In a specific example, when the mixing mode is used on the current block, it is determined whether a LIC process has been applied to the encoding of the surrounding encoded block, selected after the derivation of the MV in the mixing mode process. According to the result, the encoding is performed by switching between the application and non-application of a LIC process. It should be noted that, also in this example, the same processes are applied to the processes on the decoder side.
[00345] One embodiment of the luminescence correction (LIC) process, described with reference to Figure 39, is described in detail below.
[00346] First, the 126 interpredictor derives a motion vector to obtain a reference image that corresponds to a current block to be encoded from a reference image that is an encoded image.
[00347] Next, the 126 interpredictor extracts information indicating how the luminescence value of the reference image has changed to the luminescence value of the current image, using the luminescence pixel value of a surrounding encoded reference region that is adjacent to the left or above the current block, and the luminescence value at the corresponding position in the image of Petition 870260103461, dated 07 / 10 / 2026, page 70 / 376 61 / 121 reference specified by a motion vector, and calculates a luminescence correction parameter. For example, assume that the luminescence pixel value of a given pixel in the surrounding reference region in the current image is p0, and that the luminescence pixel value of the pixel corresponding to the determined pixel in the surrounding reference region in the reference image is p1. The 126 interpredictor calculates the coefficients A and B to optimize A x p1 + B = p0 as the luminescence correction parameter for a plurality of pixels in the surrounding reference region.
[00348] Next, the 126 interpredictor performs a luminescence correction process using the luminescence correction parameter for the reference image in the reference image specified by the motion vector, to generate a prediction image for the current block. For example, assume that the luminescence pixel value in the reference image is p2, and that the luminescence-corrected luminescence pixel value of the prediction image is p3. The 126 interpredictor generates the prediction image after it has undergone the luminescence correction process by calculating A x p2 + B = p3 for each of the pixels in the reference image.
[00349] It should be noted that the shape of the surrounding reference region illustrated in Figure 39 is an example; a different shape in addition to the shape of the surrounding reference region may be used. Additionally, part of the surrounding reference region illustrated in Figure 39 may be used. For example, a region having a determined number of pixels drawn from each of a top neighbor pixel and a left neighbor pixel may be used as a surrounding reference region. The determined number of pixels may be predetermined.
[00350] Additionally, the surrounding reference region is not limited to a region that is adjacent to the current block, and may be a region that is not adjacent to the current block. In the example illustrated in Figure 39, the surrounding reference region in the reference image is a region specified by a motion vector in a current image, from a surrounding reference region in the current image. However, a region specified by another motion vector is also possible. For example, the other motion vector may be a motion vector in a surrounding reference region in the current image. Petition 870260103461, dated 07 / 10 / 2026, page 71 / 376 62 / 121
[00351] Although operations performed by encoder 100 have been described here, it should be noted that decoder 200 typically performs similar operations.
[00352] It should be noted that the LIC process can be applied not only to luminescence, but also to chrominance. At this point, a correction parameter can be derived individually for each of Y, Cb, and Cr, or a common correction parameter can be used for any of Y, Cb, and Cr.
[00353] Additionally, the LIC process can be applied to units of a subblock. For example, a correction parameter can be derived using a surrounding reference region in a current subblock and a surrounding reference region in a reference subblock in a reference image specified by a MV of the current subblock. Forecasting Controller
[00354] Interpredictor 128 selects one of an intraprediction signal (a signal sent from intrapredictor 124) and an interprediction signal (a signal sent from interpredictor 126), and sends the selected signal to subtractor 104 and adder 116 as a prediction signal.
[00355] As illustrated in Figure 1, in various types of encoder examples, the prediction controller 128 can send a prediction parameter that is registered in the entropy encoder 110. The entropy encoder 110 can generate a sequence of encoded bits (or a sequence), based on the prediction parameter that is registered from the prediction controller 128 and quantized coefficients that are registered from the quantizer 108. The prediction parameter can be used in a decoder. The decoder can receive and decode the sequence of encoded bits, and perform the same processes as the prediction processes performed by the intrapredictor 124, interpredictor 126, and prediction controller 128.The forecast parameter may include (i) a selection forecast signal (for example, a motion vector, a forecast type, or a forecast mode used by intrapredictor 124 or interpredictor 126) or (ii) an optional index, indicator, or value that is based on a forecasting process performed in each of the intrapredictor 124, interpredictor 126, and forecast controller 128, or that indicates the forecasting process. Petition 870260103461, dated 07 / 10 / 2026, page 72 / 376 63 / 121 Example of Encoder Assembly
[00356] Figure 40 is a block diagram illustrating an example of encoder 100 assembly. Encoder 100 includes processor a1 and memory a2. For example, the plurality of constituent elements of encoder 100 illustrated in Figure 1 are assembled in processor a1 and memory a2, illustrated in Figure 40.
[00357] Processor a1 is a set of circuits that performs information processing and is accessible to memory a2. For example, processor a1 is a dedicated or general set of electronic circuits that encodes a video. Processor a1 can be a processor, such as a CPU. Additionally, processor a1 can be an aggregate of a plurality of electronic circuits. Additionally, for example, processor a1 can assume the roles of two or more constituent elements from the plurality of constituent elements of encoder 100 illustrated in Figure 1, etc.
[00358] Memory a2 is a dedicated or general memory for storing information used by processor a1 to encode a video. Memory a2 can be a set of electronic circuits and can be connected to processor a1. Additionally, memory a2 can be included in processor a1. Additionally, memory a2 can be an aggregate of a plurality of electronic circuits. Additionally, memory a2 can be a magnetic disk, an optical disk, or similar, or can be represented as a storage device, a recording medium, or similar. Additionally, memory a2 can be non-volatile memory or volatile memory.
[00359] For example, memory a2 can store a video to be encoded or a bit sequence corresponding to an encoded video. Additionally, memory a2 can store a program to cause processor a1 to encode a video.
[00360] Additionally, for example, memory a2 can assume the roles of two or more constituent elements for storing information from the plurality of constituent elements of encoder 100 illustrated in figure 1, etc. For example, memory a2 can assume the roles of block memory 118 and frame memory 122 illustrated in figure 1. More specifically, memory a2 can store a reconstructed block, a reconstructed image, etc. Petition 870260103461, dated 07 / 10 / 2026, page 73 / 376 64 / 121
[00361] It should be noted that, in encoder 100, not all of the plurality of constituent elements indicated in Figure 1, etc. may be implemented, and not all of the processes described above may be performed. Some of the constituent elements indicated in Figure 1, etc. may be included in another device, or some of the processes described above may be performed by other devices. Decoder
[00362] Next, a decoder capable of decoding an encoded signal (encoded bit sequence) sent, for example, from the encoder 100 described above, will be described. Figure 41 is a block diagram illustrating a functional configuration of the decoder 200 according to one mode. The decoder 200 is a video decoder that decodes a video into one-block units.
[00363] As illustrated in figure 41, the decoder 200 includes the entropy decoder 202, the inverse quantizer 204, the inverse transformer 206, the adder 208, the block memory 210, the circuit filter 212, the frame memory 214, the intrapredictor 216, the interpredictor 218, and the prediction controller 220.
[00364] The decoder 200 is implemented as, for example, a generic processor and memory. In this case, when a software program stored in memory is executed by the processor, the processor functions as the entropy decoder 202, the inverse quantizer 204, the inverse transformer 206, the adder 208, the circuit filter 212, the intrapredictor 216, the interpredictor 218, and the prediction controller 220. Alternatively, the decoder 200 can be implemented as one or more dedicated electronic circuits corresponding to the entropy decoder 202, the inverse quantizer 204, the inverse transformer 206, the adder 208, the circuit filter 212, the intrapredictor 216, the interpredictor 218, and the prediction controller 220.
[00365] Hereafter, a general flow of the processes performed by the 200 decoder is described, and then each of the constituent elements included in the 200 decoder will be described. General Flow of the Decoding Process
[00366] Figure 42 is a flowchart illustrating an example of a general decoding process performed by decoder 200. Petition 870260103461, dated 07 / 10 / 2026, page 74 / 376 65 / 121
[00367] First, the entropy decoder 202, from decoder 200, identifies a block splitting pattern having a fixed size (e.g., 128 x 128 pixels) (Step Sp_1). This splitting pattern is a splitting pattern selected by encoder 100. Decoder 200 then performs the processes from Step Sp_2 to Sp_6 for each of a plurality of blocks of the splitting pattern.
[00368] In other words, the entropy decoder 202 decodes (specifically, decodes by entropy) the encoded quantized coefficients and a prediction parameter of a current block to be decoded (also referred to as a current block) (Sp_2 Step).
[00369] Next, the inverse quantizer 204 performs inverse quantization among the plurality of quantized coefficients and the inverse transformer 206 performs inverse transformation of the result, to restore a plurality of prediction residuals (i.e., a difference block) (Step Sp_3).
[00370] Next, the forecast processor, including all or part of the intrapredictor 216, the interpredictor 218, and the forecast controller 220, generates a forecast signal (also referred to as a forecast block) from the current block (Step Sp_4).
[00371] Next, adder 208 adds the prediction block to the difference block to generate a reconstructed image (also referred to as a decoded image block) of the current block (Step Sp_5).
[00372] When the reconstructed image is generated, circuit filter 212 performs the filtering of the reconstructed image (Step Sp_6).
[00373] Decoder 200 then determines whether the decoding of the entire image has been completed (Step Sp_7). When it is determined that the decoding has not yet been completed (Not at Step Sp_7), decoder 200 repeatedly executes the processes starting with Step Sp_1.
[00374] As illustrated, the processes of Steps Sp_1 to Sp_7 are performed sequentially by decoder 200. Alternatively, two or more of the processes can be performed in parallel, the processing order of two or more of the processes can be modified, etc. Petition 870260103461, dated 07 / 10 / 2026, page 75 / 376 66 / 121 Entropy Decoder
[00375] The entropy decoder 202 entropy decodes an encoded bit sequence. More specifically, for example, the entropy decoder 202 arithmetically decodes an encoded bit sequence into a binary signal. The entropy decoder 202 then de-binarizes the binary signal. Thus, the entropy decoder 202 sends the quantized coefficients of each block to the inverse quantizer 204. The entropy decoder 202 can send a prediction parameter included in an encoded bit sequence (see Figure 1) to the intrapredictor 216, the interpredictor 218, and the prediction controller 220. The intrapredictor 216, the interpredictor 218, and the prediction controller 220 in one mode can perform the same prediction processes as those performed by the intrapredictor 124, the interpredictor 126, and the prediction controller 128 on the encoder side. Inverse Quantizer
[00376] The inverse quantizer 204 inversely quantizes the quantized coefficients of a block to be decoded (hereinafter referred to as a current block) that are registered from the entropy decoder 202. More specifically, the inverse quantizer 204 inversely quantizes the quantized coefficients of the current block, based on the quantization parameters corresponding to the quantized coefficients. The inverse quantizer 204 then sends the inversely quantized transformation coefficients of the current block to the inverse transformer 206. Reversing Transformer
[00377] The inverse transformer 206 restores the prediction errors by inversely transforming the transformation coefficients that are recorded from the inverse quantizer 204.
[00378] For example, when the information analyzed from a sequence of encoded bits indicates that EMT or AMT should be applied (for example, when an AMT indicator is true), the inverse transformer 206 inversely transforms the transformation coefficients of the current block based on the information, indicating the type of transformation analyzed.
[00379] Moreover, for example, when the information analyzed from a Petition 870260103461, dated 07 / 10 / 2026, page 76 / 376 The 67 / 121 encoded bit sequence indicates that NSST should be applied; the inverse transformer 206 applies a secondary inverse transformation to the transformation coefficients. Adder
[00380] The adder 208 reconstructs the current block by adding prediction errors that are recorded from the inverting transformer 206 and prediction samples that are recorded from the prediction controller 220. The adder 208 then sends the reconstructed block to the block memory 210 and the circuit filter 212. Block Memory
[00381] Block memory 210 is the storage for storing the blocks in an image to be encoded (hereinafter referred to as a current image) and referenced in intraprediction. More specifically, block memory 210 stores the reconstructed blocks sent from adder 208. Circuit Filter
[00382] Circuit filter 212 applies a circuit filter to the blocks reconstructed by the adder 208, and sends the filtered reconstructed blocks to the frame memory 214, the display device, etc.
[00383] When the information indicating whether an ALF analyzed from a sequence of encoded bits is ON or OFF indicates that an ALF is ON, a filter, from among a plurality of filters, is selected based on the direction and activity of the local gradients, and the selected filter is applied to the reconstructed block. Frame Memory
[00384] Frame memory 214 is, for example, the storage device for storing reference images for use in interprediction, and is also referred to as a frame storage device. More specifically, frame memory 214 stores a reconstructed block filtered by circuit filter 212. Forecast Processor (Intrapredictor, Interpredictor, Forecast Controller)
[00385] Figure 43 is a flowchart illustrating an example of a process performed by a prediction processor of the 200 decoder. It is noted that the Petition 870260103461, dated 07 / 10 / 2026, page 77 / 376 The 68 / 121 forecast processor includes all or part of the following constituent elements: intrapredictor 216; interpredictor 218 and forecast controller 220.
[00386] The forecast processor generates a forecast image of a current block (Step Sq_1). This forecast image is also referred to as a forecast signal or a forecast block. It should be noted that the forecast signal is, for example, an intra-forecast signal or an inter-forecast signal. Specifically, the forecast processor generates the forecast image of the current block using a reconstructed image that has already been obtained through forecast block generation, difference block generation, coefficient block generation, difference block restoration, and decoded image block generation.
[00387] The reconstructed image can be, for example, an image within a reference image, or an image of a decoded block within a current image that is the image that includes the current block. The decoded block in the current image is, for example, a block neighboring the current block.
[00388] Figure 44 is a flowchart illustrating another example of a process performed by the prediction processor of decoder 200.
[00389] The forecast processor determines a method or mode for generating a forecast image (Step Sr_1). For example, the method or mode can be determined based on, for example, a forecast parameter, etc.
[00390] When a first method is determined as a mode for generating a prediction image, the prediction processor generates a prediction image according to the first method (Step Sr_2a). When a second method is determined as a mode for generating a prediction image, the prediction processor generates a prediction image according to the second method (Step Sr_2b). When a third method is determined as a mode for generating a prediction image, the prediction processor generates a prediction image according to the third method (Step Sr_2c).
[00391] The first method, the second method, and the third method may be mutually different methods for generating a forecast image. Each of the first through third methods may be an interforecast method, an intraforecast method, or another forecast method. The reconstructed image described above may be used in these forecast methods. Petition 870260103461, dated 07 / 10 / 2026, page 78 / 376 69 / 121 Intrapredictor
[00392] The intrapredictor 216 generates a prediction signal (intraprediction signal) by performing intraprediction with reference to a block or blocks in the current image stored in block memory 210, based on the intraprediction mode analyzed from the encoded bit sequence. More specifically, the intrapredictor 216 generates an intraprediction signal by performing intraprediction with reference to samples (e.g., luminescence and / or chrominance values) of a block or blocks neighboring the current block, and then sends the intraprediction signal to the prediction controller 220.
[00393] It should be noted that when an intraprediction mode, in which a luminescence block is referenced in the intraprediction of a chrominance block, is selected, the intrapredictor 216 can predict the chrominance component of the current block based on the luminescence component of the current block.
[00394] Furthermore, when the information analyzed from an encoded bit sequence indicates that PDPC should be applied, the intrapredictor 216 corrects the intrapredictor pixel values based on the horizontal / vertical pixel gradients. Interprevisor
[00395] Interpredictor 218 predicts the current block by reference to a reference image stored in frame memory 214. The interprediction is performed in units of a current block or a sub-block (e.g., a 4 x 4 block) within the current block. For example, interpredictor 218 generates an interprediction signal of the current block or sub-block by performing motion compensation using motion information (e.g., a motion vector) analyzed from an encoded bit sequence (e.g., a prediction parameter sent from the entropy decoder 202), and sends the interprediction signal to the prediction controller 220.
[00396] It should be noted that when the information analyzed from the encoded bit sequence indicates that the OBMC mode should be applied, the 218 interpredictor generates the interprediction signal using the motion information of a neighboring block in addition to the motion information of the current block obtained from the motion estimate.
[00397] Furthermore, when the information is analyzed from the bit sequence Petition 870260103461, dated 07 / 10 / 2026, page 79 / 376 70 / 121 encoded bits indicate that FRUC mode should be applied; the 218 interpredictor derives motion information by performing motion estimation according to the pattern matching method (bilateral matching or template matching) analyzed from the encoded bit sequence. The 218 interpredictor then performs motion compensation (prediction) using the derived motion information.
[00398] Furthermore, when the BIO mode should be applied, the 218 interpredictor derives a motion vector based on a model assuming uniform linear motion. Additionally, when the information analyzed from the encoded bit sequence indicates that the affine mode compensation prediction mode should be applied, the 218 interpredictor derives a motion vector for each sub-block based on the motion vectors of neighboring blocks. MV Derivation > Normal Intermode
[00399] When the information analyzed from a sequence of encoded bits indicates that normal intermode should be applied, the interpredictor 218 derives a motion compensation (MV) based on the information analyzed from the encoded bit sequence and performs motion compensation (prediction) using the MV.
[00400] Figure 45 is a flowchart illustrating an example of interprediction in normal intermode on decoder 200.
[00401] The 218 interpredictor of the 200 decoder performs motion compensation for each block. The 218 interpredictor obtains a plurality of MV candidates for a current block based on information, such as MVs from a plurality of decoded blocks, temporally or spatially surrounding the current block (Step Ss_1). In other words, the 218 interpredictor generates a list of MV candidates.
[00402] Next, the interpredictor 218 extracts N (an integer equal to 2 or more) MV candidates from the plurality of MV candidates obtained in Step Ss_1, as motion vector predictor candidates (also referred to as MV predictor candidates), according to a predetermined priority order (Step Ss_2). It should be noted that the priority order can be determined in advance for each of the N MV predictor candidates.
[00403] Next, interpredictor 218 decodes the selection information of Petition 870260103461, dated 07 / 10 / 2026, page 80 / 376 71 / 121 motion vector predictor from an input sequence (i.e., a sequence of encoded bits) and selects a motion vector predictor candidate from the N motion vector predictor candidates using the decoded motion vector predictor selection information, such as a motion vector (also referred to as a motion vector predictor) from the current block (Step Ss_3).
[00404] Next, the interpredictor 218 decodes an MV difference from the input sequence, and derives an MV for a current block by adding a difference value that is the decoded MV difference and a selected motion vector predictor (Step Ss_4).
[00405] Finally, the interpredictor 218 generates a prediction image for the current block by performing motion compensation of the current block using the derived MV and the decoded reference image (Step Ss_5). Forecasting Controller
[00406] The prediction controller 220 selects the intraprediction signal or the interprediction signal, and sends the selected prediction signal to the adder 208. As a whole, the settings, functions, and processes of the prediction controller 220, the intrapredictor 216, and the interpredictor 218 on the decoder side can correspond to the settings, functions, and processes of the prediction controller 128, intrapredictor 124, and interpredictor 126 on the encoder side. Example of Decoder Assembly
[00407] Figure 46 is a block diagram illustrating an example of the assembly of decoder 200. Decoder 200 includes processor b1 and memory b2. For example, the plurality of constituent elements of decoder 200 illustrated in Figure 41 is assembled in processor b1 and memory b2 illustrated in Figure 46.
[00408] Processor b1 is a set of circuits that performs information processing and is accessible to memory b2. For example, processor b1 is a dedicated or general set of electronic circuits that decodes a video (that is, a sequence of encoded bits). Processor b1 can be a processor, such as a CPU. Additionally, processor b1 can be an aggregate of a plurality of electronic circuits. Additionally, for example, processor b1 can assume the roles of two or more constituent elements from the plurality of constituent elements of decoder 200 illustrated in Figure 41. Petition 870260103461, dated 07 / 10 / 2026, page 81 / 376 72 / 121 etc.
[00409] Memory b2 is a dedicated or general memory for storing information used by processor b1 to decode a sequence of encoded bits. Memory b2 can be a set of electronic circuits and can be connected to processor b1. Additionally, memory b2 can be included in processor b1. Additionally, memory b2 can be an aggregate of a plurality of electronic circuits. Additionally, memory b2 can be a magnetic disk, an optical disk, or similar, or can be represented as a storage device, a recording medium, or similar. Additionally, memory b2 can be non-volatile memory or volatile memory.
[00410] For example, memory b2 can store a video or a bit sequence. Additionally, memory b2 can store a program to cause processor b1 to decode an encoded bit sequence.
[00411] Additionally, for example, memory b2 can assume the roles of two or more constituent elements to store information from the plurality of constituent elements of the decoder 200 illustrated in figure 41, etc. Specifically, memory b2 can assume the roles of block memory 210 and frame memory 214 illustrated in figure 41. More specifically, memory b2 can store a reconstructed block, a reconstructed image, etc.
[00412] It should be noted that, in decoder 200, not all of the plurality of constituent elements illustrated in figure 41, etc. may be implemented, and not all of the processes described above may be performed. Some of the constituent elements indicated in figure 41, etc. may be included in another device, or some of the processes described above may be performed by another device. Definitions of Terms
[00413] The respective terms can be defined, as indicated below, by way of example.
[00414] An image is a set of luminescence samples in monochrome format or a set of luminescence samples and two corresponding sets of chrominance samples in 4:2:0 and 4:4:4 color format. An image can be a frame or a field. Petition 870260103461, dated 07 / 10 / 2026, page 82 / 376 73 / 121
[00415] A table is composed of an upper field and a lower field, where sample rows 0, 2, 4,... originate from the upper field and sample rows 1, 3, 5... originate from the lower field.
[00416] A slice is an integer number of encoding tree units contained within an independent slice segment and all subsequent dependent slice segments (if any) that precede the next independent slice segment (if any) within the same access unit.
[00417] A “tile” is a rectangular region of the encoding tree blocks within a particular “tile” column and a particular “tile” row in an image. A “tile” may be a rectangular region of the frame that is designated to be independently decoded and encoded, although circuit filtering across the “tile” edges may still be applied.
[00418] A block is an M x N (M columns by N rows) set of samples, or an M x N set of transformation coefficients. A block can be a square or rectangular region of pixels including a Luminance matrix and two Chrominance matrices.
[00419] A coding tree unit (CTU) can be a coding tree block of luminance samples from an image that has three sets of samples, or two corresponding coding tree blocks of chrominance samples. Alternatively, a CTU can be a coding tree block of samples from one of a monochrome image and an image that is coded using three separate color planes and syntax structures used to encode the samples. A superblock can be a 64 x 64 pixel square block consisting of 1 or 2 mode information blocks or is recursively divided into four 32 x 32 blocks, which themselves can be further divided. Non-Rectangular Partition
[00420] In the prediction controller 128, coupled to the intrapredictor 124 and the interpredictor 126 on the encoder side (see figure 1), as well as in the prediction controller 220, coupled to the intrapredictor 216 and the interpredictor 218 on the decoder side (see figure 41), partitions so far (or blocks or sub-blocks of variable size) obtained from the partitioning of each block, for which the Petition 870260103461, dated 07 / 10 / 2026, page 83 / 376 74 / 121 motion information (e.g., motion vectors) is obtained, and is invariably rectangular, as illustrated in Figure 3. The inventors discovered that generating partitions with a non-rectangular shape, such as a triangular shape, results in improved image quality and encoding efficiency depending on the content of an image within an image in various implementations. Below, several embodiments will be described in which at least one partition division, from an image block, for prediction purposes, has a non-rectangular shape. Note that these embodiments are equally applicable on the encoder side (prediction controller 128 coupled to intrapredictor 124 and interpredictor 126) and on the decoder side (prediction controller 220 coupled to intrapredictor 216 and interpredictor 218), and can be implemented in the encoder of Figure 1 or similar, or in the decoder of Figure 41 or similar.
[00421] Figure 47 is a flowchart illustrating an example of a process for dividing an image block into partitions that include at least one partition that has a non-rectangular shape (e.g., a triangle) and a second partition, and performing additional processing that includes encoding (or decoding) the image block as a reconstructed combination of the first and second partitions.
[00422] In step S1001, an image block is divided into partitions that include a first partition, which has a non-rectangular shape, and a second partition, which may or may not have a non-rectangular shape. For example, as illustrated in Figure 48, an image block can be divided from an upper-left corner of the image block to a lower-right corner of the image block to create a first partition and a second partition, both of which have a non-rectangular shape (e.g., a triangle), or an image block can be divided from an upper-right corner of the image block to a lower-left corner of the image block to create a first partition and a second partition, both of which have a non-rectangular shape (e.g., a triangle). Several examples of non-rectangular partitioning will be described below with reference to Figures 48 and 53 to 55.
[00423] In step S1002, the process predicts a first motion vector for the first partition and predicts a second motion vector for the second partition. Petition 870260103461, dated 07 / 10 / 2026, page 84 / 376 75 / 121 For example, predicting the first and second motion vectors might involve selecting the first motion vector from a first set of candidate motion vectors and selecting the second motion vector from a second set of candidate motion vectors.
[00424] In step S1003, a motion compensation process is performed to obtain the first partition using the first motion vector, which is derived in step S1002 above, and to obtain the second partition using the second motion vector, which is derived in step S1002 above.
[00425] In step S1004, a prediction process is performed for the image block as a (reconstructed) combination of the first partition and the second partition. The prediction process may include a boundary smoothing process to smooth the boundary between the first partition and the second partition. For example, the boundary smoothing process may involve weighting first boundary pixel values, predicted based on the first partition, and second boundary pixel values, predicted based on the second partition. Several implementations of the boundary smoothing process will be described below with reference to Figures 49, 50, 56, and 57A-57D.
[00426] In step S1005, the process encodes or decodes the image block using one or more parameters, including a partition parameter that indicates the division of the image block into the first partition, which has a non-rectangular shape, and the second partition. As summarized in a table in Figure 51, for example, the partition parameter (the first index value) can encode together, for example, a split direction applied to the split (e.g., from the upper left corner to the lower right corner or from the upper right corner to the lower left corner, as illustrated in Figure 48) and the first and second motion vectors derived in step S1002 above. Details of such a partition syntax operation, involving the one or more parameters, including the partition parameter, will be described in detail below with reference to Figures 51, 52, and 58 to 61.
[00427] Figure 53 is a flowchart illustrating a 2000 process for splitting an image block. In step S2001, the process splits an image into a plurality of partitions including a first partition, which has a non-standard format. Petition 870260103461, dated 07 / 10 / 2026, page 85 / 376 76 / 121 rectangular, and a second partition, which may or may not have a non-rectangular shape. As illustrated in Figure 48, an image block can be divided into a first partition, which has a triangular shape, and a second partition, which also has a triangular shape. There are numerous other examples in which an image block is divided into a plurality of partitions, including a first partition and a second partition, of which at least the first partition has a non-rectangular shape. The non-rectangular shape can be a triangle, a trapezoid, and a polygon with at least five sides and angles.
[00428] For example, as illustrated in figure 54, an image block can be divided into two triangular-shaped partitions; an image block can be divided into more than two triangular-shaped partitions (for example, three triangular-shaped partitions); an image block can be divided into a combination of triangular-shaped partitions and rectangular-shaped partitions; or an image block can be divided into a combination of triangular-shaped partitions and polygonal-shaped partitions.
[00429] As further illustrated in figure 55, an image block can be divided into an L-shaped partition (polygon shape) and a rectangular partition; an image block can be divided into a pentagonal (polygon) partition and a triangular partition; an image block can be divided into a hexagonal (polygon) partition and a pentagonal (polygon) partition; or an image block can be divided into multiple polygonal partitions.
[00430] Referring again to Figure 53, in step S2002, the process predicts a first motion vector for the first partition, for example, by selecting the first partition from a first set of motion vector candidates, and predicts a second motion vector for the second partition, for example, by selecting the second partition from a second set of motion vector candidates. For example, the first set of motion vector candidates might include the motion vectors of the partitions neighboring the first partition, and the second set of motion vector candidates might include the motion vectors of the partitions neighboring the second partition. The neighboring partitions might be either spatially neighboring partitions or temporally neighboring partitions. Some examples of spatially neighboring partitions Petition 870260103461, dated 07 / 10 / 2026, page 86 / 376 77 / 121 neighboring partitions include a partition located in the left, bottom left, bottom, bottom right, right, top right, top, or top left corner of the partition being processed. Examples of temporally neighboring partitions are partitions in the same location in the image block reference images.
[00431] In several implementations, the neighboring partitions of the first partition and the neighboring partitions of the second partition may be outside the image block from which the first and second partitions were split. The first set of motion vector candidates may be equal to, or different from, the second set of motion vector candidates. Additionally, at least one of the first set of motion vector candidates and the second set of motion vector candidates may be equal to another third set of motion vector candidates prepared for the image block.
[00432] In some implementations, in step S2002, in response to the determination that the second partition, similar to the first partition, also has a non-rectangular shape (e.g., a triangle), process 2000 creates the second set of motion vector candidates (for the second non-rectangular partition) that includes the motion vectors of the partitions neighboring the second partition excluding the first partition (excluding the motion vector of the first partition). Conversely, in response to the determination that the second partition, unlike the first partition, has a rectangular shape, process 2000 creates the second set of motion vector candidates (for the second rectangular partition) that includes the motion vectors of the partitions neighboring the second partition including the first partition.
[00433] In step S2003, the process encodes or decodes the first partition using the first motion vector derived in step S2002 above, and encodes or decodes the second partition using the second motion vector derived in step S2002 above.
[00434] An image block division process, such as process 2000 in Figure 53, can be performed by an image encoder, as illustrated in Figure 1, for example, which includes the circuit assembly and a memory coupled to the circuit assembly. The circuit assembly, in operation, performs: the division of an image block into a plurality of partitions, including a first partition, Petition 870260103461, dated 07 / 10 / 2026, page 87 / 376 78 / 121 which has a non-rectangular shape, and a second partition (step S2001); the prediction of a first motion vector for the first partition and a second motion vector for the second partition (step S2002); and the encoding of the first partition using the first motion vector and the second motion vector using the second motion vector (step S2003).
[00435] According to another embodiment, as illustrated in figure 1, an image encoder is provided including a divider 102 which, in operation, receives and divides an original image into blocks; an adder 104 which, in operation, receives the blocks from the divider and predictions from a prediction controller 128, and subtracts each prediction from its corresponding block to send a residual; a transformer 106 which, in operation, performs a transformation of residuals sent from the adder 104 to send the transformation coefficients; a quantizer 108 which, in operation, quantizes the transformation coefficients to generate the quantized transformation coefficients; an entropy encoder 110 which, in operation, encodes the quantized transformation coefficients to generate a bit sequence;and the prediction controller 128 coupled to an interpredictor 1267, an intrapredictor 124 and a memory 118, 122, where the interpredictor 126, in operation, generates a prediction of a current block based on a reference block in an encoded reference image and the intrapredictor 124, in operation, generates a prediction of a current block based on a reference block encoded in a current image. The prediction controller 128, in operation, divides the blocks into a plurality of partitions, including a first partition, which has a non-rectangular shape, and a second partition (Figure 53, step S2001); predicts a first motion vector for the first partition and a second motion vector for the second partition (step S2002); and encodes the first partition using the first motion vector and the second partition using the second motion vector (step S2003).
[00436] According to another embodiment, an image decoder, as illustrated in Figure 41, for example, is provided and includes the circuitry and a memory coupled to the circuitry. The circuitry, in operation, performs the division of an image block into a plurality of partitions including a first partition that has a non-rectangular shape and a second partition (Figure 53, step S2001); the prediction of a first motion vector for the Petition 870260103461, dated 07 / 10 / 2026, page 88 / 376 79 / 121 first partition and a second motion vector for the second partition (step S2002); and decoding the first partition using the first motion vector and the second partition using the second motion vector (step S2003).
[00437] According to an additional embodiment, an image decoder, as illustrated in figure 41, is provided including an entropy decoder 202 which, in operation, receives and decodes a sequence of encoded bits to obtain the quantized transformation coefficients; an inverse quantizer 204 and transformer 206 which, in operation, inversely quantizes the quantized transformation coefficients to obtain the transformation coefficients and inversely transforms the transformation coefficients to obtain residuals; an adder 208 which, in operation, adds the residuals sent from the inverse quantizer 204 and the transformer 206 and predictions sent from a prediction controller 220 to reconstruct the blocks;and the prediction controller 220 coupled to an interpredictor 218, an intrapredictor 216, and a memory 210, 214, where the interpredictor 218, in operation, generates a prediction of a current block based on a reference block in a decoded reference image, and the intrapredictor 216, in operation, generates a prediction of a current block based on a decoded reference block in a current image. The prediction controller 220, in operation, divides an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition (Figure 53, step S2001); predicts a first motion vector for the first partition and a second motion vector for the second partition (step S2002); and decodes the first partition using the first motion vector and the second partition using the second motion vector (step S2003). Boundary Smoothing
[00438] As described above in Figure 47, the S1004 step, according to various modalities, performing a prediction process for the image block as a combination (reconstructed) of the first partition which has a non-rectangular shape and the second partition may involve applying a boundary smoothing process along the boundary between the first partition and the second partition.
[00439] For example, Figure 57B illustrates an example of a process of Petition 870260103461, dated 07 / 10 / 2026, page 89 / 376 80 / 121 boundary smoothing involves weighting the first boundary pixel values, which are predicted first based on the first partition, and the second boundary pixel values, which are predicted second based on the second partition.
[00440] Figure 56 is a flowchart illustrating a general boundary smoothing process 3000 involving weighting first predicted boundary pixel values based on the first partition and second predicted boundary pixel values based on the second partition, according to one embodiment. In step S3001, an image block is split into a first partition and a second partition along a boundary where at least the first partition has a non-rectangular shape, as illustrated in Figure 57A or in Figures 48, 54, and 55 described above.
[00441] In step S3002, the first values (e.g., color, luminance, transparency, etc.) of a set of pixels (boundary pixels in Figure 57A) from the first partition along the boundary are predicted first, where the first values are predicted first using information from the first partition. In step S3003, the second values of the (same) set of pixels from the first partition along the boundary are predicted second, where the second values are predicted second using information from the second partition. In some implementations, at least one of the first and second predictions is an interprediction process that predicts the first and second values based on a reference partition in an encoded reference image.With reference to Figure 57D, in some implementations, the prediction process forecasts the first values of all pixels in the first partition (the first sample set), including the set of pixels through which the first and second partitions overlap, and forecasts the second values of only the set of pixels (the second sample set) through which the first and second partitions overlap. In another implementation, at least one of the first and second predictions is an intraprediction process that forecasts the first and second values based on a reference partition encoded in a current image. In some implementations, a forecasting method used in the first prediction is different from a forecasting method used in the second prediction. For example, the first prediction might include a process of... Petition 870260103461, dated 07 / 10 / 2026, p. 90 / 376 81 / 121 interprediction and the second prediction may include an intraprediction process. The information used to first predict the first values or to secondly predict the second values may be motion vectors, intraprediction directions, etc. of the first or second partition.
[00442] In step S3004, the first values, predicted using the first partition, and the second values predicted using the second partition are weighted. In step S3005, the first partition is encoded or decoded using the first and second weighted values.
[00443] Figure 57B illustrates an example of a boundary smoothing operation in which the first partition and the second partition overlap by a maximum of five pixels in each row or column. That is, the number of pixel sets in each row or column for which the first values are predicted based on the first partition, and the second values are predicted based on the second partition, is equal to a maximum of 5. Figure 57C illustrates another example of a boundary smoothing operation in which the first partition and the second partition overlap by a maximum of three pixels in each row or column. That is, the number of pixel sets in each row or column for which the first values are predicted based on the first partition, and the second values are predicted based on the second partition, is equal to a maximum of three.
[00444] Figure 49 illustrates another example of the boundary smoothing operation, where the first partition and the second partition overlap by a maximum of four pixels in each row or column. That is, the number of pixels in each row or column for which the first values are predicted, based on the first partition, and the second values are predicted, based on the second partition, is equal to a maximum of four. In the illustrated example, weights of 1 / 8, 1 / 4, 3 / 4, and 7 / 8 can be applied to the first values of the four pixels in the set, respectively, and weights of 7 / 8, 3 / 4, 1 / 4, and 1 / 8 can be applied to the second values of the four pixels in the set, respectively.
[00445] Figure 50 illustrates additional examples of a boundary smoothing operation where the first partition and the second partition overlap zero pixels from each row or each column (do not overlap), overlap one pixel (at most) from each row or each column, and overlap two pixels (at most) Petition 870260103461, dated 07 / 10 / 2026, page 91 / 376 82 / 121 of each row or each column, respectively. In the example where the first and second partitions do not overlap, zero weights are applied. In the example where the first and second partitions overlap by one pixel in each row or each column, a weight of 1 / 2 can be applied to the first pixel values in the predicted set based on the first partition, and a weight of 1 / 2 can be applied to the second pixel values in the predicted set based on the second partition. In the example where the first and second partitions overlap by two pixels in each row or each column, weights of 1 / 3 and 2 / 3 can be applied to the first values of the two pixels in the predicted set based on the first partition, respectively, and weights of 2 / 3 and 1 / 3 can be applied to the second values of the two pixels in the predicted set based on the second partition, respectively.
[00446] According to the modalities described above, the number of pixels in the set, through which the first partition and the second partition overlap, is an integer. In other implementations, the number of overlapping pixels in the set may be non-integer and may be fractional, for example. Furthermore, the weights applied to the first and second values of the pixel set may be fractional or integer depending on each application.
[00447] A boundary smoothing process, such as process 3000 in Figure 56, can be performed by an image encoder, as illustrated in Figure 1, for example, which includes the circuit assembly and a memory coupled to the circuit assembly. The circuit assembly, in operation, performs a boundary smoothing operation along a boundary between a first partition that has a non-rectangular shape and a second partition, which are split from an image block (Figure 56, step S3001).The boundary smoothing operation includes: first, predicting the first values of a set of pixels from the first partition along the boundary, using information from the first partition (step S3002); second, predicting the second values of the set of pixels from the first partition along the boundary, using information from the second partition (step S3003); weighting the first and second values (step S3004); and encoding the first partition using the weighted first and second values (step S3005).
[00448] According to another embodiment, as illustrated in figure 1, an image encoder is provided including a 102 divider which, in operation, receives Petition 870260103461, dated 07 / 10 / 2026, page 92 / 376 83 / 121 and divides an original image into blocks; an adder 104 that, in operation, receives the blocks from the divider and predictions from a prediction controller 128, and subtracts each prediction from its corresponding block to send a residual; a transformer 106 that, in operation, performs a transformation on the residuals sent from the adder 104 to send the transformation coefficients; a quantizer 108 that, in operation, quantizes the transformation coefficients to generate the quantized transformation coefficients; an entropy encoder 110 that, in operation, encodes the quantized transformation coefficients to generate a bit sequence;and the prediction controller 128 coupled to an interpredictor 126, an intrapredictor 124 and a memory 118, 122, where the interpredictor 126, in operation, generates a prediction of a current block based on a reference block in an encoded reference image and the intrapredictor 124, in operation, generates a prediction of a current block based on a reference block encoded in a current image. The prediction controller 128, in operation, performs a boundary smoothing operation along a boundary between a first partition, which has a non-rectangular shape, and a second partition, which are split from an image block (figure 56, step S3001). The boundary smoothing operation includes predicting first the first values of a set of pixels from the first partition along the boundary, using information about the first partition (step S3002);First, predict the second values of the pixel set from the first partition along the boundary, using the information from the second partition (step S3003); then weight the first and second values (step S3004); and finally, encode the first partition using the weighted first and second values (step S3005).
[00449] According to another embodiment, an image decoder is provided, as illustrated in Figure 41 for example, including a set of circuits and a memory coupled to the set of circuits. The set of circuits, in operation, performs a boundary smoothing operation along a boundary between a first partition, which has a non-rectangular shape, and a second partition, which are split from an image block (Figure 56, steps S3001). The boundary smoothing operation includes predicting, firstly, the first values of a set of pixels from the first partition along the boundary, using the information from the first partition (step S3002), and secondly, predicting the second values of the Petition 870260103461, dated 07 / 10 / 2026, page 93 / 376 84 / 121 pixel set of the first partition along the boundary, using the information from the second partition (step S3003); weight the first values and the second values (step S3004); and decode the first partition using the weighted first values and the weighted second values (step S3005).
[00450] According to another embodiment, an image decoder, as illustrated in figure 41, is provided including an entropy decoder 202 which, in operation, receives and decodes an encoded bit sequence to obtain the quantized transformation coefficients; an inverse quantizer 204 and transformer 206 which, in operation, inversely quantizes the quantized transformation coefficients to obtain transformation coefficients and inversely transforms the transformation coefficients to obtain residuals; an adder 208 which, in operation, adds the residuals sent from the inverse quantizer 204 and the transformer 206 and predictions sent from a prediction controller 220 to reconstruct the blocks;and the prediction controller 220 coupled to an interpredictor 218, an intrapredictor 216, and a memory 210, 214, where the interpredictor 218, in operation, generates a prediction of a current block based on a reference block in a decoded reference image, and the intrapredictor 216, in operation, generates a prediction of a current block based on a decoded reference block in a current image. The prediction controller 220, in operation, performs a boundary smoothing operation along a boundary between a first partition, which has a non-rectangular shape, and a second partition, which are split from an image block. (Figure 56, step S3001). The boundary smoothing operation includes first predicting the first values of a set of pixels from the first partition along the boundary, using information from the first partition (step S3002); secondly, predicting the second values of the set of pixels from the first partition along the boundary;Using the information from the second partition (step S3003); weighting the first values and the second values (step S3004); and decoding the first partition using the weighted first values and the weighted second values (step S3005). Entropy Encoding and Decoding using Partition Parameter Syntax
[00451] As described in Figure 47, step S1005, according to various modalities, the image block is divided into a first partition, which has a Petition 870260103461, dated 07 / 10 / 2026, page 94 / 376 The 85 / 121 non-rectangular format, and a second partition, can be encoded or decoded using one or more parameters that include a partition parameter indicating the non-rectangular division of the image block. In several embodiments, such a partition parameter can jointly encode, for example, a division direction applied to the division (e.g., from the upper left corner to the lower right corner or from the upper right corner to the lower left corner, see Figure 48) and the first and second motion vectors predicted in step S1002, as will be fully described below.
[00452] Figure 51 is a table of sample partition parameters (the first index value) and sets of information encoded together by the partition parameters, respectively. The partition parameters (the first index values) range from 0 to 6 and encode together the direction of division of an image block into a first partition and a second partition, both of which are triangles (see Figure 48), the first predicted motion vector for the first partition (Figure 47, step S1002), and the second predicted motion vector for the second partition (Figure 47, step S1002).Specifically, partition parameter 0 encodes the split direction from the upper left corner to the lower right corner, the first motion vector is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition.
[00453] Partition parameter 1 encodes the split direction from the upper right corner to the lower left corner; the first movement vector is the first movement vector listed in the first set of candidate movement vectors for the first partition, and the second movement vector is the second movement vector listed in the second set of candidate movement vectors for the second partition. Partition parameter 2 encodes the split direction from the upper right corner to the lower left corner; the first movement vector is the second movement vector listed in the first set of candidate movement vectors for the first partition, and the second movement vector is the first movement vector listed in the second set of candidate movement vectors for the second partition. Partition parameter 3 encodes the split direction from the upper left corner to the lower right corner; the first movement vector is Petition 870260103461, dated 07 / 10 / 2026, p. 95 / 376 86 / 121 is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the second motion vector listed in the second set of motion vector candidates for the second partition. Partition parameter 4 encodes the split direction from the upper right corner to the lower left corner, the first motion vector is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the third motion vector listed in the second set of motion vector candidates for the second partition.Partition parameter 5 encodes the split direction from the upper left corner to the lower right corner; the first motion vector is the third motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition. Partition parameter 6 encodes the split direction from the upper left corner to the lower right corner; the first motion vector is the fourth motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition.
[00454] Figure 58 is a flowchart illustrating a 4000 method performed on the encoder side. In step S4001, the process divides an image block into a plurality of partitions including a first partition, which has a non-rectangular shape, and a second partition, based on a partition parameter indicating the division. For example, as illustrated in Figure 51 described above, the partition parameter may indicate the direction of division of an image block (e.g., from the upper right corner to the lower left corner or from the upper left corner to the lower right corner). In step S4002, the process encodes the first partition and the second partition.In step S4003, the process writes one or more parameters, including the partition parameter, in a bit sequence, which the decoder side can receive and decode to obtain the one or more parameters to perform the same prediction process (as performed on the encoder side) for the first and second partitions on the decoder side. The one or more parameters, including the partition parameter, can be encoded together or... Petition 870260103461, dated 07 / 10 / 2026, page 96 / 376 87 / 121 separately, various pieces of information, such as the non-rectangular format of the first partition, the format of the second partition, the splitting direction used to divide an image block to obtain the first and second partitions, the first motion vector of the first partition, the second motion vector of the second partition, etc.
[00455] Figure 59 is a flowchart illustrating a 5000 method performed on the decoder side. In step S5001, the process analyzes one or more parameters from a bit sequence, where the one or more parameters include a partition parameter indicating the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition.One or more parameters, including the partition parameter parsed from the bit sequence, can encode, jointly or separately, various pieces of information necessary for the decoder side to perform the same prediction process as that performed on the encoder side, such as the non-rectangular shape of the first partition, the shape of the second partition, the splitting direction used to divide an image block to obtain the first and second partitions, the first motion vector of the first partition, the second motion vector of the second partition, etc. In step S5002, process 5000 splits the image block into a plurality of partitions based on the partition parameter parsed from the bit sequence. In step S5003, the process decodes the first and second partitions as split from the image block.
[00456] Figure 60 is a table of sample partition parameters (the first index value) and information sets encoded together by the partition parameters, respectively, similar in nature to the sample table described above in Figure 51. In Figure 60, the partition parameters (the first index values) range from 0 to 6 and together encode the shape of the first and second partitions divided from an image block, the direction of division of an image block into the first and second partitions, the first predicted motion vector for the first partition (Figure 47, step S1002), and the second predicted motion vector for the second partition (Figure 47, step S1002). Specifically, partition parameter 0 encodes that neither of the first nor second partitions has a triangular shape, and thus the information of Petition 870260103461, dated 07 / 10 / 2026, page 97 / 376 88 / 121 splitting direction is N / A, the first motion vector information is N / A, and the second motion vector information is N / A.
[00457] Partition parameter 1 encodes the first and second partitions as triangles, the splitting direction is from the upper left corner to the lower right corner, the first motion vector is the second motion vector in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition. Partition parameter 2 encodes the first and second partitions as triangles, the splitting direction is from the upper right corner to the lower left corner, the first motion vector is the first motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the second motion vector listed in the second set of motion vector candidates for the second partition.Partition parameter 3 encodes the first and second partitions as triangles, the splitting direction is from the upper right corner to the lower left corner, the first motion vector is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition. Partition parameter 4 encodes the first and second partitions as triangles, the splitting direction is from the upper left corner to the lower right corner, the first motion vector is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the second motion vector listed in the second set of motion vector candidates for the second partition.Partition parameter 5 encodes the first and second partitions as triangles, the splitting direction is from the upper right corner to the lower left corner, the first motion vector is the second motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the third motion vector listed in the second set of motion vector candidates for the second partition. Partition parameter 6 encodes the first and second partitions as triangles, the splitting direction... Petition 870260103461, dated 07 / 10 / 2026, page 98 / 376 89 / 121 occurs from the upper left corner to the lower right corner, the first motion vector is the third motion vector listed in the first set of motion vector candidates for the first partition, and the second motion vector is the first motion vector listed in the second set of motion vector candidates for the second partition.
[00458] According to some implementations, partition parameters (index values) can be binarized according to a binarization scheme, which is selected depending on the value of at least one or one or more parameters. Figure 52 illustrates a sample binarization scheme for binarizing index values (partition parameter values).
[00459] Figure 61 is a table of sample combinations of a first parameter and a second parameter, where one of which is a partition parameter indicating the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition. In this example, the partition parameter can be used to indicate the division of an unencoded image block in conjunction with other information, which is encoded by one or more of the other parameters.
[00460] In the first example in Figure 61, the first parameter is used to indicate an image block size, and the second parameter is used as the partition parameter (an indicator) to indicate that at least one of a plurality of partitions, divided from an image block, has a triangular shape. This combination of the first and second parameters can be used to indicate, for example, (1) when the image block size is greater than 64 x 64, there is no triangular partition, or (2) when the width-to-height ratio of an image block is greater than 4 (e.g., 64 x 64), there is no triangular partition.
[00461] In the second example in Figure 61, the first parameter is used to indicate a prediction mode, and the second parameter is used as the partition parameter (an indicator) to indicate that at least one of a plurality of partitions, divided from an image block, has a triangular shape. This combination of the first and second parameters can be used to indicate, for example, (1) when an image block is encoded in intramode, there is no Petition 870260103461, dated 07 / 10 / 2026, page 99 / 376 90 / 121 triangular partition.
[00462] In the third example in Figure 61, the first parameter is used as the partition parameter (an indicator) to indicate that at least one of a plurality of partitions, divided from an image block, has a triangular shape, and the second parameter is used to indicate a prediction mode. This combination of the first and second parameters can be used to indicate, for example, (1) when at least one of the plurality of partitions, divided from an image block, has a triangular shape, the image block should be intercoded.
[00463] In the fourth example in figure 61, the first parameter indicates the motion vector of a neighboring block, the second parameter is used as the partition parameter which indicates the direction of division of an image block into two triangles. This combination of the first and second parameters can be used to indicate, for example, (1) when the motion vector of a neighboring block is a diagonal direction, the direction of division of the image block into two triangles is from the upper left corner to the lower right corner.
[00464] In the fifth example in Figure 61, the first parameter indicates the intraprediction direction of a neighboring block, and the second parameter is used as the partition parameter that indicates the direction of division of an image block into two triangles. This combination of the first and second parameters is used as the partition parameter that indicates the direction of division of an image block into two triangles. This combination of the first and second parameters can be used to indicate, for example, (1) when the intraprediction direction of a neighboring block is a reverse diagonal direction, the direction of division of the image block into two triangles occurring from the upper right corner to the lower left corner.
[00465] It should be understood that the tables of one or more parameters, including the partition parameter, and the information encoded together or separately, as illustrated in Figures 51, 60, and 61, are presented as examples only, and numerous other forms of encoding, together or separately, of various information as part of the partition syntax operation described above are within the scope of this description. For example, the partition parameter might indicate that the first partition is a triangle, a Petition 870260103461, dated 07 / 10 / 2026, page 100 / 376 91 / 121 trapezoid, or a polygon with at least five sides and angles. The partition parameter can indicate that the second partition has a non-rectangular shape, such as a triangle, a trapezoid, and a polygon with at least five sides and angles. The partition parameter can indicate one or more pieces of information about the division, such as the non-rectangular shape of the first partition, the shape of the second partition (which can be non-rectangular or rectangular), the direction of division applied to dividing an image block into a plurality of partitions (for example, from an upper-left corner of the image block to a lower-right corner of the same, and from an upper-right corner of the image block to a lower-left corner of the same).The partition parameter can collectively encode additional information, such as the first motion vector of the first partition, the second motion vector of the second partition, the image block size, the prediction mode, the motion vector of a neighboring block, the intraprediction direction of a neighboring block, etc. Alternatively, any additional information can be encoded separately by one or more parameters in addition to the partition parameter.
[00466] A partition syntax operation, such as process 4000 of Figure 58, can be performed by an image encoder, as illustrated in Figure 1, for example, which includes the circuit assembly and a memory coupled to the circuit assembly. The circuit assembly, in operation, performs a partition syntax operation including dividing an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition based on a partition parameter indicative of the division (Figure 58, step S4001); encoding the first partition and the second partition (S4002); and writing one or more parameters including the partition parameter in a bit sequence (S4003).
[00467] According to another embodiment, as illustrated in Figure 41, an image encoder is provided including a divider 102 which, in operation, receives and divides an original image into blocks; an adder 104 which, in operation, receives the blocks from the divider and the predictions from a prediction controller 128, and subtracts each prediction from its corresponding block to send a residual; a transformer 106 which, in operation, performs a transformation on the residuals sent from the adder 104 to send the transformation coefficients; a quantizer 108 which, Petition 870260103461, dated 07 / 10 / 2026, page 101 / 376 92 / 121 in operation, quantizes the transformation coefficients to generate the quantized transformation coefficients; an entropy encoder 110 which, in operation, encodes the quantized transformation coefficients to generate a bit sequence; and the prediction controller 128 coupled to an interpredictor 126, an intrapredictor 124 and a memory 118, 122, where the interpredictor 126, in operation, generates a prediction of a current block based on a reference block in an encoded reference image and the intrapredictor 124, in operation, generates a prediction of a current block based on a reference block encoded in a current image.The prediction controller 128, in operation, divides an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition, based on a partition parameter indicative of the division (Figure 58, step S4001), and encodes the first partition and the second partition (step S4002). The entropy encoder 110, in operation, writes one or more parameters, including the partition parameter, to a bit sequence (step S4003).
[00468] According to another embodiment, an image decoder is provided, as illustrated in Figure 41, for example, including the circuit assembly and a memory coupled to the circuit assembly. The circuit assembly, in operation, performs a partition syntax operation including the parsing of one or more parameters from a bit sequence, where the one or more parameters include a partition parameter indicating the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition (Figure 59, step S5001); the division of the image block into the plurality of partitions based on the partition parameter (S5002); and the decoding of the first partition and the second partition (S5003).
[00469] According to a further embodiment, an image decoder, as illustrated in Figure 41, is provided including an entropy decoder 202 which, in operation, receives and decodes a sequence of encoded bits to obtain quantized transformation coefficients; an inverse quantizer 204 and inverse transformer 206 which, in operation, inversely quantize the quantized transformation coefficients to obtain transformation coefficients, and inversely transform the transformation coefficients to obtain residuals; an adder 208 which, in operation, adds the residuals sent from the Petition 870260103461, dated 07 / 10 / 2026, page 102 / 376 93 / 121 inverse quantizer 204 and inverse transformer 206 and predictions sent from a prediction controller 220 to reconstruct the blocks; and the prediction controller 220 coupled to an interpredictor 218, an intrapredictor 216 and a memory 210, 214, where the interpredictor 218, in operation, generates a prediction of a current block based on a reference block in a decoded reference image and the intrapredictor 216, in operation, generates a prediction of a current block based on a decoded reference block in a current image.The entropy decoder 202, in operation, analyzes one or more parameters from a bit sequence, where the one or more parameters include a partition parameter indicating the division of an image block into a plurality of partitions, including a first partition that has a non-rectangular shape and a second partition (Figure 59, step S5001); divides the image block into the plurality of partitions, based on the partition parameter (S5002); and decodes the first partition and the second partition (S5003) in cooperation with the prediction controller 220 in some implementations.
[00470] According to other examples, an interpredictor can perform the following process.
[00471] All motion vector candidates included in the first set of motion vector candidates may be uniprediction motion vectors. That is, the interpredictor can only determine uniprediction motion vectors as motion vector candidates in the first set of motion vector candidates.
[00472] The interpredictor can only select single-prediction motion vector candidates from the first set of motion vector candidates.
[00473] In one embodiment, only the uniprediction motion vector can be used to predict a small block. The biprediction motion vector can be used to predict a large block. As an example, the prediction process may include judging an image block size. When the image block size is considered larger than a threshold, the prediction may include selecting the first motion vector from a first set of motion vector candidates, and the first set of motion vector candidates. Petition 870260103461, dated 07 / 10 / 2026, page 103 / 376 94 / 121 motion may contain uniprediction and / or biprediction motion vectors. When the image block size is considered to be below the limit, the prediction may include selecting the first motion vector from an initial set of motion vector candidates, and the initial set of motion vector candidates may contain only uniprediction motion vectors. Method for storing Motion Vectors for the Weighted Area
[00474] Figure 62 is a flowchart illustrating an example of a 6000 process flow to predict a first set of samples for a first partition of a current image with a first motion vector, obtain a second motion vector from a second partition different from the first partition, predict a second set of samples for a first part included in the first partition with the second motion vector, weight the first and second sets of samples, store at least one of the first and second motion vectors for the first partition, and encode or decode the first partition using the weighted samples, and perform further processing according to a modality. The 6000 process flow can be performed, for example, by encoder 100 of Figure 1, by decoder 200 of Figure 41, etc.
[00475] In step S6001, a first set of samples for a first partition of a current image is predicted with one or more motion vectors including a first motion vector. The first partition may or may not have a non-rectangular shape. The first set of samples can be, for example, pixel blocks possessing associated motion vectors, and luminance and chrominance pixel components. For example, each sample can be a 4 x 4 pixel block associated with the first motion vector and possessing associated luminance and chrominance pixel components.
[00476] The first partition can be, for example, a partition of an image block that has been divided into a plurality of partitions. Figure 63 is a conceptual diagram to illustrate illustrative methods of dividing an image block into a first partition and a second partition. See also Figures 48, 54, and 55. For example, as illustrated in Figure 63, an image block can be divided into two or more partitions having various shapes. The illustrative illustrations in Figure Petition 870260103461, dated 07 / 10 / 2026, page 104 / 376 95 / 121 include an image block split from an upper-left corner of the image block to a lower-right corner of the image block to create a first partition and a second partition, both having a non-rectangular shape (e.g., a triangular shape); an image block split into an L-shaped partition and a rectangular partition; an image block split into a pentagonal partition and a triangular partition; an image block split into a hexagonal partition and a pentagonal partition; and an image block split into two polygonal partitions. The various partition shapes illustrated can be formed by splitting an image block in other ways.For example, two triangular partitions can be formed by dividing an image block from an upper-right corner of the image block to a lower-left corner of the image block to create a first partition and a second partition, both having a triangular shape. In some embodiments, two or more partitions of an image block may have an overlapping part.
[00477] In some modes, the first motion vector can be obtained using a motion estimation process. In some modes, the first motion vector can be analyzed from a bit sequence. In some modes, the first motion vector can be a predicted motion vector from a set of motion vector candidates for at least the first partition. The motion vector candidates from a list of motion vector candidates may include motion vector candidates derived from spatially or temporally neighboring partitions of at least one first partition, or they may derive from a list of motion vector candidates for an image block for a prediction mode, such as a blend mode, a jump mode, or an intermode.
[00478] Figure 64 is a conceptual diagram to illustrate adjacent and non-adjacent spatially neighboring partitions of a first partition of a current image. The adjacent spatially neighboring partition is a partition adjacent to the first partition in the current image. The non-adjacent spatially neighboring partition is a partition spaced from the first partition in the current image. In some embodiments, a set of motion vector candidates can be derived from the spatially neighboring partitions of at least one first partition in the image. Petition 870260103461, dated 07 / 10 / 2026, page 105 / 376 96 / 121 current.
[00479] The first motion vector can be, for example, a uniprediction motion vector or a biprediction motion vector. Figure 65 is a conceptual diagram to illustrate uniprediction and biprediction motion vector candidates for an image block of a current image. A uniprediction motion vector candidate is a singular motion vector for the current block in the current image relative to a single reference image. As illustrated in the upper part of Figure 65, the uniprediction motion vector candidate is a motion vector from a block of a current image to a block of a reference image, with the reference image appearing before the current image in a display order. In some embodiments, the reference image may appear after the current image in the display order.
[00480] A candidate biprediction motion vector comprises two motion vectors: a first motion vector for the current block relative to a first reference image and a second motion vector for the current block relative to a second reference image. As illustrated in Figure 65, the candidate biprediction motion vector in the lower left corner has a first motion vector, from the current image block to a block of a first reference image, and a second motion vector from the current image block to a block of a second reference image. As illustrated, the first and second reference images appear before the current image in a display order.The candidate biprediction motion vector in the lower right corner of Figure 65 has a first motion vector from the current image block to a block in a first reference image, and a second motion vector from the current image block to a block in a second reference image. The first reference image appears before the current image in display order, and the second reference image appears after the current image in display order.
[00481] In some modalities, certain partition formats may be associated with a type of prediction motion vector. For example, in some modalities, triangular-shaped partitions may be associated with uniprediction motion vector candidates. Petition 870260103461, dated 07 / 10 / 2026, page 106 / 376 97 / 121
[00482] The prediction of the first set of samples may include a motion compensation process with one or more motion vectors including the first motion vector, and the motion compensation process may predict samples from a current image (intrablock) or predict samples from a second different image (interblock), or combinations thereof.
[00483] In step S6002, one or more motion vectors, including a second motion vector, are obtained from a second partition, different from the first partition. The second partition may be, for example, a partition spatially or temporally located together with the first partition. A part of the second partition may spatially overlap a part of the first partition. The second motion vector may be obtained using a motion estimation process. In some embodiments, the second motion vector may be analyzed from a bit sequence. In some embodiments, the second motion vector may be a predicted motion vector from a set of motion vector candidates for at least the second partition.The motion vector candidates from a list of motion vector candidates may include motion vector candidates derived from spatially or temporally neighboring partitions of the second partition, or they may derive from a list of motion vector candidates for an image block for a prediction mode, such as a blend mode, a jump mode, or other intermodes. The second motion vector may be a uniprediction motion vector or a biprediction motion vector.
[00484] In step S6003, a second set of samples is predicted for a portion of the first partition with one or more motion vectors, including the second motion vector. The first portion of the first partition is smaller than the first partition and is positioned adjacent to an edge of the first partition and overlaps the first set of samples. The prediction of the second set of samples may include a motion compensation process with one or more motion vectors, including the second motion vector, and the motion compensation process may predict samples from a current image (intrablock) or predict samples from a different image (interblock), or combinations thereof. The second set of samples may be, for example, pixel blocks possessing associated motion vectors, and pixel components of luminescence and Petition 870260103461, dated 07 / 10 / 2026, page 107 / 376 98 / 121 chrominance. For example, each sample from the second set of samples could be a 4 x 4 pixel block associated with the second motion vector and possessing associated luminance and chrominance pixel components.
[00485] Figure 66 is a conceptual diagram to illustrate examples of the first part of the first partition and the first and second sets of samples. The first part may be, for example, one-quarter of the width or height of the first partition. In another example, the first part may have a width corresponding to N samples adjacent to an edge of the first partition, where N is an integer greater than zero, for example, N may be an integer equal to 2. As illustrated, the example to the right of Figure 66 illustrates a rectangular partition having a rectangular part with a width that is one-quarter of the width of the first partition, with the first set of samples including samples outside the first part and samples inside the first part, and the second set of samples including samples inside the first part.The example in the center of Figure 66 illustrates a rectangular partition having a rectangular part with a height that is one-quarter the height of the first partition, with the first set of samples including samples outside the first part and samples inside the first part, and the second set of samples including samples inside the first part. The example to the left of Figure 66 illustrates a triangular partition having a polygonal part with a height that corresponds to two samples, with the first set of samples including samples outside the first part and samples inside the first part, and the second set of samples including samples inside the first part.
[00486] The first part can be a part of the first partition that overlaps an adjacent partition. Figure 67 is a conceptual diagram to illustrate a first part of a first partition, which is a part of the first partition that overlaps a part of an adjacent partition. For ease of illustration, a rectangular partition, having a part overlapping a spatially adjacent rectangular partition, is illustrated. Partitions having other shapes, such as triangular partitions, can be used, and the overlapping parts can overlap a spatially or temporally adjacent partition. The adjacent partition can be the second partition.
[00487] In step S6004, a weighting process is performed for the first part included in the first partition using a subset of the first set of Petition 870260103461, dated 07 / 10 / 2026, page 108 / 376 99 / 121 samples are included in the first part, and the second set of samples is included in the first part. Weighting can be performed as part of a boundary smoothing process, such as an overlapping block motion compensation (OBMC, see Figures 35 and 36 and their description), the boundary smoothing processes described with reference to Figures 47 to 61, etc. For example, the pixels of the first set of samples in the first part can be assigned various weights, for example, based on a sample distance from an edge of the first partition with which the first part is aligned. Samples closer to the edge can be assigned lower weights than samples farther from the edge. Similarly, the pixels of the second set of samples in the first part of the first partition can be assigned various weights, for example, based on a sample distance from an edge of the first partition with which the first part is aligned.Samples closer to the edge can receive greater weights than samples farther from the edge.
[00488] In step S6005, one or more motion vectors are stored for the first part of the first partition, the stored motion vectors being based on one or both the first motion vector and the second motion vector. In some embodiments, the stored motion vectors may take into account the reference image lists to which the first and second motion vectors point.
[00489] For example, Figure 68 is a conceptual diagram to illustrate an illustrative case in which the first motion vector and the second motion vector are uniprediction motion vectors, pointing to different images in different reference image lists. In some embodiments, when the first motion vector and the second motion vector are uniprediction motion vectors pointing to different images in different reference image lists, the first and second motion vectors can be combined and stored as a biprediction motion vector for the first part of the first partition. As illustrated in Figure 68, the current image is POC 4. The first motion vector Mv1 is a uniprediction motion vector pointing to the reference image POC 0 from the reference image list L0, which includes reference images POC 0 to POC 8.The second motion vector Mv2 is a uniprediction motion vector pointing towards the reference image. Petition 870260103461, dated 07 / 10 / 2026, page 109 / 376 100 / 121 POC 16 from the L1 reference image list, which includes reference images POC 8 through POC 16. The first motion vector Mv1 and the second motion vector Mv2 are combined and stored as a biprediction motion vector for the first part of the first partition.
[00490] In another example, Figure 69 is a conceptual diagram to illustrate a case in which the first motion vector and the second motion vector are uniprediction motion vectors pointing to images in a single reference image list. A biprediction motion vector points to two different reference image lists. Thus, simply combining the first and second vectors will not produce a biprediction motion vector.In some embodiments, when the first motion vector and the second motion vector are single-view motion vectors pointing to different reference images from a single list of reference images, if one of the reference images being pointed to is, for example, included in another list of reference images, the motion vector associated with that image (for example, the second motion vector if the reference image it points to is included in both lists of reference images) can be represented by a motion vector pointing to the same reference image in the other list of reference images.The first and second motion vectors can be combined by replacing the motion vector associated with the reference image with a replacement motion vector pointing to the reference image in the other list of reference images, and stored as a biprediction motion vector for the first part of the first partition. As illustrated in Figure 69, the current image is POC 4. The first motion vector Mv1 is a uniprediction motion vector pointing to the POC 0 reference image from the L0 reference image list, which includes POC 0 to POC 8 reference images. The second motion vector Mv2 is a uniprediction motion vector pointing to the POC 8 reference image from the L0 reference image list. The POC 8 reference image is also included in the L1 reference image list.The second motion vector Mv2 is replaced by the replacement motion vector Mv2', which points to the POC 8 reference image in the L1 reference image list. The first motion vector Mv1 and the second replacement motion vector Mv2' are combined and stored as one motion vector. Petition 870260103461, dated 07 / 10 / 2026, page 110 / 376 101 / 121 biprediction for the first part of the first partition. In other words, different lists can be used and point to the same image.
[00491] In another example, Figures 70, 71, and 72 are conceptual diagrams to illustrate an illustrative case in which the first motion vector and the second motion vector are uniprediction motion vectors pointing to the same reference image in the same list of reference images. In some embodiments, when the first motion vector and the second motion vector are uniprediction motion vectors pointing to the same reference image in the same list of reference images, the first motion vector is stored as a uniprediction motion vector for the first part of the first partition. As illustrated in Figure 70, the current image is POC 4. The first motion vector Mv1 is a uniprediction motion vector pointing to the POC 0 reference image from the L0 list of reference images, which includes POC 0 to POC 8 reference images.The second motion vector Mv2 is a uniprediction motion vector pointing to the reference image POC 0 of the list of reference images L0. The first motion vector Mv1 is stored as a uniprediction motion vector for the first part of the first partition.
[00492] In some embodiments, when the first motion vector and the second motion vector are uniprediction motion vectors pointing to the same reference image in the same list of reference images, the second motion vector is stored as a uniprediction motion vector for the first part of the first partition. As illustrated in Figure 71, the current image is POC 4. The first motion vector Mv1 is a uniprediction motion vector pointing to the POC 0 reference image in the L0 list of reference images, which includes POC 0 to POC 8 reference images. The second motion vector Mv2 is a uniprediction motion vector pointing to the POC 0 reference image in the L0 list of reference images. The second motion vector Mv2 is stored as a uniprediction motion vector for the first part of the first partition.
[00493] In some modalities, when the first motion vector and the second motion vector are single-view motion vectors, pointing to the same reference image in the same list of reference images, the Petition 870260103461, dated 07 / 10 / 2026, page 111 / 376 The second motion vector, 102 / 121, is replaced by a flipped motion vector pointing to a reference image from another list of reference images, and the first motion vector and the flipped motion vector are combined and stored as a biprediction motion vector for the first part of the first partition. As illustrated in Figure 72, the current image is POC 4. The first motion vector, Mv1, is a uniprediction motion vector pointing to reference image POC 0 from the list of reference images L0, which includes reference images POC 0 to POC 8. The second motion vector, Mv2, is a uniprediction motion vector pointing to reference image POC 0 from the list of reference images L0. The second motion vector, Mv2, is replaced by a flipped motion vector, Mv2', pointing to a reference image in the list of reference images L1, illustrated as reference image POC 9 from the list of reference images L1.The first motion vector Mv1 and the flipped motion vector Mv2' are combined and stored as the biprediction motion vector for the first part of the first partition. The reference image from another list of reference images can be selected in various ways. For example, a reference image at the same position in the list of reference images L1 as the position of the reference image to which the second motion vector points in the list of reference images L0 can be selected; a reference image in the list of reference images L1 that is at the same distance in display order from the current image as the distance in display order from the reference image to which the second motion vector points in the list of reference images L0 can be selected, etc.The flipped motion vector can be scaled as needed depending on the reference image from another list of reference images to which the flipped motion vector points.
[00494] In some embodiments, a two-way prediction vector, based on the first and second motion vectors, is stored for the first part of the first partition when it is possible to do so without flipping or scaling one of the first and second motion vectors (see, for example, the embodiments in Figure 68 and Figure 69), and a one-prediction motion vector is stored for the first part of the first partition when it is not possible to generate a two-prediction motion vector without flipping or scaling one of the first Petition 870260103461, dated 07 / 10 / 2026, page 112 / 376 103 / 121 and second motion vectors (see, for example, figures 70 and 71), where either the first motion vector or the second motion vector is stored as a single-prediction motion vector for the first part of the first partition.
[00495] In another example, when the first and second motion vectors are uniprediction motion vectors pointing to the same reference image, the first and second motion vectors can be calculated to generate a uniprediction motion vector that is stored as a uniprediction motion vector for the first part of the first partition.
[00496] In another example, one of the first and second motion vectors may be a biprediction motion vector, and the other may be a uniprediction motion vector. The biprediction motion vector may be stored as the motion vector for the first part of the first partition.
[00497] In another example, both the first and second motion vectors can be biprediction motion vectors, and the reference images of the first motion vector can be the same as those of the second motion vector. The first and second motion vectors can be calculated to generate a biprediction motion vector that is stored as a biprediction motion vector for the first part of the first partition.
[00498] In another example, both the first and second motion vectors can be biprediction motion vectors, and the reference images of the first motion vector can be different from those of the second motion vector. The second motion vector can be scaled to the reference images of the first motion vector, and computed with the first motion vector, generating an average biprediction motion vector, which can be stored as a biprediction motion vector of the first part of the first partition.
[00499] In another example, the first motion vector can be stored as the motion vector of the first part of the first partition, for example, always, or when certain conditions exist. In another example, the second motion vector can be stored as the motion vector of the first part of the first partition, for example, always, or when certain conditions exist. In step S6006, the first partition is encoded or decoded using Petition 870260103461, dated 07 / 10 / 2026, page 113 / 376 104 / 121 if at least the weighted samples from the first part of the first partition. For example, the weighted pixel component values (e.g., luminance values, chrominance values) of a sample from the first set of samples and the weighted pixel component values of a corresponding sample from the second set of samples can be combined to determine the pixel component values used in the encoding or decoding process. See, for example, figures 47 to 61 and their corresponding descriptions.
[00500] The described modalities introduce methods of storing a motion vector for a weighted area (e.g., the first part of the first partition in the examples described). This can improve the accuracy of motion vector prediction for the next encoding partition, which can improve encoding efficiency. It should be noted that motion vectors for a weighted area can typically be stored for blocks of four pixels. For example, each sample can be a 4 x 4 pixel block for which a motion vector is stored. A uniprediction motion vector or a biprediction motion vector for a weighted area can be stored without resulting in a significant increase, or any increase at all, in the amount of memory employed to store the motion vectors for the weighted area.
[00501] It is noted that the term encoding used in the description for the encoding method and encoding process performed by an image encoding device may be replaced by the term decoding for a decoding method and decoding process performed by an image decoding device. Not all processes and elements described are always necessary, and one or more of the processes described may be omitted in some embodiments.
[00502] One or more of the aspects described here may be achieved by combining at least part of other aspects in this description. Additionally, one or more of the aspects described here may be achieved by combining, with other aspects, part of the processes indicated in any of the flowcharts, according to the aspects, part of the configuration of any of the devices, part of the syntax, etc. Petition 870260103461, dated 07 / 10 / 2026, page 114 / 376 105 / 121 Implementations and Applications
[00503] As described in each of the embodiments above, each functional or operational block can typically be implemented as an MPU (microprocessing unit) and memory, for example. Furthermore, the processes performed by each of the functional blocks can be implemented as a program execution unit, such as a processor that reads and executes software (a program) stored on a storage medium, such as ROM. The software can be distributed. The software can be stored on a variety of storage media, such as semiconductor memory. Note that each functional block can also be implemented as hardware (dedicated circuit). Various combinations of hardware and software can be employed.
[00504] The processing described in each of the embodiments can be performed through integrated processing using a single device (system), and, alternatively, it can be performed through decentralized processing using a plurality of devices. Furthermore, the processor that executes the program described above can be a single processor or a plurality of processors. In other words, integrated processing can be performed, and, alternatively, decentralized processing can be performed.
[00505] The embodiments in this description are not limited to the illustrative embodiments above; various modifications may be made to the illustrative embodiments, the results of which are also included within the scope of the embodiments in this description.
[00506] Next, examples of applications of the moving image encoding method (image encoding method) and the moving image decoding method (image decoding method) described in each of the above embodiments will be described, in addition to various systems that implement the application examples. Such a system can be characterized as including an image encoder employing the image encoding method, an image decoder employing the image decoding method, or an encoder-decoder that includes both the image encoder and the image decoder. Other configurations of such a system may be modified on a case-by-case basis. Petition 870260103461, dated 07 / 10 / 2026, page 115 / 376 106 / 121 Examples of Use
[00507] Figure 73 illustrates a general configuration of the ex100 content delivery system suitable for implementing a content distribution service. The area in which the communication service is provided is divided into cells of desired sizes, and the base stations ex106, ex107, ex108, ex109 and ex110, which are fixed wireless stations in the illustrated example, are located in their respective cells.
[00508] In the content delivery system ex100, devices including the computer ex111, the gaming device ex112, the camera ex113, the household appliance ex114, and the smartphone ex115 are connected to the Internet ex101 via the Internet service provider ex102 or communications network ex104 and base stations ex106 to ex110. The content delivery system ex100 can combine and connect any combination of the above devices. In various implementations, the devices can be connected directly or indirectly together via a telephony or near-field communication network instead of via base stations ex106 to ex110. Additionally, the transmission server ex103 can be connected to the devices including the computer ex111, the gaming device ex112, the camera ex113, the household appliance ex114, and the smartphone ex115 via, for example, the Internet ex101.The ex103 transmission server can also be connected, for example, to a terminal at a hotspot on the ex117 aircraft via the ex116 satellite.
[00509] Note that instead of base stations ex106 to ex110, wireless access points or hotspots can be used. The transmission server ex103 can be connected to the communications network ex104 directly instead of via the Internet ex101 or the Internet service provider ex102, and can be connected to the aircraft ex117 directly instead of via the satellite ex116.
[00510] The ex113 camera is a device capable of capturing still images and video, just like a digital camera. The ex115 smartphone is a smartphone device, a mobile phone, or a personal handyphone system (PHS) phone that can operate under the mobile communications system standards of 2G, 3G, 3.9G and 4G systems, as well as the next-generation 5G system.
[00511] The ex114 appliance is, for example, a refrigerator or a Petition 870260103461, dated 07 / 10 / 2026, page 116 / 376 107 / 121 device included in a domestic fuel cell joint generation system.
[00512] In the content delivery system ex100, a terminal, including an image and / or video capture function, is capable of, for example, live transmission by connection to the transmission server ex103 through, for example, the base station ex106. During live transmission, a terminal (for example, a computer ex111, a gaming device ex112, a camera ex113, a household appliance ex114, a smartphone ex115 or a terminal on the aircraft ex117) can perform the encoding processing described in the embodiments above on still image or video content captured by a user through the terminal, can multiplex video data obtained through encoding and audio data obtained through audio encoding corresponding to the video, and can transmit the obtained data to the transmission server ex103. In other words, the terminal functions as the image encoder according to one aspect of the present description.
[00513] The transmission server ex103 transmits the streamed content data to clients requesting the transmission. Examples of clients include the computer ex111, the gaming device ex112, the camera ex113, the household appliance ex114, the smartphone ex115, and the terminals inside the aircraft ex117, which can decode the encoded data described above. Devices receiving the streamed data can decode and play back the received data. In other words, the devices can each function as the image decoder, according to one aspect of the present description. Decentralized Processing
[00514] The ex103 broadcast server can be implemented as a plurality of servers or computers among which tasks such as data processing, recording, and transmission are divided. For example, the ex103 broadcast server can be implemented as a content delivery network (CDN) that transmits content through a network connecting multiple edge servers located around the world. In a CDN, an edge server physically close to the client can be dynamically assigned to the client. Content is temporarily stored and transmitted to the server. Petition 870260103461, dated 07 / 10 / 2026, page 117 / 376 108 / 121 edge to reduce load times. In the event of, for example, some kind of error or change in connectivity resulting from, for example, a traffic spike, it is possible to transmit data stably at high speeds, since it is possible to avoid the affected parts of the network, for example, by dividing the processing among a plurality of edge servers, or switching transmission tasks to a different edge server and continuing with the transmission.
[00515] Decentralization is not limited to just dividing processing for transmission; the encoding of captured data can be divided between and performed by the terminals, on the server side, or both. In one example, in typical encoding, processing is performed in two circuits. The first circuit serves to detect how complex the image is, frame by frame, or scene by scene, or to detect the encoding load. The second circuit serves for processing that maintains image quality and improves encoding efficiency. For example, it is possible to reduce the processing load on the terminals and improve the quality and efficiency of content encoding by having the terminals perform the first encoding circuit and having the server side that received the content perform the second encoding circuit.In such a case, after receiving a decoding request, it is possible for the encoded data, resulting from the first circuit performed by one terminal, to be received and reproduced at another terminal almost in real time. This enables smooth real-time transmission.
[00516] In another example, the ex113 camera, or similar, extracts a feature quantity (a number of features) from an image, compresses the data related to the feature quantity as metadata, and transmits the compressed metadata to a server. For example, the server determines the significance of an object based on the feature quantity and alters the quantization precision accordingly to perform compression appropriate to the meaning (or content significance) of the image. Feature quantity data is particularly efficient in providing accuracy and efficiency of motion vector prediction during the second compression pass performed by the server. Furthermore, encoding that has a relatively low processing load, such as variable-length encoding, is also suitable. Petition 870260103461, dated 07 / 10 / 2026, page 118 / 376 109 / 121 (VLC) can be handled by the terminal, and encoding that has a relatively high processing load, such as context-adaptive binary arithmetic encoding (CABAC), can be handled by the server.
[00517] In yet another example, there are cases in which a plurality of videos of almost the same scene are captured by a plurality of terminals, for example, in a stage, shopping center, or factory. In such a case, for example, encoding can be decentralized by dividing processing tasks among the plurality of terminals that captured the videos and, if necessary, other terminals that did not capture the videos, and the server, on a unit basis. The units can be, for example, groups of images (GOPs), images, or “tiles” resulting from the division of an image. This can make it possible to reduce loading times and achieve transmission that is closer to real-time.
[00518] Since the videos are approximately from the same scene, the server can manage and / or provide instructions so that the videos captured by the terminals can be cross-referenced. Additionally, the server can receive encoded data from the terminals, alter the reference relationship between the data items, or correct or replace images themselves, and then perform the encoding. This allows for the generation of a sequence with increased quality and efficiency for the individual data items.
[00519] Additionally, the server can transmit video data after transcoding has been performed to convert the video data encoding format. For example, the server can convert the encoding format from MPEG to VP (e.g., VP9), it can convert H.264 to H.265, etc.
[00520] Thus, encoding can be performed by a terminal or one or more servers. Accordingly, although the device that performs the encoding is referred to as a server or terminal in the following description, some or all of the processes performed by the server can be performed by the terminal, and similarly, some or all of the processes performed by the terminal can be performed by the server. This also applies to decoding processes. 3D, Multiple Angles
[00521] There has been an increase in the use of combined images or videos from images or videos of different scenes captured simultaneously, or Petition 870260103461, dated 07 / 10 / 2026, page 119 / 376 110 / 121 of the same scene captured from different angles by a plurality of devices, such as the ex113 camera and / or ex115 smartphone. The videos captured by the devices can be combined based, for example, on the relative position ratio obtained separately between the devices, or regions in a video possessing coincident feature points.
[00522] In addition to encoding two-dimensional moving images, the server can encode a still image based on scene analysis of a moving image, automatically or at a time specified by the user, and transmit the encoded still image to a receiving terminal. Additionally, when the server can obtain the relative position relationship between the video capture terminals, in addition to two-dimensional moving images, the server can generate the three-dimensional geometry of a scene based on video of the same scene, captured from different angles.The server can separately encode the three-dimensional data generated from, for example, a point cloud, and, based on a recognition or tracking result of a person or object using the three-dimensional data, it can select or reconstruct and generate a video to be transmitted to a receiving terminal from videos captured by a plurality of terminals.
[00523] This allows the user to enjoy a scene by freely selecting videos that correspond to the video capture terminals, and allows the user to enjoy the content obtained by extracting a video from a selected viewpoint from three-dimensional data reconstructed from a plurality of images or videos. Additionally, as with video, sound can be recorded from relatively different angles, and the server can multiplex the audio from a specific angle or space with the corresponding video, and transmit the multiplexed video and audio.
[00524] Recently, content that is a composite of the real world and a virtual world, such as virtual reality (VR) and augmented reality (AR) content, has also become popular. In the case of VR images, the server can create images from the viewpoint of both the left and right eyes, and perform encoding that tolerates reference between the two viewpoint images, such as multiview coding (MVC), or alternatively, it can encode the images as separate sequences without reference. When the images are Petition 870260103461, dated 07 / 10 / 2026, page 120 / 376 111 / 121 decoded as separate sequences, the sequences can be synchronized when played back, so as to recreate a virtual three-dimensional space according to the user's point of view.
[00525] In the case of AR images, the server can overlay virtual object information existing in a virtual space onto camera information representing a space in the real world, based on a three-dimensional position or movement from the user's perspective. The decoder can obtain or store virtual object information and three-dimensional data, generate two-dimensional images based on the user's perspective movement, and then generate the overlaid data by seamlessly connecting the images. Alternatively, the decoder can transmit the user's perspective movement to the server in addition to a request for virtual object information. The server can generate overlaid data based on three-dimensional data stored on the server, according to the received movement, and encode and transmit the generated overlaid data to the decoder.Note that the overlaid data typically includes, in addition to the RGB values, an α value indicating transparency, and the server sets the α value for sections beyond the object generated from the three-dimensional data to, for example, 0, and can perform encoding while these sections are transparent. Alternatively, the server can set the background to a specific RGB value, such as a chrominance key, and generate data in areas where other values besides the object are set as the background. The specific RGB value can be predetermined.
[00526] The decoding of similarly transmitted data can be performed by the client (e.g., terminals), on the server side, or divided between them. In one example, a terminal can transmit a reception request to a server, the requested content can be received and decoded by another terminal, and a decoded signal can be transmitted to a device with a monitor. It is possible to reproduce data with high image quality by decentralizing processing and selecting the content appropriately, regardless of the processing capacity of the communications terminal itself. In another example, while a TV, for instance, is receiving larger image data, a region of an image, such as a "tile" obtained by dividing the image, can be decoded and displayed on a viewer's or viewers' personal terminal(s). Petition 870260103461, dated 07 / 10 / 2026, page 121 / 376 112 / 121 TV. This allows viewers to share a large image view, and each viewer can check their designated area or inspect a region in greater detail.
[00527] In situations where multiple wireless connections are possible over short, intermediate, and long distances, indoors or outdoors, it may be possible to continuously receive content using a transmission system standard such as MPEG-DASH. The user can switch between data in real time while freely selecting a decoder or display device, including the user's terminal, monitors located indoors or outdoors, etc. Furthermore, using, for example, information about the user's location, decoding can be performed while switching which terminal handles decoding and which terminal handles content display. This allows for mapping and displaying information while the user is moving en route to a destination, on the wall of a nearby building in which a device capable of displaying the content is embedded, or on part of the floor.Furthermore, it is also possible to switch the bit rate of received data based on the accessibility of encoded data on a network, such as when encoded data is temporarily stored on a server readily accessible from the receiving terminal, or when encoded data is copied to an edge server in a content delivery service. Scalable Coding
[00528] Content switching will be described with reference to a scalable transmission, illustrated in Figure 74, which is encoded by compression through the implementation of the moving image coding method described in the modalities above. The server may have a configuration in which the content is switched while making use of the temporal and / or spatial scaling capability of a transmission, which is achieved by layer division and encoding, as illustrated in Figure 74. Note that there may be a plurality of individual transmissions that are equal to the same content but of different quality. In other words, by determining which layer to decode, based on internal factors such as processing capacity on the decoder side, and external factors such as communication bandwidth, the decoder side can freely switch between low-resolution content and high-resolution content. Petition 870260103461, dated 07 / 10 / 2026, page 122 / 376 113 / 121 high-resolution content during decoding. For example, in a case where the user wants to continue watching, for example, at home on a device such as an internet-connected TV, a video that the user was previously watching on their smartphone ex115 while on the go, the device can simply decode the same stream to a different layer, which reduces the load on the server side.
[00529] Additionally, in addition to the configuration described above, in which scalability is achieved as a result of images being encoded layer by layer, with the enhancement layer being above the base layer, the enhancement layer may include metadata based, for example, on statistical information in the image. The decoder side may generate high-quality image content by performing super-resolution image creation on an image in the base layer based on the metadata. Super-resolution image creation may enhance the SN ratio while maintaining and / or increasing the resolution. The metadata includes information to identify a linear or non-linear filter coefficient, as used in super-resolution processing, or information identifying a parameter value in filter processing, machine learning, or mean squares method used in super-resolution processing.
[00530] Alternatively, a configuration can be avoided in which an image is divided, for example, into “tiles” according to, for example, the meaning of an object in the image. On the decoder side, only a partial region is decoded by selecting a “tile” to be decoded. Additionally, by storing an attribute of the object (person, car, ball, etc.) and a position of the object in the video (coordinates in identical images) as metadata, the decoder side can identify the position of a desired object based on the metadata and determine which “tile” or “tiles” include that object. For example, as illustrated in Figure 75, metadata can be stored using a data storage structure different from pixel data, such as a SEI (supplementary enhancement information) message in HEVC. This metadata indicates, for example, the position, size, or color of the main object.
[00531] Metadata can be stored in units among a plurality of images, such as a stream, sequence, or units of Petition 870260103461, dated 07 / 10 / 2026, page 123 / 376 114 / 121 random access. The decoder side can obtain, for example, the moment when a specific person appears in the video, and by matching the time information with the image unit information, it can identify an image in which the object is present, and can determine the object's position in the image. Website Optimization
[00532] Figure 76 illustrates an example of a web page display screen on computer ex111, for example. Figure 77 illustrates an example of a web page display screen on smartphone ex115, for example. As illustrated in Figure 76 and Figure 77, a web page may include a plurality of image links that are links to image content, and the appearance of the web page may differ depending on the device used to view the web page.When a plurality of image links is viewable on the screen, until the user explicitly selects an image link, or until the image link is approximately centered on the screen, or until the entire image link fits on the screen, the display device (decoder) may display, as the image links, static images included in the content or I-images; it may display video, such as an animated GIF, using a plurality of static images or I-images; or it may receive only the base layer, and decode and display the video.
[00533] When an image link is selected by the user, the display device performs decoding while, for example, giving the highest priority to the base layer. Note that if there is information in the web page's HTML code indicating that the content is scalable, the display device may decode up to the enhancement layer. Additionally, in order to guarantee real-time playback, before a selection is made or when bandwidth is severely limited, the display device may reduce the delay between the moment the foreground image is decoded and the moment the decoded image is displayed (i.e., the delay between the start of content decoding and the display of the content) by decoding and displaying only foreground reference images (foreground reference image I, image P, foreground reference image B).Additionally, the display device may intentionally ignore the reference relationship between the images, and roughly decode all B and P images as front reference images, and then perform normal decoding as the number of images received increases over time. Petition 870260103461, dated 07 / 10 / 2026, page 124 / 376 115 / 121 decreases. Autonomous Driving
[00534] When transmitting and receiving still image or video data, such as two- or three-dimensional map information, for autonomous driving or assisted driving of a car, the receiving terminal may receive, in addition to the image data belonging to one or more layers, information, for example, about the weather or road works, as metadata, and associate the metadata with the image data after decoding. Note that the metadata may be called a layer and, alternatively, may simply be multiplexed with the image data.
[00535] In such a case, since the automobile, drone, aircraft, etc., containing the receiving terminal, is mobile, the receiving terminal can continuously receive and perform decoding while switching between base stations, from base stations ex106 to ex110, by transmitting information indicating the position of the receiving terminal. Furthermore, according to the user's selection, the user's situation, and / or the connection bandwidth, the receiving terminal can dynamically select the extent to which metadata is received, or the extent to which map information, for example, is updated.
[00536] In the ex100 content delivery system, the client can receive, decode and reproduce, in real time, the encoded information transmitted by the user. Individual Content Transmission
[00537] In the ex100 content delivery system, in addition to long, high-quality image content distributed by a video distribution entity, single-stream or multi-stream transmission of short, low-quality image content from an individual is also possible. Such content from individuals has a chance of growing in popularity. The server can first perform an editing process on the content before encoding processing in order to refine the individual's content. This can be achieved using the following configuration, for example.
[00538] In real time, while video or image content is being captured, or after the content has been captured and accumulated, the server performs the Petition 870260103461, dated 07 / 10 / 2026, page 125 / 376 116 / 121 Recognition processing based on raw or encoded data, such as capture error processing, scene search processing, meaning analysis, and / or object detection processing. Then, based on the result of the recognition processing, the server – either when notified or automatically – edits the content, examples of which include: correction, such as focus correction and / or motion blur correction; removal of low-priority scenes, such as scenes that are low in brightness compared to other images, or out of focus; object edge adjustment; and color tone adjustment. The server encodes the edited data based on the result of the editing. It is known that excessively long videos tend to receive fewer views.Accordingly, in order to keep the content within a specific length comparable to the length of the original video, the server may, in addition to the low-priority scenes described above, automatically remove scenes with little movement, based on an image processing result. Alternatively, the server may generate and encode a video summary based on an analysis of the significance of a scene.
[00539] There are cases in which individual content may include content that infringes copyright, moral rights, image rights, etc. Such cases may result in an unfavorable situation for the creator, such as when the content is shared beyond the scope intended by the creator. Accordingly, before encoding, the server may, for example, edit images to blur the faces of people on the periphery of the screen or blur the interior of a house, for example. Additionally, the server may be configured to recognize the faces of people, in addition to a person registered in the images to be encoded, and when such faces appear in an image, it may apply a mosaic-type filter, for example, to the person's face. Alternatively, as pre- or post-processing for encoding, the user may specify, for copyright reasons, a region of an image that includes a person or a region of the background to be processed.The server can process the specified region, for example, by replacing the region with a different image, or by blurring the region. If the region includes a person, the person can be tracked in the moving image, and the region of the person's head can be replaced with another image as the person moves. Petition 870260103461, dated 07 / 10 / 2026, page 126 / 376 117 / 121
[00540] Given the demand for real-time viewing of content produced by individuals, which tends to be small in data size, the decoder may first receive the base layer as the highest priority, and perform decoding and playback, although this may differ depending on bandwidth. When the content is played back two or more times, such as when the decoder receives the enhancement layer during the encoding and playback of the base layer and creates a playback loop, the decoder can play back a video with high image quality including the enhancement layer. If the transmission is encoded using such scalable encoding, the video may have low quality when in an unselected state or at the beginning of the video, but it can offer an experience in which the image quality of the transmission progressively increases in an intelligent way.This is not limited to scalable encoding; the same experience can be offered by setting up a single stream from a low-quality stream played for the first time and a second stream encoded using the first stream as a reference. Other Implementations and Application Examples
[00541] Encoding and decoding can be performed by an LSI (large-scale integration circuit array) ex500 (see Figure 73), which is typically included in each terminal. The LSI ex500 can be configured from a single chip or a plurality of chips. The software for encoding and decoding moving images can be integrated into some type of recording medium (such as a CD-ROM, a floppy disk, or a hard disk) that is readable, for example, by the computer ex111, and encoding and decoding can be performed using software. Additionally, when the smartphone ex115 is equipped with a camera, the video data obtained by the camera can be transmitted. In this case, the video data can be encoded by the LSI ex500 included in the smartphone ex115.
[00542] Note that the LSI ex500 can be configured to download and activate an application. In such a case, the terminal first determines whether it is compatible with the scheme used to encode the content, or whether it is capable of running a specific service. When the terminal is not compatible with the content encoding scheme, or when the terminal is not capable of running a specific service, the Petition 870260103461, dated 07 / 10 / 2026, p. 127 / 376 The 118 / 121 terminal can first download a codec or application software and then obtain and play the content.
[00543] In addition to the example of an ex100 content delivery system using the Internet ex101, at least the moving picture encoder (picture encoder) or the moving picture decoder (picture decoder), described in the embodiments above, can be implemented in a digital broadcast system. The same encoding and decoding processing can be applied to transmit and receive broadcast radio waves superimposed on multiplexed audio and video data using, for example, a satellite, although it is geared towards multicasting, whereas single-casting is easier with the ex100 content delivery system. Hardware Configuration
[00544] Figure 78 illustrates additional details of the ex115 smartphone illustrated in Figure 73. Figure 79 illustrates an example of an ex115 smartphone configuration. The ex115 smartphone includes the ex450 antenna, for transmitting and receiving radio waves to and from the ex110 base station, the ex465 camera, capable of capturing video and still images, and the ex458 monitor, which displays the decoded data, such as the video captured by the ex465 camera and the video received by the ex450 antenna.The ex115 smartphone additionally includes the ex466 user interface, such as a touch panel; an ex457 audio output unit, such as a speaker for sending speech or other audio; an ex456 audio input unit, such as a microphone for audio input; ex467 memory capable of storing decoded data, such as captured video or still images, recorded audio, received video or still images, and email, in addition to decoded data; and the ex464 partition, which is an interface for the ex468 SIM card to authorize access to a network and various data. Note that external memory can be used in place of the ex467 memory.
[00545] The main controller ex460, which can deeply control the monitor ex458 and the user interface ex466, the power supply circuit ex461, the user interface input controller ex462, the video signal processor ex455, the camera interface ex463, the display controller ex459, the modulator / demodulator ex452, the multiplexer / demultiplexer ex453, the Petition 870260103461, dated 07 / 10 / 2026, page 128 / 376 The 119 / 121 audio signal processor ex454, the partition ex464, and the memory ex467 are connected via the bus ex470.
[00546] When the user turns on the power button of the ex461 power supply circuit, the ex115 smartphone is powered into an operational state, and each component is supplied with power from a battery pack.
[00547] The ex115 smartphone performs processing, for example, for calls and data transmission, based on control by the ex460 main controller, which includes a CPU, ROM, and RAM. When making calls, an audio signal recorded by the ex456 audio input unit is converted into a digital audio signal by the ex454 audio signal processor, to which spread spectrum processing is applied by the ex452 modulator / demodulator and digital-to-analog conversion, and frequency conversion processing is applied by the ex451 transmitter / receiver, and the resulting signal is transmitted through the ex450 antenna. The received data is amplified, frequency-converted, and converted from analog to digital, processed by inverse spread spectrum by the ex452 modulator / demodulator, converted into an analog audio signal by the ex454 audio signal processor, and then sent from the ex457 audio output unit.In data transmission mode, text, still image, or video data can be transmitted under the control of the main controller ex460 via the interface input controller ex462 based on the operation of the user interface ex466 of the main body, for example. Similar transmission and reception processing is performed. In data transmission mode, when sending a video, still image, or video and audio, the video signal processor ex455 encodes by compression, using the moving image encoding method described in the embodiments above, a video signal stored in the memory ex467 or a video signal input from the camera ex465, and transmits the encoded video data to the multiplexer / demultiplexer ex453.The ex454 audio signal processor encodes an audio signal recorded by the ex456 audio input unit while the ex465 camera is capturing video or a still image, and transmits the encoded audio data to the ex453 multiplexer / demultiplexer. The ex453 multiplexer / demultiplexer multiplexes the encoded video data and the encoded audio data using a defined scheme, modulates, and converts. Petition 870260103461, dated 07 / 10 / 2026, page 129 / 376 120 / 121 the data using the modulator / demodulator (modulator / demodulator circuit) ex452 and the transmitter / receiver ex451, and transmits the result through the antenna ex450. The determined scheme can be predetermined.
[00548] When video attached to an email or chat, or a video linked from a web page, is received, for example, in order to decode the multiplexed data received through the ex450 antenna, the ex453 multiplexer / demultiplexer demultiplexes the multiplexed data to divide the multiplexed data into a sequence of video data bits and a sequence of audio data bits, supplies the encoded video data to the ex455 video signal processor via the ex470 synchronized bus, and supplies the encoded audio data to the ex454 audio signal processor via the ex470 synchronized bus.The ex455 video signal processor decodes the video signal using a motion picture decoding method corresponding to the motion picture encoding method described in the embodiments above, and the video or a still image included in the linked motion picture file is displayed on the ex458 monitor via the ex459 display controller. The ex454 audio signal processor decodes the audio signal and sends the audio from the ex457 audio sending unit. Since real-time transmission is becoming increasingly popular, there may be instances where audio playback could be socially inappropriate, depending on the user's environment.Accordingly, as an initial value, a configuration in which only the video data is played, that is, the audio signal is not played, may be preferable; the audio can be synchronized and played only when a record, such as when the user clicks on the video data, is received.
[00549] Although the ex115 smartphone was used in the example above, other implementations are conceivable; a transceiver terminal, including both an encoder and a decoder; a transmitter terminal, including only an encoder; and a receiver terminal, including only a decoder. In the description of the digital broadcasting system, an example is provided in which multiplexed data, obtained as a result of video data being multiplexed with audio data, is received or transmitted. The multiplexed data, however, can be video data multiplexed with data other than audio data, such as text data related to the video. Additionally, the video data Petition 870260103461, dated 07 / 10 / 2026, page 130 / 376 121 / 121 data, as such, rather than multiplexed data, can be received or transmitted.
[00550] Although the main ex460 controller, including a CPU, is described as controlling encoding or decoding processes, many terminals often include GPUs. Accordingly, a configuration is acceptable in which a large area is processed at once by making use of the GPU's performance capacity through memory shared by the CPU and GPU, or memory including an address that is managed in such a way as to allow common use by the CPU and GPU. This enables shortening of encoding time, maintenance of the real-time nature of the transmission, and reduction of latency. In particular, processing relating to motion estimation, deblock filtering, adaptive sample shifting (SAO), and transformation / quantization can be effectively performed by the GPU instead of the CPU in units of images, for example, all at once. Petition 870260103461, dated 07 / 10 / 2026, page 131 / 376
Claims
1 / 2 CLAIMS 1. An image encoder, characterized in that it comprises: a set of circuits; and a memory coupled to the set of circuits; wherein the set of circuits, in operation: provides a first set of pixels for a first partition of a current image with a first motion vector, the first motion vector being a uniprediction motion vector; provides a second set of pixels for a second partition with a second motion vector; weights pixels from the first set of pixels and from the second set of pixels for a first part, wherein the first partition is superimposed on the second partition; stores, for the first part, one of the first motion vector and the second motion vector, in response to a first condition; stores, for the first part, both the first motion vector and the second motion vector, in response to a second condition different from the first condition;and encodes the first partition using at least the weighted pixels.
2. Image decoder, characterized in that it comprises: a set of circuits; and a memory coupled to the set of circuits; wherein the set of circuits, in operation: provides a first set of pixels for a first partition of a current image with a first motion vector, the first motion vector being a single-prediction motion vector; provides a second set of pixels for a second partition. Petition 870260103461, dated 07 / 10 / 2026, p.132 / 376 2 / 2 with a second motion vector; weights pixels from the first set of pixels and the second set of pixels into a first partition, where the first partition is overlapped with the second partition; stores, for the first partition, one of the first motion vector and the second motion vector, in response to a first condition; stores, for the first partition, both the first motion vector and the second motion vector, in response to a second condition different from the first condition; and decodes the first partition using at least the weighted pixels.
3. Non-transient computer-readable medium characterized in that it stores a bit sequence, the bit sequence including information indicating a first condition or a second condition different from the first condition and according to which a decoder performs a partition decoding process, in the partition decoding process: a first motion vector for a first partition and a second motion vector for a second partition being derived to decode a first partition in which the first partition is overlapped with the second partition in a current block, the first motion vector being a uniprediction motion vector; one of the first motion vector and the second motion vector being stored for the first partition in response to the first condition; and both the first motion vector and the second motion vector being stored for the first partition in response to the second condition.Petition 870260103461, dated 07 / 10 / 2026, pp. 133 / 376.
