Methods, apparatus, and computer program products for video encoding and decoding.

By transforming the mixture matrix into a triangular form and using row-based operations, the method addresses computational inefficiencies in determining filter parameters for cross-component prediction, improving encoding efficiency and stability in video encoding technologies.

JP7954301B2Active Publication Date: 2026-11-19NOKIA TECHNOLOGIES OY
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2025535118
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-12-15
Filing Date
2023-11-09
Publication Date
2026-11-19
Estimated Expiration
2043-11-09

AI Technical Summary

Technical Problem

Existing video encoding technologies face challenges in efficiently determining filter parameters for cross-component prediction, particularly in large autocorrelation matrices, leading to numerical stability issues and computational inefficiencies.

Method used

A method is introduced to determine filter parameters by generating a mixture matrix from an autocorrelation matrix and cross-correlation vectors, transforming it into a triangular form using row-based operations, and solving for filter coefficients through back substitution, with mechanisms to prevent division by zero and control dynamic range.

Benefits of technology

This approach reduces computational complexity and improves numerical stability, enabling efficient calculation of filter parameters for cross-component prediction in video encoding, enhancing encoding efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007954301000009
    Figure 0007954301000009
  • Figure 0007954301000010
    Figure 0007954301000010
  • Figure 0007954301000011
    Figure 0007954301000011
Patent Text Reader

Abstract

The present embodiment relates to a method for encoding and a technical apparatus for implementing the method, which includes: processing an input video / image (310); determining a set of filter coefficients (320), where the method includes determining a mixing matrix including an autocorrelation matrix and at least one cross-correlation vector (330); modifying the determined mixing matrix into a triangular formula by determining at least one scale parameter between a source row and a destination row in the mixing matrix (340); determining the modified destination row in the mixing matrix by multiplying the source row with the scale parameter and subtracting the scaled source row from the destination row (350); determining a set of filter parameters from the triangular formula of the mixing matrix (360); and using the set of filter coefficients in a filter (370).
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] This solution generally concerns video encoding and decoding. In particular, it concerns determining the set of filter parameters used for encoding / decoding. [Background technology]

[0002] This section is intended to provide background or context to the inventions enumerated in the claims. The descriptions herein may include concepts that may be pursued but have not necessarily been previously assumed or pursued. Therefore, unless otherwise indicated herein, nothing described herein is prior art to the description and claims of this application, nor is it deemed to be prior art by being included herein.

[0003] A video encoding system may include an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that can restore the compressed video representation into a viewable format. The encoder may discard some information in the original video sequence to represent the video in a more compact format, for example, to enable storage / transmission of video information at a lower bitrate than necessary. [Overview of the Initiative]

[0004] The scope of protection required for various embodiments of the invention is presented by the independent claims. Any embodiments and features described herein that do not fall within the scope of the independent claims should be interpreted as useful examples for understanding various embodiments of the invention.

[0005] Various embodiments include methods, apparatus, and computer-readable media containing stored computer programs, which are characterized by those described in the independent claims. Various embodiments are disclosed in the dependent claims.

[0006] According to a first aspect, an apparatus is provided comprising means for processing an input video / image, and means for determining a set of filter coefficients, wherein a mixture matrix including an autocorrelation matrix and at least one cross-correlation vector is determined, the determined mixture matrix is ​​modified into a triangular formula by determining at least one scale parameter between source rows and destination rows in the mixture matrix, the modified destination rows in the mixture matrix are determined by multiplying the source rows by the scale parameter and subtracting the scaled source rows from the destination rows, and the set of filter parameters is determined from the triangular formula of the mixture matrix, the apparatus comprising means for using the set of filter coefficients in a filter.

[0007] A second aspect provides a method for determining a set of filter coefficients, which includes: processing an input video / image; determining a set of filter coefficients, which includes determining a mixture matrix comprising an autocorrelation matrix and at least one cross-correlation vector; modifying the determined mixture matrix into a triangular formula by determining at least one scale parameter between source rows and destination rows in the mixture matrix; determining the modified destination rows in the mixture matrix by multiplying the source rows by the scale parameter and subtracting the scaled source rows from the destination rows; determining a set of filter parameters from the triangular formula of the mixture matrix; and using the set of filter coefficients in a filter.

[0008] According to a third aspect, a device is provided comprising at least one processor and a memory containing computer program code, wherein the memory and the computer program code are configured to use at least one processor to cause the device to perform at least the following: process an input video / image; determine a set of filter coefficients, which includes determining a mixture matrix containing an autocorrelation matrix and at least one cross-correlation vector; modify the determined mixture matrix into a triangular expression by determining at least one scale parameter between source rows and destination rows in the mixture matrix; determine the modified destination rows in the mixture matrix by multiplying the source rows by the scale parameter and subtracting the scaled source rows from the destination rows; determine a set of filter parameters from the triangular expression of the mixture matrix; and use the set of filter coefficients in a filter.

[0009] A fourth aspect provides a computer program product which, when executed on at least one processor, includes computer program code configured to cause a device or system to process an input video / image and to use the set of filter coefficients in a filter, including determining a set of filter coefficients, which means determining a mixture matrix including an autocorrelation matrix and at least one cross-correlation vector, modifying the determined mixture matrix into a triangular formula by determining at least one scale parameter between source rows and destination rows in the mixture matrix, determining the modified destination rows in the mixture matrix by multiplying the source rows by the scale parameter and subtracting the scaled source rows from the destination rows, and determining a set of filter parameters from the triangular formula of the mixture matrix.

[0010] According to an embodiment, the mixing matrix is modified by respective means by scaling at least one value of the determined row using a scaling parameter, a rounding parameter, and a bit shift parameter, and the scaling parameter, the rounding parameter, and the bit shift parameter are determined from the diagonal elements of the mixing matrix of the determined row.

[0011] According to an embodiment, the set of filter coefficients is used in convolutional cross-component prediction, luma samples (luminance samples) are used as input, and predicted chroma samples (chrominance difference samples) are used as output.

[0012] According to an embodiment, the mixing matrix is determined to include two cross-correlation vectors, the first cross-correlation vector relates to the cross-correlation between a luma sample and a first chroma sample, and the second cross-correlation vector relates to the cross-correlation between the luma sample and a second chroma sample.

[0013] According to an embodiment, the set of filter parameters is determined from the triangular form of the mixing matrix using a back substitution process.

[0014] According to an embodiment, the back substitution includes at least one of the filter parameters that is set equal to an element in the mixing matrix.

[0015] According to an embodiment, when modifying the determined mixing matrix into a triangular form, at least for one diagonal sample, it is checked whether the value of the diagonal sample or the absolute value of the diagonal sample is less than a predetermined threshold, and if it is less than the predetermined threshold, the value of the diagonal sample is set equal to the predetermined threshold.

[0016] According to the embodiment, when the determined mixture matrix is ​​modified into a triangular matrix, it is checked whether at least the value of the diagonal sample or the absolute value of the diagonal sample is less than a predetermined threshold for each diagonal sample. If it is less than the predetermined threshold, the modification process is terminated and the filter coefficients are set to a predetermined set of values.

[0017] According to the embodiment, the process is encoding or decoding.

[0018] According to one embodiment, the computer program product is implemented on a non-temporary computer-readable medium.

[0019] Various embodiments are described in more detail below with reference to the attached drawings. [Brief explanation of the drawing]

[0020] [Figure 1] This figure shows the encoding process according to the embodiment. [Figure 2] This figure shows the decoding process according to the embodiment. [Figure 3] This is a flowchart illustrating the method according to the embodiment. [Figure 4] This figure shows an apparatus according to an embodiment. [Modes for carrying out the invention]

[0021] The following descriptions and drawings are illustrative and should not be construed as unnecessarily restrictive. Certain details are provided for a full understanding of the disclosure. However, in some cases, well-known or conventional details are omitted so as not to obscure the description. References to one or more embodiments in this disclosure may refer to the same embodiment, but not necessarily, and such references mean at least one of the embodiments.

[0022] Any reference herein to “one embodiment” or “embodiment” means that certain features, structures, or characteristics described in relation to an embodiment are included in at least one embodiment of the disclosure.

[0023] Several embodiments are described below in the context of a single video encoding configuration. However, it should be noted that these embodiments are not necessarily limited to this particular configuration. The embodiments relate to cross-component filter parameter calculation using row-based matrix operations.

[0024] The Advanced Video Coding standard (AVC or H.264 / AVC) was developed by the Joint Video Team (JVT) of the Video Coding Expert Group (VCEG) of the Telecommunications Standardization Sector of the International Telecommunication Union (ITU-T) and the Video Expert Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). The H.264 / AVC standard is published by both parent standardization organizations and is also known as ITU-T Recommendation H.264 and ISO / IEC International Standard 14496-10, and MPEG-4 Part 10 Advanced Video Coding (AVC). Multiple versions of the H.264 / AVC standard exist, each incorporating new extensions or features into the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0025] The High Efficiency Video Coding standard (HEVC or H.265 / HEVC) was developed by the Joint Video Coding Team (JCT-VC) of VCEG and MPEG. The standard is published by both parent standardization organizations and is known as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also called MPEG-H Part 2 High Efficiency Video Coding (AVC). Extensions to H.265 / HEVC include scalable, multi-view, 3D, and fidelity range extensions, which may be referred to as SHVC, MV-HEVC, 3D-HEVC, and REXT, respectively. References to H.265 / HEVC, SHVC, MV-HEVC, 3D-HEVC, and REXT in this description are made for the purpose of understanding the definitions, and the structure or concepts of these standard specifications should be understood as references to the latest versions of these standards that were available prior to the filing date of this application, unless otherwise indicated.

[0026] Multipurpose Video Coding (VVC, H.266, or sometimes abbreviated as H.266 / VVC) is a video compression standard developed as a successor to HEVC. VVC is specified in ITU-T Recommendation H.266, equivalently in ISO / IEC 23090-3, also known as MPEG-I Part 3.

[0027] The AV1 bitstream format and decoding process specifications were developed by the Alliance of Open Media (AOM). The AV1 specification was published in 2018. AOM is reportedly working on the AV2 specification.

[0028] Several important definitions, bitstreams and encoding structures, as well as the concepts of H.264 / AVC, HEVC, VVC and / or AV1, and some of their extensions, are described in this article as examples of video encoders, decoders, encoding methods, decoding methods, and bitstream structures, and embodiments may be implemented. While the various embodiments are not limited to H.264 / AVC, HEVC, VVC and / or AV1, or their extensions, the description is given for one possible basis on which these embodiments may be realized partially or completely.

[0029] A video codec may include an encoder that converts input video into a compressed representation suitable for storage / transmission, and a decoder that can restore the compressed video representation into a viewable format. The compressed representation may be called a bitstream or video stream. The video encoder and / or video decoder may also be separate from each other, i.e., they do not need to form a codec. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form (i.e., at a lower bitrate). The notation "(de)coder" refers to both the encoder and / or decoder.

[0030] Hybrid video codecs, such as ITU-T H.263, H.264 / AVC, and HEVC, may encode video information in two stages. First, pixel values in a certain picture region (or "block") are predicted, for example, by motion compensation means (finding and indicating a region in one of the previously encoded video frames that closely corresponds to the block being encoded) or by spatial means (using pixel values around the block to be encoded in a specified manner). Thereafter, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is encoded. This may be performed by transforming the difference in pixel values using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant thereof), quantizing the coefficients, and entropy encoding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and the size of the resulting encoded video representation (file size or transmission bit rate). The encoding process according to the embodiments is illustrated in Figure 1.

[0031] Figure 1 shows an image to be encoded (I n ), a predictive representation of an image block (P' n ), a prediction error signal (D n ), a reconstructed prediction error signal (D' n ), a preliminary reconstructed image (I' n ), a final reconstructed image (R' n ), a transform (T) and an inverse transform (T -1 ), a quantization (Q) and an inverse quantization (Q -1 ), entropy encoding (E), a reference frame memory (RFM), inter prediction (P inter ), intra prediction (P intra ), mode selection (MS) and filtering (F).

[0032] In some video codecs, such as H.265 / HEVC, a video picture is divided into coding units (CUs) that cover the area of ​​the picture. A CU consists of one or more prediction units (PUs) that define a prediction process for the samples within the CU, and one or more transformation units (TUs) that define a prediction error coding process for the samples within the CU. A CU may consist of square blocks of samples with sizes selectable from a predefined set of possible CU sizes. A CU with the maximum allowable size may be called an LCU (Largest Coding Unit) or CTU (Coding Tree Unit), and the video picture is divided into non-overlapping CTUs. A CTU may be further divided into smaller combinations of CUs, for example, by recursively splitting the CTU and the resulting CU. Each resulting CU may have at least one PU and at least one associated TU. Each PU and TU may be further divided into smaller PUs and TUs to improve the granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it (e.g., motion vector information for inter-predicted PUs and intra-prediction directionality information for intra-predicted PUs) that defines which type of prediction should be applied to the pixels within that PU. Similarly, each TU is associated with information describing the prediction error decoding process for the samples within the TU (e.g., including DCT coefficient information). Whether or not prediction error coding is applied to each CU may be signaled at the CU level. If there is no prediction error residual associated with a CU, it may be assumed that there is no TU for that CU. The division of an image into CUs, and the division of CUs into PUs and TUs, may be signaled in the bitstream to allow the decoder to reproduce the intended structure of these units.

[0033] The basic unit of input to an encoder and output to a decoder is, in most cases, a picture. The picture given as input to the encoder may also be called the source picture, and the picture decoded by the decoder may be called the decoded picture or reconstructed picture.

[0034] The source picture and the decoded picture each consist of one or more sample arrays, such as one of the following sets of sample arrays. - Luma (Y) only (monochrome) - Luma and two chroma (YCbCr or YCgCo) - Green, blue, and red (also known as GBR or RGB) - Arrays representing other unspecified monochrome or tristimulus color samples (e.g., YZX, also known as XYZ)

[0035] Hereinafter, these arrays may also be called luma (i.e., L or Y) and chroma, and the two chroma arrays may be called Cb and Cr, regardless of the actual color representation method used. The actual color representation method used may be indicated, for example, in the encoded bitstream, using, for example, the HEVC Video Usability Information (VUI) syntax or similar. A component may be defined as a single sample from an array or one of three sample arrays (luma and two chroma), or as a single sample from an array that constitutes a picture in an array or monochrome format.

[0036] A picture can be defined as either a frame or a field. A frame contains a matrix of luma samples and possibly corresponding chroma samples. A field is a set of alternating sample rows from a frame and can be used as an encoder input when the source signal is interlaced. The chroma sample array may not exist (and therefore monochromatic sampling may be used), or the chroma sample array may be subsampled compared to the luma sample array.

[0037] The decoder reconstructs the output video by applying encoder-like prediction means to form a predictive representation of pixel blocks (using motion or spatial information created by the encoder and stored in a compressed representation), and prediction error decoding (the inverse operation of prediction error coding, which restores the quantized prediction error signal in the spatial pixel domain). After applying the prediction and prediction error decoding means, the decoder sums the prediction and prediction error signals (pixel values) to form an output video frame. The decoder (and encoder) may also apply additional filtering means to improve the quality of the output video before handing it over for display and / or before storing it as a predictive reference for future frames in the video sequence. The decoding process according to an embodiment is shown in Figure 2.

[0038] Figure 2 shows the predicted representation (P') of the image block. n ), reconstructed prediction error signal (D' n ), preliminary reconstructed image (I' n ), final reconstructed image (R' n ), inverse transform (T -1 ), inverse quantization (Q -1 ), entropy decoding (E -1 This shows the reference frame memory (RFM), prediction (either inter or intra) (P), and filtering (F).

[0039] Color palette-based coding may be used instead of, or in addition to, methods that utilize sample value prediction and transformation coding to indicate encoded sample values. Palette-based coding refers to a family of methods in which a palette, i.e., a set of colors and their associated indices, is defined, and the value of each sample within a coding unit is represented by indicating its index in the palette. Palette-based coding can achieve good coding efficiency in coding units with a relatively small number of colors (such as image regions representing computer screen content like text or simple graphics). To improve the coding efficiency of palette coding, different types of palette index prediction methods may be used, or the palette indices may be run-length coded to enable efficient representation of larger homogeneous image regions. Escape coding may also be used when a CU contains sample values ​​that do not cycle within the CU. Escaped samples are transmitted without referencing any palette index. Instead, their values ​​are indicated independently for each escaped sample.

[0040] Motion information can be indicated using motion vectors associated with each motion-compensated image block in the video codec. Each of these motion vectors represents the displacement of the image block in the picture being encoded (on the encoder side) or decoded (on the decoder side), and the predicted source block in one of the previously encoded or decoded pictures. To efficiently represent motion vectors, they can be encoded differently for each predicted motion vector-specific block. Predicted motion vectors can be created in a predefined way, for example, by calculating the median of the encoded or decoded motion vectors of adjacent blocks. Another way to create motion vector predictions is to generate a list of candidate predictions from adjacent and / or co-located blocks in the temporal reference picture, and to signal the candidates selected as motion vector predictors. In addition to predicting motion vector values, the reference index of a previously encoded / decoded picture can be predicted. The reference index can be predicted from adjacent and / or co-located blocks in the temporal reference picture. Furthermore, high-efficiency video codecs may employ an additional motion information encoding / decoding mechanism, often called merging mode, in which all motion field information, including motion vectors and corresponding reference picture indices for each available reference picture list, is predicted and used without any modification / correction. Similarly, predicting motion field information is performed using motion field information from adjacent blocks and / or collated blocks within a temporal reference picture, and the motion field information used is signaled among a list of motion field candidate lists filled with motion field information from available adjacent blocks / collated blocks.

[0041] A bitstream can be defined as a sequence of bits or a sequence of syntactic structures. A bitstream format can restrict the order of syntactic structures within a bitstream.

[0042] Syntactic elements can be defined as elements of data represented in a bitstream. Syntactic structures can be defined as zero or more syntactic elements present together in a bitstream in a specified order.

[0043] In some encoding formats or standards, the bitstream may be in the form of a unit stream or byte stream of the network abstraction layer (NAL), which forms a representation of encoded pictures and associated data that make up one or more encoded video sequences.

[0044] A NAL unit can be defined as a syntactic structure containing bytes that include the data in the form of an RBSP, interspersed with an indication of the type of data that follows and, if necessary, a start code emulation prevention byte. A raw byte sequence payload (RBSP) can be defined as a syntactic structure containing an integer number of bytes encapsulated in a NAL unit. An RBSP has the form of a string of data bits containing syntactic elements, which is either empty or followed by an RBSP stop bit and followed by zero or one or more zero-equal-values.

[0045] A NAL unit includes a header and a payload. The NAL unit header specifically indicates the type of the NAL unit.

[0046] In some encoding formats, such as AV1, a bitstream may contain a sequence of open bitstream units (OBUs). An OBU contains a header and a payload; the header identifies the type of OBU. Furthermore, the header may contain the size of the payload in bytes.

[0047] Phrases aligned with a bitstream (e.g., indicating along the bitstream) or along a bitstream encoding unit (e.g., indicating along an encoding tile) may be used in the claims and the embodiments described to refer to transmission, signaling, or storage in a manner in which “out-of-band” data is associated with the bitstream or encoding unit, respectively, but is not contained within it. Phrase decoding aligned with a bitstream or along a bitstream encoding unit or similar may refer to decoding referenced out-of-band data (which may be retrieved from out-of-band transmission, signaling, or storage) associated with the bitstream or encoding unit, respectively. For example, phrases aligned with a bitstream may be used when the bitstream is contained in a container file such as a file conforming to an ISO-based media file format, and certain file metadata is stored in the file such that metadata is associated with the bitstream, such as a box in a sample entry for a track containing the bitstream, a sample group for a track containing the bitstream, or a time-specified metadata track associated with a track containing the bitstream.

[0048] The video codec may support motion-compensated predictions from one source image (single-prediction) and two sources (double-prediction). In the case of single-prediction, a single motion vector is applied; in the case of double-prediction, two motion vectors are signaled, and the motion-compensated predictions from the two sources are averaged to create the final sample prediction. In the case of weighted prediction, the relative weights of the two predictions may be adjusted, or a signaled offset may be added to the prediction signal.

[0049] In addition to applying motion compensation for interpicture prediction, a similar technique can be applied to intrapicture prediction. In this case, the displacement vector indicates where blocks of a sample may be copied from the same picture to form predictions of the blocks to be encoded or decoded. This type of intrablock copying method can significantly improve encoding efficiency when repetitive structures, such as text or other graphics, exist within a frame.

[0050] The predicted residuals after motion compensation or intra-prediction can first be transformed using a transform kernel (such as DCT) and then encoded. This is because, often, some correlation still exists between the residuals, and the transform can often help reduce this correlation and provide more efficient encoding.

[0051] A video encoder may utilize a Lagrangian cost function to find the optimal encoding mode, for example, a desired macroblock mode and associated motion vectors. This type of cost function uses a weighting coefficient λ to tie together the (exact or estimated) image distortion resulting from the lossy encoding method with the amount of (exact or estimated) information required to represent the pixel values ​​in the image region. C=D+λR (formula 1) Here, C is the Lagrange cost to be minimized, D is the image distortion (e.g., mean squared error) using the modes and motion vectors considered, and R is the number of bits required to represent the data needed to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).

[0052] Scalable video coding refers to an encoding structure in which a single bitstream may contain multiple representations of content at different bitrates, resolutions, or frame rates. In these cases, a receiver may extract the desired representation depending on its characteristics (e.g., the resolution that best matches the display device). Alternatively, a server or network element may extract a portion of the bitstream sent to the receiver, depending, for example, on network characteristics or the receiver's processing power. A scalable bitstream may consist of a “base layer” that provides the lowest available video quality and one or more enhancement layers that, when received and decoded together with the lower layers, enhance the video quality. To improve the encoding efficiency for the enhancement layers, the encoded representation of those layers may depend on the lower layers. For example, motion and mode information in the enhancement layers may be predicted from the lower layers. Similarly, pixel data from the lower layers may be used to make predictions about the enhancement layers.

[0053] Scalable video codecs for quality scalability (also known as signal-to-noise ratio or SNR) and / or spatial scalability can be implemented as follows: For the base layer, conventional non-scalable video encoders and decoders are used. The reconstructed / decoded picture from the base layer is placed in a reference picture buffer for the extension layer. In H.264 / AVC, HEVC, and similar codecs that use a reference picture list for interprediction, the base layer decoded picture may be inserted into the reference picture list to encode / decode an extension layer picture similar to the decoded reference picture of the extension layer. As a result, the encoder may select the base layer reference picture as the interprediction reference and may indicate its use, for example, by the reference picture index in the encoded bitstream. The decoder decodes from the bitstream where the base layer picture is used as the interprediction reference for the extension layer, for example, by the reference picture index. When the decoded base layer picture is used as the prediction reference for the extension layer, it is called the interlayer reference picture.

[0054] In addition to quality scalability, the following scalability modes exist: ● Spatial scalability: Base layer pictures are encoded at a lower resolution than extended layer pictures. ● Bit depth scalability: Base layer pictures are encoded with a lower bit depth (e.g., 8 bits) than extension layer pictures (e.g., 10 or 12 bits). ● Chroma format scalability: Extended layer pictures offer higher chroma fidelity (e.g., encoded in a 4:4:4 chroma format) than base layer pictures (e.g., 4:2:0 format).

[0055] In all of the above scalability cases, the base layer information can be used to encode the extension layer and minimize additional bitrate overhead.

[0056] Scalability can be enabled in two basic ways: by introducing a new encoding mode to perform prediction of pixel values ​​or syntax from lower layers of a scalable representation, or by placing lower-layer pictures in a reference picture buffer (decoded picture buffer, DPB) of the upper layer. The first method is more flexible and therefore can provide better encoding efficiency in most cases. However, the second, reference frame-based scalability method can be implemented very efficiently with minimal changes to a single-layer codec while still achieving most of the available improvements in coding efficiency. A reference frame-based scalability codec can be implemented by simply handling DPB management by external means and utilizing the same hardware or software embodiment for all layers.

[0057] To enable parallel processing, an image may be divided into independently encodeable and decodeable image segments (slices or tiles). In this description, a “slice” may refer to an image segment consisting of a certain number of basic encoding units processed in a default encoding or decoding order, and a “tile” may refer to an image segment defined as a rectangular area of ​​image that is processed at least to some extent as a separate frame.

[0058] Video may be encoded in the YUV or YCbCr color space because this color space was chosen to reflect some characteristics of the human visual system, and because human perception is less sensitive to the chrominance fidelity represented by the Cb and Cr channels, thus tolerating lower quality representation.

[0059] Some video encoding tools perform filtering operations. Filtering operations involve convolving a set of reference samples with a set of filter parameters to output a predicted value for a given sample in a picture. In some cases, the filter parameters may be predetermined or signaled in the bitstream. In other cases, such as when cross-component linear (CCLM) or cross-component convolutional (CCCM) predictions are used, the parameters are computed using the set of reference samples in both the encoder and decoder. Generally, the computation of such filter parameters involves the inverse of the autocorrelation matrix, which is a computationally difficult operation. Furthermore, when there are many filter parameters to be determined, the size of the autocorrelation matrix becomes large, which can lead to numerical stability problems (overflow or underflow).

[0060] The filter parameters for filters used in video coding are calculated in different ways. For example, CCLM in VVC / H.266 calculates its parameters by identifying two chroma sample values ​​and their corresponding chroma sample values, and then using those two sample pairs to determine the sloe and offset parameters. A different variation of CCLM is used in Enhanced Compression Model 6 (ECM6). In this case, the least squares mean method is used to find the parameters. For CCCM prediction in ECM6, an autocorrelation matrix is ​​generated and decomposed into a triangular matrix using a method similar to LDL decomposition, and then the filter coefficients are solved by a sequence including two backward substitution and scaling steps.

[0061] This embodiment aims to determine a set of filter parameters by generating a mixture matrix by combining an autocorrelation matrix with one or more cross-correlation vectors, and by reducing the generated mixture matrix to a diagonal form using row-direction matrix operations. The process is completed with a single set of back substitution operations for each set of filter parameters being solved. In addition, a preemptive mechanism is disclosed that is applied after one iteration of row-direction matrix operations to avoid division by zero between further iterations and between the back substitution steps.

[0062] In this embodiment, the above set of filter parameters may be used, for example, by a cross-component prediction filter for video and image codecs. It should be understood that the set of parameters may also be used to determine parameters for other types of filters, such as filters used in motion-compensated prediction or filters used to improve sample values ​​on sample block boundaries.

[0063] The determined filter parameters can be represented by a vector x. x = [x 0 · · · x N-1 ] T

[0064] The filter parameters may be convolved with the input vector z, which can include, for example, luminance input samples and can be written as follows: z=[z0··· z N-1 ] T

[0065] The resulting value p can represent, for example, the predicted sample value in one of the chrominance channels and can be calculated using the filter parameter x and the input sample z as follows:

[0066]

number

[0067] The input vector z may be configured to include, for example, luminance sample values, a function of luminance sample values ​​or a constant, or a combination thereof. Including a constant in the input vector z corresponds to adding a constant to the output of the filter p. This type of constant may also be called a bias term or bias parameter and may be used to represent an offset between the input and output values.

[0068] The filter parameter x can generally be computed by finding a solution to a set of equations that can be expressed in matrix form as follows: Ax=y Here, A represents the autocorrelation matrix of the input reference samples or the determined set of "training" samples used in the process, and y represents the cross-correlation vector between the input training samples and the corresponding output training samples. The cross-correlation vector y, which has items in the autocorrelation matrix A (size N × N) and N values, can be calculated, for example, as follows:

[0069]

number

[0070] To solve the equation Ax=y for the filter parameter x, a mixture matrix B containing elements of both the autocorrelation matrix A and the cross-correlation vector y can be constructed as follows:

[0071]

number

[0072] This can be triangularized (also called "diagonalized") using a method similar to Gaussian elimination, which involves multiplying the elements of a particular row and subtracting the resulting values ​​of another row, until the mixture matrix B is reduced to an upper triangular equation B' of the mixture matrix B, where all elements below the main diagonal are zero.

[0073]

number

[0074] By creating a mixture matrix containing multiple cross-correlation vectors, the parameters of two or more filters can be calculated simultaneously. This can be useful when calculating the luma for a chromacross component predictor filter for two chroma sample blocks when both of those chroma sample blocks use the same luma reference sample. This allows the same autocorrelation matrix A to be used in both cases, and thus a single mixture matrix can reduce the row-direction operations required in the triangulation process of the mixture matrix by half. In such a case, the mixture matrix C is the luma-based autocorrelation matrix A and the cross-correlation vectors y for the respective color channels Cb and Cr. cb and y cr It can be configured as follows:

[0075]

number

[0076] After triangulation,

number

[0077] Since the confusion matrix C is reduced to an upper triangular matrix, its submatrix A is reduced to an upper triangular matrix A', and two filters x cb and x cr The filter coefficients for can be solved by back substitution using two triangular systems. A'x cb =y' cb A'x cr =y' cr

[0078] A mixture matrix of dimension N×N+2 (where N rows correspond to the rows of the autocorrelation matrix A, N+2 columns correspond to the N columns of the autocorrelation matrix A, and two additional columns are the two cross-correlation vectors y included in the mixture matrix). cb and y crThe corresponding triangulation may be calculated, for example, using the following exemplary pseudo-code. for(i=0;i<N-1;i++) { for(j=i+1;j<N;j++) { scale=divide(a[j][i],a[i][i]) for(k=i+1;k<N+2;k++) { a[j][k]-=multiply(scale,a[i][k]) } } }

[0079] The division and multiplication functions may be implemented in various ways that contemplate, for example, floating-point, fixed-point, or integer implementations, or a mixture thereof. For example, the multiplication may be implemented using a fixed-point implementation, and the division operation may be implemented using the approximate fixed-point approach proposed in JVET document JVET-AB0174.

[0080] For the sake of brevity, row-oriented pointers src and dst may be used to describe the row-based nature of the process. In the following example, src points to the i-th row of the mixing matrix, and dst points to each of the rows therebelow in sequence. For each destination row j indicated by dst, a scale value may be calculated based on the ratio of the i-th element of the destination row and the source row. By multiplying the source row with the scale and subtracting the result from the destination row j, the i-th element of the j-th row is pushed to zero. To reduce the number of computational operations used in the process, the calculation of that element may be omitted since the result is determined to be zero. Similarly, since all values of k<i have already been determined to be zero in both source row i and destination row j, there is no need to calculate any element of k<i in the j-th destination row. for(i=0;i<N-1;i++) { src=a[i] for(j=i+1;j<N;j++) { dst=a[j] scale=divide(dst[i],src[i]) for(k=i+1;k <N+2;k++) { dst[k]-=multiply(scale,src[k]) } } } In other words, Iterate over the rows (source rows) of the mixture matrix. { Iterate across lines below the source line (destination line). { Calculate the scale between the destination row and the source row. Multiply the source line by the scale. Subtract the multiplied source line from the destination line. } }

[0081] To avoid division by zero and to maintain positive diagonal values ​​in the autocorrelation matrix A after triangularization, different methods may be used. As shown below, the check for the diagonal elements of the next processing row may be performed after iterating over rows below the current source row. for(i=0;i <N-1;i++) { for(j=i+1;j <N;j++) { scale=divide(a[j][i],a[i][i]) for(k=i+1;k <N+2;k++) { a[j][k]-=multiply(scale,a[i][k]) } } a[i+1][i+1]=a[i+1][i+1] <zeroThr?zeroThr:a[i+1][i+1] } In other words, Iterate over the rows (source rows) of the mixture matrix. { Iterate across lines below the source line (destination line). { Calculate the scale between the destination row and the source row. Multiply the source line by the scale. Subtract the multiplied source line from the destination line. } Set the diagonal element of the next source line to a value that is not less than zeroThr. }

[0082] This ensures that the diagonal element a[i+1][i+1] of the next source line i+1 has at least the value of zeroThr. zeroThr may be determined to be, for example, 1, or some other positive number greater than zero. As another example, the absolute value of the diagonal element of the next source line may be used for comparison with the threshold zeroThr.

[0083] In some embodiments, it may be desirable to know during triangulation whether the autocorrelation matrix A is degenerating (or irreversible) and whether there is no solution to the system of equations. If, at any point during triangulation, all rows of the autocorrelation matrix A are zero (or equivalently, each element of the above rows is less than zeroThr) and each element of the cross-correlation vector is non-zero (or greater than zeroThr), then the autocorrelation matrix A can be considered degenerating and the elimination can be terminated early. In such a case, a fixed set of filter coefficients (e.g., all zero) may be output. Similarly, if the diagonal elements of the autocorrelation matrix A are zero or negative, the matrix can be considered degenerating and the elimination can be terminated early.

[0084] After the confusion matrix is ​​reduced to a triangular shape, the filter parameter x cb and x cr Back assignment can be used to solve this. This can be done, for example, as shown by the following pseudocode. x[N-1]=divide(a[N-1][col],a[N-1][N-1]) for(i=N-2;i>=0;i--) { val=a[i][col] for(j=i+1;j <N;j++) { val-=multiply(a[i][j],x[j]) } x[i]=divide(val,a[i][i]) }

[0085] As shown above, all division operations use the diagonal elements a[i][j] of the triangularized mixture matrix as the denominator, and if they are performed during the triangulation step as proposed above, no additional checks are needed for denominators with zero values. The column parameter col determines which column of the triangularized mixture matrix is ​​used as the target vector in the back substitution process. Select the cross-correlation vector y from the example above. cb Let be the Nth column of the mixture matrix, and the cross-correlation vector y cr If we include it as the N+1th column of the mixture matrix, col is x for the filter used for sample prediction of the first color component Cb. cb The coefficient can be set to N when solving, and col is x for the filter used for sample prediction of the second color component Cr. cr The coefficient can be set to N+1 when solving for it.

[0086] During the triangulation process, the dynamic range of the intermediate values ​​of the mixture matrix can be controlled in different ways. For example, the values ​​of a particular row of the mixture matrix may be shifted up or down to avoid overflow that may occur when the scale parameter becomes large due to the difference between the value of source row i and the value of destination row j. An example of such a control mechanism is given in the pseudocode below. In this example, the log2 operation may also include pointing to the base 2 logarithm of the input and truncating the output value to an integer. The difference between the source row scale srcScale and the destination row scale dstScale is used to determine the effective scale value used in the row-wise multiplication. Also, before the multiplication operation, the k-th value of destination row j (a[j][k]) is updated by shifting it down by the difScale bit to compensate for the modified scale value. for(i=0;i <N-1;i++) { for(j=i+1;j <N;j++) { dstScale=log2(a[j][i]<0?-a[j][i]:a[j][i]) srcScale=log2(a[i][i]<0?-a[i][i]:a[i][i]) difScale=dstScale-srcScale>0?dstScale-srcScale:0 difRound=difScale?1<<(difScale-1):0 scale=divide(a[j][i],a[i][i])< <difScale) for(k=i+1;k <N+2;k++) { a[j][k]=(a[j][k]+difRound)>>difScale a[j][k]-=multiply(scale,a[i][k]) } } a[i+1][i+1]=a[i+1][i+1] <zeroThr?zeroThr:a[i+1][i+1] }

[0087] It should be understood that different techniques may also be used for dynamic range control. For example, when calculating the scale parameter, instead of shifting down the source row value a[i][i] to achieve a scale close to 1, the destination row value a[j][i] may be shifted down. Similarly, when subtracting the scaled version of the source row from the destination row, the values of the source row may be shifted up to compensate for the modification of the scale parameter.

[0088] In general, since the mixing matrix represents a system of linear equations, the rows of the matrix may be further multiplied, or columns scaled by a multiplication operation may be added to other rows to enable additional functionality or to control the dynamic range of matrix elements.

[0089] As a further example, the rows of the mixing matrix may be normalized such that the diagonal elements have a value of 1 (or equivalently 1<<DECIM_BITS, or 2^DECIM_BITS, where DECIM_BITS is the number of bits representing the fractional part of a fixed-point number). This may be useful because subsequent steps of the process require division by the diagonal elements. If the diagonal elements are each forced to have a value of 1, the division operation becomes trivial, and the division can be replaced by taking the value of the numerator as the output of the division operation. In such cases, the result of the triangulation operation may be given as follows.

[0090] [Equation]

[0091] This can be calculated, for example, using the following pseudocode. In this case, since all diagonal elements are determined to be 1, there is no need to calculate the diagonal elements, and operations that require diagonal element values, such as division in the back assignment process, can ignore the diagonal elements and use the value 1 instead. First, the diagonal values ​​are determined in the i-th row of the mixture matrix. Its values ​​may be truncated to a specific range to avoid division by zero. For example, it may be determined to have a minimum value of 1 or 10 when fixed-point arithmetic is used, or it may be determined to have different minimum values ​​such as 0.001 or 0.0001 when floating-point arithmetic is used. Following this, the remaining set of values ​​in that row are scaled by the determined diagVal, but the results of the values ​​of elements at index k less than i+1 are known without having to calculate them (elements at index k less than i are pushed to zero, and elements at index k equal to i are pushed to have a value of 1 because they are divided by themselves when arithmetic operations are performed). for(i=0;i <N;i++) { diagVal=a[i][i]<1?1:a[i][i] for(k=i+1;k <N+2;k++) { a[i][k]=divide(a[i][k],diagVal) } for(j=i+1;j <N;j++) { scale=a[j][i] for(k=i+1;k <N+2;k++) { a[j][k]-=multiply(scale,a[i][k]) } } } In other words, Iterate over the rows (source rows) of the mixture matrix. { Scale source rows using the diagonal elements of the row. Iterate across lines below the source line (destination line). { Determine the scale value to be equal to the reading value of the destination line. Multiply the source line by the scale. Subtract the multiplied source line from the destination line. } }

[0092] Division operations are often difficult to perform in practice, and different approximations may be used to speed up the operation. For example, JVET contribution JVET-AB0174 teaches how division operations can be approximated by determining the scaling parameter (scale), rounding parameter (round), and bit shift parameter (shift). In such cases, the division of the numerator (num) and denominator (denom) can be approximated as follows: result=(num*scale+round)>>shift Here, the scale, round, and shift parameters depend on the denominator denom and may be determined by a function that generally estimates these parameters with desired precision using a desired method (as proposed, for example, in JVET-AB0174), which can here be given as the output of the function getDivParams. scale,round,shift=getDivParams(denom)

[0093] Using that notation, if we assume that all elements in row i of the mixture matrix are divided by the same value diagVal, it may suffice to compute the parameters necessary to approximate the division operation only once per row. The actual scaling operation can, fortunately, be performed using only multiplication, addition, and bit shift operations, as shown in the following exemplary pseudocode. for(i=0;i <N;i++) { diagVal=a[i][i]<1?1:a[i][i] scale,round,shift=getDivParams(diagVal) for(k=i+1;k <N+2;k++) { a[i][k]=(a[i][k]*scale+round)>>shift } for(j=i+1;j <N;j++) { scale=a[j][i] for(k=i+1;k <N+2;k++) { a[j][k]-=multiply(scale,a[i][k]) } } }

[0094] Since the above processing for a mixture matrix ensures that the diagonal elements of the matrix have a value of 1, the back substitution process can also be simplified to take this restriction into account. More specifically, division by the diagonal element a[i][i] can, advantageously, be replaced by a simple substitution operation. For example, the filter parameter x[N-1] can be directly set to equal the corresponding a[N-1][col] in the mixture matrix. x[N-1]=a[N-1][col] for(i=N-2;i>=0;i--) { val=a[i][col] for(j=i+1;j <N;j++) { val-=multiply(a[i][j],x[j]) } x[i]=val } Alternatively, without using the intermediate parameter val, x[N-1]=a[N-1][col] for(i=N-2;i>=0;i--) { x[i]=a[i][col] for(j=i+1;j <N;j++) { x[i]-=multiply(a[i][j],x[j]) } }

[0095] As an alternative to the back substitution process, the parameter vector x can also be computed directly from the confusion matrix, even if the upper triangular elements of the matrix are pushed to zero by further row-based scaling and subtraction operations. This can be done, for example, using the following pseudocode: for(i=N-2;i>=0;i--) { for(j=i;j>=0;j--) { scale=a[j][i+1] for(k=N;k <N+2;k++) { a[j][k]-=multiply(scale,a[i+1][k]) } } }

[0096] The resulting mixture matrix, in this example, is a diagonal form with only non-zero elements on the diagonal and an extended column of N or more (columns N and N+1 in the case of two filters), and since all diagonal elements have a value of 1, the solution for the filter coefficient vector is trivial, and the resulting filter parameters can be read directly from the extended column. For example, in the case where two filters are calculated, one for the first color component Cb and one for the second color component Cr, the filter coefficients can be assigned as follows: x cb =y' cb x cr =y' cr

[0097] As shown in the pseudocode below, the filter parameters can be read from the extended columns N and N+1. This represents a simple back substitution, where the filter parameters are directly set to equal the corresponding elements in the confusion matrix. for(i=0;i <N;i++) { xCb[i]=a[i][N] xCr[i]=a[i][N+1] }

[0098] It should be understood that the mixed matrix format is given here as an example to clarify the operations used to calculate the filter coefficients in the proposed method. In fact, mixed matrices can be implemented in different ways. For example, instead of placing the data in a matrix format, the elements may be maintained in a row or column vector format, or the elements may be maintained as separate items. Also, some elements may be stored in matrix format, and some elements may be stored in vector format or as separate elements. For example, the autocorrelation matrix A and its triangulation result A' may be stored in matrix format, and the cross-correlation vector y after triangulation of the autocorrelation matrix A may be stored. cb and y cr and their new forms y' cb and y' cr This may be stored in vector format.

[0099] It should also be understood that, in practice, triangulation of an autocorrelation matrix A using row-based scaling operations can be performed using additional row reordering (partial pivoting). Changing the row order does not alter the results of the system of equations. However, such reordering may be beneficial for the numerical stability of the practical embodiment.

[0100] Some of the examples show how triangularization sets the elements of a matrix below the main diagonal to zero, but of course, the operation can also be applied to the reversed row order to produce a triangular matrix with zeros above the main diagonal.

[0101] In the embodiment, the set of cross-component predictive filter coefficients is computed at least partially by using a set of row-direction scaling operations to reduce the matrix to its triangular form.

[0102] In one embodiment, the set of filter coefficients is determined using matrix triangulation, which includes calculating a scaling parameter between an element of a determined row of the matrix and an element of another row of the matrix, multiplying at least one element of the determined row of the matrix by the scaling parameter, and subtracting the result from at least one element of the other row of the matrix.

[0103] In this embodiment, at least two sets of cross-component predictive filter coefficients are calculated by combining an autocorrelation matrix with two or more cross-correlation vectors and then reducing the resulting mixture matrix to a triangular form using row-based scaling operations.

[0104] In some embodiments, if the autocorrelation matrix is ​​considered to be degenerate (i.e., irreversible), triangulation can be terminated early. In such cases, a fixed set of filter coefficients (e.g., each coefficient is zero) is produced.

[0105] A method according to an embodiment is shown in Figure 3. The method generally includes: processing an input video / image 310; determining a set of filter coefficients 320, determining a mixture matrix 330 which includes an autocorrelation matrix and at least one cross-correlation vector; modifying the determined mixture matrix into a triangular formula by determining at least one scale parameter between source rows and destination rows in the mixture matrix 340; determining the modified destination row in the mixture matrix by multiplying the source row by the scale parameter and subtracting the scaled source row from the destination row 350; determining a set of filter parameters from the triangular formula of the mixture matrix 360; and using the set of filter coefficients in a filter 370.

[0106] The apparatus according to the embodiment includes means for processing an input video / image; means for determining a set of filter coefficients, the set of filter coefficients including means for determining a mixture matrix comprising an autocorrelation matrix and at least one cross-correlation vector; means for modifying the determined mixture matrix into a triangular expression by determining at least one scale parameter between source rows and destination rows in the mixture matrix; means for determining the modified destination row in the mixture matrix by multiplying the source row by the scale parameter and subtracting the scaled source row from the destination row; means for determining a set of filter parameters from the triangular expression of the mixture matrix; and means for using the set of filter coefficients in a filter.

[0107] The means includes at least one processor and memory containing computer program code, the processor may further include processor circuitry. The memory and computer program code are configured to cause the device to perform the method of Figure 3 according to various embodiments using at least one processor.

[0108] An example of a data processing system for a device is shown in Figure 4. Several functions may be performed on a single physical device. For example, all calculation procedures may be performed on a single processor as needed. The data processing system includes a main processing unit 100, memory 102, storage device 104, input device 106, output device 108, and graphics subsystem 110, all of which are connected to each other via a data bus 112.

[0109] The main processing unit 100 is a conventional processing unit configured to process data within a data processing system. The main processing unit 100 may include one or more processors or processor circuits, or may be implemented as one or more processors or processor circuits. The memory 102, storage device 104, input device 106, and output device 108 may include conventional components as recognized by those skilled in the art. The memory 102 and storage device 104 store data in the data processing system 100.

[0110] Computer program code resides in memory 102 to implement, for example, the method shown in the flowchart of Figure 3 according to various embodiments. Input device 106 inputs data into the system, and output device 108 receives data from the data processing system and transfers the data, for example, to a display. Data bus 112 is a conventional data bus and is shown as a single line, but may be any combination of processor bus, PCI bus, graphical bus, or ISA bus. Thus, those skilled in the art will readily recognize that the device may be any data processing device such as a computer device, personal computer, server computer, mobile phone, smartphone, or internet access device, such as an internet tablet computer.

[0111] Various embodiments can be implemented with the help of computer program code that resides in memory and causes the associated device to perform the method. For example, a device may include circuits and electronics for handling, receiving, and transmitting data, computer program code in memory, and a processor that, when the computer program code is executed, causes the device to perform features of the embodiment. Furthermore, a network device such as a server may include circuits and electronics for handling, receiving, and transmitting data, computer program code in memory, and a processor that, when the computer program code is executed, causes the network device to perform features of various embodiments.

[0112] If necessary, the different functions described herein may be executed in different orders and / or simultaneously with each other. Furthermore, if necessary, one or more of the functions and embodiments described above may be arbitrary or in combination.

[0113] Various aspects of the embodiments are presented in the independent claims, but other aspects include other combinations of features from dependent claims with features of the described embodiments and / or independent claims, and are not limited to the combinations expressly presented in the claims.

[0114] While the above describes exemplary embodiments, it should be noted herein that these descriptions should not be interpreted as restrictive. Rather, several variations and modifications may be made without departing from the scope of this disclosure as defined in the appended claims.

Claims

1. It is a device, - Means for processing input video / images, - A means for determining the set of filter coefficients by the aforementioned device, ○ Means for determining a mixture matrix including an autocorrelation matrix and at least one cross-correlation vector, ○ Means for modifying the determined mixing matrix into a triangular form by determining at least one scale parameter between the source row and the destination row in the mixing matrix, ○ Means for determining the modified destination row in the blend matrix by multiplying the source row by the scale parameter and subtracting the scaled source row from the destination row, ○ Means for determining the set of filter coefficients, including means for determining the set of filter parameters from the triangular formula of the mixture matrix, - Means for using the set of filter coefficients in the filter, A device equipped with the following features.

2. The apparatus according to claim 1, further comprising means for modifying the mixture matrix by scaling at least one value of a determined row using a scaling parameter, a rounding parameter, and a bit shift parameter, wherein the scaling parameter, the rounding parameter, and the bit shift parameter are determined from the diagonal elements of the mixture matrix of the determined row.

3. The apparatus according to claim 1 or 2, wherein the set of filter coefficients is used in a convolutional cross-component prediction, a chroma sample is used as input, and a predicted chroma sample is used as output.

4. The apparatus according to claim 3, further comprising means for determining the mixing matrix to include two cross-correlation vectors, wherein the first cross-correlation vector relates to the cross-correlation between the luma sample and the first chroma sample, and the second cross-correlation vector relates to the cross-correlation between the luma sample and the second chroma sample.

5. The apparatus according to claim 1 or 2, further comprising means for determining the set of filter parameters from the triangular formula of the mixture matrix using a back substitution process.

6. The apparatus according to claim 5, wherein the back substitution process includes at least one of the filter parameters set to be equal to the elements in the mixture matrix.

7. The apparatus according to claim 1 or 2, wherein the means for modifying the determined mixture matrix into the triangular formula comprises, for each diagonal sample, means for checking whether the value of the diagonal sample or the absolute value of the diagonal sample is less than a predetermined threshold, and means for setting the value of the diagonal sample to be equal to the predetermined threshold if it is less than the predetermined threshold.

8. The apparatus according to claim 1 or 2, wherein the means for modifying the determined mixture matrix into the triangular formula comprises, for at least one diagonal sample, means for checking whether the value of the diagonal sample or the absolute value of the diagonal sample is less than a predetermined threshold, and, if it is less than the predetermined threshold, means for terminating the modification process and means for setting the filter coefficients to a predetermined set of values.

9. The apparatus according to claim 1 or 2, wherein the processing is encoding or decoding.

10. It is a method, - Processing input video / images, - This involves determining the set of filter coefficients, ○ Determine the autocorrelation matrix and the mixture matrix which includes at least one cross-correlation vector. ○ Modify the determined mixing matrix into a triangular form by determining at least one scale parameter between the source row and the destination row in the mixing matrix. ○ The modified destination row in the blend matrix is ​​determined by multiplying the source row by the scale parameter and subtracting the scaled source row from the destination row. ○ Determine the set of filter parameters from the triangular equation of the mixture matrix. Determining the set of filter coefficients, including, - Using the set of filter coefficients in the filter, Methods that include...

11. The method according to claim 10, further comprising modifying the mixture matrix by scaling at least one value of a determined row using a scaling parameter, a rounding parameter, and a bit shift parameter, wherein the scaling parameter, the rounding parameter, and the bit shift parameter are determined from the diagonal elements of the mixture matrix of the determined row.

12. The method according to claim 10 or 11, wherein the set of filter coefficients is used in a convolutional cross-component prediction, a chroma sample is used as input, and a predicted chroma sample is used as output.

13. The method according to claim 12, further comprising determining the mixing matrix to include two cross-correlation vectors, the first cross-correlation vector relating to the cross-correlation between the luma sample and the first chroma sample, and the second cross-correlation vector relating to the cross-correlation between the luma sample and the second chroma sample.

14. The method according to claim 10 or 11, further comprising determining the set of filter parameters from the triangulation of the mixture matrix using a back substitution process.

15. The method according to claim 14, wherein the back substitution process includes at least one of the filter parameters set to be equal to the elements in the mixture matrix.

Citation Information

Patent Citations

  • Multi-input adaptive filter based on a combination of sum-modified Laplacian filter indexing and quadtree partitioning.

    JP2013539287A

  • Image encoding / decoding method and apparatus using in-loop filtering

    JP2021505074A

  • Filter coefficient derivation simplification for cross-component prediction

    WO2024081011A1