Video decoding method, apparatus and storage medium

Abstract

A filter having a center-symmetric shape is applied to the current block. The filter includes a plurality of coefficients. Each of the plurality of coefficients is associated with a respective first coordinate value along a first axis and a respective second coordinate value along a second axis. The plurality of coefficients of the filter is grouped into a plurality of groups based on a transpose operation to be performed on the filter. The coefficients in each of the plurality of groups are the same distance from a center position of the filter. The coefficients of a first group of the plurality of groups are reordered based on the transpose operation. Filtered samples of the current block are reconstructed by applying the filter having the center-symmetric shape to the current block based on the reordered coefficients of the filter.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 983921, filed November 9, 2022, entitled “ARRANGEMENT OF ADAPTIVE LOOPFILTER COEFFICIENTS FOR FAST VECTORIZED TRANSPOSITIONS”, which in turn claims the benefit of priority to U.S. Provisional Application No. 63 / 286530, filed December 6, 2021, entitled “Arrangement of Adaptive Loop Filter Coefficients for Fast Vectorized Transpositions”, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This disclosure describes embodiments that are generally related to video encoding and decoding. Background Technology

[0004] The background description provided herein is for the purpose of presenting the general content of this disclosure. The extent of the work of the currently named inventors described in the background section and various aspects of this specification does not indicate that it was prior art at the time of filing of this application, nor is it expressly or implied that it was acknowledged as prior art to this disclosure.

[0005] Uncompressed digital video can comprise a series of images, each with a spatial size of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of images can have a fixed or variable image rate (alternatively called the frame rate), for example, 60 images per second or 60 frames per second. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (at a frame rate of 60Hz, with a luminance sample resolution of 1920×1080) requires approximately 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 GB of storage space.

[0006] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be employed. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. In the case of video, lossy compression is widely used. The tolerable amount of distortion depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio reflects this: higher allowable / tolerable distortion results in a higher compression ratio.

[0007] Video encoders and decoders can utilize techniques from a wide range of categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques may include techniques called intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the picture can be an intra-frame picture. Intra-frame pictures and their derivatives (e.g., independent decoder refresh pictures) can be used to reset the decoder state and thus can be used as the first picture in an encoded video stream and video session, or as a still image. Samples of an intra-frame block can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0009] For example, traditional intra-frame coding techniques known from MPEG-2 generation coding technologies do not use intra-frame prediction. However, some newer video compression techniques include attempts based on, for example, surrounding sample data and / or metadata obtained during spatially adjacent encoding and / or decoding, preceding data blocks in the decoding order. Such techniques are referred to below as "intra-frame prediction." It should be noted that, at least in some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and not reference data from a reference picture.

[0010] Intra-frame prediction can take many different forms. When more than one such technique is available in a given video coding technique, the technique in use can be coded in an intra-frame prediction mode. In some cases, a mode may have sub-modes and / or parameters, and may be encoded individually or contained within a mode codeword. Which codeword is used for a given combination of mode, sub-mode, and / or parameters can affect the coding efficiency gain through intra-frame prediction, and therefore may affect the entropy coding technique used to convert the codeword into a bitstream.

[0011] H.264 introduced an intra-frame prediction mode, which was refined in H.265 and further improved in updated coding techniques such as the Joint Exploration Model (JEM), Next-Generation Video Coding (VVC), and Baseline Spectrum (BMS). Prediction blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the prediction block according to a certain direction. The reference to the direction used can be encoded in the bitstream or predicted itself.

[0012] refer to Figure 1A The lower right corner depicts a subset of nine prediction directions known from the 33 possible prediction directions of H.265 (corresponding to 33 angular modes of 35 intra-frame modes). The point (101) where the arrows converge represents the sample being predicted. The arrow indicates the direction along which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples in the upper right corner at a 45-degree angle to the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples in the lower left corner of sample (101) at a 22.5-degree angle to the horizontal.

[0013] Still referencing Figure 1A A square block (104) of 4×4 samples is depicted in the upper left (indicated by a bold dashed line). The square block (104) comprises 16 samples, each labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is located in the lower right corner. Reference samples following a similar numbering scheme are also shown. Reference samples are labeled with R, their Y position (e.g., row index) relative to the block (104), and their X position (column index). In H.264 and H.265, predicted samples adjacent to the block being reconstructed are used; negative values ​​are not required.

[0014] Intra-frame image prediction can begin by copying reference sample values ​​from neighboring samples based on a prediction direction indicated by a signal. For example, suppose the encoded video stream includes signaling that indicates a prediction direction consistent with arrow (102) for the block, i.e., predicting samples from one or more prediction samples located at the upper right, at a 45-degree angle to the horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples can be combined, for example, by interpolation, to calculate a reference sample; especially when the directions cannot be evenly separated at 45 degrees.

[0016] As video coding techniques continue to evolve, the number of possible directions increases. In H.264 (2003), nine different directions could be represented. In H.265 (2013), this increased to 33 directions, and at the time of this disclosure, JEM / VVC / BMS supports up to 65 directions. Experiments have been conducted to identify the most likely directions, and some techniques in entropy coding are used to represent those possible directions with a small number of bits, accepting a certain cost for less likely directions. Furthermore, the direction itself can sometimes be predicted from neighboring directions used in already decoded neighboring blocks.

[0017] Figure 1B A schematic diagram (110) is shown, depicting 65 intra-frame predicted directions according to JEM, to illustrate that the number of predicted directions increases over time.

[0018] The mapping of intra-predicted direction bits representing direction in an encoded video bitstream can vary depending on the video coding technique; for example, it can range from a simple direct mapping of the predicted direction to the intra-predicted mode, to a codeword, to a complex adaptive scheme involving the most probable mode, and similar techniques. However, in all cases, there may be some directions that are statistically less likely to occur in the video content compared to others. Since the goal of video compression is to reduce redundancy, in a well-designed video coding technique, less likely directions will be represented by more bits than more likely directions.

[0019] Video encoding and decoding can be performed using inter-frame image prediction with motion compensation. Motion compensation can be a lossy compression technique and can involve using sample data blocks from a previously reconstructed image or a portion thereof (the reference image) spatially offset along a direction indicated by a motion vector (hereinafter referred to as MV) to predict the newly reconstructed image or image portion. In some cases, the reference image may be the same as the image currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, with the third dimension indicating the reference image being used (indirectly, the third dimension may be a temporal dimension).

[0020] In some video compression techniques, the MV applicable to a region of the sample data can be predicted based on other MVs, such as those related to another region of sample data that is spatially adjacent to the region being reconstructed and precedes the MV in the decoding order. This significantly reduces the amount of data required to encode the MV, thereby eliminating redundancy and increasing compression. MV prediction works effectively, for example, because when encoding the input video signal (called natural video) obtained from the camera, there is a statistical probability that a larger region than the region applicable to a single MV moves along similar directions. Therefore, in some cases, the larger region can be predicted using similar motion vectors derived from the MVs of adjacent regions. This makes the MV considered for a given region similar or identical to the MV predicted based on the surrounding MVs, and thus, after entropy coding, the MV can be represented with fewer bits than when directly encoding the MV. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating predictions based on multiple surrounding MVs.

[0021] H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this paper describes the technique referred to below as “spatial combining”.

[0022] Referring to Figure 2, the current block (201) includes samples found by the encoder during the motion search process, which can be predicted based on previous blocks of the same size that have generated spatial offsets. Instead of directly encoding the MV, it can be derived from metadata associated with one or more reference images, for example, using the MV associated with any of five surrounding samples labeled A0, A1 and B0, B1, B2 (corresponding to 202 to 206 respectively), derived from the nearest reference image (in decoding order). In H.265, MV prediction can use predictions from the same reference image being used by adjacent blocks. Summary of the Invention

[0023] This disclosure provides methods and apparatus for video encoding or decoding. In some examples, the apparatus for video decoding includes processing circuitry.

[0024] According to one aspect of this disclosure, a method for performing video decoding in a video decoder is provided. In this method, prediction information for a current block in a current image can be received from an encoded video bitstream. The prediction information can indicate the application of a filter with a centrally symmetric shape to the current block. The filter can include multiple coefficients. Each of the multiple coefficients can be associated with a corresponding first coordinate value along a first axis and a corresponding second coordinate value along a second axis. Based on a transpose operation to be performed on the filter, the multiple coefficients of the filter can be grouped into multiple groups, where the coefficients in each group are equidistant from the center of the filter. Based on the transpose operation, the coefficients of the first group of the multiple groups can be reordered. Filtered samples of the current block can be reconstructed by applying a filter with a centrally symmetric shape to the current block based on the reordered coefficients of the filter.

[0025] In this method, the reordered coefficients of the first group and the coefficients of the second group in multiple groups can be swapped. The reordered coefficients of the first group and the coefficients of the second group are located at a first distance from the center of the filter.

[0026] In some embodiments, the transpose operation can be one of a rotation operation, a vertical flip operation, or a diagonal flip operation.

[0027] In some embodiments, the number of coefficients in the first group can be the same as the number of coefficients in the second group.

[0028] The first distance can be equal to (i) the absolute value of the first coordinate value of the first coefficient of the first group and (ii) the absolute value of the second coordinate value of the first coefficient of the first group.

[0029] The sum of the first coordinate value of the first coefficient in the first group and the second coordinate value of the first coefficient in the second group can be equal to 0. The second coordinate value of the first coefficient in the first group can be equal to the first coordinate value of the first coefficient in the second group.

[0030] In some embodiments, the coefficients of each of the plurality of groups can be stored in a corresponding storage space. The corresponding storage space can have one of (i) a first storage capacity equal to or less than 128 bits and (ii) a second storage capacity equal to or less than 256 bits.

[0031] In some embodiments, the coefficients of one or more of the plurality of groups are stored in a storage space having one of (i) a first storage capacity of 128 bits and (ii) a second storage capacity of 256 bits.

[0032] In this method, each of the multiple groups can have a corresponding group located within the multiple groups. The coefficients of the corresponding group within the multiple groups and the coefficients of the corresponding group located within the multiple groups can be at the same distance from the center of the filter. The corresponding group within the multiple groups and the corresponding group located within the multiple groups include the same number of coefficients.

[0033] In this method, multiple coefficients of the filter can be determined based on prediction information in the order in which signaling notifications are sent within the prediction information. The determined multiple coefficients of the filter can be grouped into multiple groups.

[0034] According to another aspect of this disclosure, an apparatus is provided. The apparatus includes processing circuitry. The processing circuitry can be configured to perform any method for video encoding / decoding.

[0035] This disclosure also provides a non-transitory computer-readable storage medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform any of the methods for video encoding / decoding. Attached Figure Description

[0036] Other features, properties, and various advantages of the disclosed subject will become more apparent from the following detailed description and accompanying drawings, in which:

[0037] Figure 1A This is a schematic diagram of an exemplary subset of intra-frame prediction modes.

[0038] Figure 1B This is an illustration of an exemplary intra-frame prediction direction.

[0039] Figure 2 is a schematic diagram of the current block and its surrounding space as candidates for merging in an example.

[0040] Figure 3 This is a simplified block diagram of a communication system (300) according to one embodiment.

[0041] Figure 4 This is a simplified block diagram of a communication system (400) according to one embodiment.

[0042] Figure 5 This is a simplified block diagram of a decoder according to one embodiment.

[0043] Figure 6 This is a simplified block diagram of an encoder according to one embodiment.

[0044] Figure 7 A block diagram of an encoder according to another embodiment is shown.

[0045] Figure 8 A block diagram of a decoder according to another embodiment is shown.

[0046] Figure 9 This is a schematic diagram of an exemplary adaptive loop filter (ALF) according to some embodiments.

[0047] Figure 10A This is a schematic diagram of the first exemplary transpose of the coefficients of ALF according to some embodiments.

[0048] Figure 10B This is a schematic diagram of a second exemplary transpose of the coefficients of ALF according to some embodiments.

[0049] Figure 10C This is a schematic diagram of a third exemplary transpose of the coefficients of ALF according to some embodiments.

[0050] Figure 11 This is a schematic diagram of grouping coefficients according to some embodiments of ALF.

[0051] Figure 12A This is a schematic diagram of the first exemplary transpose of the grouping coefficients of ALF according to some embodiments.

[0052] Figure 12B This is a schematic diagram of a second exemplary transpose of the grouping coefficients of an ALF according to some embodiments.

[0053] Figure 12C This is a schematic diagram of a third exemplary transpose of the grouping coefficients of an ALF according to some embodiments.

[0054] Figure 13 This is a schematic diagram of the coordinates of the grouping coefficients of ALF according to some embodiments.

[0055] Figure 14 This is a schematic diagram of the coordinates of the grouping coefficients of ALF according to some embodiments.

[0056] Figure 15 This is a schematic diagram of the coordinates of the grouping coefficients of ALF according to some embodiments.

[0057] Figure 16 A flowchart outlining exemplary decoding processes according to some embodiments of this disclosure is shown.

[0058] Figure 17 A flowchart outlining exemplary encoding processes according to some embodiments of this disclosure is shown.

[0059] Figure 18 This is a schematic diagram of a computer system according to one embodiment. Detailed Implementation

[0060] Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3 In the example, the first terminal device performs unidirectional data transmission between (310) and (320). For example, terminal device (310) may encode video data (e.g., a video image stream captured by terminal device (310)) for transmission over a network (350) to another terminal device (320). The encoded video data may be transmitted in the form of one or more encoded video streams. Terminal device (320) may receive the encoded video data from the network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0061] In another example, the communication system (300) includes a pair of second terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, in one example, each of the terminal devices (330) and (340) may encode video data (e.g., a video picture stream captured by the terminal device) for transmission over a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) may also receive encoded video data transmitted by the other terminal device (330) and (340), and may decode the encoded video data to recover video pictures, and may display the video pictures on an accessible display device based on the recovered video data.

[0062] exist Figure 3In the examples, terminal devices (310), (320), (330), and (340) may be shown as servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure are found in laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) refers to any number of networks that transmit encoded video data between terminal devices (310), (320), (330), and (340), including, for example, wired (connected) and / or wireless communication networks. Communication networks (350) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, the architecture and topology of the network (350) may be irrelevant to the operation of this disclosure unless explained below.

[0063] As an example of the application to the disclosed topic, Figure 4 The placement of a video encoder and video decoder in a streaming environment is illustrated. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0064] The streaming system may include an acquisition subsystem (413) that may include a video source (401) such as a digital camera, which creates, for example, an uncompressed video image stream (402). In one example, the video image stream (402) includes samples taken by a digital camera. The video image stream (402), depicted as a thick line to emphasize its high data volume, may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401), compared to the encoded video data (404) (or encoded video stream). The video encoder (403) may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data (404) (or encoded video stream), depicted as a thin line to emphasize its lower data volume, may be stored on a streaming server (405) for future use, compared to the video image stream (402). One or more streaming client subsystems, such as Figure 4Client subsystems (406) and (408) can access a streaming server (405) to retrieve copies (407) and (409) of encoded video data (404). Client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and produces an output video picture stream (411) that can be displayed on a monitor (412) (e.g., a display screen) or other presentation device (not depicted). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video stream) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Next Generation Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0065] It should be noted that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0066] Figure 5 A block diagram of a video decoder (510) according to one embodiment of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may be used in place of Figure 4 The example video decoder (410).

[0067] The receiver (531) may receive one or more encoded video sequences to be decoded by the video decoder (510); in the same embodiment or another embodiment, one encoded video sequence may be decoded at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. Encoded video sequences may be received from a channel (501), which may be a hardware / software link leading to a storage device storing the encoded video data. The receiver (531) may receive encoded video data and other data, such as encoded audio data and / or auxiliary data streams, that may be forwarded to their respective user entities (not depicted). The receiver (531) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other applications, the buffer memory (515) may be located external to the video decoder (510) (not depicted). In still other applications, a buffer memory (not depicted) may be located external to the video decoder (510) to, for example, prevent network jitter, and another additional buffer memory (515) may be located internally to, for example, handle broadcast timing. When the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer memory (515) may not be necessary, or it may be made smaller. For use on packet-switched networks such as the Internet, a buffer memory (515) may be required, which may be relatively large, advantageously having an adaptive size, and may be implemented at least partially in an operating system or a similar element (not depicted) external to the video decoder (510).

[0068] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (510), and potential information for controlling a presentation device such as a presentation device (512) (e.g., a display screen), which is not integral to the electronic device (530) but may be coupled to it. Figure 5As shown. The control information used for the presentation device may be in the form of Supplemental Enhancement Information (SEI) messages or Video Availability Information (VUI) parameter set fragments (not depicted). The parser (520) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroups of the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. Subgroups may include picture groups (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (520) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0069] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0070] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (521) may involve multiple different units. Which units are involved and how they are involved can be controlled by the parser (520) through subgroup control information parsed from the encoded video sequence. For clarity, the flow of such subgroup control information between the parser (520) and the multiple units described below is not depicted.

[0071] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into multiple functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into multiple functional units as described below.

[0072] The first unit is the scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives quantization transform coefficients as symbols (521) from the parser (520) and control information, including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output a block containing sample values, which can be input into the aggregator (555).

[0073] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to intra-coded blocks; that is, blocks that do not use prediction information from previously reconstructed images, but can use prediction information from previously reconstructed portions of the current image. Such prediction information may be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses surrounding reconstructed information extracted from the current image buffer (558) to generate blocks of the same size and shape as the blocks being reconstructed. For example, the current image buffer (558) buffers partially reconstructed current images and / or fully reconstructed current images. In some cases, the aggregator (555) adds the prediction information generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) based on each sample.

[0074] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to blocks of inter-frame coding and potential motion compensation. In this case, the motion compensation prediction unit (553) may access the reference image memory (557) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (521) belonging to the block, these samples may be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (in this case, referred to as residual samples or residual signals) to generate output sample information. The extraction of prediction samples by the motion compensation prediction unit (553) from the address in the reference image memory (557) may be controlled by motion vectors, which may be provided to the motion compensation prediction unit (553) in the form of symbols (521), which may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (557) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0075] The output samples of the aggregator (555) can be subjected to various loop filtering techniques in the loop filter unit (556). The video compression technique may include an in-loop filter technique controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream) and used as symbols (521) from the parser (520) for the loop filter unit (556). However, the video compression technique may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0076] The output of the loop filter unit (556) can be a sample stream that can be output to the presentation device (512) and stored in the reference image memory (557) for future inter-frame image prediction.

[0077] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded images corresponding to the current image have been fully reconstructed and the encoded images (by, for example, the parser (520)) are identified as reference images, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0078] The video decoder (510) can perform decoding operations according to a predetermined video compression technique as specified in a standard such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under that configuration file. For compliance, it may also be necessary that the complexity of the encoded video sequence be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy may be further limited by the hypothetical reference decoder (HRD) specification and metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0079] In one embodiment, the receiver (531) may receive additional (redundant) data when receiving encoded video. This additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0080] Figure 6 A block diagram of a video encoder (603) according to one embodiment of the present disclosure is shown. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., transmission circuitry). The video encoder (603) can be used in place of... Figure 4 The example video encoder (403).

[0081] The video encoder (603) can obtain data from the video source (601) (not...). Figure 6In one example, an electronic device (620) receives video samples, and the video source (601) can capture video images that will be encoded by a video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0082] A video source (601) can provide a sequence of source video samples to be encoded by a video encoder (603) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCB, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (601) can be a storage device storing previously prepared video. In a video conferencing system, the video source (601) can be a camera that captures local image information as a video sequence. The video data can be provided as multiple individual pictures, which are given motion when viewed in sequence. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. The relationship between pixels and samples will be readily understood by those skilled in the art. The following focuses on describing samples.

[0083] According to one embodiment, the video encoder (603) can encode and compress images of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (650). In some embodiments, the controller (650) controls and is functionally coupled to other functional units described below. For clarity, the coupling is not depicted in the figures. Parameters set by the controller (650) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of images (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other suitable functions related to the video encoder (603) optimized for a particular system design.

[0084] In some embodiments, the video encoder (603) is configured to operate within an encoding loop. As an oversimplification, in one example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder can also create sample data (since any compression between the symbols and the encoded video stream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference image memory (634). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory (634) are also bit-precisely corresponding between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0085] The operation of the “local” decoder (633) can be combined with, for example, those already mentioned above. Figure 5 The video decoder (510) described in detail is the same as the "remote" decoder. However, a further brief reference is provided. Figure 5 Since symbols are available and the entropy encoder (645) and parser (520) are able to encode / decode symbols into encoded video sequences without loss, the entropy decoding portion of the video decoder (510), which includes the buffer (515) and parser (520), may not be fully implemented in the local decoder (633).

[0086] The observation that can be made at this point is that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. Therefore, the subject matter presented focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are inverses of the fully described decoder techniques. Only in certain areas is a more detailed description necessary and provided below.

[0087] During operation, in some examples, the source encoder (630) may perform motion-compensated predictive coding, which predictively encodes the input image by referencing one or more previously encoded images from the video sequence designated as "reference images." In this way, the encoding engine (632) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0088] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When the encoded video data can be decoded by the video decoder (630)... Figure 6 When decoded in a (not shown) source video sequence, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0089] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search in the reference image memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. The predictor (635) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0090] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0091] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (645). The entropy encoder (645) performs lossless compression on the symbols generated by the various functional units using techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into an encoded video sequence.

[0092] The transmitter (640) can buffer the encoded video sequence created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device capable of storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0093] The controller (650) manages the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0094] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (“IDR” pictures. Those skilled in the art will understand these variations of I-pictures and their corresponding applications and characteristics.

[0095] A predictive picture (P-picture) can be a picture that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0096] Bidirectional predictive images (B-images) can be images that can be encoded and decoded using intra-frame or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can be used to reconstruct a single block using more than two reference images and associated metadata.

[0097] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignment of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or blocks of an I-image can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0098] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard such as ITU-T H.265 Recommendation. In operation, the video encoder (603) can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0099] In one embodiment, the transmitter (640) may transmit additional data while transmitting encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0100] The captured video can be presented as multiple source images (video images) in a time-series format. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, a specific image being encoded / decoded is divided into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference image.

[0101] In some embodiments, bidirectional prediction techniques can be used for inter-frame image prediction. According to this bidirectional prediction technique, two reference images are used, such as a first reference image and a second reference image that precede the current image in the video in decoding order (but may be past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0102] In addition, merging mode techniques can be used for inter-frame image prediction to improve coding efficiency.

[0103] According to some embodiments of this disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a video picture sequence are divided into Coding Tree Units (CTUs) for compression, with CTUs in the pictures having the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Typically, a CTU comprises three Coding Tree Blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively partitioned into one or more Coding Units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be partitioned into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In one example, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Based on temporal and / or spatial predictability, the CU is divided into one or more Prediction Units (PUs). Typically, each PU comprises a luma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation in the encoding (encoding / decoding) is performed on a per-prediction-block basis. Using a luminance prediction block as an example, the prediction block comprises a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0104] Figure 7 A diagram of a video encoder (703) according to another embodiment of the present disclosure is shown. The video encoder (703) is configured to receive a processing block (e.g., a prediction block) of sample values ​​within a current video image in a video image sequence, and to encode the processing block into an encoded image that is part of an encoded video sequence. In one example, the video encoder (703) is used instead of Figure 4 The example video encoder (403).

[0105] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (703) uses, for example, rate-distortion optimization to determine whether to use intra-frame mode, inter-frame mode, or bidirectional prediction mode to optimally encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (703) can use intra-frame prediction techniques to encode the processing block into an encoded picture; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (703) can use inter-frame prediction or bidirectional prediction techniques respectively to encode the processing block into an encoded picture. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector predictors without the aid of encoded motion vector components outside the predictor. In some other video coding techniques, motion vector components applicable to the subject block may exist. In one example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0106] exist Figure 7 In the example, the video encoder (703) includes, for example, Figure 7 The inter-frame encoder (730), intra-frame encoder (722), residual calculator (723), switch (726), residual encoder (724), general controller (721) and entropy encoder (725) are shown coupled together.

[0107] An inter-frame encoder (730) is configured to receive samples of the current block (e.g., the processing block), compare that block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., redundancy descriptions based on inter-frame coding techniques, motion vectors, merging mode information), and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on encoded video information.

[0108] The intra encoder (722) is configured to receive samples of the current block (e.g., the processing block), compare the block with encoded blocks in the same image in some cases, generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and a reference block in the same image.

[0109] A general controller (721) is configured to determine general control data and control other components of the video encoder (703) based on that general control data. In one example, the general controller (721) determines the mode of a block and provides control signals to a switch (726) based on that mode. For example, when the mode is intra-frame mode, the general controller (721) controls the switch (726) to select the intra-frame mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include the intra-frame prediction information in the bitstream; and when the mode is inter-frame mode, the general controller (721) controls the switch (726) to select the inter-frame prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0110] A residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction selected from the intra encoder (722) or inter encoder (730). A residual encoder (724) is configured to operate based on the residual data to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be used appropriately by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) may generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (722) may generate a decoded block based on the decoded residual data and intra-frame prediction information. In some examples, the decoded blocks are appropriately processed to generate a decoded image, which can be buffered in a memory circuit (not shown) and used as a reference image.

[0111] The entropy encoder (725) is configured to format the bitstream to include encoded blocks. According to a suitable standard such as the HEVC standard, the entropy encoder (725) is configured to include various information. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0112] Figure 8A diagram of a video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In one example, the video decoder (810) is used instead of Figure 4 The video decoder (410) in the example.

[0113] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The entropy decoder (871), inter-frame decoder (880), residual decoder (873), reconstruction module (874), and intra-frame decoder (872) are shown coupled together.

[0114] The entropy decoder (871) can be configured to reconstruct certain symbols from an encoded picture, these symbols representing the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode encoded for the block (e.g., intra-mode, inter-mode, bidirectional prediction mode, a combined sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can be identified for use by the intra-decoder (872) or inter-decoder (880) for prediction, residual information in the form of, for example, quantized transform coefficients, etc. In one example, when the prediction mode is inter-mode or bidirectional prediction mode, inter-prediction information is provided to the inter-decoder (880); and when the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information may be inversely quantized and provided to the residual decoder (873).

[0115] The inter-frame decoder (880) is configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0116] The intra-frame decoder (872) is configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0117] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require some control information (to include quantizer parameters (QP)), which can be provided by the entropy decoder (871) (the data path is not depicted because this is only low-volume control information).

[0118] The reconstruction module (874) is configured to combine the residual output by the residual decoder (873) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module, as appropriate) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0119] It should be noted that any suitable technology can be used to implement the video encoder (403), video encoder (603), and video encoder (703), as well as the video decoder (410), video decoder (510), and video decoder (810). In one embodiment, one or more integrated circuits can be used to implement the video encoder (403), video encoder (603), and video encoder (703), as well as the video decoder (410), video decoder (510), and video decoder (810). In another embodiment, one or more processors that execute software instructions can be used to implement the video encoder (403), video encoder (603), and video encoder (603), as well as the video decoder (410), video decoder (510), and video decoder (810).

[0120] This disclosure includes embodiments related to adaptive loop filtering in video coding. These embodiments may include pre-arranging the adaptive loop filter (ALF) coefficients such that the transpose of the ALF coefficients can be performed with minimal operations when the transpose is implemented in a single instruction, multiple data (SIMD) vectorized form.

[0121] In April 2018, JVET officially launched the next-generation video coding standardization process, naming the new standard Versatile Video Coding (VVC), and JVET was renamed the Joint Video Experts Team. In 2020, ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) released the H.266 / VVC video coding standard (version 1).

[0122] For example, the adaptive loop filter in the H.266 / VVC standard can be applied to reconstructed pixels as the dot product of pixels on a rhombus pattern. For example, ALF coefficients can have a bit depth of 16 bits. The correspondence between pixel values ​​and coefficients on the rhombus pattern can be described in formulas, such as formula (1438) in the VVC standard, and depends on the transpose index. The transpose index can be obtained by an algorithm, such as the algorithm described in section 8.8.5.3 of the VVC standard. The ALF can be centrally symmetric. Therefore, the ALF transpose defined in H.266 / VVC can involve 12 coefficients on the rhombus pattern. For example, in H.266 / VVC, four transposes or geometric transformations of the ALF can be defined. For example, the transpose with index 0 can be the default transpose or "initial" transpose to preserve the coefficients in parsing / decoding order, the transpose with index 1 can be a "diagonal flip," the transpose with index 2 can be a "vertical flip," and the transpose with index 3 can be a "rotation."

[0123] Figure 9 The initial or default position of the ALF coefficients on the diamond pattern can be provided. For example... Figure 9 As shown, ALF(900) can have coefficients C0-C12 positioned in a diamond pattern. According to the definition in the encoded bitstream of the H.266 / VVC standard, it can be based on... Figure 9 The initial pattern shown encodes and decodes the coefficients of ALF (900) in order from 0 (e.g., C0) to 11 (e.g., C11). The 12th coefficient (e.g., C12) may be located at the center of ALF (900) and is derived from the values ​​of the 0th to 11th coefficients for normalization of the filter (900).

[0124] In the H.266 / VVC standard, the transpose can be defined in formulas (1), (2), (3), and (4) according to the mapping table. The luminance filter coefficients and clipping value index idx can be derived from the transpose Idx[x][y], as shown below:

[0125] If the transpose Idx[x][y] equals 1, then the following formula applies:

[0126] idx[ ] = { 9, 4, 10, 8, 1, 5, 11, 7, 3, 0, 2, 6} Formula (1)

[0127] Otherwise, if the transpose Idx[x][y] equals 2, then the following formula applies:

[0128] idx[ ] = { 0, 3, 2, 1, 8, 7, 6, 5, 4, 9, 10, 11} Formula (2)

[0129] Otherwise, if the transpose Idx[x][y] equals 3, then the following formula applies:

[0130] idx[ ]={ 9, 8, 10, 4, 3, 7, 11, 5, 1, 0, 2, 6} Formula (3)

[0131] Otherwise, apply the following formula:

[0132] idx[]={ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11} Formula (4)

[0133] exist Figures 10A to 10C The text describes transposes based on diagonal flips, vertical flips, and rotations defined by formulas (1)-(3), as well as commutation coefficients based on the corresponding transposes. Figure 10A An exemplary diagonal flip of the coefficients of ALF(1002) is shown. For example, coefficient C0 can be swapped with coefficient C9, and coefficient C2 can be swapped with coefficient C10. Figure 10B An exemplary vertical flip of the coefficients of ALF(1004) is shown. Figure 10B As shown, coefficient C0 can remain in its original position. For example, coefficient C1 can be swapped with coefficient C3. Figure 10C An exemplary rotation of the coefficients of ALF(1006) is shown.

[0134] In some software implementations, the ALF filter coefficients can be placed in the same order as the initial "untranspose" (e.g., Figure 9 The initial placement (in the code) matches the decoding order stored in contiguous blocks of memory (or memory space with contiguous blocks). Before filtering, coefficients can be swapped according to a specific transpose operation. The transpose operation can be indicated by transpose indices of, for example, 0, 1, 2, or 3. Transpose index 0 indicates "no transpose". Transpose index 1 indicates "diagonal flip". Transpose index 2 indicates "vertical flip", and transpose index 3 indicates "rotation". In H.266 / VVC compliant codecs, the transpose process can be frequently applied to ALF filters during decoding or encoding. However, transpose can be a series of swaps of coefficients in memory locations where coefficients are stored. Transpose may be unfriendly to SIMD (single instruction / multiple data), such as when swapping coefficients that are far apart in memory. For example, if each coefficient has a bit depth of 16 bits and all coefficients are stored in contiguous memory blocks, coefficients that are far apart in contiguous memory blocks (or contiguous blocks of memory) must be swapped by diagonal flipping or rotation.

[0135] Intel's current AVX architecture or similar architecture has 256-bit registers that can be split into two channels. If 12 coefficients (e.g., 12 x 16 bits = 192 bits) are read from memory into such a register in their initial default arrangement and transposed in a vectorized manner, then values ​​(or coefficients) need to be swapped across channels. For example, the 0th coefficient used for "diagonal flip" or "rotation" might have to be swapped with the 9th coefficient, which is located 128 bits from the 0th coefficient. Swapping values ​​across channels can be much slower than swapping values ​​within a channel, which can be done with just one SIMD instruction, while swapping values ​​across channels may not be possible with just one SIMD instruction. To filter each 4x4 luma block, such as in H.266 / VVC, transposition can become a very time-consuming process.

[0136] If the rhombus pattern is large, for example larger than an H.266 / VVC ALF pattern, the problem of fast transposition implemented in vectorized SIMD becomes more difficult. This ALF design was proposed earlier and is currently being studied by the JVET group as a new video coding algorithm for better compression. Embodiments of this disclosure can be used to efficiently handle cases with large rhombus patterns.

[0137] In this disclosure, ALF coefficients can be placed in contiguous blocks of memory in a specially designed order. For example, subsets of ALF coefficients to be swapped can be placed (or grouped) close to each other in contiguous memory. Grouping ALF coefficients can be implemented by placing coefficients during decoding or before filtering but after sequential decoding. This order can be defined by the encoding standard, such as the H.266 / VVC standard. Figure 11 An exemplary pre-arrangement of ALF coefficients with desired properties can be shown in the diagram. For example... Figure 11 As shown, the coefficients of an ALF filter (1100) can be grouped. The ALF filter (1100) can have a diamond shape and include coefficients C0-C12, where, for example, C0 can be labeled "0" and C1 can be labeled "1". C0 can be located at the center of the ALF filter (1100). Coefficients C1-C12 can be arranged into multiple groups. Figure 11 In one example, coefficients C1-C12 can be arranged into three groups. Coefficients C1-C2 can be arranged in the first group, coefficients C3-C6 in the second group, and coefficients C7-C12 in the third group. Furthermore, transposes (e.g., rotation, vertical flip, and diagonal flip) can be performed on the groups. These groups can be transposed in the same or different combinations.

[0138] Common properties of a defined set of transposes (e.g., rotation, vertical flip, and diagonal flip) may be applied to only exchange coefficients that are at the same distance from the center of the pattern of the ALF filter. For example, Figure 11 the coefficients shown may initially be placed in an order that allows "diagonal flip", "vertical flip" or "rotation" to be performed. Groups of coefficients (or subsets of coefficients) may then be exchanged. For example, based on the bit depth of each coefficient, the groups of coefficients may be located at positions no more than a certain number of coefficients apart from each other. In one example, the groups of coefficients may be located at positions no more than 3 coefficients apart from each other, or if the bit depth of each coefficient is 16 bits and the coefficients are placed in consecutive blocks of memory, the groups of coefficients may be located at positions no more than 48 bits apart from each other. In the AVX architecture, the exchanged groups of coefficients may be placed in 128-bit registers, or in one lane of a 256-bit register. Thus, a single SIMD instruction may be applied to exchange the groups of coefficients. By grouping coefficients that are within a certain distance of each other, and placing the groups of coefficients into 128-bit registers or lanes of 256-bit registers, the transposes are performed much faster than on widely used architectures. Thus, the defined transposes may be implemented in software while minimizing the number of SIMD instructions.

[0139] Figure 12A , Figure 12B and Figure 12C illustrate exemplary ALF transposes (e.g., "diagonal flip", "vertical flip" and "rotation"), as well as the proposed initial arrangement of ALF coefficients, for example the ALF coefficients provided by Figure 11 . Figure 12A shows the ALF (1202) on which the diagonal flip is performed. As shown in Figure 12A , the coefficients C1-C12 of the ALF (1202) may include three groups. Group 1 may include C1-C2, group 2 may include C3-C6, and group 3 may include C7-C12. Compared with each group in Figure 11 , due to the applied transpose (e.g., diagonal flip), the coefficient arrangement (or placement) of each group in Figure 12A and the corresponding group in Figure 11 may be different. For example, the positions of coefficients C1-C2 of group 1 in Figure 12A may be different from the positions of coefficients C1-C2 of group 1 in Figure 11 . Still referring to Figure 12A , the diagonal flip may be performed on the coefficients of each group. For example, in group 1, based on the diagonal flip, coefficient C1 and coefficient C2 may be exchanged with each other. It should be noted that in some examples, the coefficients of these two groups may not be exchanged.

[0140] Figure 12B illustrates the ALF (1204) on which the vertical flip is performed. As shown in Figure 12B as shown, the coefficients C1-C12 of the ALF (1204) may include three groups. Group 1 may include C1-C2, group 2 may include C3-C6, and group 3 may include C7-C12. Compared with Figure 12A each of the groups in, due to the applied transpose (e.g., vertical flip), Figure 12B each group in and Figure 12A the coefficient arrangement (or placement) of the corresponding group in may be different. For example, Figure 12B the positions of coefficients C1-C2 of group 1 in may be different from Figure 12A the positions of coefficients C1-C2 of group 1 in. Still referring to Figure 12B , a vertical flip may be performed on the coefficients of each group. For example, in group 1, based on the vertical flip, coefficient C1 and coefficient C2 remain in their original positions.

[0141] Figure 12C shows the ALF (1206) on which rotation is performed. As shown in Figure 12C , the coefficients C1-C12 of the ALF (1206) may include three groups. Group 1 may include C1-C2, group 2 may include C3-C6, and group 3 may include C7-C12. Compared with Figure 12A and Figure 12B each of the groups in, due to the applied transpose (e.g., rotation), Figure 12C each group in and Figure 12A or Figure 12B the coefficient arrangement (or placement) of the corresponding group in may be different. For example, Figure 12C the positions of coefficients C1-C2 of group 1 in may be different from Figure 12A or Figure 12B the positions of coefficients C1-C2 of group 1 in. A rotation may be performed on the coefficients of each group. For example, in group 2, based on the transpose of rotation, coefficient C3 and coefficient C4 may be swapped with each other.

[0142] grouping the coefficients of an ALF such as the H.266 / VVC ALF with a diamond pattern, a corresponding pair of indices (or a pair of coordinate values along the first axis and the second axis) ( x, y ) may be applied to each coefficient. The pair of indices may indicate the position of the coefficient in the ALF. Figure 13 shows exemplary indices (or coordinate values) of the coefficients of the ALF (1300). As shown in Figure 13 , the center coefficient of the ALF (1300) (e.g., ) may have a pair of indices equal to (0, 0). The other coefficients of the ALF (1300) may have relative indices matching the horizontal and vertical offsets starting from the center index (0, 0) ( x, yALF(1300) can have a centrally symmetric pattern (or centrally symmetric shape). This centrally symmetric pattern (or centrally symmetric shape) indicates that the coefficient values ​​of ALF(1300) can follow a function. Therefore, each coefficient in ALF(1300) can have an opposite / symmetric coefficient relative to the central coefficient. Corresponding coefficients and their opposites can be at the same distance from the central coefficient, but on opposite sides.

[0143] Coefficient-based index (or coordinate value) Figure 11 The preset arrangement shown can be provided based on the index of the coefficients, for example... Figure 13 The index shown. (As shown in the image) Figure 13 As shown, each of the coefficients C7-C12 in the third group of ALF(1300) can have a corresponding pair of indices. x and y The index pairs for coefficients C7-C12 can be {(-3, 0), (-2, 1), (-1, 2), (0, 3), (1, 2), (2, 1)}. Each index pair in the third group... x and y The sum of the absolute values ​​can equal 3 (for example, ).have coefficient It can belong to the second group of ALF(1300). The index pairs of coefficients C2-C6 in the second group can be {(-2, 0), (-1, 1), (0, 2), (1, 1)}. The first group of ALF(1300) can contain two coefficients C1-C2. Coefficients C1 and C2 can have the index pair (-1, 0) and the index pair (0, 1) respectively. The index pairs (-1, 0) and (0, 1) x and y The sum of the absolute values ​​can equal .

[0144] For each ALF transpose, such as "Diagonal Flip," "Vertical Flip," and "Rotation" as defined in the H266 / VVC ALF, the coefficients of the ALF can only be swapped within each group. In one example, for "Diagonal Flip," the coefficients in each group of ALF(1300) can be swapped in pairs as follows:

[0145] (1) The first group can have 3 swap pairs: (-3, 0)(0, 3); (-2, 1)(-1, 2); (1, 2)(2, 1)

[0146] (2) The second group can have 1 swap pair: (-2, 0) (0, 2)

[0147] (3) The third group can have 1 swap pair: (-1, 0) (0, 1).

[0148] For transpose "vertical flip" and "rotation", the coefficients can only be applied within each group.

[0149] If each group is stored in a contiguous block of computer memory, and the coefficients have a bit depth of 16 bits, then the six coefficients of the first group can be loaded into a channel of a 256-bit register or a 128-bit register using a single SIMD load instruction for that group. A single SIMD transform position instruction for that group can also be used to perform any defined transpose (e.g., rotation, vertical flip, or diagonal flip). Similarly, the six coefficients of the second and third groups can be loaded into a channel of a 256-bit register or a 128-bit register using a single SIMD instruction. Alternatively, as shown in Table 1, the coefficients of the first and third groups can be loaded into a channel of a 128-bit register or a 256-bit register, and the second group can be loaded into another channel of a 128-bit register or a 256-bit register. Likewise, each transpose can be performed without mixing registers or channels. This helps to provide a minimum number of SIMD operations to perform the defined transposes.

[0150] Table 1: Storage of ALF coefficients in registers

[0151]

[0152] Note: This represents the uninitialized 16-bit space in the register.

[0153] for Larger rhombuses, such as the ALF pattern in newer video compression technologies, have the same... 16-bit coefficient of the value In contiguous memory, it may occupy more than 128 bits. However, it is still beneficial to group the ALF coefficients, which have a large diamond shape.

[0154] Figure 14 An example of grouping coefficients in an ALF (1400) with a large rhombus shape is shown. The ALF (1400) can receive the same set of transposes as the H.266 / VVC ALF. (e.g.) Figure 14 As shown, ALF(1400) can have a rhombus pattern including 42 coefficients that can be interchanged according to transpose (e.g., vertical flip, diagonal flip, and rotation).

[0155] The position of the coefficients of ALF(1400) can be determined using indices (or coordinate values). To identify. Coefficients can be grouped, making... and The coefficient belongs to the first group. and The coefficient can belong to the second group. and The coefficient can belong to the third group. and The coefficient can belong to the fourth group, and so on. Therefore, as Figure 14 As shown, a total of 12 groups can be defined. Furthermore, The group can be classified as an odd group, while Groups can be categorized as even-numbered groups. For example, the first group could be an odd-numbered group containing 6 coefficients with indices {(-6, 0), (-5, 1), (-4, 2), (-3, 3), (-2, 4), (-1, 5)}. The second group could be an even-numbered group containing 6 coefficients with indices {(0, 6), (1, 5), (2, 4), (3, 3), (4, 2), (5, 1)}. Still refer to... Figure 16 Each odd array can have a corresponding even array. The odd array and its corresponding even array can have the same number of coefficients and be equidistant from the center of the ALF filter (e.g., For example, the second group could be the group corresponding to the first group. The fourth group could be the group corresponding to the third group.

[0156] For each defined group, the coefficients can be rearranged (or reordered) within each group based on a defined transpose (e.g., "diagonal flip", "vertical flip", or "rotation"). Based on the defined transpose, an entire group (e.g., group 1) can be swapped with its corresponding group (e.g., group 2). Therefore, only in some examples can each odd array swap coefficients with its matching (or corresponding) even array. In one example, for "rotation", the coefficients of group 1 can be swapped only with the coefficients of group 2. For "diagonal flip" or "vertical flip", the coefficients can first be rearranged within each group, and then the odd array can only swap coefficients with its matching (or corresponding) even array. The swap between the odd array and its corresponding even array can be independent of other groups. Vectorized swapping can be performed more efficiently by swapping coefficients only within a single group and between the odd array and its corresponding even array.

[0157] For compact storage and more efficient use of the channels in 128-bit or 256-bit registers, the groups defined above can be stored as follows: Odd-numbered groups, such as group 1 with indices {(-6, 0), (-5, 1), (-4, 2), (-3, 3), (-2, 4), (-1, 5)} and group 9 with indices {(-2, 0), (-1, 1)}, can be placed in a single 128-bit register. Matching even-numbered groups, such as group 2 with indices {(0, 6), (1, 5), (2, 4), (3, 3), (4, 2), (5, 1)} and group 10 with indices {(0, 2), (1, 1)}, can be stored in another 128-bit register. Thus, the entire space of the register can be used for 16-bit coefficients. A similar technique for grouping groups into registers can be used for other groups: group 3 with 5 16-bit coefficients can be stored together with group 7 with 3 16-bit coefficients in a single 128-bit register. Finally, the coefficient positions in the 128-bit register can be as shown in Table 2 below:

[0158] Table 2: 128-bit odd and even registers of the fast vectorized transpose of an ALF containing 42 coefficients.

[0159]

[0160] Note: This represents the uninitialized 16-bit space in the register.

[0161] Then, within each defined transpose, two operations can be involved: (1) a vectorized positional transformation (e.g., rotation, diagonal flip, or vertical flip) can be applied within each register, and (2) a vectorized exchange of groups can be implemented between odd and even registers. Both operations can be implemented very efficiently using SIMD instructions on six registers that encapsulate all 42 required 16-bit values.

[0162] Grouping coefficients at the same distance from the center of the ALF filter into groups is beneficial for large ALF filters with a modified centrosymmetric shape. For example... Figure 15As shown, ALF (1500) can provide an example of a modified centrally symmetric shape. ALF (1500) may include a portion (1502) having a rhombus shape and portions (1504)-(1507) extending from the sides of portion (1502). Portion (1502) may be similar to ALF (1300) such that the coefficients of portion (1502) can be pre-arranged into 3 groups. Components (1504)-(1507) may extend from the four sides of component (1502) respectively. In some embodiments, the coefficients of component (1502) may be grouped into rhombus-shaped coefficients provided in H.266 / VVC. For example, coefficients with indices {(-3, 0), (-2, 1), (-1, 2), (0, 3), (1, 2), (2, 1)} may be grouped into a first group. Coefficients with indices {(-2, 0), (-1, 1), (0, 2), (1, 1)} may be grouped into a second group. The third group may include coefficients with indices {(-1, 0), (0, -1)}. The remaining 8 coefficients in parts (1504) and (1505) may be grouped into a fourth group with indices {(-7, 0), (-6, 0), (-5, 0), (-4, 0), (0, 4), (0, 5), (0, 6), (0, 7)}. The fourth group may be accommodated in a 128-bit register and may remain unchanged for defined transposes (e.g., vertical flips, diagonal flips, or rotations).

[0163] In this disclosure, based on the distance from the center of ALF Grouping the coefficients of ALF can be applied to patterns of various shapes, including arbitrarily large rhombuses, truncated rhombuses, or centrally symmetric shapes. Transposes, such as "diagonal flip," "vertical flip," "rotational" transposes, or any other transpose that does not change the distance from the center to the coefficients, can then be applied to the grouped coefficients.

[0164] The decoding order and / or parsing order of ALF coefficients in a bitstream can be matched with locality coefficient groups. Therefore, ALF coefficients can be decoded based on group information. For example, the coefficients of the first group of ALFs can be decoded first, and the coefficients of the second group of ALFs can be decoded subsequently.

[0165] In the encoding process for searching coefficients among ALF coefficient candidates in an encoder such as an H.266 / VVC compliant encoder, pre-arranged positions or default initial positions can be used. After parsing (or decoding) the ALF coefficients, in new video encoding and decoding algorithms, or in devices involving transposes on centrally symmetric shapes, the decoded ALF coefficients can be placed in contiguous memory blocks.

[0166] Coefficients can be applied to various patterns. The arrangement includes various patterns such as rhombuses, truncated rhombuses, or those with a central coefficient. The center-symmetric filter pattern. The defined transpose can then be applied to the arranged (or grouped) coefficients.

[0167] Signaling can be sent to inform the order of coefficients in a centrally symmetric filter (or a filter with a centrally symmetric shape). Based on the signal order, the filter coefficients can be divided into several groups. The coefficients in each group... The distance from the center of a centrally symmetric shape filter can be the same. .

[0168] In some embodiments, the coefficients of each group can be placed in contiguous blocks of memory (or contiguous blocks of memory). The coefficients of a group can be rearranged (or reordered) based on a defined transpose (e.g., vertical flip, diagonal flip, or rotation). Therefore, for any defined transpose, coefficients from another group cannot be swapped with any coefficients of that group. Thus, the coefficients of that group can still remain within that group and stored in the same contiguous memory.

[0169] In some embodiments, the coefficients can be grouped such that the coefficients of the ALF can be arranged into an even number of groups. The even number of groups can be divided into two disjoint sets: a first set and a second set. For each defined transpose, each group from the first set can exchange coefficients with a matching (or corresponding) group from the second set. The groups in the first set can have indices x. N coefficients , making The groups in the second set can have an index x. N There are coefficients such that, The first and second sets can each contain the same number of groups. Each group in the first set can have a matching (or corresponding) group in the second set. Groups in the first set and corresponding groups in the second set can have the same number of coefficients and be equidistant from the center of the ALF. For the defined transpose, one or more coefficients of a group in the first set can be swapped with one or more coefficients of a matching group in the second set.

[0170] In some embodiments, the coefficients of each group can be placed in consecutive blocks of memory such that each block (or block) of consecutive memory is no larger than, for example, 128 bits of storage capacity.

[0171] In some embodiments, the coefficients of several groups can be stored in contiguous blocks of memory, such as 128-bit blocks of contiguous memory.

[0172] In some embodiments, the coefficients of each group can be placed in consecutive memory blocks such that each block of memory has a storage capacity of no more than, for example, 256 bits.

[0173] In some embodiments, the coefficients of several groups can be stored in contiguous blocks of memory, such as 256-bit blocks of contiguous memory.

[0174] In some embodiments, the signaling notifications can be sent in any order (e.g., Figure 9 (The order of the coefficients shown) After decoding the ALF coefficients, the ALF coefficients are immediately grouped. Therefore, the ALF coefficients can be decoded based on the order of signaling notification. The coefficients can then be further grouped.

[0175] In some embodiments, the ALF coefficients can be grouped immediately after decoding them according to the H.266 / VVC decoding order. For example, according to Figure 9 The initial pattern shown decodes coefficients in order from number 0 (e.g., C0) to number 11 (e.g., C11). The coefficients can then be further grouped.

[0176] In some embodiments, the ALF coefficients can be grouped before searching for optimal (or selected) ALF coefficients on the centrally symmetric pattern of the encoder. For example, a transpose (e.g., vertical flip, diagonal flip, or rotation) can be determined first. Signaling can be sent accordingly to indicate the transpose index of the determined transpose. The positions of the ALF coefficients can be grouped based on the determined transpose. Furthermore, the ALF coefficients can be determined from the candidate coefficients based on a search process. The optimal (or selected) ALF coefficients can be determined by minimizing the prediction error based on a covariance matrix (or cost value), such as the covariance matrix in the formula for the mean squared error (MSE) of a Wiener filter. For each transpose, a corresponding search can be performed to find the optimal ALF coefficients from the candidate coefficients.

[0177] In some embodiments, ALF coefficients can be grouped based on pixel data of blocks at matching positions (e.g., positions corresponding to filter coefficients) on a centrally symmetric pattern before searching for the optimal ALF coefficients in the encoder.

[0178] In some embodiments, grouping of ALF coefficients with a centrally symmetric shape, such as a rhombus shape, can be performed before searching for the optimal ALF coefficients in an encoder such as an H.266 / VVC compliant encoder.

[0179] In some embodiments, ALF coefficients having a centrally symmetric shape, such as a rhombus shape, can be grouped based on the pixel values ​​of blocks at matching positions of filter coefficients on a centrally symmetric shape before searching for ALF coefficients in an encoder such as an H.266 / VVC compliant encoder.

[0180] Figure 16 A flowchart outlining an exemplary decoding process (1600) according to some embodiments of this disclosure is shown. Figure 17 A flowchart outlining an exemplary encoding process (1700) according to some embodiments of this disclosure is shown. The proposed processes can be used individually or in combination in any order. Furthermore, each of the processes (or embodiments), encoders, and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium.

[0181] The operations in the processes (e.g., (1600) and (1700)) can be combined or arranged in any number or order as needed. In an embodiment, two or more operations in the processes (e.g., (1600) and (1700)) can be performed in parallel.

[0182] These processes (e.g., (1600) and (1700)) can be used for block reconstruction and / or encoding to generate predictive blocks for the blocks being reconstructed. In various embodiments, the processes (e.g., (1600) and (1700)) are performed by processing circuitry, such as processing circuitry in terminal devices (310), (320), (330), and (340), processing circuitry performing the functions of a video encoder (403), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of a video decoder (510), processing circuitry performing the functions of a video encoder (603), etc. In some embodiments, the processes (e.g., (1600) and (1700)) are implemented with software instructions, so that the processing circuitry performs the processes (e.g., (1600) and (1700)) when the processing circuitry executes the software instructions.

[0183] like Figure 16 As shown, the process (1600) can begin at (S1601) and proceed to (S1610). In S1610, prediction information for the current block in the current image is received from the encoded video bitstream. The prediction information indicates that a filter with a centrally symmetric shape should be applied to the current block. The filter includes multiple coefficients. Each of the multiple coefficients is associated with a corresponding first coordinate value along a first axis and a corresponding second coordinate value along a second axis.

[0184] In S1620, based on the transpose operation to be performed on the filter, the multiple coefficients of the filter are grouped into multiple groups, and the coefficients in each group are at the same distance from the center of the filter.

[0185] In S1630, the coefficients of the first group in the multiple groups are reordered based on the transpose operation.

[0186] In S1640, based on the reordered coefficients of the filter, the filtered samples of the current block are reconstructed by applying a filter with a centrally symmetric shape to the current block.

[0187] In processing 1600, the reordered coefficients of the first group in multiple groups are swapped with the coefficients of the second group in multiple groups. The reordered coefficients of the first group and the coefficients of the second group are located at a first distance from the center of the filter.

[0188] In one example, the transpose operation can be one of a rotation operation, a vertical flip operation, or a diagonal flip operation.

[0189] In one example, the number of coefficients in the first group is the same as the number of coefficients in the second group.

[0190] In one example, the first distance is equal to the sum of (i) the absolute value of the first coordinate value of the first coefficient of the first group and (ii) the absolute value of the second coordinate value of the first coefficient of the first group.

[0191] In one example, the sum of the first coordinate value of the first coefficient of the first group and the second coordinate value of the first coefficient of the second group is equal to 0. In another example, the second coordinate value of the first coefficient of the first group is equal to the first coordinate value of the first coefficient of the second group.

[0192] In one example, the coefficients of each of the multiple groups can be stored in a corresponding storage space. In one example, the corresponding storage space has one of (i) a first storage capacity equal to or less than 128 bits and (ii) a second storage capacity equal to or less than 256 bits.

[0193] In one example, the coefficients of one or more of the multiple groups are stored in a storage space having one of (i) a first storage capacity of 128 bits and (ii) a second storage capacity of 256 bits.

[0194] In one example of processing (1600), each of the multiple groups has a corresponding group located within the multiple groups, and the coefficients of the corresponding group within the multiple groups and the coefficients of the corresponding group within the multiple groups are at the same distance from the center of the filter. The corresponding group within the multiple groups and the corresponding group within the multiple groups include the same number of coefficients.

[0195] In one example of processing (1600), based on prediction information, multiple coefficients of the filter can be determined in the order in which signaling notifications are sent in the prediction information. The determined multiple coefficients of the filter can be grouped into multiple groups.

[0196] The process (1600) can be adjusted as appropriate. One or more steps in the process (1600) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0197] like Figure 17 As shown, the process (1700) can begin at (S1701) and proceed to (S1710). In (S1710), based on the transpose operation to be performed on the filter, the multiple coefficients of the filter having a centrally symmetric shape are grouped into multiple groups. The filter includes multiple coefficients. Each of the multiple coefficients is associated with a corresponding first coordinate value along a first axis and a corresponding second coordinate value along a second axis. The coefficients in each of the multiple groups are at the same distance from the center of the filter.

[0198] In (S1720), based on the transpose operation, the coefficients of the first group in the multiple groups are reordered.

[0199] In (S1730), based on the reordered coefficients of the filter, the filtered prediction samples of the current block are generated by applying a filter with a centrally symmetric shape to the current block.

[0200] At (S1740), encoding information is generated. The encoding information indicates that a filter with a centrally symmetric shape will be applied to the current block, and that the multiple coefficients of the filter will be grouped into multiple groups.

[0201] Then, the process proceeds to (S1799) and terminates.

[0202] The process (1700) can be adjusted as appropriate. One or more steps in the process (1600) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0203] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 18 A computer system (1800) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0204] Computer software can be coded using any suitable machine code or computer language. Any suitable machine code or computer language can be assembled, compiled, linked, or similarly processed to create code containing instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpretation, microcode execution, or other means.

[0205] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0206] Figure 18 The components of the computer system (1800) shown are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependencies or requirements relating to any component or combination of components shown in the exemplary embodiments of the computer system (1800).

[0207] The computer system (1800) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, movement of a data glove), audio input (e.g., speech, clapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, images captured from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0208] Human-machine interface input devices may include one or more of the following (only one of each is shown): keyboard (1801), mouse (1802), touchpad (1803), touch screen (1810), data glove (not shown), joystick (1805), microphone (1806), scanner (1807), camera (1808).

[0209] The computer system (1800) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback of a touchscreen (1810), data gloves (not shown), or joysticks (1805), but may also be tactile feedback devices that are not input devices), audio output devices (e.g., speakers (1809), headphones (not depicted)), and visual output devices (e.g., screens (1810) including CRT screens, LCD screens, plasma screens, OLED screens, each screen may or may not have touchscreen input functionality, each screen may or may not have tactile feedback functionality, some of which are capable of outputting two-dimensional or more three-dimensional visual outputs through devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays and smoke boxes (not depicted), and printers (not depicted).

[0210] The computer system (1800) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1820) having media such as CD / DVD (1821), finger drives (1822), removable hard disk drives or solid-state drives (1823), conventional magnetic media such as magnetic tape and floppy disks (not depicted), devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted), etc.

[0211] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0212] The computer system (1800) may also include an interface (1854) to one or more communication networks (1855). The network may be, for example, a wireless network, a wired network, or an optical network. The network may further be a local area network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter (e.g., a USB port of the computer system (1800)) attached to some general-purpose data port or peripheral bus (1849); other network interfaces are typically integrated into the core of the computer system (1800) by being attached to a system bus (e.g., an Ethernet interface connected to a PC computer system or a cellular network interface connected to a smartphone computer system), as described below. The computer system (1800) can use any of these networks to communicate with other entities. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., a CANBus connected to certain CANBus devices), or bidirectional, such as connecting to other computer systems using a local area network (LAN) or wide area network (WAN) digital network. As mentioned above, certain protocols and protocol stacks can be used on each of those networks and network interfaces.

[0213] The aforementioned human-machine interface device, human-machine accessible storage device, and network interface can be attached to the kernel (1840) of the computer system (1800).

[0214] The core (1840) may include one or more central processing units (CPU) (1841), graphics processing units (GPUs) (1842), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (1843), hardware accelerators (1844) for certain tasks, graphics adapters (1850), etc. These devices, as well as read-only memory (ROM) (1845), random access memory (1846), and internal mass storage (1847) such as internal non-user-accessible hard disk drives, SSDs, etc., may be connected via the system bus (1848). In some computer systems, the system bus (1848) may be accessed in the form of one or more physical plugs to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (1848) or attached to the core's system bus (1848) via a peripheral bus (1849). In one example, a screen (1810) may be connected to a graphics adapter (1850). Peripheral bus architectures include PCI, USB, etc.

[0215] The CPU (1841), GPU (1842), FPGA (1843), and accelerator (1844) can execute certain instructions that can be combined to form the aforementioned computer code. This computer code can be stored in ROM (1845) or RAM (1846). Transient data can be stored in RAM (1846), while permanent data can be stored, for example, in internal mass storage (1847). Fast storage and retrieval to any storage device can be achieved by using a cache, which can be closely associated with one or more CPUs (1841), GPUs (1842), mass storage (1847), ROM (1845), RAM (1846), etc.

[0216] Computer-readable media may have computer code thereon that performs various computer-implemented operations. The media and computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0217] By way of example, and not limitation, software contained in one or more tangible computer-readable media can be executed by one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) to enable a computer system having an architecture (1800), particularly a kernel (1840), to provide functionality. Such computer-readable media can be media associated with user-accessible mass storage as described above, and memory of certain non-transitory kernels (1840), such as internal kernel mass storage (1847) or ROM (1845). Software implementing various embodiments of this disclosure can be stored in such devices and executed by the kernel (1840). Depending on specific needs, the computer-readable media may include one or more storage devices or chips. The software can cause the kernel (1840), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1846) and modifying such data structures according to software-defined processing. Additionally or alternatively, the computer system may be made functional by logic hardwired or otherwise embodied in circuitry (e.g., an accelerator (1844)), which may replace or operate with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0218] Appendix A: Acronyms

[0219] ALF: Adaptive Loop Filter

[0220] AVX: Advanced Vector Extensions Architecture

[0221] JVET: Joint Video Experts Group

[0222] SIMD: Single Instruction Multiple Data Architecture

[0223] H.266 / VVC: The Next Generation Video Coding Standard

[0224] JEM: Joint Exploration Model

[0225] VVC: Next-Generation Video Coding

[0226] BMS: Benchmark Set

[0227] MV: Motion Vector

[0228] HEVC: High-Efficiency Video Coding

[0229] SEI: Auxiliary Enhancement Information

[0230] VUI: Video Availability Information

[0231] GOP: Image Group

[0232] TU: Transformation Unit

[0233] PU: Prediction Unit

[0234] CTU: Coding Tree Unit

[0235] CTB: Coded Tree Block

[0236] PB: Prediction Block

[0237] HRD: Hypothetical Reference Decoder

[0238] SNR: Signal-to-noise ratio

[0239] CPU: Central Processing Unit

[0240] GPU: Graphics Processing Unit

[0241] CRT: Cathode Ray Tube

[0242] LCD: Liquid Crystal Display

[0243] OLED: Organic Light Emitting Diode

[0244] CD: CD-ROM

[0245] DVD: Digital Video Disc

[0246] ROM: Read-Only Memory

[0247] RAM: Random Access Memory

[0248] ASIC: Application-Specific Integrated Circuit

[0249] PLD: Programmable Logic Device

[0250] LAN: Local Area Network

[0251] GSM: Global System for Mobile Communications

[0252] LTE: Long Term Evolution

[0253] CANBus: Controller Area Network Bus

[0254] USB: Universal Serial Bus

[0255] PCI: Interconnect Peripheral Components

[0256] FPGA: Field Programmable Gate Domain

[0257] SSD: Solid State Drive

[0258] IC: Integrated Circuit

[0259] CU: Encoding Unit

[0260] While several exemplary embodiments have been described in this disclosure, modifications, substitutions, and various equivalent alternatives fall within the scope of this disclosure. Therefore, it should be understood that those skilled in the art will be able to design numerous systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and thus fall within its spirit and scope.

Claims

1. A video decoding method, characterized in that, The method includes: Prediction information for the current block in the current image is received from the encoded video bitstream. The prediction information indicates that a filter with a centrally symmetric shape is applied to the current block. The filter includes a plurality of coefficients, each of which is associated with a corresponding first coordinate value along a first axis and a corresponding second coordinate value along a second axis. Based on the transpose operation to be performed on the filter, the multiple coefficients of the filter are grouped into multiple groups, the coefficients in each of the multiple groups are equidistant from the center of the filter, and the coefficients of each of the multiple groups are stored in their respective contiguous memory blocks. Based on the transpose operation, the coefficients of the first group among the plurality of groups are reordered; and The filtered samples of the current block are reconstructed by applying the filter, which has a centrally symmetric shape, to the current block based on the reordered coefficients of the filter.

2. The method according to claim 1, characterized in that, The method further includes: The coefficients of the first group and the coefficients of the second group in the plurality of groups are swapped, and the coefficients of the first group and the second group are located at a first distance from the center of the filter.

3. The method according to claim 1, characterized in that, The transpose operation is one of the following: rotation, vertical flip, or diagonal flip.

4. The method according to claim 2, characterized in that, The number of coefficients in the first group is the same as the number of coefficients in the second group.

5. The method according to claim 2, characterized in that, The first distance is equal to (i) the absolute value of the first coordinate value of the first coefficient of the first group and (ii) the absolute value of the second coordinate value of the first coefficient of the first group.

6. The method according to claim 2, characterized in that, The sum of the first coordinate value of the first coefficient of the first group and the second coordinate value of the first coefficient of the second group is equal to 0. The second coordinate value of the first coefficient of the first group is equal to the first coordinate value of the first coefficient of the second group.

7. The method according to claim 1, characterized in that, The respective consecutive memory blocks have one of (i) a first storage capacity equal to or less than 128 bits and (ii) a second storage capacity equal to or less than 256 bits.

8. The method according to claim 1, characterized in that, The coefficients of one or more of the multiple groups are located in a contiguous memory block having one of (i) a first storage capacity of 128 bits and (ii) a second storage capacity of 256 bits.

9. The method according to any one of claims 1 to 8, characterized in that, Each of the plurality of groups has a corresponding group located within the plurality of groups, and the coefficients of the corresponding group within the plurality of groups and the coefficients of the corresponding group located within the plurality of groups are at the same distance from the center of the filter. The corresponding group in the plurality of groups and the corresponding group located in the plurality of groups include the same number of coefficients.

10. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Based on the prediction information, the plurality of coefficients of the filter are determined in the order in which signaling notifications are sent in the prediction information; and The determined coefficients of the filter are grouped into the multiple groups.

11. A video decoding device, characterized in that, The device includes: Memory, used to store instructions; and A processor for invoking instructions stored in the memory to implement the method according to any one of claims 1-10.

12. A non-transitory computer-readable storage medium for storing instructions, characterized in that, When executed by at least one processor, the instructions cause the at least one processor to perform the method according to any one of claims 1-10.

Images