Video decoding method, computer device, equipment and storage medium

Abstract

Aspects of the present disclosure include methods, devices, and non-transitory computer-readable storage media for video encoding / decoding. The device includes a processing circuit system that decodes prediction information of a current block in a current picture that is part of a video bitstream. The processing circuit system determines to enable a first plurality of coding tools and disable a second plurality of coding tools for the current block based on syntax elements included in the video bitstream. The first plurality of coding tools includes a deblocking filter. The second plurality of coding tools includes at least one of a sample adaptive offset filter, intra-frame subdivision, and matrix-based intra-frame prediction. The processing circuit system reconstructs the current block based on the first plurality of coding tools being enabled and the second plurality of coding tools being disabled.

Description

[0001] Incorporation by reference

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 466,910, filed on September 3, 2021, "METHOD AND APPARATUS FOR VIDEO CODING FOR MACHINE," which claims the benefit of priority to U.S. Provisional Application No. 63 / 129,496, filed on December 22, 2020, "ON VIDEO CODING FOR MACHINE." The entire disclosures of these prior applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure describes implementations generally related to machine-oriented video decoding, and more particularly to a video decoding method, a computer device, an apparatus, and a computer-readable storage medium. Background Art

[0004] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent that the work of the presently named inventors is described in this background section, the work of the presently named inventors and aspects of the description that may not otherwise be considered prior art at the time of filing are neither explicitly nor implicitly admitted as prior art against the present disclosure.

[0005] In recent years, with the rise of machine learning applications and the abundance of sensors, many intelligent platforms use videos for machine vision tasks such as object detection, segmentation or tracking. How to encode videos or images for machine tasks has become an interesting and challenging problem. This has led to the introduction of machine-oriented video decoding (Video Coding for Machine, VCM) research. To achieve this goal, the international standards organization MPEG created a special organization "Video Coding for Machine (VCM)" to standardize related technologies to achieve better interoperability between different devices.

[0006] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. An uncompressed digital video may include a series of pictures, each with a spatial size of, for example, 1920×1080 luma samples and associated chroma samples. The series of pictures may have a fixed or variable picture rate (also informally referred to as a frame rate), such as 60 pictures per second or 60 Hz. Uncompressed video has significant bit rate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luma sample resolution at 60 Hz frame rate) requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 GBytes of storage space.

[0007] One purpose of video encoding and decoding can be to reduce the redundancy of the input video signal by compression. Compression can help reduce the above-mentioned bandwidth requirements or storage space requirements, in some cases by two orders of magnitude or more. Both lossless compression and lossy compression and their combinations can be used. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal based on the compressed original signal. When lossy compression is used, the reconstructed signal may be different from the original signal, but the distortion between the original signal and the reconstructed signal is small enough to enable the reconstructed signal to be used for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect that higher allowable / tolerable distortion can produce higher compression ratios.

[0008] Video encoders and video decoders may utilize techniques from several broad categories including, for example, motion compensation, transforms, quantization, and entropy coding.

[0009] Video codec techniques may include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures are spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-frame mode, the picture may be an intra-frame picture. Intra-frame pictures and their derivatives (e.g., independent decoder refresh pictures) may be used to reset decoder states, and may therefore be used as the first picture in an encoded video bitstream and video session or as a still image. Samples of intra-frame blocks may be subjected to transformation, and transform coefficients may be quantized before entropy coding. Intra-frame prediction may be a technique for minimizing sample values ​​in a pre-transform domain. In some cases, the smaller the DC value after transformation, and the smaller the AC coefficient, the fewer bits are required to represent the block after entropy coding at a given quantization step size.

[0010] Conventional intra-frame coding, such as that known from MPEG-2 generation coding techniques, does not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to do so based on metadata and / or surrounding sample data obtained during encoding and / or decoding of data blocks that are spatially adjacent and preceding in decoding order. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and not reference data from reference pictures.

[0011] There may be many different forms of intra-frame prediction. When more than one such technique may be used in a given video coding technique, the technique in use may be encoded in intra-frame prediction mode. In some cases, a mode may have sub-modes and / or parameters, and these sub-modes and / or parameters may be encoded separately or included in a mode codeword. Which codeword is used for a given mode, sub-mode and / or parameter combination may affect the coding efficiency gain through intra-frame prediction, and therefore the entropy coding technique used to convert the codeword into a bitstream may also affect it.

[0012] Some modes of intra prediction were introduced with H.264, refined in H.265, and further refined in newer coding techniques such as the joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). Neighboring sample values ​​belonging to samples that are already available can be used to form a prediction block. Sample values ​​of neighboring samples are copied into the prediction block according to the direction. A reference to the direction in use can be encoded in the bitstream, or it can itself be predicted. Summary of the invention

[0013] According to an embodiment, a method for video decoding is provided, characterized in that the method includes: decoding prediction information of a current block in a current picture that is part of a video bitstream; determining, for the current block, based on syntax elements included in the video bitstream, to enable a first plurality of encoding tools and to disable a second plurality of encoding tools, wherein the first plurality of encoding tools includes a deblocking filter encoding tool, and the second plurality of encoding tools includes at least one of a sample adaptive offset filter (SAO) encoding tool, an intra sub-partitioning (ISP) encoding tool, and a matrix-based intra prediction (MIP) encoding tool; and reconstructing the current block based on the first plurality of encoding tools being enabled and the second plurality of encoding tools being disabled, wherein one of the first plurality of encoding tools and the second plurality of encoding tools includes both a multiple transform set (MTS) encoding tool and a low frequency non-separable transform (LFNST) encoding tool.

[0014] According to an embodiment, a computer device is provided, characterized in that the computer device includes: one or more computer-readable non-transitory storage media, which are configured to store computer program codes; and one or more computer processors, which are configured to access the computer program codes and execute the above-mentioned video decoding method according to the instructions of the computer program codes.

[0015] According to an embodiment, a device is provided, characterized in that the device includes: a processing circuit system, the processing circuit system is configured to: decode prediction information of a current block in a current picture as part of a video bitstream; determine to enable a first plurality of encoding tools and disable a second plurality of encoding tools for the current block based on syntax elements included in the video bitstream, the first plurality of encoding tools including a deblocking filter encoding tool, the second plurality of encoding tools including at least one of a sample adaptive offset filter (SAO) encoding tool, an intra sub-partitioning (ISP) encoding tool, and a matrix-based intra prediction (MIP) encoding tool; and reconstruct the current block based on the first plurality of encoding tools being enabled and the second plurality of encoding tools being disabled, wherein one of the first plurality of encoding tools and the second plurality of encoding tools includes both a multiple transform set (MTS) encoding tool and a low frequency non-separable transform (LFNST) encoding tool.

[0016] According to an embodiment, a non-transitory computer-readable storage medium is provided, which stores instructions, which, when executed by at least one processor, cause the at least one processor to perform the above-mentioned video decoding method.

[0017] According to the video decoding method, computer device, apparatus and computer-readable storage medium of the present invention, taking VVC as an example, a method for selecting a subset of coding tools in a video coding standard for machine vision is described. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0019] FIG1A is a schematic illustration of an exemplary subset of intra prediction modes;

[0020] FIG1B is a diagram of exemplary intra prediction directions;

[0021] FIG1C is a schematic diagram of a current block and its surrounding spatial merging candidates in an example;

[0022] Figure 2 is a schematic illustration of a simplified block diagram of a communication system according to an embodiment;

[0023] Figure 3 is a schematic illustration of a simplified block diagram of a communication system according to another embodiment;

[0024] Figure 4 is a schematic illustration of a simplified block diagram of a decoder according to an embodiment;

[0025] Figure 5 is a schematic illustration of a simplified block diagram of an encoder according to an embodiment;

[0026] Figure 6 shows a block diagram of an encoder according to another embodiment;

[0027] Figure 7 shows a block diagram of a decoder according to another embodiment;

[0028] Figure 8 An exemplary architecture for use in video decoding for machines (VCM) according to an embodiment is shown;

[0029] Fig. 9 An exemplary encoder for use in Versatile Video Coding (VVC) according to an embodiment is shown;

[0030] Fig.10 shows an exemplary flow chart according to an embodiment; and

[0031] Fig.11 is a schematic illustration of a computer system according to an embodiment. DETAILED DESCRIPTION

[0032] 1A, a subset of nine prediction directions known from the 33 possible prediction directions of H.265 (corresponding to 33 angular modes in 35 intra modes) is depicted at the bottom right. The point (101) where the arrows converge represents the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted based on one or more samples at an angle of 45 degrees to the horizontal at the top right. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more samples at an angle of 22.5 degrees to the horizontal at the bottom left of sample (101).

[0033] Still referring to FIG. 1A , a square block (104) of 4×4 samples is depicted at the upper left (indicated by the bold dashed line). The square block (104) includes 16 samples, each of which is labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in both the Y and X dimensions in block (104). Since the size of the block is 4×4 samples, S44 is at the lower right. Also shown are reference samples that follow a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index), and X position (column index) relative to the block (104). In both H.264 and H.265, the prediction samples are adjacent to the block being reconstructed; therefore, there is no need to use negative values.

[0034] Intra-picture prediction can work by appropriately copying reference sample values ​​from neighboring samples according to the prediction direction indicated by the signal. For example, assume that the encoded video bitstream includes signaling that, for this block, the prediction direction is consistent with arrow (102) - that is, the sample is predicted based on one or more prediction samples at a 45 degree angle to the horizontal to the upper right. In this case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Sample S44 is then predicted based on reference sample R08.

[0035] In some cases, the values ​​of multiple reference samples may be combined, such as by interpolation, in order to calculate the reference sample; in particular, when the direction is not divisible by 45 degrees.

[0036] As video coding technology has developed, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. The number of possible directions increased to 33 in H.265 (2013), and when disclosed JEM / VVC / BMS can support up to 65 directions. Experiments have been conducted to identify the most likely directions, and certain techniques in entropy coding are used to represent these possible directions with a small number of bits, accepting certain penalties for less likely directions. In addition, the direction itself can sometimes be predicted based on the neighboring directions used in adjacent decoded blocks.

[0037] FIG. 1B shows a schematic diagram ( 105 ) depicting 65 intra prediction directions according to JEM to illustrate that the number of prediction directions increases over time.

[0038] The mapping of intra-prediction direction bits representing directions in the coded video bitstream may vary from one video coding technique to another; and the mapping may range, for example, from a simple direct mapping of prediction directions to intra-prediction modes, to codewords, to complex adaptive schemes involving most probable modes, and the like. However, in all cases, there may be some directions that are statistically less likely to appear in the video content than some other directions. Since the goal of video compression is to reduce redundancy, in a well-functioning video coding technique, those less probable directions will be represented by a larger number of bits than more probable directions.

[0039] Motion compensation may be a lossy compression technique and may involve the following technique: a block of sample data from a previously reconstructed picture or portion thereof (reference picture) is used to predict a newly reconstructed picture or picture portion after being spatially shifted in a direction indicated by a motion vector (hereinafter referred to as MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference picture in use (the third dimension may indirectly be a temporal dimension).

[0040] In some video compression techniques, an MV applicable to a particular region of sample data can be predicted based on other MVs, for example, based on an MV of another region of sample data that is spatially adjacent to the region being reconstructed and that precedes the MV in decoding order. Doing so can significantly reduce the amount of data required to encode the MV, thereby eliminating redundancy and increasing compression. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera (called natural video), there is a statistical possibility that a larger region than the region to which a single MV applies moves in a similar direction, and therefore in some cases similar MVs derived from MVs of adjacent regions can be used to predict the larger region. This makes the MV obtained for a given region similar or identical to the MV predicted from the surrounding MVs, and in turn, the MV can be represented with a smaller number of bits after entropy coding than would be used if the MV was encoded directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from the original signal (i.e., a sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating a predictor based on several surrounding MVs.

[0041] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Recommendation H.265, "High Efficiency Video Coding", December 2016). This article describes a technique referred to as "spatial merging" among the various MV prediction mechanisms provided by H.265.

[0042] 1C, the current block (111) may include samples obtained by the encoder during a motion search process that can be predicted from a previous block of the same size that has been spatially shifted. Instead of encoding the MV directly, the MV associated with any of the five surrounding samples represented by A0, A1 and B0, B1, B2 (corresponding to 112 to 116, respectively) can be used to derive the MV from metadata associated with one or more reference pictures, for example, from the most recent (in decoding order) reference picture. In H.265, MV prediction can use a predictor from the same reference picture that is also being used by neighboring blocks.

[0043] I. Video Decoder and Encoder Systems

[0044] Figure 2A simplified block diagram of a communication system (200) according to an embodiment of the present disclosure is shown. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and (220) interconnected via the network (250). Figure 2 In the example of , a first pair of terminal devices (210) and (220) perform unidirectional transmission of data. For example, the terminal device (210) can encode video data (e.g., a video picture stream captured by the terminal device (210)) for transmission to another terminal device (220) via the network (250). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. The terminal device (220) can receive the encoded video data from the network (250), decode the encoded video data to recover the video picture, and display the video picture based on the recovered video data. Unidirectional data transmission can be common in media service applications and the like.

[0045] In another example, the communication system (200) includes a second pair of terminal devices (230) and (240) that perform bidirectional transmission of encoded video data that may occur, for example, during a video conference. For the bidirectional transmission of data, in the example, each of the terminal devices (230) and (240) can encode video data (e.g., a video picture stream captured by the terminal device) for transmission to the other of the terminal devices (230) and (240) via the network (250). Each of the terminal devices (230) and (240) can also receive the encoded video data transmitted by the other of the terminal devices (230) and (240), and can decode the encoded video data to restore the video picture, and can display the video picture at an accessible display device based on the restored video data.

[0046] exist Figure 2In the example of , terminal devices (210), (220), (230) and (240) can be shown as servers, personal computers and smart phones, but the principles of the present disclosure may not be so limited. Implementations of the present disclosure are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. Network (250) represents any number of networks that transmit encoded video data between terminal devices (210), (220), (230) and (240), including, for example, wired (wired) and / or wireless communication networks. Communication network (250) can exchange data in circuit switching channels and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purposes of this discussion, unless otherwise specified below, the architecture and topology of network (250) may be irrelevant to the operation of the present disclosure.

[0047] As examples of applications of the disclosed subject matter, Figure 3 Arrangements of video encoders and video decoders in a streaming environment are shown. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0048] The streaming system may include a capture subsystem (313) that may include a video source (301), such as a digital camera, that creates, for example, an uncompressed video picture stream (302). In an example, the video picture stream (302) includes samples captured by the digital camera. The video picture stream (302) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (304) (or encoded video bitstream), which may be processed by an electronic device (320) coupled to the video source (301) and including a video encoder (303). The video encoder (303) may include hardware, software, or a combination thereof to implement or embody aspects of the disclosed subject matter as described in more detail below. The encoded video data (304) (or encoded video bitstream (304)) is depicted as a thin line to emphasize the lower amount of data when compared to the video picture stream (302), and the encoded video data (304) can be stored on the streaming server (305) for future use. One or more streaming client subsystems (e.g. Figure 3The client subsystems (306) and (308) in the video transmission system can access the streaming server (305) to retrieve copies (307) and (309) of the encoded video data (304). The client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and creates an outgoing video picture stream (311) that can be presented on a display (312) (e.g., a display screen) or another presentation device (not depicted). In some streaming transmission systems, the encoded video data (304), (307) and (309) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of these standards include ITU-T Recommendation H.265. In the example, the developing video coding standard is informally referred to as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0049] Note that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0050] Figure 4 A block diagram of a video decoder (410) according to an embodiment of the present disclosure is shown. The video decoder (410) may be included in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used to replace Figure 3 A video decoder (310) in an example of FIG.

[0051] A receiver (431) may receive one or more encoded video sequences to be decoded by a video decoder (410); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data and other data such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not depicted). The receiver (431) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and an entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other applications, the buffer memory (415) may be external to the video decoder (410) (not depicted). In still other applications, there may be a buffer memory (not depicted) external to the video decoder (410) to, for example, prevent network jitter, and further there may be another buffer memory (415) internal to the video decoder (410) to, for example, handle playout timing. When the receiver (431) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (415) may not be required, or the buffer memory (415) may be smaller. For use over a best effort packet network such as the Internet, the buffer memory (415) may be required, may be relatively large and may advantageously have an adaptive size, and may be implemented at least in part in an operating system or similar element (not depicted) external to the video decoder (410).

[0052] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. The categories of these symbols include: information for managing the operation of the video decoder (410), and may include information for controlling a presentation device such as a presentation device (412) (e.g., a display screen), which is not part of the electronic device (430) but can be coupled to the electronic device (430), such as Figure 4As shown. The control information for (one or more) rendering devices may take the form of a Supplemental Enhancement Information (SEI message) or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser (420) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may comply with a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set for at least one subgroup of the pixel subgroups in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include: Group of Pictures (GOP), picture, tile, slice, macroblock, Coding Unit (CU), block, Transform Unit (TU), Prediction Unit (PU), etc. The parser (420) can also extract information such as transform coefficients, quantizer parameter values, MVs, etc. from the encoded video sequence.

[0053] The parser (420) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415), thereby creating symbols (421).

[0054] The reconstruction of the symbol (421) may involve a number of different units depending on the type of the coded video picture or portion thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser (420). For the sake of brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not depicted.

[0055] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into a number of functional units as described below. In a practical implementation working under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0056] The first unit is a sealer / inverse transform unit (451). The sealer / inverse transform unit (451) receives quantized transform coefficients as symbol(s) (421) from the parser (420) and control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (451) may output blocks including sample values, which may be input into an aggregator (455).

[0057] In some cases, the output samples of the scaler / inverse transform (451) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) generates a block of the same size and shape as the block being reconstructed using surrounding reconstructed information obtained from a current picture buffer (458). For example, the current picture buffer (458) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (455) adds the prediction information already generated by the intra-prediction unit (452) to the output sample information as provided by the scaler / inverse transform unit (451) on a per-sample basis.

[0058] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to a block that has been inter-coded and possibly motion compensated. In such a case, the motion compensated prediction unit (453) may access the reference picture memory (457) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (421) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (451) (in this case referred to as residual samples or residual signals) by the aggregator (455), thereby generating output sample information. The address from which the motion compensated prediction unit (453) obtains the predicted samples in the reference picture memory (457) may be controlled by the MV, which may be obtained by the motion compensated prediction unit (453) in the form of symbols (421), which may have, for example, an X component, a Y component and a reference picture component. Motion compensation may also include interpolation of sample values ​​obtained from the reference picture memory (457) when sub-sample accurate MVs are used, MV prediction mechanisms, etc.

[0059] The output samples of the aggregator (455) may be subjected to various loop filtering techniques in a loop filter unit (456). The video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video sequence (also referred to as a coded video bitstream) and obtained by the loop filter unit (456) as symbols (421) from the parser (420), but the in-loop filter techniques may also be responsive to meta-information obtained during decoding of a previous portion (in decoding order) of the coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.

[0060] The output of the loop filter unit (456) may be a sample stream that may be output to a rendering device (412) and stored in a reference picture memory (457) for future inter-picture prediction.

[0061] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (420)), the current picture buffer (458) may become part of the reference picture memory (457) and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0062] The video decoder (410) may perform decoding operations according to a predetermined video compression technique in a standard such as ITU-T H.265 Recommendation. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the encoded video sequence follows both the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as tools available only under the profile. For compliance, it is also required that the complexity of the encoded video sequence is within the limits defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (measured in, for example, millions of samples per second), the maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0063] In an embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence (one or more). The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0064] Figure 5 A block diagram of a video encoder (503) according to an embodiment of the present disclosure is shown. The video encoder (503) is included in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit system). The video encoder (503) can be used to replace Figure 3 A video encoder (303) in an example.

[0065] The video encoder (503) can be used to obtain the video source (501) (not Figure 5 In an example of an electronic device (520) receiving video samples, a video source (501) can capture (one or more) video images to be encoded by a video encoder (503). In another example, the video source (501) is a part of the electronic device (520).

[0066] The video source (501) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (503), the digital video sample stream may have any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (501) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (501) may be a camera device that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The picture itself may be organized as a spatial pixel array, wherein each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on samples.

[0067] According to an embodiment, the video encoder (503) can encode and compress the pictures of the source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of the controller (550). In some embodiments, the controller (550) controls other functional units as described below and is functionally coupled to the other functional units. The coupling is not depicted for simplicity. The parameters set by the controller (550) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology...), picture size, picture group (GOP) layout, maximum MV allowed reference area, etc. The controller (550) can be configured to have other suitable functions that are related to the video encoder (503) optimized for a specific system design.

[0068] In some embodiments, the video encoder (503) is configured to operate in an encoding loop. As an extremely simplified description, in an example, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and (one or more) reference pictures) and a (local) decoder (533) embedded in the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to the way the (remote) decoder creates sample data (because in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (534). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (534) are also bit-accurate between the local encoder and the remote encoder. In other words, the prediction part of the encoder "sees" sample values ​​that are exactly the same as the sample values ​​that the decoder "sees" when using prediction during decoding as reference picture samples. This basic principle of reference picture synchronization (and the resulting offset if synchronization cannot be maintained, e.g. due to channel errors) is also used in some related techniques.

[0069] The operation of the "local" decoder (533) can be combined with the "remote" decoder such as has been described above. Figure 4 The operation of the video decoder (410) described in detail is the same. However, reference is briefly made to Figure 4 , since the symbols are available and the encoding of the symbols into an encoded video sequence by the entropy encoder (545) and the decoding of the symbols by the parser (420) can be lossless, the entropy decoding portion of the video decoder (410) including the buffer memory (415) and the parser (420) may not be fully implemented in the local decoder (533).

[0070] At this point, it can be observed that any decoder technology other than the parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on the decoder operation. The description of the encoder technology can be simplified because the encoder technology is contrary to the decoder technology described comprehensively. A more detailed description is only needed in some aspects, and the description is provided below.

[0071] In some examples, during operation, the source encoder (530) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (532) encodes the differences between pixel blocks of an input picture and pixel blocks of reference picture(s) that may be selected as prediction reference(s) for the input picture.

[0072] The local video decoder (533) can decode the encoded video data of the picture that can be designated as the reference picture based on the symbol created by the source encoder (530). The operation of the encoding engine (532) can advantageously be a lossy process. When the encoded video data can be decoded at the video decoder ( Figure 5 When the video encoder (503) is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture cache (534). In this way, the video encoder (503) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture that will be obtained by the remote video decoder.

[0073] The predictor (535) may perform a prediction search for the encoding engine (532). That is, for a new picture to be encoded, the predictor (535) may search the reference picture memory (534) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture MVs, block shapes, etc., that may be used as suitable prediction references for the new picture. The predictor (535) may operate on a sample-block-by-pixel-block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (535), the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (534).

[0074] The controller (550) can manage encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0075] The outputs of all the above-mentioned functional units may be subjected to entropy coding in an entropy encoder (545). The entropy encoder (545) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0076] The transmitter (540) can buffer the encoded video sequence(s) created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which can be a hardware / software link to a storage device storing the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0077] The controller (550) may manage the operation of the video encoder (503). During encoding, the controller (550) may assign a certain coded picture type to each coded picture, which may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned one of the following picture types:

[0078] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of those variations of I pictures and their corresponding applications and features.

[0079] A predictive picture (P picture) may be a picture that can be encoded and decoded using inter prediction or intra prediction to predict sample values ​​of each block using at most one MV and a reference index.

[0080] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using inter-prediction or intra-prediction, which uses up to two MVs and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0081] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively) and coded block by block. The blocks may be predictively coded with reference to other (already coded) blocks, which are determined by the coding allocation applied to the corresponding picture of the block. For example, blocks of an I picture may be non-predictively coded, or may be predictively coded (spatial prediction or intra-prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0082] The video encoder (503) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In its operation, the video encoder (503) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0083] In an embodiment, the transmitter (540) may transmit additional data along with the encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0084] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified as intra-prediction) exploits spatial correlation in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In the example, a particular picture being encoded / decoded, which is referred to as the current picture, is divided into blocks. In the case where a block in the current picture is similar to a reference block in a reference picture that has been previously encoded and buffered in the video, the block in the current picture can be encoded by a vector called an MV. The MV points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the MV may have a third dimension that identifies the reference picture.

[0085] In some embodiments, bidirectional prediction techniques may be used for inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first MV pointing to a first reference block in the first reference picture and a second MV pointing to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.

[0086] In addition, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0087] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, a picture in a video picture sequence is divided into coding tree units (CTUs) for compression, and the CTUs in the picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), namely a luminance CTB and two chrominance CTBs. Each CTU can be recursively divided into one or more coding units (CUs) in a quadtree. For example, a 64×64 pixel CTU can be divided into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In the example, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. The CU is divided into one or more prediction units (PUs) based on temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in the codec (encoding / decoding) is performed in units of prediction blocks. Using the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values) such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0088] Figure 6 A diagram of a video encoder (603) according to another embodiment of the present disclosure is shown. The video encoder (603) is configured to receive a processed block (e.g., a predicted block) of sample values ​​within a current video picture in a sequence of video pictures, and encode the processed block into an encoded picture that is part of an encoded video sequence. In an example, the video encoder (603) is used instead of Figure 3 A video encoder (303) in an example.

[0089] In the HEVC example, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (603) uses, for example, rate-distortion optimization to determine whether to use intra mode, inter mode, or bidirectional prediction mode to best encode the processing block. In the case where the processing block is to be encoded in intra mode, the video encoder (603) may encode the processing block into an encoded picture using intra prediction techniques; and in the case where the processing block is to be encoded in inter mode or bidirectional prediction mode, the video encoder (603) may encode the processing block into an encoded picture using inter prediction or bidirectional prediction techniques, respectively. In some video coding techniques, the merge mode may be an inter-picture prediction submode, in which the MV is derived from the predictor without the aid of an encoded MV component external to one or more MV predictors. In some other video coding techniques, there may be an MV component applicable to the object block. In the example, the video encoder (603) includes other components, such as a mode decision module (not shown) that determines the mode of the processing block.

[0090] exist Figure 6 In the example of FIG. 6 , the video encoder ( 603 ) includes: Figure 6 An inter-frame encoder (630), an intra-frame encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), an overall controller (621), and an entropy encoder (625) are shown coupled together.

[0091] The inter-frame encoder (630) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-frame prediction information (e.g., description of redundant information, MV, merge mode information according to an inter-frame coding technique), and calculate an inter-frame prediction result (e.g., a prediction block) based on the inter-frame prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the encoded video information.

[0092] The intra encoder (622) is configured to: receive samples of a current block (e.g., a processing block); compare the block with an already encoded block in the same picture in some cases; generate quantization coefficients after transformation; and in some cases also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In an example, the intra encoder (622) also calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same picture.

[0093] The overall controller (621) is configured to determine overall control data and control other components of the video encoder (603) based on the overall control data. In an example, the overall controller (621) determines a mode of a block and provides a control signal to a switch (626) based on the mode. For example, when the mode is an intra-frame mode, the overall controller (621) controls the switch (626) to select an intra-frame mode result for use by a residual calculator (623), and controls the entropy encoder (625) to select intra-frame prediction information and include the intra-frame prediction information in a bitstream; and when the mode is an inter-frame mode, the overall controller (621) controls the switch (626) to select an inter-frame prediction result for use by a residual calculator (623), and controls the entropy encoder (625) to select inter-frame prediction information and include the inter-frame prediction information in a bitstream.

[0094] The residual calculator (623) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra-frame encoder (622) or the inter-frame encoder (630). The residual encoder (624) is configured to operate based on the residual data to encode the residual data to generate a transform coefficient. In an example, the residual encoder (624) is configured to convert the residual data from the spatial domain to the frequency domain and generate a transform coefficient. The transform coefficient is then subjected to quantization to obtain a quantized transform coefficient. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra-frame encoder (622) and the inter-frame encoder (630). For example, the inter encoder (630) can generate a decoded block based on the decoded residual data and the inter prediction information, and the intra encoder (622) can generate a decoded block based on the decoded residual data and the intra prediction information. In some examples, the decoded blocks are appropriately processed to generate decoded pictures, and these decoded pictures can be buffered in a memory circuit (not shown) and used as reference pictures.

[0095] The entropy encoder (625) is configured to format the bitstream to include the encoded blocks. The entropy encoder (625) is configured to include various information according to a suitable standard, such as HEVC. In an example, the entropy encoder (625) is configured to include overall control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Note that according to the disclosed subject matter, when the block is encoded in inter-frame mode or the merge sub-mode of the bidirectional prediction mode, there is no residual information.

[0096] Figure 7A diagram of a video decoder (710) according to another embodiment of the present disclosure is shown. The video decoder (710) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In an example, the video decoder (710) is used instead of Figure 3 A video decoder (310) in an example of FIG.

[0097] exist Figure 7 In the example of FIG. 7 , the video decoder ( 710 ) includes Figure 7 An entropy decoder (771), an inter-frame decoder (780), a residual decoder (773), a reconstruction module (774), and an intra-frame decoder (772) are shown coupled together.

[0098] The entropy decoder (771) can be configured to reconstruct certain symbols from the encoded picture, which represent syntax elements constituting the encoded picture. Such symbols may include, for example, a mode for encoding a block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can identify certain samples or metadata used by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, such as residual information in the form of quantized transform coefficients, etc. In an example, when the prediction mode is an inter-frame mode or a bidirectional prediction mode, the inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is an intra-frame prediction type, the intra-frame prediction information is provided to the intra-frame decoder (772). The residual information can be subjected to inverse quantization and provided to the residual decoder (773).

[0099] The inter-frame decoder (780) is configured to receive the inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information.

[0100] The intra decoder (772) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0101] The residual decoder (773) is configured to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require certain control information (to include quantizer parameters (QuantizerParameter, QP)), and the information may be provided by the entropy decoder (771) (the data path is not depicted because this may only be a small amount of control information).

[0102] The reconstruction module (774) is configured to combine the residual output by the residual decoder (773) with the prediction result (output by the inter-frame prediction module or the intra-frame prediction module as appropriate) in the spatial domain to form a reconstructed block, which can be part of a reconstructed picture, which can be part of a reconstructed video. Note that other suitable operations such as deblocking operations can be performed to improve visual quality.

[0103] Note that the video encoders (303), (503) and (603) and the video decoders (310), (410) and (710) may be implemented using any suitable technology. In an embodiment, the video encoders (303), (503) and (603) and the video decoders (310), (410) and (710) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (303), (503) and (603) and the video decoders (310), (410) and (710) may be implemented using one or more processors that execute software instructions.

[0104] II. Machine-oriented video decoding

[0105] Videos or images can be used by humans for various purposes, such as entertainment, education, etc. Therefore, video coding or image coding usually exploits the characteristics of the human visual system to obtain better compression efficiency while maintaining good subjective quality.

[0106] In recent years, with the rise of machine learning applications and equipped with a large number of sensors, many intelligent platforms use videos or images for machine vision tasks such as object detection, segmentation or tracking. How to encode videos or images for use in machine vision tasks can be an interesting and challenging problem. This has led to the introduction of machine-oriented video decoding (VCM) research. To achieve this goal, the international standards organization MPEG created a special organization "Video Decoding for Machines (VCM)" to standardize related technologies to achieve better interoperability between different devices.

[0107] Figure 8 An exemplary VCM architecture according to an embodiment of the present disclosure is shown. Note that Figure 8 The output of the video decoding module in is mainly used for machine use, that is, machine vision. However, in some cases, the output of the video decoding module can also be used for human vision, such as Figure 8 Additionally, an interface for a neural network (NN) module may be included in the exemplary VCM architecture.

[0108] III. General Video Coding

[0109] VVC was recently developed jointly by two international standard organizations, namely ITU and ISO / IEC. A version of VVC was completed in July 2020 and is one of the most advanced video coding standards.

[0110] Fig. 9 An exemplary VVC encoder according to an embodiment of the present disclosure is shown. The VVC encoder includes a transform and quantization (TR+Q) module, an entropy coding module, a motion estimation (ME) module and a decoder. The decoder includes an inverse transform and inverse quantization (iTR+iQ) module, an intra-frame prediction module, an inter-frame prediction module, a combined prediction module, a deblocking filter (Deblk) module, a sample adaptive offset filter (SAO) module, an adaptive loop filter (ALF) module and a buffer. The encoder receives a coding block (CB) of an input picture and outputs an encoded bitstream and a reconstructed picture. The transform and quantization module and the inverse transform and inverse quantization module both process the transform block (TB). The intra-frame prediction module and the inter-frame prediction module both process the PB. The combined prediction module combines inter-frame prediction and intra-frame prediction.

[0111] For intra prediction, various coding tools may be included, for example, cross-component linear mode (CCLM), multiple reference line prediction (MRLP), intra sub-partitioning (ISP), matrix-based intra prediction (MIP), etc.

[0112] For inter-frame prediction, a collection of coding tools may be included, such as affine motion model (AFFINE), subblock-based temporal merging candidate (SbTMC), adaptive motion vector resolution (AMVR), geometric partition mode (GPM), combined intra / inter prediction (CIIP), merge mode with motion vector difference (MMVD), bi-predictive with CU weight (BCW), decoder motion vector refinement (DMVR), bi-directional optical flow (BDOF), prediction refinement using optical flow (PROF), etc.

[0113] Other tools such as transforms, quantization, and in-loop filters may be included, for example, dependent quantization (DQ), multiple transform sets (MTS), low frequency non-separable transform (LFNST), luma mapping and chroma scaling (LMCS), cross-component adaptive loop filter (CCALF), adaptive loop filter (ALF), sample adaptive offset filter (SAO), deblocking filter (Deblk), etc.

[0114] Note that other video coding standards (such as H.264 or HEVC) may use a subset of the above-mentioned but modified coding tools. For example, HEVC uses SAO, Deblk, intra-frame prediction and inter-frame prediction, multiple transform sets, etc. However, except for the SAO filter, other coding tools (such as Deblk filter, intra-frame prediction and inter-frame prediction, multiple transform sets) are not the same as those used in VVC, although the design principles are the same.

[0115] IV. Selection of Coding Tools for Machine Vision

[0116] In some related examples, video coding standards (e.g., H.264, HEVC, or VVC) utilize a collection of coding tools designed for human use and optimized for the human vision system (HVS). However, these coding tools may not be suitable for machine vision, which may exploit different characteristics of video signals.

[0117] Aspects of the present disclosure include machine-oriented video decoding. In the present disclosure, using VVC as an example, a method of selecting a subset of coding tools in a video coding standard for machine vision is included.

[0118] Note that the methods included in the present disclosure are not limited to the VVC standard. The principles of the methods can also be applied to other video coding standards, such as H.264, HEVC, or AV1 developed by the Alliance for Open Media.

[0119] In some related examples, coding tools in a coding standard (eg, VVC) are selected based on a trade-off between factors such as subjective performance, bit rate, and implementation complexity.

[0120] For machine vision, the selection criteria can be modified to reflect trade-offs between factors such as machine vision performance, bitrate, and implementation complexity. Machine vision performance refers to the performance of various machine tasks, such as mean average precision (mAP) for object detection and segmentation, multiple objects tracking accuracy (MOTA) for object tracking, etc.

[0121] Therefore, the coding tools in video coding standards (such as VVC) can be divided into several categories. For example, the coding tools in category 1 mainly benefit human vision and have a small adverse effect on machine vision. The coding tools in category 2 mainly benefit human vision and have a small positive effect on machine vision. The coding tools in category 3 mainly benefit human vision and have a significant adverse effect on machine vision. The coding tools in category 4 benefit both human vision and machine vision.

[0122] In an example such as the VVC standard, SAO and ISP may belong to category 1, ALF and MIP may belong to category 2, LFNST may belong to category 3, and MTS, LMCS, and Deblk may belong to category 4. Among these coding tools, Deblk is an important tool for both human vision and machine vision. Experiments show that if Deblk is turned off, the mAPBD rate performance of object detection drops by more than 10%.

[0123] According to aspects of the present disclosure, one or more categories may be turned on, while one or more other categories may be turned off. For example, in the case of VCM, coding tools in categories 1 and 3 may be turned off, and coding tools in categories 2 and 4 may be turned on. Thus, a new subset of coding tools in a video coding standard (e.g., VVC) may form a new codec for machine vision.

[0124] According to aspects of the present disclosure, coding tools in category 2 can be further divided into two subcategories: relatively low complexity coding tools (e.g., MIP) and relatively high complexity coding tools (e.g., ALF). Thus, in one embodiment, coding tools in categories 1 and 3 can be turned off, coding tools in category 4 can be turned on, coding tools with relatively low complexity in category 2 can be turned on, and coding tools with relatively high complexity in category 2 can be turned off. In this embodiment, complexity is traded off against a slight performance loss in machine vision.

[0125] V. Interactions between Coding Tools

[0126] In some related examples, coding tools interact in related coding standards for human vision. According to aspects of the present disclosure, similar interactions can also be shown in experiments for machine vision.

[0127] In one embodiment, an experiment for object detection shows that turning off ALF and Deblk may cause mAP BD rate performance to drop by 0.8% and 14.8% respectively. However, if both ALF and Deblk are turned off, the mAP BD rate performance may drop by more than 41%.

[0128] In one embodiment, an experiment for object detection shows that turning off MTS may cause the mAP BD rate performance to drop by 1.8%, while turning off LFNST may improve the mAP BD rate performance by 2.5%. However, if both MTS and LFNST are turned off, the overall impact becomes negative. That is, the mAP BD rate performance may drop by 7%. In this case, the overall encoding time may be reduced by more than 42%.

[0129] In one embodiment, an experiment for object detection shows that turning off MIP may cause a 0.4% decrease in mAP BD rate performance, while turning off ISP can improve mAP BD rate performance by 0.8%. However, if both ISP and MIP are turned off, the overall mAP BD rate performance can be improved by 2.4%. In this case, the overall encoding time can be reduced by more than 25%.

[0130] Therefore, based on, for example, the above observations, when selecting a subset of encoding tools for machine vision, the interactions between encoding tools can be considered. In one example, if the combined encoding tools can improve the overall machine vision performance, then the combination of encoding tools can be turned on, such as the combination of MTS and LFNST. In another example, if the combined encoding tools may reduce the overall machine vision performance, then the combination of encoding tools can be turned off, such as the combination of ISP and MIP.

[0131] In one embodiment, a trade-off between implementation complexity and machine vision performance may be considered. For example, a combination of coding tools may be turned off as long as turning off the combination of coding tools may result in a significant reduction in complexity, even if turning off the combined coding tools results in a significant loss in machine vision performance. For example, a combination of MTS and LFNST may be turned off to reduce implementation complexity.

[0132] VI. Syntax Changes for Machine Vision

[0133] According to aspects of the present disclosure, syntax elements in prediction information of a video bitstream may be used to enable a first set of coding tools for machine vision and / or disable a second set of coding tools. In some embodiments, to select a set of coding tools for machine vision, syntax elements such as flags (e.g., sps_vcm_enable_flag) may be inserted into a syntax table in the VVC specification. For example, the syntax table for a sequence parameter set (SPS) may be modified, as shown in Table 1.

[0134] Table 1

[0135]

[0136] In Table 1, the syntax element sps_vcm_enable_flag is used to enable a first set of coding tools for machine vision and / or disable a second set of coding tools. In some embodiments, sps_vcm_enable_flag=1 indicates that the first set of coding tools for machine vision is enabled and / or the second set of coding tools is disabled. In addition, the syntax elements sps_mts_enabled_flag, sps_lfnst_enabled_flag, sps_alf_enabled_flag, sps_sao_enabled_flag, sps_isp_enabled_flag, and sps_mip_enabled_flag are used to enable and disable MTS, LFNST, ALF, SAO, ISP, and MIP coding tools, respectively. In some embodiments, for each of these syntax elements, a value of 1 is set to enable the corresponding coding tool, and a value of 0 is set to disable the corresponding coding tool. For example, sps_mts_enabled_flag=1 indicates that the MTS coding tool is enabled, and sps_mts_enabled_flag=0 indicates that the MTS coding tool is disabled.

[0137] Turning the deblocking filter on and off may be indicated by a syntax element not shown in Table 1. In some embodiments, the syntax element for turning the deblocking filter on and off may be a picture parameter set (PPS) flag (e.g., pps_deblocking_filter_disabled_flag). When pps_deblocking_filter_disabled_flag=0, the deblocking filter is turned on. When pps_deblocking_filter_disabled_flag=1, the deblocking filter is turned off.

[0138] In one embodiment, when sps_vcm_enable_flag is set to 1, the first set of coding tools enabled for machine vision includes MTS, LFNST, ALF, and deblocking filter, and the second set of coding tools disabled for machine vision includes SAO, ISP, and MIP. Therefore, in the bitstream consistency of the video bitstream, sps_mts_enabled_flag=1, sps_lfnst_enabled_flag=1, sps_alf_enabled_flag=1, sps_sao_enabled_flag=0, sps_isp_enabled_flag=0, sps_mip_enabled_flag=0, and pps_deblocking_filter_disabled_flag=0.

[0139] In one embodiment, ALF is turned off for machine vision. Thus, when sps_vcm_enable_flag is set to 1, the first set of coding tools turned on for machine vision includes MTS, LFNST, and deblocking filter, and the second set of coding tools turned off for machine vision includes ALF, SAO, ISP, and MIP. In the bitstream consistency of the video bitstream, sps_mts_enabled_flag = 1, sps_lfnst_enabled_flag = 1, sps_alf_enabled_flag = 0, sps_sao_enabled_flag = 0, sps_isp_enabled_flag = 0, sps_mip_enabled_flag = 0, and pps_deblocking_filter_disabled_flag = 0.

[0140] Note that in some implementations, based on the above method, relevant flags may be set under other configurations to balance the performance impact on machine vision and the implementation complexity.

[0141] VII. Profiles for Machine Vision

[0142] According to aspects of the present disclosure, a profile can be used to select a set of encoding tools for a machine vision task. In some embodiments, the profile can be referred to as a master 10VCM profile or a master 10VCM static picture profile. For example, using a profile can reduce the number of bits required and processed by the signal to improve compression and processing efficiency.

[0143] In one embodiment, a video bitstream conforming to the Main 10VCM Profile and / or the Main 10VCM Still Picture Profile may comply with the following constraints.

[0144] (1) The ptl_multilayer_enabled_flag of the referenced SPS is equal to 0.

[0145] (2) In a bitstream conforming to the main 10VCM static picture profile, the bitstream contains only one picture.

[0146] (3) sps_chroma_format_idc of the referenced SPS is equal to 0 or 1.

[0147] (4) sps_bitdepth_minus8 of the referenced SPS is in the range of 0 to 2 (inclusive).

[0148] (5) The sps_palette_enabled_flag of the referenced SPS is equal to 0.

[0149] (6) The sps_mts_enabled_flag of the referenced SPS is equal to 1.

[0150] (7) The sps_lfnst_enabled_flag of the referenced SPS is equal to 1.

[0151] (8) The sps_sao_enabled_flag of the referenced SPS is equal to 0.

[0152] (9) The sps_alf_enabled_flag of the referenced SPS is equal to 1.

[0153] (10) The sps_isp_enabled_flag of the referenced SPS is equal to 0.

[0154] (11) The sps_mip_enabled_flag of the referenced SPS is equal to 0.

[0155] (12) The pps_deblocking_filter_disabled_flag of the referenced PPS is equal to 0.

[0156] (13) In a bitstream conforming to the Main 10VCM profile, the general level number general_level_idc and the sublayer level number sublayer_level_idc for all i values ​​in the referenced video parameter set (VPS) and the referenced SPS, where applicable, are not equal to 255, indicating a specified level (e.g., level 15.5).

[0157] (14) Where applicable, satisfy the tier and level constraints specified in the Tiers and Levels section of the VVC specification (e.g., subclause A.4 in VVC draft 10) for the Main 10VCM Profile or Main 10VCM Still Picture Profile.

[0158] According to aspects of the present disclosure, the consistency of a video bitstream with a profile may be indicated by a profile number (eg, general_profile_idc). For example, a main 10VCM profile or a main 10VCM still picture profile may be indicated by a profile number.

[0159] In an embodiment, the conformance of a video bitstream to the main 10VCM profile may be indicated by a first profile number (eg, general_profile_idc=2).

[0160] In an embodiment, the conformance of the video bitstream to the main 10VCM still picture profile may be indicated by a second profile number (eg, general_profile_idc=66).

[0161] Note that the condition indicating the consistency of the video bitstream with the main 10VCM still picture profile is also met when the consistency of the video bitstream with the main 10VCM profile is indicated by a second profile number (e.g., general_profile_idc=66) and the indicated level is not a specified level (e.g., level 15.5).

[0162] In one embodiment, a decoder compliant with the Main 10VCM profile at a particular level of a particular tier is able to decode all bitstreams for which all of the following conditions apply.

[0163] (1) Indicates that the bitstream conforms to the Main 10VCM Profile or the Main 10VCM Still Picture Profile.

[0164] (2) Indicates that the bitstream conforms to a level lower than or equal to the specified level.

[0165] (3) Indicates that the bitstream conforms to a level other than level 15.5 and lower than or equal to the specified level.

[0166] In one embodiment, a decoder compliant with the Main 10 VCM Still Picture Profile at a particular level of a particular tier is able to decode all bitstreams for which all of the following conditions apply.

[0167] (1) Indicates that the bitstream conforms to the main 10VCM still picture profile.

[0168] (2) Indicates that the bitstream conforms to a level lower than or equal to the specified level.

[0169] (3) Indicates that the bitstream conforms to a level other than level 15.5 and lower than or equal to the specified level.

[0170] In one embodiment, a decoder compliant with the Main 10VCM still picture profile at a specific level of a specific tier is also able to decode the first picture of the bitstream when both of the following conditions apply.

[0171] (1) Indicates that the bitstream conforms to the main 10VCM profile, to a tier lower than or equal to the specified tier, and to a tier other than level 15.5 and lower than or equal to the specified tier.

[0172] (2) The first picture is an intra random access picture (IRAP) or a progressive decoding refresh (GDR) picture in the output layer with ph_recovery_poc_cnt equal to 0, and ph_pic_output_flag of the first picture is equal to 1.

[0173] In the above implementation, the syntax element ph_recovery_poc_cnt specifies the recovery point of a decoded picture in output order. If the current picture is a GDR picture and ph_recovery_poc_cnt is equal to 0, the current picture itself is called the recovery point picture. The syntax element ph_pic_output_flag affects the decoded picture output and removal process. When ph_pic_output_flag is not present, it is inferred to be equal to 1. A requirement for bitstream conformance is that the bitstream contains at least one picture in the output layer with pic_output_flag equal to 1.

[0174] Note that the term output layer refers to a layer that is intended for output. In an example, a layer that is not intended for output may still be decodable, for example, when used for inter-layer (or inter-view) prediction.

[0175] In one embodiment, in the above-mentioned main 10VCM profile or main 10VCM static picture profile, the sps_alf_enabled_flag of the referenced SPS may be equal to 0. The remaining constraints may be the same as those of the above-mentioned main 10VCM profile or main 10VCM static picture profile.

[0176] Note that in the present disclosure, the values ​​of the profile numbers (eg, general_profile_idc) of the main 10VCM profile and the main 10VCM static picture profile are only examples. The values ​​of the profile numbers may be different as long as they are not used by the relevant standard specifications and follow the corresponding rules of the standard specifications.

[0177] In addition, in some implementations, based on the above method, relevant flags may be set under other configurations to balance the performance impact on machine vision and the implementation complexity.

[0178] VIII. Flowchart

[0179] Fig.10 A flow chart outlining an exemplary process (1000) according to an embodiment of the present disclosure is shown. In various embodiments, the process (1000) is performed by a processing circuit system, such as a processing circuit system in a terminal device (210), (220), (230) and (240), a processing circuit system that performs the functions of a video encoder (303), a processing circuit system that performs the functions of a video decoder (310), a processing circuit system that performs the functions of a video decoder (410), a processing circuit system that performs the functions of an intra-frame prediction module (452), a processing circuit system that performs the functions of a video encoder (503), a processing circuit system that performs the functions of a predictor (535), a processing circuit system that performs the functions of an intra-frame encoder (622), a processing circuit system that performs the functions of an intra-frame decoder (772), etc. In some embodiments, the process (1000) is implemented in software instructions, so that when the processing circuit system executes the software instructions, the processing circuit system performs the process (1000).

[0180] The process (1000) may generally start at step (S1010), where the process (1000) decodes prediction information for a current block in a current picture that is part of a video bitstream. The process (1000) then proceeds to step (S1020).

[0181] At step (S1020), the process (1000) determines to enable a first plurality of coding tools and disable a second plurality of coding tools for the current block based on syntax elements included in the video bitstream. The first plurality of coding tools includes a deblocking filter coding tool, and the second plurality of coding tools includes at least one of a SAO coding tool, an ISP coding tool, and a MIP coding tool. Then, the process (1000) proceeds to step (S1030).

[0182] At step (S1030), the process (1000) reconstructs the current block based on the first plurality of coding tools being enabled and the second plurality of coding tools being disabled. The MTS coding tool and the LFNST coding tool are both included in one of the first plurality of coding tools and the second plurality of coding tools.

[0183] Then, the process (1000) terminates.

[0184] In one embodiment, the ALF encoding tool is included in one of the first plurality of encoding tools and the second plurality of encoding tools.

[0185] In one embodiment, the syntax element is an SPS flag indicating that VCM is enabled for the current block.

[0186] In one embodiment, the syntax element includes a profile number indicating that the video bitstream conforms to one of the Main 10VCM Profile and the Main 10VCM Still Picture Profile.

[0187] In one embodiment, a first value (eg, 2) of the profile number indicates that the video bitstream conforms to the main 10VCM profile, and a second value (eg, 66) of the profile number indicates that the video bitstream conforms to the main 10VCM still picture profile.

[0188] In one embodiment, a decoder associated with the processing circuitry conforms to the Main 10VCM still picture profile, the profile number indicates that the video bitstream conforms to the Main 10VCM profile, and the current picture is the first picture in the video bitstream and is one of an IRAP and a GDR picture.

[0189] In one embodiment, the tier number and level number of the profile to which the video bitstream conforms are respectively less than or equal to the tier number and level number of the profile to which the decoder conforms.

[0190] IX. Computer Systems

[0191] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig.11 A computer system (1100) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0192] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, linking, etc. to create code comprising instructions that may be executed directly by one or more computer central processing units (CPU), graphics processing units (GPU), etc., or through interpretation, microcode execution, etc.

[0193] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0194] Fig.11The components for the computer system (1100) shown in the example are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the exemplary embodiment of the computer system (1100).

[0195] The computer system (1100) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to inputs implemented by one or more human users through, for example, tactile inputs (e.g., keystrokes, swipes, data glove movements), audio inputs (e.g., voice, tapping), visual inputs (e.g., gestures), olfactory inputs (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to intentional human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0196] The input human-machine interface devices may include one or more of the following (only one of each is depicted): keyboard (1101), mouse (1102), trackpad (1103), touch screen (1110), data gloves (not shown), joystick (1105), microphone (1106), scanner (1107) and camera (1108).

[0197] The computer system (1100) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include: tactile output devices (e.g., tactile feedback through a touch screen (1110), a data glove (not shown), or a joystick (1105), but there may also be tactile feedback devices that are not used as input devices); audio output devices (e.g., speakers (1109), headphones (not depicted)); visual output devices (e.g., screens (1110), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be able to output two-dimensional visual output or more than three-dimensional output through methods such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted)); and printers (not depicted). These visual output devices, such as a screen ( 1110 ), may be connected to the system bus ( 1148 ) via a graphics adapter ( 1150 ).

[0198] The computer system (1100) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1120) with CD / DVD etc. media (1121), thumb drives (1122), removable hard drives or solid-state drives (1123), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD based devices such as security dongles (not depicted), etc.

[0199] Those skilled in the art will also appreciate that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0200] The computer system (1100) may also include a network interface (1154) to one or more communication networks (1155). The one or more communication networks (1155) may be, for example, wireless, wired, optical. The one or more communication networks (1155) may also be local area networks, wide area networks, metropolitan area networks, vehicle and industrial networks, real-time networks, delay-tolerant networks, etc. Examples of the one or more communication networks (1155) include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter attached to some general data port or peripheral bus (1149) (e.g., a USB port of the computer system (1100)); other networks are typically integrated into the core of the computer system (1100) by attaching to a system bus as described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). The computer system (1100) can communicate with other entities using any of these networks. Such communications can be one-way receive-only (e.g., broadcast television), one-way send-only (e.g., CANbus to some CANbus devices), or two-way, such as to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0201] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core ( 1140 ) of the computer system ( 1100 ).

[0202] The core (1140) may include one or more central processing units (CPUs) (1141), graphics processing units (GPUs) (1142), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1143), hardware accelerators (1144) for certain tasks, graphics adapters (1150), etc. These devices as well as read-only memory (ROM) (1145), random access memory (1146), internal large-capacity storage devices (1147) such as internal non-user accessible hard drives, SSDs, etc. can be connected through a system bus (1148). In some computer systems, the system bus (1148) can be accessed in the form of one or more physical plugs to achieve expansion through additional CPUs, GPUs, etc. Peripheral devices can be attached to the core's system bus (1148) directly or through a peripheral bus (1149). In an example, a screen (1110) can be connected to a graphics adapter (1150). The architecture of the peripheral bus includes PCI, USB, etc.

[0203] The CPU (1141), GPU (1142), FPGA (1143) and accelerator (1144) can execute certain instructions, which can be combined to form the computer code mentioned above. The computer code can be stored in ROM (1145) or RAM (1146). Transient data can also be stored in RAM (1146), while permanent data can be stored in, for example, an internal mass storage device (1147). Fast storage and retrieval of any storage device in the storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1141), GPUs (1142), mass storage devices (1147), ROMs (1145), RAMs (1146), etc.

[0204] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or the medium and computer code may be of a type well known and available to those skilled in the art of computer software.

[0205] As an example and not limitation, a computer system (1100) having an architecture and in particular a core (1140) can provide functionality provided as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, as well as certain storage devices of the core (1140) having a non-transitory nature, such as a mass storage device (1147) or ROM (1145) inside the core. Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (1140). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can enable the core (1140) - and in particular the processor therein (including a CPU, GPU, FPGA, etc.) - to perform specific processing or specific parts of specific processing described herein, including defining data structures stored in RAM (1146) and modifying such data structures according to processing defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1144)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and conversely references to logic may include software. Where appropriate, references to computer-readable media may include circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits implementing logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0206] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. It will therefore be appreciated that, although not explicitly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that implement the principles of the present disclosure and are therefore within its spirit and scope.

[0207] Appendix A: Acronyms

[0208] ALF: Adaptive Loop Filter

[0209] AMVP: Advanced Motion Vector Prediction

[0210] APS: Adaptive Parameter Set

[0211] ASIC: Application-Specific Integrated Circuit

[0212] ATMVP: Alternative / Advanced Temporal Motion Vector Prediction

[0213] AV1: AOMedia Video 1

[0214] AV2: AOMedia Video 2

[0215] BMS: Benchmark Set

[0216] BV: Block Vector

[0217] CANBus: Controller Area Network Bus

[0218] CB: Coded Block

[0219] CC-ALF: Cross-Component Adaptive Loop Filter

[0220] CD: Compact Disc

[0221] CDEF: Constrained Directional Enhancement Filter

[0222] CPR: Current Picture Reference

[0223] CPU: Central Processing Unit

[0224] CRT: cathode ray tube

[0225] CTB: Coding Tree Block

[0226] CTU: Coding Tree Unit

[0227] CU: Coding Unit

[0228] DPB: Decoder Picture Buffer

[0229] DPCM: Differential Pulse Code Modulation

[0230] DPS: Decoding Parameter Set

[0231] DVD: Digital Video Disc

[0232] FPGA: Field Programmable Gate Array

[0233] JCCR: Joint CbCr Coding Residual

[0234] JVET: Joint Video Exploration Team

[0235] GOP: Group of Pictures

[0236] GPU: Graphics Processing Unit

[0237] GSM: Global System for Mobile Communications

[0238] HDR: High Dynamic Range

[0239] HEVC: High Efficiency Video Coding

[0240] HRD: Hypothetical Reference Decoder

[0241] IBC: Intra Block Copy

[0242] IC: Integrated Circuit

[0243] ISP: Intra-frame sub-partitioning

[0244] JEM: Joint Development Model

[0245] LAN: Local Area Network

[0246] LCD: Liquid Crystal Display

[0247] LR: Loop Restoration Filter

[0248] LRU: Loop Recovery Unit

[0249] LTE: Long Term Evolution

[0250] MPM: Most Probable Mode

[0251] MV: Motion Vector

[0252] OLED: Organic Light Emitting Diode

[0253] PB: prediction block

[0254] PCI: Peripheral Component Interconnect

[0255] PDPC: Position-Dependent Prediction Combination

[0256] PLD: Programmable Logic Device

[0257] PPS: Picture Parameter Set

[0258] PU: prediction unit

[0259] RAM: Random Access Memory

[0260] ROM: Read Only Memory

[0261] SAO: Sample Adaptive Offset

[0262] SCC: Screen Content Coding

[0263] SDR: Standard Dynamic Range

[0264] SEI: Supplemental Enhancement Information

[0265] SNR: Signal to Noise Ratio

[0266] SPS: Sequence Parameter Set

[0267] SSD: Solid State Drive

[0268] TU: Transform Unit

[0269] USB: Universal Serial Bus

[0270] VPS: Video Parameter Set

[0271] VUI: Video Availability Information

[0272] VVC: Versatile Video Coding

[0273] WAIP: Wide Angle Intra Prediction

Claims

1. A video decoding method, characterized in that: The method comprises: decoding prediction information for a current block in a current picture as part of a video bitstream; Determining, for the current block based on syntax elements included in the video bitstream, to enable a first plurality of encoding tools and to disable a second plurality of encoding tools, the first plurality of encoding tools comprising a deblocking filter encoding tool, the second plurality of encoding tools comprising at least one of a sample adaptive offset filter (SAO) encoding tool, an intra sub-partitioning (ISP) encoding tool, and a matrix-based intra prediction (MIP) encoding tool; and reconstructing the current block based on the first plurality of encoding tools being enabled and the second plurality of encoding tools being disabled, One of the first plurality of coding tools and the second plurality of coding tools includes both a multiple transform set (MTS) coding tool and a low frequency non-separable transform (LFNST) coding tool.

2. The method according to claim 1, characterized in that: One of the first plurality of encoding tools and the second plurality of encoding tools includes an adaptive loop filter (ALF) encoding tool.

3. The method according to claim 1 or 2, characterized in that: The syntax element is a sequence parameter set (SPS) flag indicating that video decoding machine-oriented (VCM) is enabled for the current block.

4. The method according to claim 1 or 2, characterized in that: The syntax element includes a profile number indicating that the video bitstream conforms to the main 10VCM profile.

5. The method according to claim 4, characterized in that The profile number, when a first value, indicates that the video bitstream complies with the main 10VCM profile, and When the profile number is a second value, it indicates that the video bitstream complies with the main 10VCM static picture profile.

6. The method according to claim 4, characterized in that The decoder complies with the main 10VCM still picture profile, The profile number indicates that the video bitstream conforms to the main 10VCM profile, and The current picture is the first picture in the video bitstream and is one of an intra random access picture (IRAP) and a progressive decoding refresh (GDR) picture.

7. The method according to claim 4, characterized in that The layer number and level number of the profile to which the video bitstream conforms are respectively less than or equal to the layer number and level number of the profile to which the decoder conforms.

8. A computer device, characterized in that: The computer device comprises: one or more computer-readable non-transitory storage media configured to store computer program code; and One or more computer processors configured to access the computer program code and execute the method according to any one of claims 1 to 7 as instructed by the computer program code.

9. A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 7.

10. A device, characterized in that: The device comprises: processing circuitry configured to: decoding prediction information for a current block in a current picture as part of a video bitstream; Determining, for the current block based on syntax elements included in the video bitstream, to enable a first plurality of encoding tools and to disable a second plurality of encoding tools, the first plurality of encoding tools comprising a deblocking filter encoding tool, the second plurality of encoding tools comprising at least one of a sample adaptive offset filter (SAO) encoding tool, an intra sub-partitioning (ISP) encoding tool, and a matrix-based intra prediction (MIP) encoding tool; and reconstructing the current block based on the first plurality of encoding tools being enabled and the second plurality of encoding tools being disabled, One of the first plurality of coding tools and the second plurality of coding tools includes both a multiple transform set (MTS) coding tool and a low frequency non-separable transform (LFNST) coding tool.

11. The device according to claim 10, characterized in that One of the first plurality of encoding tools and the second plurality of encoding tools includes an adaptive loop filter (ALF) encoding tool.

12. The device according to claim 10 or 11, characterized in that The syntax element is a sequence parameter set (SPS) flag indicating that video decoding machine-oriented (VCM) is enabled for the current block.

13. The device according to claim 10 or 11, characterized in that The syntax element includes a profile number indicating that the video bitstream conforms to the main 10VCM profile.

14. The device according to claim 13, characterized in that The profile number, when a first value, indicates that the video bitstream complies with the main 10VCM profile, and When the profile number is a second value, it indicates that the video bitstream complies with the main 10VCM static picture profile.

15. The device according to claim 13, characterized in that The decoder associated with the device complies with the main 10VCM still picture profile, The profile number indicates that the video bitstream conforms to the main 10VCM profile, and The current picture is the first picture in the video bitstream and is one of an intra random access picture (IRAP) and a progressive decoding refresh (GDR) picture.

16. The device according to claim 13, characterized in that The tier number and level number of the profile to which the video bitstream conforms are respectively less than or equal to the tier number and level number of the profile to which a decoder associated with the device conforms.

17. A method for storing or sending a video bitstream, characterized in that: The video bit stream is decoded according to the method according to any one of claims 1 to 7.

Images