Method and device for video encoding and decoding, and storage medium

Abstract

The embodiment of the invention provides a video encoding and decoding method and device and a storage medium. The method comprises: decoding configuration file information for a plurality of image slices in prediction information of an encoded video bitstream, the configuration file information comprising a plurality of groups of syntax elements and indicating a configuration file for the encoded video bitstream; according to a profile tierlevel () grammar structure indicated by the high-level grammar, decoding the configuration file information; decoding the first syntax element and the second syntax element when the first variable is equal to 1, and receiving generic constraint information; decoding the third syntax element; decoding a fourth syntax element in the general constraint information, and if the fourth syntax element is equal to 1, determining that the slice type is an intra-frame slice; when the fifth syntax element in the high-level syntax is zero, all image slices received from the encoded video bitstream are intra slices.

Description

[0001] Case Analysis This application is a divisional application of Chinese patent application No. 202180005301.1, entitled "Method and Apparatus for Video Coding", which entered the Chinese national phase of PCT international patent application PCT / US2021 / 025870, filed on April 6, 2021. Technical Field

[0002] This application describes embodiments related to video encoding and decoding. Background Technology

[0003] The background description provided herein is intended to introduce the background of the embodiments of this application. Within the scope described in this background section, neither the work of the currently named inventors nor any aspects of the description that may not be considered prior art at the time of submission are express or implied acknowledgments of prior art as embodiments of this application.

[0004] Video encoding and decoding can be performed using inter-frame picture prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with a spatial size of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of pictures can have a fixed or variable picture rate (also informally referred to as the frame rate), such as 60 pictures per second or 60Hz. Uncompressed video has very high bitrate requirements. For example, at 8 bits per sample, 1080p60 4:2:0 video (with a 1920×1080 luminance sample resolution at a 60Hz frame rate) requires nearly 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 GB of storage space.

[0005] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be employed. Lossless compression refers to a technique that reconstructs an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. In the case of video, lossy compression is widely used. The tolerable amount of distortion depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can be reflected in the fact that higher permissible / acceptable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec techniques can include techniques known as intra-frame coding. In intra-frame coding, the representation of sample values ​​does not reference samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the picture can be an intra-frame picture. Intra-frame pictures and their derivatives, such as stand-alone decoder refresh pictures, can be used to reset the decoder state and thus can be used as the first picture in an encoded video bitstream and video session, or as a still picture. Samples of an intra-frame block can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value, the smaller the AC coefficient, and the fewer bits are needed to represent the entropy-coded block for a given quantization step size.

[0008] For example, traditional intra-frame coding known from MPEG-2 generation coding techniques does not use intra-frame prediction. However, some newer video compression techniques include attempts to use sample data and / or metadata from surrounding areas, obtained, for example, during spatially adjacent encoding / decoding and preceding data blocks in the decoding order. This technique is now referred to as "intra-frame prediction." Note that, at least in some cases, intra-frame prediction uses only reference data from the current image being reconstructed, rather than reference data from a reference image.

[0009] There can be many different forms of intra-prediction. When more than one such technique is available in a given video coding technique, the technique used can be encoded in an intra-prediction mode. In some cases, a mode can have sub-modes and / or parameters, which can be encoded separately or included in the mode codeword. For a given combination of mode / sub-mode / parameters, which codeword is used can affect the coding efficiency gain obtained through intra-prediction, as can the entropy coding technique used to convert the codeword into a bitstream.

[0010] H.264 introduced an intra-frame prediction mode, which was improved in H.265 and further refined in new coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). Prediction blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the prediction block according to the direction. The reference for the direction used can be encoded in the bitstream, or the reference for the direction used can be predicted itself.

[0011] refer to Figure 1A The lower right corner shows a subset of nine known prediction directions from the 33 possible prediction directions of H.265 (corresponding to 33 angular modes out of 35 intra-frame modes). The point (101) where the arrows converge represents the sample being predicted. The arrow indicates the direction from which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples at a 45-degree angle to the horizontal from the upper right. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples at a 22.5-degree angle to the horizontal from the lower left of sample (101).

[0012] Still referencing Figure 1A A square block (104) of 4×4 samples is shown in the upper left corner (indicated by bold dashed lines). The square block (104) comprises 16 samples, each labeled “S”, its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top to bottom) and the first sample in the X dimension (from left to right). Similarly, sample S44 is the fourth sample in both the Y and X dimensions of the block (104). Since the block size is 4×4 samples, S44 is located in the lower right corner. Reference samples following a similar numbering scheme are further shown. Reference samples are labeled R, with their Y position (e.g., row index) and X position (column index) relative to the block (104). In H.264 and H.265, predicted samples are adjacent to the blocks in the reconstruction; therefore, negative values ​​are not required.

[0013] Intra-frame image prediction can work by copying reference sample values ​​from neighboring samples determined by the prediction direction indicated by a signal. For example, suppose the encoded video bitstream includes signaling that indicates a prediction direction consistent with arrow (102) for that block—that is, from one or more prediction samples at a 45-degree angle to the horizontal from the upper right. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

[0014] In some cases, the values ​​of multiple reference samples can be combined by interpolation to calculate a reference sample, especially when the directions cannot be evenly separated by 45 degrees.

[0015] With the development of video coding technology, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. This increased to 33 directions in H.265 (2013), and JEM / VVC / BMS, in its public release, supported up to 65 directions. To identify the most likely directions, experiments were conducted, using certain techniques from entropy coding to represent those possible directions with a small number of bits, while accepting a certain cost for less likely directions. Furthermore, sometimes the direction itself can be predicted from adjacent directions used in already decoded blocks.

[0016] Figure 1B A schematic diagram (105) is shown illustrating 65 intra-frame prediction directions based on JEM to show that the number of prediction directions increases over time.

[0017] The mapping of intra-predicted direction bits representing direction in a encoded video bitstream can vary depending on the video coding technique; and its range can extend from a simple, direct mapping of the predicted direction to the intra-predicted mode, to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques. However, in all cases, there may be some directions that are statistically less likely to occur in the video content compared to others. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less likely directions will be represented with a greater number of bits than the more likely directions.

[0018] Motion compensation can be a lossy compression technique and can involve using sample data blocks from a previously reconstructed image or a portion thereof (the reference image) after spatial offset along a direction indicated by a motion vector (hereinafter referred to as MV) for prediction of a newly reconstructed image or image portion. In some cases, the reference image can be the same as the image currently being reconstructed. Multiple MVs can have two or three dimensions, X and Y, with the third dimension being an indication of the reference image in use (which can indirectly be a temporal dimension).

[0019] In some video compression techniques, the motion vector (MV) applicable to a region of sample data can be predicted based on multiple other MVs, such as those related to another sample data region spatially adjacent to the region being reconstructed and preceding that MV in the decoding order. This substantially reduces the amount of data required to encode the MVs, thereby eliminating redundancy and increasing compression ratio. MV prediction can work effectively, for example, because when encoding an input video signal derived from a camera (called natural video), there is a statistical probability that a larger region than the region applicable to a single MV has moved in a similar direction. Therefore, in some cases, similar motion vectors derived from MVs of adjacent regions can be used for prediction. This results in the MV found for a given region being similar or identical to the MV predicted based on multiple surrounding MVs, and after entropy encoding, this, in turn, can be represented with fewer bits than would be used to directly encode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., multiple MVs) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors in calculating predictions from several surrounding MVs.

[0020] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the various MV prediction mechanisms provided by H.265, the one described here is the technique formerly known as "spatial combining".

[0021] Reference Figure 1C The current block (111) includes samples discovered by the encoder during motion search, which can be predicted from previous blocks of the same size that have been spatially shifted. Instead of directly encoding the MV, the MV can be derived from metadata associated with one or more reference images, such as from the most recent (in decoding order) reference image, using the MV associated with any one of the five surrounding samples (denoted as A0, A1, B0, B1, B2 (from 112 to 116 respectively)). In H.265, MV prediction can use a predictor from the same reference image being used by adjacent blocks. Summary of the Invention

[0022] This application provides a method, apparatus, and storage medium for video decoding in a decoder.

[0023] In some embodiments, this application provides a method for video decoding in a decoder, including: Decode the configuration information for multiple image slices in the prediction information of the encoded video bitstream, the configuration information including multiple sets of syntax elements, the configuration information being used to indicate the configuration for the encoded video bitstream; The profile information is decoded according to the profile_tier_level() syntax structure indicated by the high-level syntax, wherein the profile_tier_level() syntax structure includes: a first variable profileTierPresentFlag and a second variable maxNumSubLayersMinus1; When the first variable profileTierPresentFlag is equal to 1, the first syntax element general_profile_idc and the second syntax element general_tier_flag are decoded, and the general constraint information of the General_constraint_info() syntax structure is received, wherein the General_constraint_info() syntax structure is contained in the profile_tier_level() syntax structure; In all cases where the first variable profileTierPresentFlag has any value, the third syntax element general_level_idc is decoded. When decoding the fourth syntax element intra_only_constraint_flag contained in the general constraint information of the General_constraint_info() syntax structure, if the fourth syntax element intra_only_constraint_flag is equal to 1, the slice type of the multiple image slices is an intra-frame slice, and the slice type is represented by (slice_type=I slice); In the high-level syntax, there is a fifth syntax element to indicate whether decoding is performed in intra-frame mode. When the fifth syntax element is 0, all image slices received from the encoded video bitstream are intra-frame slices.

[0024] In some embodiments, this application also provides a video encoding method, including: Configuration information for multiple image slices is encoded, the configuration information including multiple sets of syntax elements, the configuration information being used to indicate the configuration file for the encoded video bitstream; The profile information is encoded according to the profile_tier_level() syntax structure indicated by the high-level syntax, wherein the profile_tier_level() syntax structure includes: a first variable profileTierPresentFlag and a second variable maxNumSubLayersMinus1; When the first variable profileTierPresentFlag equals 1, the first syntax element general_profile_idc and the second syntax element general_tier_flag are encoded, and the general constraint information of the General_constraint_info() syntax structure is obtained, wherein the General_constraint_info() syntax structure is contained in the profile_tier_level() syntax structure; In all cases where the first variable profileTierPresentFlag has any value, the third syntax element general_level_idc is encoded. When encoding the fourth syntax element intra_only_constraint_flag contained in the general constraint information of the General_constraint_info() syntax structure, if the fourth syntax element intra_only_constraint_flag is equal to 1, the slice type of the plurality of image slices is an intra-frame slice, and the slice type is represented by (slice_type=I slice); In the high-level syntax, there is a fifth syntax element to indicate whether encoding is in intra-frame mode. When the fifth syntax element is 0, all image slices obtained from the encoded video bitstream are intra-frame slices.

[0025] In some embodiments, this application also provides an apparatus for video decoding in a decoder, the apparatus including processing circuitry configured to perform the video decoding method for video decoding in a decoder according to embodiments of this application.

[0026] In some embodiments, this application also provides a video encoding apparatus, the apparatus including a processing circuit configured to perform the video encoding method according to the embodiments of this application.

[0027] In some embodiments, this application also provides a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a video decoding method in a decoder according to an embodiment of this application, or a video encoding method according to an embodiment of this application.

[0028] In some embodiments, this application also provides a method for storing a bitstream, the bitstream being generated based on the video encoding method described in this application; and storing the bitstream.

[0029] In some embodiments, this application also provides a method for transmitting a bitstream, the bitstream being generated based on the video encoding method described in this application; and transmitting the bitstream. Attached Figure Description

[0030] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which: Figure 1A This is a schematic diagram of an exemplary subset of intra-frame prediction modes; Figure 1B This is an illustration of an exemplary intra-frame prediction direction; Figure 1C This is a schematic diagram of the current block and its surrounding space merge candidates in an example; Figure 2 This is a simplified block diagram of a communication system according to an embodiment; Figure 3 This is a simplified block diagram of a communication system according to an embodiment; Figure 4 This is a simplified block diagram of the decoder according to an embodiment; Figure 5 This is a simplified block diagram of the encoder according to an embodiment; Figure 6 A block diagram of an encoder according to another embodiment is shown; Figure 7 A block diagram of a decoder according to another embodiment is shown; Figure 8 An exemplary flowchart according to an embodiment is shown; Figure 9 Another exemplary flowchart according to an embodiment is shown; and Figure 10 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0031] I. Video Decoder and Encoder Systems Figure 2A simplified block diagram of a communication system (200) according to one embodiment of this application is shown. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and (220) interconnected via the network (250). Figure 2 In the example, the first pair of terminal devices (210) and (220) perform one-way data transmission. For example, terminal device (210) may encode video data (e.g., a video image stream captured by terminal device (210)) for transmission over a network (250) to another terminal device (220). The encoded video data may be transmitted in the form of one or more encoded video bitstreams. Terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. One-way data transmission is common in media service applications and other applications.

[0032] In another example, the communication system (200) includes a second pair of terminal devices (230) and (240) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, in one example, each of the terminal devices (230) and (240) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission over a network (250) to the other terminal device (230) and (240). Each of the terminal devices (230) and (240) may also receive encoded video data transmitted by the other terminal device (230) and (240), and may decode the encoded video data to recover video images, and may display the video images on an accessible display device based on the recovered video data.

[0033] exist Figure 2In the examples, terminal devices (210), (220), (230), and (240) may be shown as servers, personal computers, and smartphones, but the principles of the embodiments of this application are not limited thereto. Embodiments of the embodiments of this application are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (250) refers to any number of networks that transmit encoded video data between terminal devices (210), (220), (230), and (240), including, for example, wired (connected) and / or wireless communication networks. Communication networks (250) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation of the embodiments of this application.

[0034] As an example of the application of the disclosed topic Figure 3 The placement of a video encoder and video decoder in a streaming environment is illustrated. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0035] The streaming system may include an acquisition subsystem (313) that may include a video source (301) such as a digital camera, which creates, for example, an uncompressed video image stream (302). In one example, the video image stream (302) includes samples captured by a digital camera. The video image stream (302), depicted as thick lines to emphasize its high data volume, may be processed by an electronic device (320) including a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data (304) (or encoded video bitstream (304)), depicted as thin lines to emphasize its lower data volume, may be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3Client subsystems (306) and (308) can access a streaming server (305) to retrieve copies (307) and (309) of encoded video data (304). Client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and produces an output video picture stream (311) that can be displayed on a display (312) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (304), encoded video data (307), and encoded video data (309) (e.g., a video bitstream) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Multi-Functional Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0036] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0037] Figure 4 A block diagram of a video decoder (410) according to one embodiment of this application is shown. The video decoder (410) may be included in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of Figure 3 The example video decoder (310).

[0038] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective user entities (not depicted). The receiver (431) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be located external to the video decoder (410) (not depicted). In other cases, a buffer memory (not depicted) may be provided outside the video decoder (410) to prevent network jitter, for example, and another buffer memory (415) may be configured inside the video decoder (410) to handle broadcast timing, for example. When the receiver (431) is receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer memory (415) may not be needed, or it may be made smaller. For optimal use on packet networks such as the Internet, a buffer memory (415) may be required, which may be relatively large and advantageously have an adaptive size, and may be implemented at least partially in a similar element (not depicted) outside the operating system or the video decoder (410).

[0039] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (410), and potential information for controlling a rendering device (412) (e.g., a display screen), which is not integral to the electronic device (430) but may be coupled to it, such as... Figure 3As shown in the diagram. The control information used for the presentation device may be in the form of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (420) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set from the encoded video sequence for at least one subgroup of pixels in the subgroups used in the video decoder, based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOPs), pictures, tiles, slices, macroblocks, Coding Units (CUs), blocks, Transform Units (TUs), Prediction Units (PUs), etc. The parser (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0040] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0041] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (420). For brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not depicted.

[0042] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In practical implementations operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0043] The first unit is the scaler / inverse transform unit (451). The scaler / inverse transform unit (451) receives quantization transform coefficients as symbols (421) and control information from the parser (420), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block containing sample values, which can be input into the aggregator (455).

[0044] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-prediction unit (452). In some cases, the intra-prediction unit (452) generates blocks of the same size and shape as the blocks being reconstructed, using surrounding reconstructed information extracted from the current image buffer (458). For example, the current image buffer (458) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (455) adds the predictive information generated by the intra-prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0045] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (453) may access the reference image memory (457) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (421) belonging to the block, these samples may be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (in this case referred to as residual samples or residual signals) to generate output sample information. The address at which the motion compensation prediction unit (453) extracts the predicted samples from the reference image memory (457) may be controlled by a motion vector, which is provided to the motion compensation prediction unit (453) in the form of symbols (421), which may have, for example, X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (457) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0046] The output samples of the aggregator (455) can be subjected to various loop filtering techniques in the loop filter unit (456). The video compression technique may include an in-loop filter technique controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and available to the loop filter unit (456) as symbols (421) from the parser (420). However, the video compression technique may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0047] The output of the loop filter unit (456) can be a sample stream, which can be output to the presentation device (412) and stored in the reference image memory (457) for subsequent inter-frame image prediction.

[0048] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded images corresponding to the current image have been fully reconstructed and the encoded images (by, for example, the parser (420)) are identified as reference images, the current image cache (458) can become part of the reference image memory (457), and a new current image cache can be reallocated before the reconstruction of subsequent encoded images begins.

[0049] The video decoder (410) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T Rec.H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile may select certain tools from all available tools in the video compression technique or standard as the only tools available under that profile. For compliance, the complexity of the encoded video sequence may also be required to be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0050] In one embodiment, the receiver (431) may receive additional (redundant) data along with the received encoded video. This additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0051] Figure 5 A block diagram of a video encoder (503) according to one embodiment of this application is shown. The video encoder (503) is included in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., transmission circuitry). The video encoder (503) can be used in place of... Figure 3 The example video encoder (303).

[0052] The video encoder (503) can obtain data from the video source (501) (not...). Figure 5 In one example, the electronic device (520) receives a video sample, the video source of which can capture video images that will be encoded by the video encoder (503). In another example, the video source (501) is part of the electronic device (520).

[0053] A video source (501) can provide a sequence of source video samples to be encoded by a video encoder (503) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (501) can be a storage device storing previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual images, which produce motion effects when viewed in sequence. The images themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can readily understand the relationship between pixels and samples. The following focuses on describing samples.

[0054] According to one embodiment, the video encoder (503) can encode and compress images of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units as described below. For simplicity, coupling is not depicted in the figures. Parameters set by the controller (550) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be configured to have other suitable functions related to the video encoder (503) optimized for a particular system design.

[0055] In some embodiments, the video encoder (503) is configured to operate within an encoding loop. As a simplified description, in one example, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (533) embedded within the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder (since any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference image memory (534). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (534) also correspond bit-accurately between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the offset that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0056] The operation of the “local” decoder (533) can be combined with, for example, those already mentioned above. Figure 4 The operation is the same as that of a "remote" decoder such as the video decoder (410) described in detail. However, a brief additional reference is provided. Figure 4 When symbols are available and the entropy encoder (545) and parser (420) can losslessly encode / decode the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (410), including the buffer (415) and parser (420), may not be fully implemented in the local decoder (533).

[0057] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the subject matter disclosed focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0058] During operation, in some examples, the source encoder (530) may perform motion-compensated predictive coding. This motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images designated as "reference images" from the video sequence. In this way, the encoding engine (532) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0059] The local video decoder (533) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (530). The operation of the encoding engine (532) can advantageously be a lossy process. When the encoded video data can be decoded by the video decoder (530), Figure 5 When decoded at (not shown), the reconstructed video sequence is typically a copy of the source video sequence, but with some errors. The local video decoder (533) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (534). In this way, the video encoder (503) can locally store a copy of the reconstructed reference image that shares the same content as the reconstructed reference image that will be obtained by the remote video decoder (without transmission errors).

[0060] The predictor (535) can perform a prediction search against the encoding engine (532). That is, for a new image to be encoded, the predictor (535) can search in the reference image memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (535) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (535), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (534).

[0061] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0062] The outputs of all the above functional units can be entropy encoded in the entropy encoder (545). The entropy encoder (545) transforms the symbols generated by various functional units into an encoded video sequence through lossless compression using techniques such as Huffman coding, variable length coding, and arithmetic coding.

[0063] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0064] The controller (550) manages the operation of the video encoder (503). During encoding, the controller (550) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types: An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics.

[0065] A predictive picture (P-picture) can be a picture that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0066] A bidirectional predictive picture (B-picture) can be a picture that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict the sample values ​​for each block. Similarly, a multiple-predictive picture can use more than two reference pictures and associated metadata to reconstruct a single block.

[0067] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks) and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to these blocks. For example, blocks of an I-image can be non-predictively coded, or the block can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0068] The video encoder (503) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T Rec.H.265. In operation, the video encoder (503) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0069] In one embodiment, the transmitter (540) may transmit encoded video along with additional data. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0070] The captured video can be presented as multiple source images (video images) in a time-series format. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, a specific image being encoded / decoded is segmented into blocks; this specific image being encoded / decoded is called the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference image.

[0071] In some embodiments, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image that precede the current image in the video in decoding order (but may be past and future in display order, respectively). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0072] In addition, merging mode techniques can be used for inter-frame image prediction to improve coding efficiency.

[0073] According to some embodiments of this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In one example, each CU is analyzed to determine the prediction type used for that CU, such as inter-frame prediction or intra-frame prediction. Depending on temporal and / or spatial predictability, CUs are split into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In one embodiment, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking the luminance prediction block as an example, the prediction block includes a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and so on.

[0074] Figure 6 A diagram of a video encoder (603) according to another embodiment of this application is shown. The video encoder (603) is configured to receive a processing block (e.g., a prediction block) of sample values ​​within a current video image in a video image sequence, and to encode the processing block into an encoded image that is part of an encoded video sequence. In one example, the video encoder (603) is used instead of Figure 3 The example video encoder (303).

[0075] In the HEVC example, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use intra-frame mode, inter-frame mode, or bidirectional prediction mode to optimally encode the processing block. When the processing block is to be encoded in intra-frame mode, the video encoder (603) can use intra-frame prediction techniques to encode the processing block into an already encoded picture; and when the processing block is to be encoded in inter-frame mode or bidirectional prediction mode, the video encoder (603) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into an already encoded picture, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector predictors without the aid of encoded motion vector components outside the predictor. In some other video coding techniques, motion vector components applicable to the subject block may exist. In one example, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0076] exist Figure 6 In the example, the video encoder (603) includes, for example, Figure 6 The inter-frame encoder (630), intra-frame encoder (622), residual calculator (623), switch (626), residual encoder (624), general controller (621) and entropy encoder (625) are shown coupled together.

[0077] The inter-frame encoder (630) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., a description of redundancy information based on the inter-frame coding technique, motion vectors, merging mode information), and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on encoded video information.

[0078] The intra encoder (622) is configured to receive samples of the current block (e.g., the processed block), in some cases compare the block with encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). In one example, the intra encoder (622) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and a reference block in the same image.

[0079] A general controller (621) is configured to determine general control data and, based on this general control data, control other components of the video encoder (603). In one example, the general controller (621) determines the mode of a block and, based on this mode, provides control signals to a switch (626). For example, when the mode is intra-frame mode, the general controller (621) controls the switch (626) to select an intra-frame mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select intra-frame prediction information and include the intra-frame prediction information in the bitstream; and when the mode is inter-frame mode, the general controller (621) controls the switch (626) to select an inter-frame prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select inter-frame prediction information and include the inter-frame prediction information in the bitstream.

[0080] A residual calculator (623) is configured to calculate the difference (residual data) between the received block and the prediction result selected from an intra-encoder (622) or an inter-encoder (630). A residual encoder (624) is configured to operate based on the residual data to encode the residual data to generate transform coefficients. In one example, the residual encoder (624) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then subjected to quantization to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be used appropriately by the intra-encoder (622) and the inter-encoder (630). For example, an inter-frame encoder (630) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and an intra-frame encoder (622) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded picture, and in some examples, the decoded picture can be buffered in a memory circuit (not shown) and used as a reference picture.

[0081] An entropy encoder (625) is configured to format the bitstream to include encoded blocks. The entropy encoder (625) is configured to include various information according to a suitable standard, such as the HEVC standard. In one example, the entropy encoder (625) is configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0082] Figure 7A diagram of a video decoder (710) according to another embodiment of this application is shown. The video decoder (710) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In one example, the video decoder (710) is used instead of Figure 3 The example video decoder (310).

[0083] exist Figure 7 In the example, the video decoder (710) includes, for example, Figure 7 The entropy decoder (771), inter-frame decoder (780), residual decoder (773), reconstruction module (774) and intra-frame decoder (772) are shown coupled together.

[0084] The entropy decoder (771) can be configured to reconstruct certain symbols from an encoded picture, representing the syntax elements constituting the encoded picture. Such symbols may include, for example, the mode used to encode the block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a combined sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can be identified for use by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In one example, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (772). The residual information may be inversely quantized and provided to the residual decoder (773).

[0085] The inter-frame decoder (780) is configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0086] The intra-frame decoder (772) is configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0087] The residual decoder (773) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to include quantizer parameters (QP)), and this information can be provided by the entropy decoder (771) (the data path is not depicted because this is only low-level control information).

[0088] The reconstruction module (774) is configured to combine the residual output by the residual decoder (773) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module, depending on the specific situation) in the spatial domain to form a reconstruction block, which may be part of a reconstructed image, which in turn may be part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0089] It should be noted that any suitable technology can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In one embodiment, one or more integrated circuits can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (303), video encoder (503), and video encoder (503), as well as the video decoder (310), video decoder (410), and video decoder (710).

[0090] II. Exemplary Advanced Syntax Elements Table 1 shows some relevant examples, such as the exemplary Sequence Parameter Set (SPS) syntax elements in VVC. Syntax elements related to intra-frame coding and inter-frame coding are all included in Table 1. Note that for intra-frame profiles that only include intra-frame slices, inter-frame coding syntax elements may exist in the SPS but are not used in the decoding process of the intra-frame profile. This also applies to any still image profile. That is, for still image profiles that only include intra-frame slices, inter-frame coding syntax elements are not used in the decoding process of the still image profile.

[0091] Table 1. Syntax of Raw Byte Sequence Payload (RBSP) for Sequence Parameter Set

[0092] Table 2 shows some relevant examples, such as the exemplary Picture Parameter Set (PPS) syntax elements in VVC. Syntax elements related to intra-frame coding and inter-frame coding are all included in Table 2. Note that for intra-frame profiles that only include intra-frame slices, inter-frame coding syntax elements may exist in the PPS but are not used in the decoding process of the intra-frame profile. This also applies to any still image profile. That is, for still image profiles that only include intra-frame slices, inter-frame coding syntax elements are not used in the decoding process of the still image profile.

[0093] Table 2 Image Parameters RBSP Syntax

[0094] Tables 3 and 4 show some relevant examples, such as the exemplary Picture Header (PH) syntax element in VVC. A picture can include a combination of intra-frame and inter-frame slices. When a picture only includes intra-frame slices, the inter-frame coding syntax element can exist in the PH but is not used in the image decoding process. To reduce overhead in the picture, in some examples, the elements in Table 4... ph_inter_slice_allowed_flag and ph_intra_slice_allowed_flag Used to conditionally signal intra-frame coding related syntax elements and inter-frame coding related syntax elements.

[0095] Table 3 Image Title RBSP Syntax

[0096] Table 4 Image Title Structure

[0097] III. Example Configuration File Information Table 5 shows some relevant examples, such as exemplary configuration file information in VVC. Configuration file information may also exist in SPS. profile_tier_level() In, and includes general constraint information. general_constraint_info （） As shown in Table 5.

[0098] Table 6 shows some relevant examples, such as exemplary general constraint information in VVC. General constraint information includes things like intra-frame-only constraint flags (e.g., those in Table 6). intra_only_constraint_flag The first flag of the configuration file can be used to indicate whether the slice type of an image slice conforming to the configuration file is an intra-slice. A first flag equal to 1 specifies that the slice type of an image slice conforming to the configuration file is an intra-slice (slice_type=I slice). A first flag equal to 0 does not impose such a constraint.

[0099] Table 5 Configuration file information in SPS

[0100] Table 6 General Constraint Information in Configuration Files

[0101] Bitstreams conforming to the Main 10 configuration file may comply with the following constraints: (i) the referenced SPS chroma_ format_idc (i) equal to 0 or 1; (ii) referenced SPS bit_depth_minus8 Within the range of 0 to 2, including 0 and 2; (iii) the reference SPS sps_palette_enabled_flag (iv) All values ​​of i in the ViewParameter Set (VPS) (if available) and the referenced SPS. general_level_idc and sublayer_level_idc[i] Not equal to 255 (which indicates level 8.5); and (v) can satisfy the layer and level constraints specified for the Main10 configuration file in VVC, if applicable.

[0102] The consistency between the bitstream and the Main 10 configuration file is determined by... general_profile_idc The value is equal to 1 to indicate that a decoder conforming to the Main 10 profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to the Main 10 profile; (ii) the bitstream is indicated to conform to a layer lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level other than 8.5 and lower than or equal to the specified level.

[0103] Bitstreams conforming to the Main 4:4:4 10 configuration file may adhere to the following constraints: (i) the referenced SPS chroma_format_id c is in the range of 0 to 3, inclusive; (ii) the reference SPS bit_depth_minus8 (iii) All values ​​of i in the VPS (if available) and the referenced SPS. general_level_ idcand sublayer_level_idc[i] (iv) equals 255 (indicating level 8.5); and (iv) can satisfy the layer and level constraints specified for the Main 4:4:4 10 configuration file in VVC, if applicable.

[0104] The consistency between the bitstream and the Main 4:4:4 10 configuration file is determined by... general_profile_idc The value is 2 to indicate this. A decoder that conforms to the Main4:4:4 10 profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to Main 4:4:4 10 or Main 10 profile; (ii) the bitstream is indicated to conform to a layer lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level other than 8.5 and lower than or equal to the specified level.

[0105] IV. Configuration file information for video sequences including intra-frame slices only. In some relevant examples, the SPS level flag (e.g., sps_inter_allowed_flag The flag () can be used to indicate the inclusion of intra-frame-only slices in an encoded video sequence or bitstream. This flag can be used to skip signaling for inter-frame coding-related syntax elements to reduce redundancy. For example, when sps_inter_allowed_flag When the value is 1, inter-frame coding related syntax elements can exist in SPS. sps_inter_allowed_flag When the value is 0, SPS can contain syntax elements that are only related to intra-frame coding.

[0106] Furthermore, in some relevant examples, the PPS level flag (e.g., pps_inter_allowed_flag The flag () can be used to indicate the inclusion of intra-frame-only slices in the encoded video sequence. This flag can be used to skip signaling for inter-frame coding-related syntax elements to reduce redundancy. For example, when pps_inter_allowed_flag When the value is 1, inter-frame coding related syntax elements can exist in PPS. pps_inter_allowed_flag When the value is 0, PPS can contain syntax elements that are only related to intra-frame coding.

[0107] Embodiments of this application include a method for using configuration file information to indicate when to include intra-frame-only slices in an encoded video sequence.

[0108] According to various aspects of embodiments of this application, all intra profiles can be used to indicate the inclusion of intra-only slices in an encoded video sequence. In all intra profiles, all slices conforming to that profile are configured for intra-coding. All intra profiles can be defined by information such as profile identification information (e.g., in Table 5). general_profile_ idc The configuration file information is used to indicate this.

[0109] Still picture profiles can be used to indicate the inclusion of intra-frame-only slices in an encoded video sequence. In a still picture profile, all slices conforming to that profile are configured for intra-frame coding. Still pictures can be identified by information such as profile identifiers (e.g., general profile IDC The information in the still image profile is used to indicate this. Still image profiles can be used for still photography acquired by a camera, computer-generated images, snapshots extracted from video sequences, etc. Still image profiles can have a subset of the capabilities of the Main 10 profiles mentioned above.

[0110] In one embodiment, for still image profiles, such as a single-image constraint flag (e.g., in Table 7) one_picture_only_constraint_flag The second flag can be included in the configuration file information. This single-picture constraint flag can indicate whether all slices are configured for intra-frame coding and whether there is only one picture in the encoded video sequence. In one example, a single-picture constraint flag equal to 1 specifies that all slices in the encoded video sequence that conform to the still-picture configuration file are configured for intra-frame coding (e.g., slice_type=I slice), and that there is only one picture in the encoded video sequence. A single-picture constraint flag equal to 0 does not impose such a constraint.

[0111] In one embodiment, configuration file information (e.g., profile_tier_level() General constraint information in ) (e.g., general_constraint_info() The only-image constraint flag is indicated by a signal. Table 7 provides examples of general constraint information including the only-image constraint flag. As mentioned above, the general constraint information in Table 7 can be included in, for example, Table 5. profile_tier_level() In the configuration file information.

[0112] Table 7 General constraint information including constraint markers for a single image

[0113] According to various aspects of the embodiments of this application, one or more constraints may be applied between the first flag and the second flag, such as an intra-frame constraint flag and a single-picture constraint flag in bitstream consistency.

[0114] In one embodiment, as shown in Table 7, the One-Picture-Only Constraint Flag is signaled before the Intra-Frame-Only Constraint Flag. The One-Picture-Only Constraint Flag indicates that all slices in the encoded video sequence conforming to the still picture profile are configured to be intra-frame coded and included in a single picture (e.g., in Table 7). one_picture_only_constraint_ flag =1), the intra-frame-only constraint flag can be set based on bitstream consistency to indicate that a slice conforming to a still image profile is an intra-frame slice (e.g., in Table 7). intra_only_constraint_flag =1).

[0115] In one embodiment, a single-image constraint flag equal to 1 specifies that the slice type conforming to the still image profile is an intra-frame slice (e.g., ...). slice_typ e=I slice), and there is only one image in the bitstream. A single-image constraint flag equal to 0 does not impose such a constraint. A single-frame-only constraint flag equal to 1 specifies that the slice type is an intra-frame slice ( slice_type =I slice). An intra-frame constraint flag equal to 0 does not impose such a constraint. Based on bitstream consistency, the intra-frame constraint flag is also true when only one image constraint flag is true.

[0116] In one embodiment, when the only picture constraint flag is equal to 1, the only picture in the encoded video sequence can be an Intra Random Access Picture (IRAP), such as an Instantaneous Decoding Refresh (IDR) picture or a Clean Random Access (CRA) picture.

[0117] In one embodiment, when the only image constraint flag is equal to 1, the Video Parameter Set (VPS) may not exist, and the number of layers of the encoded video sequence may be equal to 1.

[0118] In one embodiment, when the only picture constraint flag is equal to 1, the Reference Picture List (RPL) and Picture Order Count (POC) may not exist in the picture title or slice title.

[0119] In one embodiment, when the constraint flag for only one image is equal to 1, the Access Unit Delimiter (AUD) and End Of Stream (EOS) Network Abstraction Layer (NAL) units may not exist in the bitstream.

[0120] In one embodiment, for all intra profiles, for example when the intra-only constraint flag is set to 1, the intra-only constraint flag indicates that all slices conforming to that profile are configured for intra-coding only. Therefore, intra-only slices can exist in the bitstream across all intra profiles.

[0121] In one embodiment, for a still image profile, both the single-image constraint flag and the intra-frame-only constraint flag can be set to 1, indicating that intra-frame-only slices and single-image segments can exist in the bitstream.

[0122] In one embodiment, for a still image profile, the single-image constraint flag can be set to 1, indicating that only intra-frame slices and only one image can exist in the bitstream.

[0123] According to various aspects of embodiments of this application, one or more non-intra-dependent syntax elements can be excluded based on the intra-only constraint flag. For example, when the intra-only constraint flag is present and indicates that all slices in the bitstream are configured for intra-coding, such as when the intra-only constraint flag is equal to 1, no signal is needed to notify non-intra-dependent syntax elements.

[0124] In one embodiment, when only the intra-frame constraint flag is present and indicates that all slices in the bitstream are configured for intra-frame coding, such as when the only-frame constraint flag is equal to 1, one or more flags can be set to 0. For example, the above-mentioned sps_inter_allowed_flag and pps_inter_allowed_flag Both can be set to 0. When only the intra-frame constraint flag is absent or equal to 0, such constraints do not apply to one or more flags, for example... sps_inter_allowed_ flag and pps_inter_allowed_flag .

[0125] According to various aspects of embodiments of this application, one or more non-intra-relevant syntax elements can be excluded based on a single-picture constraint flag. For example, when the single-picture constraint flag is present and indicates that all slices in the bitstream are configured for intra-frame coding and that there is only one picture in the bitstream, such as when the single-picture constraint flag is equal to 1, no signal is needed to notify non-intra-relevant syntax elements.

[0126] In one embodiment, when the Only One Picture constraint flag exists and indicates that all slices in the bitstream are configured for intra-frame coding and that there is only one picture in the bitstream, for example, when the Only One Picture constraint flag is equal to 1, one or more flags can be set to 0. For example, the above... sps_inter_allowed_flag and pps_inter_ allowed_flag All can be set to 0. When only one image constraint flag is missing or equal to 0, such a constraint does not apply to one or more flags, for example... sps_inter_allowed_flag and pps_inter_allowed_flag .

[0127] According to various aspects of embodiments of this application, a third flag can be used to indicate that all slices in the encoded video sequence are configured to be intra-frame coded and included in a single frame. The third flag can be an SPS (Single Frame Only) frame-only flag, and is signaled separately from configuration file information. For example, in SPS, the SPS frame-only flag can be signaled. An SPS frame-only flag equal to 1 indicates that the slice type in the encoded video sequence is an intra-frame slice (…). slice_ type =I slice), and the sequence contains only one image. An SPS equal to 0 indicates that only one image exists, and this constraint is not imposed.

[0128] In one embodiment, one or more syntax elements can be excluded based on a one-picture-only constraint flag. For example, when the one-picture-only constraint flag is present and indicates that all slices in the bitstream are configured for intra-frame coding and that there is only one picture in the bitstream, such as when the one-picture-only constraint flag is equal to 1, no signal is used to notify non-intra-relevant syntax elements and / or syntax elements regarding POC values ​​and RPL.

[0129] In one embodiment, when the only-image-existence flag exists and is equal to 1, the SPS only-image-existence flag can be set to the same value as the only-image-existence flag based on bitstream consistency.

[0130] Table 8 shows some exemplary syntax elements in the general constraint information, including a picture constraint flag and an intra-frame-only constraint flag. As mentioned above, the general constraint information in Table 8 can be included in configuration file information such as Table 5. profile_tier_level() In Table 8, the constraint flag for an image is... general_one_picture_only_ constraint_flag. The general_one_picture_only_constraint_flag equal to 1 The specified bitstream contains only one encoded image, and the value equals 0. general_one_picture_only_constraint_flag Then no such constraint is imposed. Furthermore, only intra-frame constraint flags are... intra_only_constraint_flag Equal to 1 intra_only_ constraint_flag The slice type specified in the slice title is an intra-frame slice. sh_slice_type =I), while equal to 0 intra_only_constraint_flag No such constraint shall be imposed. When general_one_picture_only_ constraint_flag When equal to 1, intra_only_constraint_flag The value can be set to 1.

[0131] Table 8 General Constraint Information Syntax

[0132] According to various aspects of the embodiments of this application, a bitstream conforming to Main 10 or Main 10 still image profile may comply with the following constraints: (i) in a bitstream conforming to Main 10 still image profile, the bitstream contains only one image; (ii) the referenced SPS... sps_chroma_format_idc (iii) The reference SPS is equal to 0 or 1. sps_bit_ depth_minus8 (iv) In the range of 0 to 2, inclusive; and (iv) in the bitstream conforming to the Main 10 still image profile, the referenced SPS. max_dec_pic_buffering_minus1[sps_max_sublayers_minus1] Equals 0; (v) reference SPS sps_palette_enabled_flag (vi) All values ​​of i in the referenced VPS (when available) and referenced SPS in a bitstream that conforms to the Main 10 profile but not the Main 10 still image profile. general_level_idc and sublayer_level_idc[i] Not equal to 255 (which indicates level 15.5); and (vii) can satisfy the layer and level constraints specified for Main 10 or Main10 still image profile in VVC, if applicable.

[0133] The consistency between the bitstream and the Main 10 configuration file can be indicated by configuration file identification information (e.g., in Table 5). general_profile_idc =1).

[0134] The consistency between the bitstream and the Main 10 still image profile can be determined by an image constraint flag (e.g., in Table 8). general_one_picture_only_constraint_flag =1) and configuration file identification information (e.g., in Table 5) general_profile_idc =1) Instruct together.

[0135] Note that the conditions for indicating consistency between the bitstream and the Main 10 still image profile are also met when the consistency between the bitstream and the Main 10 profile is indicated as specified above, and the indicated level is not level 15.5.

[0136] A decoder that conforms to the Main 10 profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to the Main 10 or Main 10 still image profile; (ii) the bitstream is indicated to conform to a layer that is lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level that is not 15.5 and is lower than or equal to the specified level.

[0137] A decoder that conforms to the Main 10 still image profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to the Main 10 still image profile; (ii) the bitstream is indicated to conform to a layer that is lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level that is not 15.5 and is lower than or equal to the specified level.

[0138] According to various aspects of the embodiments of this application, a bitstream conforming to a Main 4:4:4 10 or Main 4:4:4 10 still image profile may comply with the following constraints: (i) in a bitstream conforming to a Main 4:4:4 10 still image profile, the bitstream contains only one image; (ii) the referenced SPS... sps_chroma_format_idc Within the range of 0 to 3, including 0 and 3; (iii) the reference SPS sps_bit_depth_minus 8 is in the range of 0 to 2, inclusive; (iv) in the bitstream conforming to the Main 4:4:4 10 still image profile, the reference SPS max_dec_pic_buffering_ minus1[sps_max_sublayers_minus1] (v) All values ​​of i in the referenced VPS (when available) and referenced SPS in a bitstream that conforms to the Main 4:4:4 10 profile but not to the Main 4:4:4 10 still image profile. general_level_idc and sublayer_level_idc[i] Not equal to 255 (which indicates level 15.5); and (vi) can satisfy the layer and level constraints specified in the VVC for Main 4:4:4 10 or Main 4:4:4 10 still image profile, if applicable.

[0139] The consistency of the bitstream with the Main 4:4:4 10 configuration file is indicated by the configuration file identification information (e.g., in Table 5). general_profile_idc =2).

[0140] The consistency of the bitstream with the Main 4:4:4 10 still image profile is constrained by an image constraint flag (e.g., in Table 8). general_one_picture_only_constraint_flag =1) and configuration file identification information (e.g., in Table 5) general_profile_idc =2) Instruct together.

[0141] Note that the conditions for indicating the consistency of the bitstream with the Main 10 4:4:4 still image profile are also met when the consistency of the bitstream with the Main 10 4:4:4 profile is indicated as specified above, and the indicated level is not level 15.5.

[0142] A decoder that conforms to the Main 4:4:4 10 profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to the Main 4:4:4 10, Main 10, Main 4:4:4 10 still image, or Main 10 still image profile; (ii) the bitstream is indicated to conform to a layer that is lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level other than 15.5 and lower than or equal to the specified level.

[0143] A decoder that conforms to the Main 4:4:4 10 still image profile at a specific level of a specific layer is able to decode all bitstreams that meet all of the following conditions: (i) the bitstream is indicated to conform to the Main 4:4:4 10 still image or the Main 10 still image profile; (ii) the bitstream is indicated to conform to a layer that is lower than or equal to the specified layer; and (iii) the bitstream is indicated to conform to a level that is not 15.5 and is lower than or equal to the specified level.

[0144] V. General Constraint Flag Group The general constraint information described above may include multiple syntax elements (e.g., the syntax elements in Table 8). However, the decoder may only need to decode a subset of the multiple syntax elements to conform to the configuration file. Embodiments of this application include a method for grouping multiple syntax elements in the general constraint information. Grouping multiple syntax elements can allow the decoder to terminate the parsing of the general constraint information early, thereby speeding up the decoding process.

[0145] According to various aspects of the embodiments of this application, multiple syntax elements in general constraint information can be grouped based on, for example, the use case of a configuration file. Subgroups may also exist within a group. Each group or subgroup of syntax elements includes a trunk of syntax elements that exist consecutively in the bitstream. Therefore, the decoder can terminate the parsing of the general constraint information in advance if the grouping is known.

[0146] In some embodiments, the decoder may know the total number and order of the groups of syntax elements included in the general constraint information.

[0147] In one embodiment, as shown in Table 9, the general constraint information includes three sets of syntax elements: non-intra-non-inter-frame groups (GROUP I), intra-frame groups (GROUP II), and inter-frame groups (GROUP III). For example, intra-frame groups (GROUP II) include the backbone of syntax elements related to intra-frame coding tools, inter-frame groups (GROUP III) include the backbone of syntax elements related to inter-frame coding tools, and non-intra-non-inter-frame groups (GROUP I) include the backbone of syntax elements unrelated to either intra-frame or inter-frame coding tools. In one example, non-intra-non-inter-frame groups (GROUP I) appear first in the bitstream, followed by intra-frame groups (GROUP II), and inter-frame groups (GROUP III) appear last.

[0148] As mentioned above, the general constraint information in Table 9 can be included in the configuration file information, such as Table 5. profile_ tier_level() middle.

[0149] Table 9 contains general constraint information for the three sets of syntax elements.

[0150] In the Main 10 still image or Main 10 4:4:4 still image configuration file, bitstream consistency can be constrained by a single image constraint flag (e.g., in Table 9). general_one_picture_only_constraint_flag =1) and configuration file identification information (e.g., in Table 5) general_profile_idc =1) Intense together. When the Only Picture Constraint flag is equal to 1, the Intense Only Constraint flag is equal to 1. In other profiles, such as those that include Intense Only Pictures, the Intense Only Constraint flag can also be equal to 1.

[0151] When only the intra-frame constraint flag is equal to 1, the values ​​of the syntax elements in group III are as shown in Table 9. no_ref_wraparound_constraint_flag , no_temporal_mvp_constraint_flag All equal 1.

[0152] Therefore, given the known grouping of syntax elements, the decoder can terminate the parsing of general constraint information early, which is beneficial to decoding speed and process.

[0153] In one embodiment, as shown in Table 10, the general constraint information includes two sets of syntax elements: non-inter-frame groups (GROUP I) and inter-frame groups (GROUP II). Inter-frame groups (GROUP II) may include a backbone of syntax elements associated with the inter-frame coding tools. Non-inter-frame groups (GROUP I) may include a backbone of syntax elements not associated with the inter-frame coding tools. In one example, non-inter-frame groups (GROUP I) exist first in the bitstream, followed by inter-frame groups (GROUP II).

[0154] As mentioned above, the general constraint information in Table 10 can be included in the configuration file information, such as Table 5. profile_tier_level() middle.

[0155] Table 10 contains general constraint information for the two sets of syntax elements.

[0156] In the Main 10 still image or Main 10 4:4:4 still image configuration file, bitstream consistency can be constrained by a single image constraint flag (e.g., in Table 10). general_one_picture_only_constraint_flag =1) and configuration file identification information (in Table 5) general_profile_idc =1) Intense together. When the Only Picture Constraint flag is equal to 1, the Intense Only Constraint flag is equal to 1. In other profiles, such as those that include Intense Only Pictures, the Intense Only Constraint flag can also be equal to 1.

[0157] When only the intra-frame constraint flag is equal to 1, the values ​​of the syntax elements in Group II are as shown in Table 10. no_ref_wraparound_constraint_flag , no_temporal_mvp_constraint_flag All equal 1.

[0158] Therefore, given the known grouping of syntax elements, the decoder can terminate the parsing of general constraint information early, which is beneficial to decoding speed and process.

[0159] According to some embodiments, whenever general constraint information (e.g., in Table 9 or Table 10) is provided... general_ constraint_info() When a new syntax element is introduced, it can be included in an existing group or subgroup, so that general constraint information does not need to form a new group relative to the existing group.

[0160] In one embodiment, for general constraint information including non-intra-non-inter-frame groups (GROUP I), intra-frame groups (GROUP II), and inter-group groups (GROUP III), when a new syntax element related to an intra-frame coding tool is to be included in the general constraint information, the new syntax element can be included in the intra-frame group (GROUP II). When a new syntax element related to an inter-frame coding tool is to be included in the general constraint information, the new syntax element can be included in the inter-frame group (GROUP III). When a new syntax element is not related to either intra-frame coding or inter-frame coding tools, the new syntax element can be included in the non-intra-non-inter-frame group (GROUP I).

[0161] In the example shown in Table 11, a new syntax element, the unweighted prediction constraint flag, is introduced in the general constraint information, which includes three sets of syntax elements, specifying whether weighted prediction can be applied to P slices (e.g., ...). no_weighted_pred_ constraint_flag New syntax elements can be included in Group III of the inter-frame group because they are syntax elements associated with inter-frame coding tools.

[0162] As mentioned above, the general constraint information in Table 11 can be included in the configuration file information, such as Table 5. profile_tier_level() middle.

[0163] Table 11 introduces new syntax elements that include general constraint information from three sets of syntax elements.

[0164] In one embodiment, for general constraint information that includes both non-inter-frame groups (GROUP I) and inter-frame groups (GROUP II), when a new syntax element related to the inter-frame coding tool is to be included in the general constraint information, the new syntax element can be included in the inter-frame group (GROUP II). When a new syntax element unrelated to the inter-frame coding tool is to be included in the general constraint information, the new syntax element can be included in the non-inter-frame group (GROUP I).

[0165] In the examples shown in Table 12, a new syntax element, the unweighted prediction constraint flag, is introduced in the general constraint information, which includes two sets of syntax elements, specifying whether weighted prediction can be applied to P slices (e.g., ...). no_weighted_pred_ constraint_flag New syntax elements can be included in inter-frame groups (GROUP II) because they are syntax elements associated with inter-frame coding tools.

[0166] As mentioned above, the general constraint information in Table 12 can be included in the configuration file information, such as Table 5. profile_tier_level() middle.

[0167] Table 12 introduces new syntax elements that include constraint information from two sets of syntax elements.

[0168] According to various aspects of the embodiments of this application, byte alignment can be checked after each group or subgroup to facilitate parsing and early termination.

[0169] Table 13 shows exemplary byte alignment for each group in the general constraint information comprising three groups of syntax elements. Byte alignment is checked at the end of each group or subgroup. If a syntax element in a group or subgroup is not byte aligned, additional bits can be signaled to ensure that the total bits used for each group are byte aligned.

[0170] As mentioned above, the general constraint information in Table 13 can be included in the configuration file information, such as Table 5. profile_tier_level() middle.

[0171] Table 13 lists the byte alignment for each of the three sets of syntax elements, including general constraint information.

[0172] Table 14 shows exemplary byte alignments for each group in the general constraint information that includes two groups of syntax elements. Byte alignment is checked at the end of each group or subgroup. If a syntax element in a group or subgroup is not byte aligned, additional bits can be signaled to ensure that the total bits used for each group are byte aligned.

[0173] As mentioned above, the general constraint information in Table 14 can include configuration file information such as that in Table 5. profile_tier_level() middle.

[0174] Table 14 lists the byte alignment for each group in the general constraint information for the two groups of syntax elements.

[0175] VII. Flowchart Figure 8A flowchart of a general example process (800) according to an embodiment of this application is shown. In various embodiments, the process (800) is executed by a processing circuit, which is, for example, a processing circuit in a terminal device (210), (220), (230), and (240), a processing circuit that performs the function of a video encoder (303), a processing circuit that performs the function of a video decoder (310), a processing circuit that performs the function of a video decoder (410), a processing circuit that performs the function of an intra-frame prediction unit (452), a processing circuit that performs the function of a video encoder (503), a processing circuit that performs the function of a predictor (535), a processing circuit that performs the function of an intra-frame encoder (622), a processing circuit that performs the function of an intra-frame decoder (772), etc. In some embodiments, the process (800) is implemented in software instructions, so the processing circuit executes the process (800) when the processing circuit executes the software instructions.

[0176] The process (800) typically begins at step (S810), where the process (800) decodes profile information for multiple image slices in the prediction information of the encoded video bitstream. The profile information includes profile identifier information for the profile, in which each image slice in the encoded video bitstream is configured for intra-frame coding. Then, the process (800) proceeds to step (S820).

[0177] In step (S820), process (800) performs intra-frame prediction on each image slice in the encoded video bitstream. Then, process (800) proceeds to step (S830).

[0178] In step (S830), the process (800) reconstructs at least one image based on intra-frame prediction. Then, the process (800) terminates.

[0179] In one embodiment, the configuration file information includes a first flag indicating whether each image slice in the encoded video bitstream is configured for intra-frame coding, and a second flag indicating whether each image slice in the encoded video bitstream is included in a picture.

[0180] In one embodiment, the first flag is decoded after the second flag and based on the second flag indicating that each image slice in the encoded video bitstream is included in a picture, the first flag indicating that each image slice in the encoded video bitstream is configured to be intra-frame coded. In one embodiment, a first flag indicates that each image slice in the encoded video bitstream is configured to be intra-coded, based on profile identification information of a profile in which each image slice in the image slices of the encoded video bitstream is configured to be intra-coded.

[0181] In one embodiment, the configuration file is a still image configuration file in which the encoded video bitstream includes only one image, and the second flag indicates that each image slice in the image slices of the encoded video bitstream is included in one image.

[0182] In one embodiment, non-intra-relevant syntax elements are not included in the prediction information based on one of the following: (i) a first flag indicating that each image slice in the encoded video bitstream is configured to be intra-coded and (ii) a second flag indicating that each image slice in the encoded video bitstream is included in a picture.

[0183] In one embodiment, the prediction information includes a third flag indicating whether each image slice in the encoded video bitstream is configured to be intra-frame coded and included in a single picture. This third flag is not included in the configuration file information.

[0184] In one embodiment, a second flag indicates that each image slice in the encoded video bitstream is included in a picture, and a third flag indicates that each image slice in the encoded video bitstream is configured to be intra-frame coded and included in a picture.

[0185] Figure 9 Another flowchart of a general example process (900) according to an embodiment of this application is shown. In various embodiments, the process (900) is executed by a processing circuit, which is, for example, a processing circuit in a terminal device (210), (220), (230), and (240), a processing circuit that performs the function of a video encoder (303), a processing circuit that performs the function of a video decoder (310), a processing circuit that performs the function of a video decoder (410), a processing circuit that performs the function of an intra-frame prediction unit (452), a processing circuit that performs the function of a video encoder (503), a processing circuit that performs the function of a predictor (535), a processing circuit that performs the function of an intra-frame encoder (622), a processing circuit that performs the function of an intra-frame decoder (772), etc. In some embodiments, the process (900) is implemented in software instructions, so that the processing circuit executes the process (900) when the processing circuit executes the software instructions.

[0186] The process (900) typically begins at step (S910), where the process (900) decodes the profile information in the prediction information of the encoded video bitstream. The profile information includes multiple sets of syntax elements and indicates the profile of the encoded video bitstream. Then, the process (900) proceeds to step (S920).

[0187] In step (S920), process (900) determines at least one set of multiple sets of syntax elements based on the configuration file indicated in the configuration file information. Then, process (900) proceeds to step (S930).

[0188] In step (S930), process (900) decodes the syntax elements included in the prediction information based on at least one of the determined sets of syntax elements. Then, process (900) proceeds to step (S940).

[0189] In step (S940), process (900) reconstructs at least one image based on the decoded syntax elements included in the prediction information.

[0190] In one embodiment, the order of at least one of the multiple sets of syntax elements in the determined configuration file is consistent with the predetermined order of the multiple sets of syntax elements in the configuration file information.

[0191] In one embodiment, byte alignment is checked for each of the multiple sets of syntax elements in the configuration file information.

[0192] VIII. Computer System The above techniques can be implemented as computer software, which uses computer-readable instructions and is physically stored in one or more computer-readable media. For example, Figure 10 A computer system (1000) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0193] Computer software can be coded using any suitable machine code or computer language. Any suitable machine code or computer language can be assembled, compiled, linked, or similarly processed to create code containing instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpretation, microcode execution, etc.

[0194] The instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, and Internet of Things devices.

[0195] Figure 10 The components of the computer system (1000) shown are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of the computer software implementing the embodiments of this application. The configuration of the components should also not be construed as having any dependencies or requirements relating to any component or combination of components shown in the exemplary embodiments of the computer system (1000).

[0196] The computer system (1000) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., speech, clapping), visual input (e.g., gestures), or olfactory input (not shown). The human-machine interface device may also be used to acquire certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0197] The input human-machine interface device may include one or more of the following (only one of each is shown): keyboard (1001), mouse (1002), touchpad (1003), touch screen (1010), data glove (not shown), joystick (1005), microphone (1006), scanner (1007), camera (1008).

[0198] The computer system (1000) may also include certain human-machine interface output devices. Such human-machine interface output devices may, for example, stimulate the senses of one or more human users through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (1010), a data glove (not shown), or a joystick (1005), but may also be tactile feedback devices that are not input devices), audio output devices (e.g., speakers (1009), headphones (not depicted)), visual output devices (e.g., screens (1010) including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input functionality, each with or without tactile feedback functionality—some of these screens are capable of outputting two-dimensional or more three-dimensional visual outputs via devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted), and printers (not depicted). These visual output devices (e.g., screens (1010)) may be connected to the system bus (1048) via a graphics adapter (1050).

[0199] The computer system (1000) may also include human-machine-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1020) having media such as CD / DVD (1021), finger drives (1022), removable hard disk drives or solid-state drives (1023), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted), etc.

[0200] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0201] The computer system (1000) may also include an interface (1054) for connecting one or more communication networks (1055). The one or more communication networks (1055) may be, for example, wireless networks, wired networks, or optical networks. The one or more communication networks (1055) may further be local networks, wide area networks, metropolitan area networks, vehicle and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of the one or more communication networks (1055) include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter (e.g., a USB port of the computer system (1000)) to connect to some general-purpose data port or peripheral bus (1049); other network interfaces are typically integrated into the core of the computer system (1000) by connecting to the system bus (e.g., an Ethernet interface in a PC computer system or a cellular network interface in a smartphone computer system), as described below. The computer system (1000) can use any of these networks to communicate with other entities. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., CANbus connected to some CANbus device), or bidirectional, such as connecting to other computer systems using a local area or wide area digital network. As mentioned above, certain protocols and protocol stacks can be used on each of these networks and network interfaces.

[0202] The aforementioned human-machine interface device, human-machine accessible storage device, and network interface can be attached to the kernel (1040) of the computer system (1000).

[0203] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), dedicated programmable processing units (1043) in the form of field-programmable gate areas (FPGAs), hardware accelerators (1044) for certain tasks, etc. These devices, along with read-only memory (ROM) (1045), random access memory (1046), and internal mass storage such as internal non-user-accessible hard disk drives (SDs), etc. (1047), may be connected via a system bus (1048). In some computer systems, the system bus (1048) may be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly connected to the core's system bus (1048) or connected via a peripheral bus (1049) to the core's system bus (1048). Peripheral bus architectures include PCI, USB, etc.

[0204] The CPU (1041), GPU (1042), FPGA (1043), and accelerator (1044) can execute certain instructions that can be combined to form the aforementioned computer code. This computer code can be stored in ROM (1045) or RAM (1046). Transient data can also be stored in RAM (1046), while permanent data can be stored, for example, in internal mass storage (1047). Fast storage and retrieval to any storage device can be achieved using a cache, which can be closely associated with one or more CPUs (1041), GPUs (1042), mass storage (1047), ROM (1045), RAM (1046), etc.

[0205] A computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be media and computer code specifically designed and constructed for the purposes of embodiments of this application, or the medium and computer code may be of types known and available to those skilled in the art of computer software.

[0206] By way of example and not limitation, a computer system having an architecture (1000), particularly a kernel (1040), can be made functional by one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software contained in one or more tangible computer-readable media. Such computer-readable media may be media associated with user-accessible mass storage as described above, and some non-transitory memory of the kernel (1040), such as internal kernel mass storage (1047) or ROM (1045). Software implementing various embodiments of the present application may be stored in such means and executed by the kernel (1040). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software may cause the kernel (1040), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute a particular process or a particular portion of a particular process described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to a software-defined process. Additionally or alternatively, the computer system may provide functionality due to logic hardwired or otherwise embodied in the circuitry (e.g., the accelerator (1044)), which may replace or operate with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. Embodiments of this application include any suitable combination of hardware and software.

[0207] It should be noted that the data collection and processing in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0208] Although several exemplary embodiments have been described in this application, modifications, substitutions, and various equivalent replacements may be made that fall within the scope of the embodiments of this application. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, although not expressly shown or described herein, embody the principles of the embodiments of this application and thus fall within the spirit and scope of the embodiments of this application.

[0209] Appendix A: Abbreviations AMVP: Advanced Motion Vector Prediction ASIC: Application-Specific Integrated Circuit ATMVP: Alternating / Advanced Temporal Motion Vector Prediction BMS: Benchmark Set BV: Block Vector CANBus: Controller Area Network Bus CB: Coded Block CD: CD-ROM CPR: Current image reference CPU: Central Processing Unit CRT: Cathode Ray Tube CTB: Coded Tree Block CTU: Coding Tree Unit CU: Encoding Unit DPB: Decoder Image Buffer DVD: Digital Video Disc FPGA: Field Programmable Gate GOP: Image Group GPU: Graphics Processing Unit GSM: Global System for Mobile Communications HEVC: High Efficiency Video Coding HRD: Hypothetical Reference Decoder IBC: Intra-Block Copy IC: Integrated Circuit JEM: Joint Exploration Model JVET: Joint Video Exploration Team LAN: Local Area Network LCD: Liquid Crystal Display LTE: Long Term Evolution MV: Motion Vector OLED: Organic Light Emitting Diode PB: Prediction Block PCI: Peripheral Component Interconnect PH: Image title PLD: Programmable Logic Device PPS: Image Parameter Settings POC: Image Sequential Counting PU: Prediction Unit RAM: Random Access Memory RBSP: Raw Byte Sequence Payload ROM: Read-Only Memory RPL: List of Reference Images SCC: Screen Content Encoding SDR: Standard Dynamic Range SEI: Supplemental Enhancement Information SNR: Signal-to-noise ratio SPS: Sequence Parameter Set SSD: Solid State Drive TU: Transformation Unit USB: Universal Serial Bus VUI: Video Availability Information VVC: Multi-functional Video Coding

Claims

1. A method for video decoding in a decoder, characterized in that, include: Decode the configuration information for multiple image slices in the prediction information of the encoded video bitstream, the configuration information including multiple sets of syntax elements, the configuration information being used to indicate the configuration for the encoded video bitstream; The profile information is decoded according to the profile_tier_level() syntax structure indicated by the high-level syntax, wherein the profile_tier_level() syntax structure includes: a first variable profileTierPresentFlag and a second variable maxNumSubLayersMinus1; When the first variable profileTierPresentFlag is equal to 1, the first syntax element general_profile_idc and the second syntax element general_tier_flag are decoded, and the general constraint information of the General_constraint_info() syntax structure is received, wherein the General_constraint_info() syntax structure is contained in the profile_tier_level() syntax structure; In all cases where the first variable profileTierPresentFlag has any value, the third syntax element general_level_idc is decoded. When decoding the fourth syntax element intra_only_constraint_flag contained in the general constraint information of the General_constraint_info() syntax structure, if the fourth syntax element intra_only_constraint_flag is equal to 1, the slice type of the plurality of image slices is intra-frame slice, and the slice type is determined by ( slice_type =I slice) represents; In the high-level syntax, there is a fifth syntax element to indicate whether decoding is performed in intra-frame mode. When the fifth syntax element is 0, all image slices received from the encoded video bitstream are intra-frame slices.

2. The method according to claim 1, characterized in that, The method further includes: Decode the sixth syntax element general_frame_only_constraint_flag contained in the configuration file information.

3. The method according to claim 1 or 2, characterized in that, The fifth syntax element is sps_inter_allowed_flag and / or pps_inter_allowed_flag; When the fourth syntax element intra_only_constraint_flag exists and is equal to 1, the sps_inter_allowed_flag and / or the pps_inter_allowed_flag are set to 0 in the encoded video bitstream.

4. The method according to claim 3, characterized in that, When the fourth syntax element intra_only_constraint_flag is equal to 1, both sps_inter_allowed_flag and pps_inter_allowed_flag are set to 0; When the fourth syntax element intra_only_constraint_flag does not exist in the encoded video bitstream, or when the fourth syntax element intra_only_constraint_flag is equal to 0, the sps_inter_allowed_flag and the pps_inter_allowed_flag are not restricted to 0; When the sps_inter_allowed_flag is equal to 0, there are no syntax elements related to inter-frame coding in the sequence parameter set, and the decoder decodes syntax elements related to intra-frame coding in the sequence parameter set. When pps_inter_allowed_flag equals 0, there are no syntax elements related to inter-frame coding in the image parameter set, and the decoder decodes syntax elements related to intra-frame coding in the image parameter set.

5. The method according to claim 1, characterized in that, The configuration file information also includes the syntax element one_picture_only_constraint_flag, which indicates whether each image slice in the image slice in the encoded video bitstream is included in a picture.

6. The method according to claim 5, characterized in that, The fourth syntax element intra_only_constraint_flag is decoded after the syntax element one_picture_only_constraint_flag; The fourth syntax element intra_only_constraint_flag is set to 1 in the encoded video bitstream when the syntax element one_picture_only_constraint_flag indicates that each image slice in the image slices in the encoded video bitstream is included in a picture.

7. The method according to claim 1, characterized in that, The method further includes: Based on the first syntax element `general_profile_idc`, it is determined that the encoded video bitstream conforms to one of the Main 10 still image profile and the Main 4:4:4 10 still image profile; and For an encoded video bitstream that contains multiple image slices and conforms to the Main 10 still image profile or the Main 4:4:410 still image profile, the encoded video bitstream is constrained to include only one image.

8. The method according to claim 1, characterized in that, The method further includes: When the fourth syntax element intra_only_constraint_flag is equal to 1, intra-frame prediction is performed on each image slice in the image slice of the encoded video bitstream. At least one image is reconstructed based on the intra-frame prediction.

9. A video encoding method, characterized in that, include: Configuration information for multiple image slices is encoded, the configuration information including multiple sets of syntax elements, the configuration information being used to indicate the configuration file for the encoded video bitstream; The profile information is encoded according to the profile_tier_level() syntax structure indicated by the high-level syntax, wherein the profile_tier_level() syntax structure includes: a first variable profileTierPresentFlag and a second variable maxNumSubLayersMinus1; When the first variable profileTierPresentFlag equals 1, the first syntax element general_profile_idc and the second syntax element general_tier_flag are encoded, and the general constraint information of the General_constraint_info() syntax structure is obtained, wherein the General_constraint_info() syntax structure is contained in the profile_tier_level() syntax structure; In all cases where the first variable profileTierPresentFlag has any value, the third syntax element general_level_idc is encoded. When encoding the fourth syntax element intra_only_constraint_flag contained in the general constraint information of the General_constraint_info() syntax structure, if the fourth syntax element intra_only_constraint_flag is equal to 1, the slice type of the plurality of image slices is an intra-frame slice, and the slice type is represented by (slice_type=I slice); In the high-level syntax, there is a fifth syntax element to indicate whether encoding is in intra-frame mode. When the fifth syntax element is 0, all image slices obtained from the encoded video bitstream are intra-frame slices.

10. An apparatus for video decoding in a decoder, characterized in that, The apparatus includes processing circuitry configured to perform a video decoding method in a decoder according to any one of claims 1 to 8.

11. A video encoding apparatus, characterized in that, The apparatus includes processing circuitry configured to perform the video encoding method according to claim 9.

12. A non-transitory computer-readable storage medium for storing instructions, characterized in that, When executed by at least one processor, the instructions cause the at least one processor to perform the video decoding method for use in a decoder according to any one of claims 1 to 8, or the video encoding method according to claim 9.

13. A method for storing a bit stream, characterized in that, The bitstream is generated based on the video encoding method of claim 9; and the bitstream is stored.

14. A method for transmitting a bit stream, characterized in that, The bitstream is generated based on the video encoding method of claim 9; and the bitstream is transmitted.

Images