Method, apparatus, storage medium and device for encoding a source video sequence

Abstract

Embodiments of the present application provide a method, device, storage medium and equipment for encoding a source video sequence. The method comprises: encoding a plurality of candidate resolutions into a parameter set of a bitstream; selecting a target candidate resolution from the plurality of candidate resolutions; resampling samples in a group of samples in the source video sequence based on an output resolution and the target candidate resolution; encoding an index into a temporal header structure applicable to the group of samples, the index indicating the target candidate resolution among the plurality of candidate resolutions; performing prediction using the resampled samples; filtering the group of samples after the resampling, and encoding the group of samples after the filtering into the bitstream according to meta information of a previous part of an encoded image or an encoded image sequence.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 816,667, filed March 11, 2019, with the United States Patent and Trademark Office, which is incorporated herein by reference in its entirety. Technical Field

[0003] The subject matter disclosed in this application relates to video encoding and decoding, and more specifically, to methods, apparatus, storage media, and devices for encoding source video sequences. Background Technology

[0004] Examples of video encoding and decoding using motion-compensated inter-picture prediction have been around for decades. Uncompressed digital video can consist of a series of pictures, each with a certain spatial dimension, such as 1920 × 1080 luminance samples and associated chrominance samples. This series of pictures can have a fixed or variable picture rate (commonly known as the frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has very high bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920 × 1080 luminance sample resolution at a 60 Hz frame rate) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 GBytes of storage.

[0005] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce bandwidth or storage requirements, in some cases by two or more orders of magnitude. Lossless compression, lossy compression, and combinations thereof can all be used for video encoding and decoding. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and the reconstructed signal is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion tolerated by lossy compression depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion compared to users of television distribution applications. The achievable compression ratio reflects the higher the allowable / acceptable distortion, the higher the compression ratio that can be produced.

[0006] Video encoders and decoders can use several major categories of techniques, including motion compensation, transform, quantization, and entropy encoding / decoding. Some of these techniques will be introduced below.

[0007] Historically, video encoders and decoders have tended to operate on a given image size, which in most cases is well-defined and constant for a Coded Video Sequence (CVS), Group of Pictures (GOP), or similar multi-image timeframe. For example, in MPEG-2, it is well-known that system design varies the horizontal resolution (and thus the image size) based on factors such as scene activity, only at the I-image level, and therefore typically for the GOP. Resampling a reference image at a different resolution within a CVS is known, for example, from Annex P of the ITU-T H.263 Recommendation. However, here, the image size is not changed; only the reference image is resampled, which may result in only using a portion of the image canvas (in the case of downsampling) or only capturing a portion of the scene (in the case of upsampling). Furthermore, H.263 Annex Q allows for a doubling of resampling up or down (in each dimension) of a single macroblock. Again, the image size remains the same. In H.263, the size of a macroblock is fixed, and therefore there is no need to signal the size of the macroblock.

[0008] In modern video codecs, predicting changes in image size within a picture has become mainstream. For example, VP9 allows for reference image resampling and changes in the resolution of the entire picture. Similarly, certain proposals have been made for VVC (including, for example, Hendry et al., “On adaptive resolution change (ARC) for VVC,” Joint Video Group document JVET-M0135-v1, January 9-19, 2019, in its entirety incorporated herein by reference), allowing the entire reference image to be resampled to different higher or lower resolutions. In that document, it is proposed to encode different candidate resolutions in a sequence parameter set and reference them by syntax elements for each picture in the picture parameter set.

[0009] To adapt to application scenarios with adaptive image sizes, how to send signals in the video stream to notify the adaptive image size in order to achieve predictive encoding and decoding of adaptive images has become an urgent technical problem to be solved. Summary of the Invention

[0010] According to an embodiment, a method for encoding a source video sequence into a bitstream includes: encoding a plurality of candidate resolutions into a parameter set of the bitstream; selecting a target candidate resolution from the plurality of candidate resolutions; resampling samples from a set of samples in the source video sequence based on an output resolution and a target candidate resolution; encoding an index into a transient header structure applicable to the set of samples, the index indicating the target candidate resolution among the plurality of candidate resolutions; performing prediction using the resampled samples; filtering the sampled set of samples; and encoding the filtered set of samples into the bitstream based on metadata of a previously encoded image or a previous portion of a sequence of encoded images.

[0011] According to an embodiment, an apparatus for encoding a source video sequence into a bitstream includes:

[0012] The encoding module is used to encode multiple candidate resolutions into a parameter set of the bitstream;

[0013] The selection module is used to select a target candidate resolution from multiple candidate resolutions;

[0014] A resampling module is used to resample samples from a set of samples in the source video sequence based on the output resolution and the target candidate resolution;

[0015] The encoding module is further configured to: encode an index into a transient header structure applicable to the set of samples, the index indicating a target candidate resolution among the plurality of candidate resolutions; perform prediction using resampled samples; filter the sampled set of samples and encode the filtered set of samples into a bitstream based on metadata of a previous portion of an encoded image or a sequence of encoded images.

[0016] According to an embodiment, a non-volatile computer-readable storage medium stores instructions and a code stream, the instructions causing a processor to perform the aforementioned method to obtain the code stream.

[0017] According to an embodiment, a computer device includes: a memory for storing computer-readable instructions; and a processor for reading the computer-readable instructions and executing the aforementioned method as instructed by the computer-readable instructions.

[0018] A method for storing or transmitting a bitstream, comprising performing the aforementioned video encoding method to generate a bitstream; and storing or transmitting the bitstream.

[0019] According to the embodiments of this application, the method, apparatus, readable storage medium, and computer device for encoding a source video sequence into a bitstream include placing the resolution of an adaptive image into the transient header structure of the bitstream. This allows the size of the signaling for the adaptive image size to vary depending on the image or a portion of the image, saving codewords and providing better fault tolerance, thus improving encoding and decoding efficiency. Attached Figure Description

[0020] Further features, essence, and various advantages of the subject matter disclosed in this application will become clearer from the following detailed description and accompanying drawings, wherein:

[0021] Figure 1 This is a simplified block diagram of a communication system according to an embodiment.

[0022] Figure 2 This is a simplified block diagram of another communication system according to an embodiment.

[0023] Figure 3 This is a simplified block diagram of the decoder according to an embodiment.

[0024] Figure 4 This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0025] Figure 5A This is a schematic diagram of the options for signaling ARC parameters, as indicated.

[0026] Figure 5B This is a schematic diagram of an option for signaling ARC parameters according to an embodiment, as indicated.

[0027] Figure 6A This is a schematic diagram according to an embodiment.

[0028] Figure 6B This is a flowchart illustrating a method for decoding encoded images of an encoded video sequence according to an embodiment.

[0029] Figure 6C This is a simplified block diagram of an apparatus for controlling the decoding of a video sequence according to an embodiment.

[0030] Figure 7 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0031] Recently, the compression-domain aggregation or extraction of multiple semantically independent image portions into a single video image has attracted some attention. Specifically, in the context of applications such as 360-degree encoding / decoding or certain other applications, multiple semantically independent source images (e.g., the six cube surfaces of a 360-degree scene in a cube projection, or individual camera inputs in a multi-camera application setting) may require separate adaptive resolution settings to handle the different scene activities at a given time point. In other words, at a given time point, the encoder may choose to use different resampling factors for the different semantically independent images that make up the entire 360-degree or other scene. When combined into a single image, this in turn requires resampling of the reference image, and adaptive resolution encoding / decoding signaling is available for each portion of the encoded image.

[0032] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of this application is illustrated. The communication system (100) may include at least two terminals (110-120) interconnected via a network (150). For one-way data transmission, a first terminal (110) may encode video data at a local location for transmission to another terminal (120) via the network (150). A second terminal (120) may receive the encoded video data from the other terminal from the network (150), decode the encoded data, and display the recovered video data. One-way data transmission may be common in media service applications, etc.

[0033] Figure 1 The illustration shows a second pair of terminals (130, 140) for supporting bidirectional transmission of encoded video, which may occur, for example, during a video conference. For bidirectional data transmission, each terminal (130, 140) can encode video data acquired at its local location for transmission to the other terminal via a network (150). Each terminal (130, 140) can also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0034] exist Figure 1In this application, terminals (110-140) may be illustrated as servers, personal computers, and smartphones, but the principles disclosed herein are not limited to these. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (150) refers to any number of networks that transmit encoded video data between terminals (110-140), including, for example, wired and / or wireless communication networks. Communication networks (150) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (150) may be irrelevant to the operation of this application.

[0035] As an example of the application of the subject matter disclosed in this application Figure 2 The placement of a video encoder and video decoder in a streaming environment is illustrated. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0036] The streaming system (200) may include an acquisition subsystem (213) that may include a video source (201) such as a digital camera, which is used to create, for example, an uncompressed video sample stream (202). The sample stream (202) is depicted as a thick line to emphasize its high data volume compared to an encoded video stream, and the sample stream (202) may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video stream (204) is depicted as a thin line to emphasize its low data volume compared to the sample stream, and the encoded video stream (204) may be stored on a streaming server (205) for future use. At least one streaming client (206, 208) can access a streaming server (205) to retrieve a copy (207, 209) of the encoded video stream (204). The client (206) may include a video decoder (210). The video decoder (210) decodes the incoming copy (207) of the encoded video stream and produces an output video sample stream (211) that can be displayed on a display (212) or other presentation device (not depicted). In some streaming systems, the video stream (204, 207, 209) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. The video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the subject matter disclosed in this application can be used in the context of VVC standards.

[0037] Figure 3 It may be a block diagram of a video decoder (210) according to an embodiment of this application.

[0038] The receiver (310) can receive at least one encoded video sequence to be decoded by the decoder (210); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences can be received from a channel (312), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (310) can receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not shown). The receiver (310) can separate the encoded video sequences from other data. To prevent network jitter, a buffer (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter referred to as the "parser"). The buffer (buffer memory) (315) may not be required when the receiver (310) receives data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isochronous synchronization network, or the buffer may be very small. For use on best-effort packet networks such as the Internet, a buffer (315) may also be required, which may be relatively large and have an adaptive size.

[0039] The video decoder (210) may include a parser (320) to reconstruct symbols (321) from an entropy-coded video sequence. These symbols may include information for managing the operation of the decoder (210), and potentially information for controlling a display device (212) that is not part of the decoder but may be coupled to it, such as... Figure 2As shown in the diagram. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be performed according to video coding techniques or standards, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence based on at least one parameter corresponding to a group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), etc. The entropy decoder / parser can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter (QP) values, motion vectors, and so on.

[0040] The parser (320) can perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to create symbols (321).

[0041] The reconstruction of the symbol (321) may involve multiple different units, depending on the type of the encoded video picture or its parts (such as inter-frame pictures and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors. Which units are involved and how they are involved can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (320). For clarity, the flow of such subgroup control information between the parser (320) and the multiple units described below is not described.

[0042] In addition to the functional blocks already mentioned, decoder 210 can be conceptually subdivided into several functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the subject matter disclosed in this application, it is appropriate to conceptually subdivide them into the functional units described below.

[0043] The first unit is the scaler / inverse transform unit (351). The scaler / inverse transform unit (351) receives quantization transform coefficients as symbols (321) and control information from the parser (320), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (351) can output a block containing sample values, which can be input into the aggregator (355).

[0044] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-prediction unit (352). In some cases, the intra-prediction unit (352) takes the surrounding reconstructed information extracted from the current (partially reconstructed) image (356) and generates blocks of the same size and shape as the blocks being reconstructed. In some cases, the aggregator (355) adds the predictive information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) based on each sample.

[0045] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (353) may access the reference image memory / buffer (357) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (321) associated with the block, these samples may be added by the aggregator (355) to the output of the scaler / inverse transform unit (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation prediction unit may obtain the predicted samples from the address in the reference image memory under motion vector control, and the motion vector is available to the motion compensation prediction unit in the form of the symbols (321), which may include, for example, X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0046] The output samples of the aggregator (355) can be processed by various loop filtering techniques in the loop filter unit (354). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream, and these parameters can be used as symbols (321) from the parser (320) in the loop filter unit (354). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0047] The output of the loop filter unit (354) can be a sample stream, which can be output to a display device (212) and stored in a reference image memory (357) for subsequent inter-frame image prediction.

[0048] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. Once the encoded images have been fully reconstructed and the encoded images (by, for example, a parser (320)) are identified as reference images, the current reference image (356) can become part of the reference image memory (357), and a new current image memory can be reallocated before the reconstruction of subsequent encoded images begins.

[0049] The video decoder 210 can perform decoding operations according to predetermined video compression techniques, such as those described in ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard, as specified in the video compression technique document or standard, and particularly in its configuration files. For compliance purposes, the complexity of the encoded video sequence must be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0050] In this embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0051] Figure 4 This is a block diagram of a video encoder (203) according to an embodiment disclosed in this application.

[0052] The encoder (203) can receive video samples from a video source (201) (not part of the encoder), which can capture video images that will be encoded by the encoder (203).

[0053] A video source (201) can provide a sequence of source videos in the form of a digital video sample stream to be encoded by an encoder (203). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601 YCrCb, RGB…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (201) can be a storage device storing previously prepared video. In a video conferencing system, the video source (201) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed sequentially. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include at least one sample, depending on the sampling structure, color space, etc., used. The following focuses on describing the samples.

[0054] According to an embodiment, the encoder (203) can encode and compress images of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (450). The controller controls and is functionally coupled to other functional units described below. For clarity, coupling is not indicated in the figures. Parameters set by the controller may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of the controller (450) may be performed, as these functions may involve a video encoder (203) optimized for a particular system design.

[0055] Some video encoders operate within an “encoding loop” readily recognizable to those skilled in the art. In a simplified description, the encoding loop may include the encoding portion of an encoder (430) (hereinafter referred to as the “source encoder”) responsible for creating symbols based on the input image to be encoded and a reference image, and a (local) decoder (433) embedded within the encoder (203). The decoder (433) reconstructs the symbols to create sample data that the (remote) decoder would also create (since any compression between the symbols and the encoded video stream is lossless in the video compression techniques considered in the subject matter disclosed herein). The reconstructed sample stream is input to a reference image memory (434). Since the decoding of the symbol stream produces bit-precise results independent of the decoder location (local or remote), the contents of the reference image buffer are also bit-precisely corresponding between the local and remote encoders. In other words, the reference image samples “seen” by the encoder’s prediction portion are exactly the same sample values ​​that the decoder will “see” during the prediction process. This is the fundamental principle of reference image synchronization (and the drift that occurs, for example, due to channel errors, when synchronization cannot be maintained).

[0056] The operation of the "local" decoder (433) can be combined with the above. Figure 3 The "remote" decoder (210) described in detail is the same. However, another brief reference is provided. Figure 3 When symbols are available and the entropy encoder (445) and parser (320) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding part of the decoder (210), including the channel (312), receiver (310), buffer (315) and parser (320), may not all be implemented in the local decoder (433).

[0057] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the subject matter of this application focuses on the operation of the decoder. The description of the encoder technique can be simplified because the encoder technique is the inverse of the fully described decoder technique. Only certain areas require more detailed description, which is provided below.

[0058] As part of its operation, the source encoder (430) can perform motion-compensated predictive coding, predictively coding the input frame with reference to at least one previously encoded frame from the video sequence designated as a "reference frame". In this way, the encoding engine (432) encodes the differences between pixel blocks of the input frame and pixel blocks of the reference frame, which can be selected as the prediction reference for the input frame.

[0059] The local video decoder (433) can decode encoded video data of a frame that can be designated as a reference frame based on symbols created by the source encoder (430). The operation of the encoding engine (432) can be a lossy process. When the encoded video data can be decoded at the video decoder, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process, which can be performed by the video decoder on the reference frame, and can store the reconstructed reference frame in a reference picture cache / memory (434). In this way, the encoder (203) can locally store a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame that will be obtained by the remote video decoder.

[0060] The predictor (435) can perform a prediction search against the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) can search in the reference image memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (435) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (435), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (434).

[0061] The controller (450) can manage the encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0062] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (445). The entropy encoder performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0063] The transmitter (440) can buffer the encoded video sequence created by the entropy encoder (445) in preparation for transmission via a communication channel (460), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (440) can combine the encoded video data from the source encoder (430) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0064] The controller (450) manages the operation of the encoder (203). During encoding, the controller (450) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following frame types:

[0065] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Variations of I-pictures and their corresponding applications and features can be used.

[0066] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0067] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0068] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively encoded with reference to other (already encoded) blocks, determined based on the encoding assignment of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively encoded, or the blocks can be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of a P-image can be predictively encoded with reference to a previously encoded reference image via spatial prediction or temporal prediction. Blocks of a B-image can be predictively encoded with reference to one or two previously encoded reference images via spatial prediction or temporal prediction.

[0069] The video encoder (203) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (203) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0070] In this embodiment, the transmitter (440) may transmit additional data while transmitting encoded video. The source encoder (430) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0071] Before describing certain aspects of the subject matter disclosed in this application in more detail, it is necessary to introduce some terms that will be used in the remainder of this specification.

[0072] In the following contexts, "sub-image" refers in certain situations to a rectangular arrangement of samples, blocks, macroblocks, coded units, or similar entities that are semantically grouped and can be independently encoded at varying resolutions. At least one sub-image can form an image. At least one encoded sub-image can form an encoded image. At least one sub-image can be assembled into an image, and at least one sub-image can be extracted from an image. In some environments, at least one encoded sub-image can be assembled into an encoded image in a compressed domain without transcoding it to the sample level; in the same or some other cases, at least one encoded sub-image can be extracted from an encoded image in a compressed domain.

[0073] "Adaptive Resolution Change (ARC)" refers to a mechanism that allows the resolution of images or sub-images within an encoded video sequence to be changed, for example, by resampling a reference image. "ARC parameters" can refer to the type of control information required to perform adaptive resolution change, and may include, for example, filter parameters, scaling factors, the resolution of the output image and / or reference image, various control flags, etc. Therefore, ARC information can refer to the data and values ​​corresponding to the ARC parameters.

[0074] The above description focuses on the encoding and decoding of individual, semantically independent encoded video images. Before describing the implications of encoding / decoding multiple sub-images with independent ARC parameters and the additional complexities they imply, the options for signaling ARC parameters will be described first.

[0075] refer to Figure 5A This document illustrates several new options for signaling ARC parameters. As each option is noted, it has certain advantages and disadvantages from the perspectives of coding efficiency, complexity, and architecture. Video coding standards or technologies may choose at least one of these options, or others, for signaling ARC parameters. These options may not be mutually exclusive and may be interchangeable depending on application requirements, the standard technologies involved, or the choice of encoder.

[0076] The categories of ARC parameters can include:

[0077] - Upsampling factors and / or downsampling factors, separated or combined, in the X and Y dimensions.

[0078] - Add an upsampling factor and / or a downsampling factor in the time dimension to indicate constant-rate zooming and / or zooming out of a given number of images.

[0079] - Either of the above may involve encoding at least one possibly shorter syntax element, which may point to a table containing the factors (the plurality of factors).

[0080] - "Resolution": The resolution of the input image, output image, reference image, or encoded image, in units of samples, blocks, macroblocks, CUs, or any other suitable granularity, either combined or separate along the X or Y dimensions. If more than one resolution exists (e.g., one resolution for the input image and one resolution for the reference image), then in some cases, another set of values ​​can be inferred from one set. For example, resolution can be gated using flags. See below for more detailed examples.

[0081] - "Warping" coordinates: Similar to the coordinates used in Appendix P of the H.263 standard, they can have the appropriate granularity described above. Appendix P of the H.263 standard defines an efficient way to encode such warping coordinates, but it is conceivable that other potentially more efficient methods can also be envisioned. For example, the variable-length, reversible "Huffman" encoding of the warping coordinates in Appendix P could be replaced by binary encoding of appropriate length, where the length of the binary codewords could be derived, for example, from the maximum image size (possibly multiplied by a factor and offset by a value) to allow "warping" outside the boundaries of the maximum image size.

[0082] - Upsampling filter parameters or downsampling filter parameters: In the simplest case, perhaps only a single filter is used for upsampling and / or downsampling. However, in some cases, allowing greater flexibility in filter design can be advantageous, which may require signaling filter parameters. Such parameters can be selected by indexing a list of possible filter designs, the filter can be adequately specified (e.g., by using a list of filter coefficients with appropriate entropy coding techniques), the filter can be implicitly selected by the upsampling ratio and / or downsampling ratio, which in turn is signaled according to any of the mechanisms mentioned above, and so on.

[0083] In the following description, it is assumed that encoding is performed on a finite set of upsampling and / or downsampling factors (using the same factors in both the X and Y dimensions) indicated by codewords. These codewords can advantageously be coded in variable length, for example using Exp-Golomb codes, which are common to certain syntax elements in video coding standards such as H.264 and H.265. A suitable mapping from values ​​to upsampling / downsampling factors can be seen, for example, in the table below.

[0084]

[0085] Many similar mappings can be designed based on the application's needs and the capabilities of upscaling and downscaling mechanisms available in video compression technologies or standards. This table can be extended to more values. Values ​​can also be represented using entropy coding mechanisms other than Exp-Golomb codes, such as binary coding. Using binary coding can have certain advantages when the resampling factor is of interest outside the video processing engine itself (primarily the encoder and decoder), such as in the MAME (Media Aware Network Element). It should be noted that for the most common case where resolution does not need to be changed, a shorter Exp-Golomb code can be chosen; in the table above, only a single bit is used. For this most common case, using Exp-Golomb codes can have the advantage of superior coding efficiency compared to using binary codes.

[0086] The number of entries in the table and their semantics can be fully or partially configurable. For example, the basic form of the table described above can be transmitted in a "higher-level" parameter set, such as a sequence parameter set or a decoder parameter set. Alternatively or additionally, at least one such table can be defined in a video coding technique or standard, and can be selected, for example, by a decoder or sequence parameter set.

[0087] The following describes how the upsampling / downsampling factors (ARC information) encoded as described above can be included in video coding techniques or standard syntax. Similar considerations can be applied to control one or more codewords of upsampling / downsampling filters. See the discussion below for when filters or other data structures require relatively large amounts of data.

[0088] Appendix P of the H.263 standard includes four forms of ARC information 502 in the image header 501, and more specifically, in the H.263 PLUSPTYPE (503) header extension. This form of ARC information 502 can be used when a) a usable image header exists, and b) frequent changes in the ARC information are expected. However, the overhead of using H.263-type signaling can be quite high, and the scaling factor of the image boundaries may be irrelevant because the image header may be transient in nature.

[0089] The JVCET-M135-v1 referenced above includes ARC reference information (505) (index) located in the picture parameter set (504) for indexing a table (506) containing the target resolution, which is located within the sequence parameter set (507). According to the authors' verbal statement, the position of the possible resolution in the table (506) within the sequence parameter set (SPS) (507) can be rationalized by using the SPS as an interoperability negotiation point during capability exchange. Within the constraints set by the values ​​in table (506), the resolution can be varied with the picture by referencing the appropriate picture parameter set (PPS) (504).

[0090] The following additional options may exist to transmit ARC information in the video stream. Each of these options has certain advantages as described above. These options can coexist in the same video coding technology or standard.

[0091] exist Figure 5B In one embodiment shown, ARC information (509), such as a resampling (scaling) factor, can appear in the strip header, GOB (Group of Blocks) header, tile header, or tile group header (hereinafter referred to as the tile group header) (508). This may be sufficient if the ARC information is small; for example, the ARC information is a single variable-length ue(v) codeword of a few bits or a fixed-length codeword, as shown above. Including the ARC information directly in the tile group header has additional advantages if the ARC information can be applied, for example, to a sub-picture represented by a tile group, rather than to the entire picture. See also below. Furthermore, even if the video compression technique or standard only envisions adaptive resolution changes for the entire picture (as opposed to, for example, adaptive resolution changes based on tile groups), from a fault-tolerance perspective, placing the ARC information in the tile group header has certain advantages over placing it in an H.263 type picture header.

[0092] In the same or another embodiment, the ARC information (512) itself may reside in a suitable parameter set (511), such as a picture parameter set, a header parameter set, a tile parameter set, an adaptation parameter set (APS), etc. (adaptation parameter sets are described herein). The scope of this parameter set may advantageously be no larger than a single picture, such as a group of tiles. The use of the ARC information (512) can be implied by activating the relevant parameter set. For example, a picture parameter set or equivalent parameter set may be applicable when the video coding technique or standard only considers picture-based ARC.

[0093] In the same or another embodiment, the ARC reference information (513) may exist in the tile group header (514) or a similar data structure. The ARC reference information (513) may refer to a subset of the ARC information (515) available in the parameter set (516) of a single picture (e.g., a sequence parameter set or a decoder parameter set).

[0094] As used in JVET-M0135-v1, the additional level of indirectly implying PPS activation from the tile group header, PPS, and SPS appears unnecessary, since the picture parameter set, like the sequence parameter set, can be used for capability negotiation or announcement (as already used in some standards such as RFC3984). PPS typically refers to a syntax structure containing syntax elements applicable to zero or more complete coded pictures, determined by the syntax elements in each stripe header. SPS typically refers to a syntax structure containing syntax elements applicable to zero or more complete coded layer videosequences (CLVS), determined by the content of the syntax elements in the PPS referred to by the syntax elements in each picture header. However, if ARC information should apply to, for example, subpicks also represented by tile groups, then limiting the activation scope to the tile group's parameter set (e.g., the adaptation parameter set or header parameter set) might be a better choice. Moreover, if the ARC information has a size that is more than negligible—for example, containing filter control information such as multiple filter coefficients—then from a coding efficiency perspective, choosing to use the parameter set may be better than choosing to directly use the tile group header (508), because those settings can be reused by future pictures or subpictures by referring to the same parameter set.

[0095] When using a sequence of parameters that spans multiple images or another higher-level parameter set, the following points may need to be considered:

[0096] 1. The parameter set (516) used to store the ARC information table may, in some cases, be a sequence parameter set, but in other cases, it may advantageously be a decoder parameter set. The decoder parameter set may have an active range of multiple CVSs (Coded Video Sequences) (i.e., encoded video streams), that is, all encoded video bits from the start of the session until the end of the session. Such a range may be more suitable because the possible ARC factors may be decoder features that can be implemented in hardware, and hardware features tend not to change with any CVS (which in at least some entertainment systems is a group of pictures of one second or less in length). That is, placing the table in a sequence parameter set is explicitly included in the placement options described in this application (in particular in conjunction with point 2 below).

[0097] 2. The ARC reference information (513) can advantageously be placed directly in the picture / strip / tile / GOB / tile group header (hereinafter referred to as the tile group header) (514), instead of in the picture parameter set as in JVCET-M0 135-v1. The reason is as follows: when the encoder wants to change a single value in the picture parameter set, such as the ARC reference information, the encoder may have to create a new PPS and reference that new PPS. Suppose only the ARC reference information changes, but other information such as the quantization matrix information in the PPS is retained. Such information can be large and will need to be retransmitted to make the new PPS complete. Since the ARC reference information may be a single codeword, such as an index in table (513), and this will be the only value that changes, retransmitting all such information, such as the quantization matrix information, would be very cumbersome and wasteful. Therefore, from the perspective of coding efficiency, it is better to avoid indirect access via the PPS as proposed in JVET-M0135-v1. Similarly, putting ARC reference information into PPS has another disadvantage, namely that the ARC information referenced by ARC reference information (513) must be applied to the entire picture rather than the subpicture, because the scope of the picture parameter set activation is the picture.

[0098] In the same embodiment or another embodiment, the signaling of the ARC parameters can follow Figure 6A Detailed examples are outlined in the text. Figure 6A This describes a syntax diagram illustrating the representations used in video coding standards. The notation in such syntax diagrams generally follows C-type programming. Bold lines indicate syntax elements present in the bitstream, while lines without bold lines typically represent control flow or variable settings.

[0099] The tile group header (601), as an exemplary syntax structure for a header applicable to a (possibly rectangular) portion of an image, may conditionally contain a variable-length Exp-Golomb-encoded syntax element dec_pic_size_idx (602) (shown in bold). The presence of this syntax element in the tile group header can be gated using an adaptive resolution (603). Here, the value of the flag is not shown in bold, meaning that the position of the flag in the bitstream is where it appears in the syntax table. The adaptive resolution can be signaled whether it is used for the image or a portion thereof using any high-level syntax structure, either inside or outside the bitstream. In the example shown, the adaptive resolution is signaled in the sequence parameter set, as described below.

[0100] Still referencing Figure 6A An excerpt of the sequence parameter set (610) is also shown. The first syntax element shown is adaptive_pic_resolution_change_flag (611). When true, this flag can indicate that an adaptive resolution is used, which in turn may require specific control information. In this example, such control information is conditionally present based on the value of the flag, which is based on the sequence parameter set (612) and the if() statement in the tile group header (601).

[0101] When using adaptive resolution, in this example, the output resolution is encoded in units of samples (613). Label 613 refers to `output_pic_width_in_luma_samples` and `output_pic_height_in_luma_samples`, which together define the resolution of the output image. Elsewhere in video coding techniques or standards, certain restrictions on either value can be defined. For example, a level definition can limit the total number of output samples, which can be the product of the values ​​of the two syntax elements mentioned above. Furthermore, certain video coding techniques or standards, or external techniques or standards (e.g., system standards), may restrict the range of values ​​(e.g., one or both dimensions must be divisible by powers of 2) or the aspect ratio (e.g., width and height must have a relationship of, for example, 4:3 or 16:9). Such restrictions can be introduced to facilitate hardware implementation or for other reasons.

[0102] In some applications, it is recommended that the encoder instruct the decoder to use a certain reference image size, rather than implicitly assuming that its size is the output image size. In this example, the syntax element reference_pic_size_present_flag (614) gates the conditional existence of the reference image size (615) (again, this number refers to both width and height).

[0103] Finally, a possible table of decoded image widths and heights is shown. Such a table can be represented, for example, by a table indication (num_dec_pic_size_in_luma_samples_minus1) (616). “minus1” (minus 1) can refer to the interpretation of the value of this syntax element. For example, if the encoded value is zero, there is one table entry. If the value is five, there are six table entries. For each “row” in the table, the decoded image width and height are then included in the syntax (617).

[0104] The presented table entries (617) can be indexed using the syntax element dec_pic_size_idx (602) in the tile group header, thereby allowing each tile group to have a different decoding size (actually a scaling factor).

[0105] Some video coding techniques or standards (such as VP9) support spatial scalability by incorporating temporal scalability and implementing certain forms of reference image resampling (in a way that is signaled very differently from that disclosed in this application). More specifically, techniques of the ARC type can be used to upsample certain reference images to a higher resolution to form the basis of the spatial enhancement layer. These upsampled images can be refined using high-resolution standard prediction mechanisms, thereby increasing detail.

[0106] The subject matter disclosed in this application can be used in such environments. In some cases, in the same or another embodiment, values ​​in the Network Abstract Layer (NAL) unit header (e.g., the Temporal ID field) can be used to indicate not only the temporal layer but also the spatial layer. This has certain advantages for some system designs; for example, existing Selected Forwarding Units (SFUs) created and optimized based on the Temporal ID value of the NAL unit header, selected for the temporal layer, can be used unmodified in scalable environments. To achieve this, it may be necessary to map between the encoded image size and the temporal layer indicated by the Temporal ID field in the NAL unit header.

[0107] Figure 6B This is a flowchart illustrating a method (620) for decoding encoded images of an encoded video sequence according to an embodiment. In some embodiments, Figure 6B At least one method block can be executed by the decoder (210). In some implementations, Figure 6B At least one method block may be performed by another device or a group of devices that are separate from or include the decoder (210), such as an encoder (203).

[0108] refer to Figure 6B Method (620) includes determining whether ARC information is available (621), and if it is determined that ARC information is not available, the method terminates (650). If it is determined that ARC information is available, the method includes decoding multiple candidate decoded resolutions from the parameter set (625).

[0109] The method (620) includes selecting a candidate decoded resolution (630) from the plurality of candidate decoded resolutions based on an index encoded in a transient head structure applicable to a set of samples.

[0110] Method (620) includes resampling (635) samples from the set of samples based on the output resolution and the selected candidate decoded resolution.

[0111] The method (620) includes using resampled samples to enable prediction (640).

[0112] The method (620) may further include: wherein the transient header structure is any one or any combination of the image header, tile group header, tile header, strip header and block group header.

[0113] The method (620) may further include: wherein the index is encoded in exponential Golomb code.

[0114] The method (620) may further include: wherein the transient header structure is a tile group header, and the tile group header includes a resampling factor.

[0115] The method (620) may further include: wherein the transient header structure is a tile group header, and the tile group header includes adaptive resolution variation (ARC) reference information.

[0116] Method (620) may also include: wherein the ARC reference information refers to a subset of ARC information available in the parameter set.

[0117] The method (620) may further include: wherein the number of the plurality of candidate decoded resolutions is encoded in a syntax element encoded in exponential Golomb code and is located before the plurality of candidate decoded resolutions.

[0118] although Figure 6B An example block diagram of method (620) is shown, but in some implementations, method (620) may include, compared to Figure 6B Other boxes besides those depicted, fewer boxes than those, different boxes from those, or boxes with different arrangements. Additionally or alternatively, two or more boxes in method (620) may be executed in parallel.

[0119] Furthermore, the methods proposed in this application can be implemented by processing circuitry (e.g., at least one processor or at least one integrated circuit). In one example, at least one processor executes a program stored in a non-volatile computer-readable medium to perform at least one of the methods proposed in this application.

[0120] Figure 6C This is a simplified block diagram of an apparatus (660) for decoding encoded images of a video sequence according to an embodiment.

[0121] refer to Figure 6C As shown, the device (660) includes decoding code (670), selection code (675), resampling code (680), and enable code (685).

[0122] The decoding code (670) is configured to decode multiple candidate decoded resolutions from the parameter set.

[0123] The selection code (675) is configured to select a candidate decoded resolution from the plurality of candidate decoded resolutions based on the index of the encoding in the transient header structure applicable to a set of samples.

[0124] The resampling code (680) is configured to resample samples from the set of samples based on the output resolution and the selected candidate decoded resolution.

[0125] Enable code (685) is configured to use resampled samples to enable prediction.

[0126] The technique described above for signaling adaptive resolution parameters can be implemented as computer software using computer-readable instructions and physically stored in at least one computer-readable storage medium. For example, Figure 7 A computer system 700 suitable for implementing certain embodiments of the subject matter of this application is shown.

[0127] The computer software can be encoded using any suitable machine code or computer language, which can be assembled, compiled, linked, or similarly processed to create code including instructions that can be executed directly or by a computer's central processing unit (CPU), graphics processing unit (GPU), or similar means, either through interpretation, microcode execution, or similar means.

[0128] The instructions can be executed on various types of computers or computer components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0129] Figure 7 The components shown for computer system 700 are exemplary in nature and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any one or combination of components shown in the exemplary embodiments of computer system 700.

[0130] Computer system 700 may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user via, for example, tactile input (e.g., key presses, swipes, data glove movements), audio input (e.g., voice, taps), visual input (e.g., gestures), or olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0131] The input human-machine interface device may include at least one of the following (only one of each is depicted): keyboard 701, mouse 702, trackpad 703, touch screen 710, data glove 704, joystick 705, microphone 706, scanner 707, and camera 708.

[0132] Computer system 700 may also include certain human-machine interface (HMI) output devices. Such HMI output devices can stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. These HMI output devices may include tactile output devices (e.g., tactile feedback from a touchscreen 710, data glove 704, or joystick 705, but may also include tactile feedback devices that do not function as input devices), audio output devices (e.g., speaker 709, headphones (not depicted)), visual output devices (e.g., screen 710, including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability—some of which can output two-dimensional or greater than three-dimensional visual output via, for example, stereoscopic flat image output; virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted)), and printers (not depicted).

[0133] The computer system 700 may also include human-accessible storage devices and associated media of the storage devices, such as optical media, including CD / DVD ROM / RW 720 with media such as CD / DVD 721, thumb drives 722, removable hard disk drives or solid-state drives 723, legacy magnetic media such as magnetic tapes and floppy disks (not depicted), dedicated devices based on ROM / ASIC / PLD, such as security protection devices (not depicted), and so on.

[0134] Those skilled in the art should also understand that the term "computer-readable medium" used in connection with the currently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0135] Computer system 700 may also include an interface to at least one communication network (755). The network (755) may be, for example, wireless, wired, or optical. The network (755) may also be local, wide area, metropolitan area, vehicle-mounted and industrial, real-time, latency-tolerant, etc. Examples of the network (755) include, for example, local area networks such as Ethernet and wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., wired or wireless wide area digital TV networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicle-mounted networks including CAN buses, and industrial networks, etc. Some networks (755) typically require an external network interface adapter (754) attached to some general-purpose data port or peripheral bus 749 (e.g., a USB port of computer system 700); other networks are typically integrated into the core of computer system 700 by attaching to a system bus as described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). By using any of these networks (755), computer system 700 can communicate with other entities. Such communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., a CAN bus connected to some CAN bus device), or bidirectional, such as using a local area digital network or a wide area digital network to connect to other computer systems. Certain protocols and protocol stacks can be used on each of the networks (755) and network interfaces (754) described above.

[0136] The aforementioned human-machine interface device, human-accessible storage device, and network interface (754) can be attached to the core 740 of the computer system 700.

[0137] The core 740 may include at least one Central Processing Unit (CPU) 741, a Graphics Processing Unit (GPU) 742, a dedicated programmable processing unit 743 in the form of a Field Programmable Gate Area (FPGA), a hardware accelerator 744 for certain tasks, and so on. These devices, along with read-only memory (ROM) 745, random access memory 746, a graphics adapter 750, and internal high-capacity storage devices 747 such as internal non-user-accessible hard drives or SSDs, can be connected via a system bus 748. In some computer systems, the system bus 748 can be accessed via at least one physical connector to allow expansion through additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus 749 to the core's system bus 748. Architectures for the peripheral bus include PCI, USB, and so on.

[0138] The CPU 741, GPU 742, FPGA 743, and accelerator 744 can execute certain instructions, which, when combined, constitute the aforementioned computer code. The computer code can be stored in ROM 745 or RAM 746. Transient data can also be stored in RAM 746, while permanent data can be stored, for example, in an internal mass storage device 747. Fast storage and retrieval of any memory device can be achieved using a cache memory, which can be closely associated with at least one CPU 741, GPU 742, mass storage device 747, ROM 745, RAM 746, etc.

[0139] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be designed and constructed specifically for the purposes of this application, or may belong to a class well-known and available to those skilled in the art of computer software.

[0140] For example, but not as a limitation, a computer system having architecture 700, and particularly core 740, can provide functionality resulting from the execution of software embodied in at least one tangible computer-readable medium by a processor (including CPU, GPU, FPGA, accelerator, etc.). Such a computer-readable medium can be a medium associated with certain storage devices (e.g., internal core mass storage 747 or ROM 745) of the non-volatile nature of core 740, as described above. Software implementing various embodiments of this application can be stored in such devices and executed by core 740. Depending on specific needs, the computer-readable medium may include at least one memory device or chip. The software can cause core 740, and specifically its processors (including CPU, GPU, FPGA, etc.), to perform specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM 746 and modifying such data structures according to the processes defined by the software. Alternatively or as an alternative, the computer system may provide functionality generated by logic hardwired or otherwise embodied in circuitry (e.g., accelerator 744), which may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may cover logic, and vice versa. Where appropriate, references to computer-readable media may cover circuitry storing software for execution (e.g., integrated circuits (ICs)), circuitry embodying logic for execution, or both. This application covers any suitable combination of hardware and software.

[0141] Although this application describes several exemplary embodiments, various modifications, arrangements, and alternative equivalents are possible within the scope of this application. Therefore, it should be understood that, within the spirit and scope of this application, those skilled in the art can devise various systems and methods that, while not expressly shown or described herein, embody the principles of this application.

Claims

1. A method for encoding a source video sequence into a bitstream, characterized in that, The method includes: Encode multiple candidate resolutions into a parameter set of the bitstream; Select the target candidate resolution from multiple candidate resolutions; Based on the output resolution and the target candidate resolution, samples in a set of samples in the source video sequence are resampled; An index is encoded into a transient head structure applicable to the set of samples, the index indicating a target candidate resolution among the plurality of candidate resolutions; Make predictions using resampled samples; The sampled set of samples is filtered, and the filtered set of samples is encoded into the bitstream based on the metadata of the encoded image or the previous part of the encoded image sequence.

2. The method according to claim 1, characterized in that, The transient header structure is any one or any combination of image header, tile group header, tile header, strip header, and block group header.

3. The method according to claim 2, characterized in that, The index is encoded in exponential Golomb code.

4. The method according to claim 2, characterized in that, The transient header structure is a tile group header, and the tile group header includes a resampling factor.

5. The method according to claim 2, characterized in that, The transient header structure is a tile group header, and the tile group header includes adaptive resolution variation ARC reference information containing the index.

6. The method according to claim 5, characterized in that, The ARC reference information refers to the subset of ARC information available in the parameter set.

7. The method according to any one of claims 1 to 6, characterized in that, The number of the plurality of candidate resolutions is encoded in a syntax element encoded in exponential Golomb code and is located before the plurality of candidate resolutions.

8. The method according to any one of claims 1 to 6, characterized in that, The parameter set is either a sequence parameter set or an image parameter set.

9. The method according to claim 8, characterized in that, The sequence parameter set or image parameter set includes ARC reference information, which includes at least a scaling factor, the resolution of the output image and / or the reference image.

10. An apparatus for encoding a source video sequence into a bitstream, characterized in that, The device includes: The encoding module is used to encode multiple candidate resolutions into a parameter set of the bitstream; The selection module is used to select a target candidate resolution from multiple candidate resolutions; A resampling module is used to resample samples from a set of samples in the source video sequence based on the output resolution and the target candidate resolution; The encoding module is also used for: An index is encoded into a transient head structure applicable to the set of samples, the index indicating a target candidate resolution among the plurality of candidate resolutions; Make predictions using resampled samples; The sampled set of samples is filtered, and the filtered set of samples is encoded into the bitstream based on the metadata of the encoded image or the previous part of the encoded image sequence.

11. A non-volatile computer-readable storage medium storing instructions and a code stream, characterized in that, The instructions cause the processor to execute the method according to any one of claims 1 to 9 to obtain the bitstream.

12. A computer device, characterized in that, include: Memory, used to store computer-readable instructions; A processor configured to read the computer-readable instructions and execute the method according to any one of claims 1 to 9 as instructed by the computer-readable instructions.

13. A method for storing or transmitting a bitstream, characterized in that, The method described in any one of claims 1 to 9 is used to generate a bitstream; and the bitstream is stored or transmitted.

Images