Methods and related devices for encoding and decoding video

Abstract

A method and apparatus for decoding an encoded image are provided. The encoded image includes an encoded video sequence comprising a first segment and a second segment. The method includes: determining a first decoding process for decoding the first segment based at least on a first syntax element of a high-level syntax structure applicable to the first and second segments, the first syntax element instructing the first decoding process to disable a first prediction; and determining a second decoding process for decoding the second segment based at least on a second syntax element of the high-level syntax structure, the second syntax element instructing the second prediction to be disabled, the second prediction being different from the first prediction, the second decoding process disabling the second prediction.

Description

[0001] Cross-references to related applications

[0002] This application is a divisional application of patent application CN2019800358915, which claims priority to U.S. Provisional Patent Application No. 62 / 727,381, filed September 5, 2018; U.S. Patent Application No. 16 / 202,949, filed November 28, 2018; and U.S. Patent Application No. 16 / 454,545, filed June 27, 2019, the entire disclosure of each of which is incorporated herein by reference. Technical Field

[0003] This application relates to image decoding technology, and more particularly to methods and related apparatus for decoding encoded images. Background Technology

[0004] Video encoding and decoding using motion-compensated inter-frame image prediction has been around for decades. Uncompressed digital video can comprise a series of images, each with spatial dimensions such as 1920×1080 luma samples and associated chroma samples. This series of images can have a fixed or variable image rate (also informally referred to as frame rate), such as 60 images per second or 60Hz. Uncompressed video has high bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luma sample resolution at 60Hz frame rate) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 GB of storage space.

[0005] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce the bandwidth or storage space requirements mentioned above, in some cases by two orders of magnitude or more. Lossless compression and lossy compression, or combinations thereof, can be used. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can be reflected in the fact that higher allowable / tolerable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding, some of which will be discussed below.

[0007] A coded video bitstream can be a compressed representation of an uncompressed source video and can be divided into coded pictures. A coded picture includes one or more picture segments. A picture segment can be, for example, a slice, a tile, a group of blocks (GOB), and so on. A picture segment can include a segment header, such as a slice header, a tile header, a GOB header, and so on, which can include information about one or more coding units (CUs) that can make up the rest of the segment. In some cases, the information in the segment header can pertain to the first CU in the segment, but can be overridden by corresponding update information located in the CU header. In other cases, the information in the segment header can pertain to all CUs in the segment.

[0008] Picture partitioning is included in video compression technology and standards for a variety of reasons. One reason for the introduction of slices in MPEG-1 was the need for Maximum Transfer Unit (MTU) size matching. In cases where a coded picture is larger than the MTU of packets in a packet-based network, it was deemed desirable to split the picture into independently decodable units to some extent, hence the introduction of slices. Another reason was the desire to simplify the composition of sub-pictures into coded pictures, possibly in the compressed domain. The group of blocks (GOB) concept of H.261, especially the GOB numbering used in H.261, was an early example of this technique, while the rectangular slices of H.263 was another example. Another reason was to enable encoding and / or decoding where multiple encoder / decoder processors or cores simultaneously decode parts of a given picture.

[0009] Tiles are one of several mechanisms available in certain video codecs that facilitate partitioning of a coded video bitstream. A tile can be a rectangular region of a picture. The scan order of CUs (also referred to as blocks or macroblocks) can be positioned to tiles from top left to right, and then from top to bottom. A picture can be divided into many tiles, where each CU can be exactly a part of one tile. Tiles are introduced to enable parallel encoding and decoding by allowing each processor or core to focus attention on only a portion of a picture to be encoded, as no communication with processors responsible for other tiles is needed except for the final bitstream generation process; however, they can also be used as a mechanism for picture composition.

[0010] A commonality of picture partitioning techniques is that segment boundaries interrupt certain prediction mechanisms. For example, in some video coding techniques and standards, segment boundaries interrupt in-picture prediction mechanisms such as motion vector prediction, intra prediction, and the like. To what extent inter-picture prediction mechanisms are allowed (e.g., using samples outside of a segment boundary for motion compensation) depends on the video coding technique or standard. For example, in H.263+, the independent segment decoding mode provides a setting to disable importing of sample values by motion compensation across segment boundaries. The constrained tile sets in H.265 have a similar purpose.

[0011] Picture partitioning techniques can also have the commonality that only certain types of coding units are allowed in a segment of a given type. For example, in some video coding techniques and standards, an intra slice can only contain CUs coded in intra mode, an inter slice can contain CUs in intra and inter mode, and a bi-predictive slice can contain CUs coded in intra, inter, and bi-predictive mode. It can be observed that in at least some video coding techniques or standards, segment types form a hierarchy, e.g., intra segments are the most restrictive, followed by inter segments, and then bi-predictive segments.

[0012] Intra segments can be used to reset the decoder state to a known state with respect to certain parts of the current decoded picture (covered by the intra segment).

[0013] Recent video coding techniques can include techniques that share certain similarities between inter, intra, and (perhaps to a lesser extent) bi-predictive. For example, the screen content coding (SCC) profile of H.265 includes a technique known as intra block copy (IBC), which can be characterized as a motion compensation mechanism in which the reference sample information is part of the same decoded picture as the sample under reconstruction. See, e.g., “HEVC Screen Content Coding Draft Text,” ITU-T / ISO / IEC, JCTVC-T1005, 2015. Without the need to access a previously decoded reference picture, this is a common function of intra coding. However, the reconstruction of a given CU can require sample information from outside the CU, which has traditionally been considered a function of inter coding in earlier video coding standards such as MPEG-2.

[0014] The interaction between intra segments and certain modern coding tools, such as IBC, can be complex. On the one hand, IBC proves to be an effective tool that can improve the coding efficiency of intra-coder regions of the picture under reconstruction at least in some cases. However, IBC works better at least in some cases (where more regions of the picture under reconstruction are available for IBC to use as reference samples) and can include samples outside the current segment. However, using samples outside the current segment for IBC reference can work against the goal of resetting a given region to a known state for an intra segment without referencing any information outside that segment.

[0015] The same is true for tools that can be said to be less advanced (or at least older), such as intra prediction.

[0016] Accordingly, video compression technologies or standards tend to prohibit any reference to outside samples and meta-information for intra slice CUs. This can be achieved, for example, by marking any samples and any CUs outside the segment boundary as unavailable for prediction. However, this decision unnecessarily reduces coding efficiency at least in some cases that are not related to the goal of resetting the decoding process for an intra segment. SUMMARY

[0017] According to an embodiment, a method of decoding a coded picture in a coded video sequence, the coded picture comprising a first partitioned segment and a second partitioned segment, the method comprising: determining a first decoding process for decoding the first partitioned segment based on at least a first syntax element, the first syntax element being used to indicate that the first decoding process disables a first prediction; determining a second decoding process for decoding the second partitioned segment based on at least a second syntax element, the second syntax element being used to indicate that the second decoding process disables a second prediction, the second prediction being different from the first prediction; decoding the first partitioned segment based on the first decoding process disabling the first prediction; and decoding the second partitioned segment based on the second decoding process disabling the second prediction; combining the first decoding process and the second decoding process to decode the first partitioned segment and the second partitioned segment if the first syntax element indicates that the first prediction is allowed and the second syntax element indicates that the second prediction is allowed; the first prediction comprises any one of: intra block copy across segment boundary, intra prediction across segment boundary, prediction picture prediction across segment boundary, and bi-prediction picture prediction across segment boundary; and the second prediction comprises a different one of: the intra block copy across segment boundary, the intra prediction across segment boundary, the prediction picture prediction across segment boundary, and the bi-prediction picture prediction across segment boundary.

[0018] According to an embodiment, there is provided a method of encoding a video, each picture in the video comprising a first partition and a second partition, the method comprising: determining a first encoding process for encoding the first partition based on at least a first syntax element, the first syntax element being used to indicate that the first encoding process disables a first prediction; determining a second encoding process for encoding the second partition based on at least a second syntax element, the second syntax element being used to indicate that the second encoding process disables a second prediction, the second prediction being different from the first prediction; encoding the first partition based on the first encoding process disabling the first prediction; and encoding the second partition based on the second encoding process disabling the second prediction; combining the first encoding process and the second encoding process to encode the first partition and the second partition if the first syntax element indicates that the first prediction is allowed and the second syntax element indicates that the second prediction is allowed; the first prediction comprises any one of: an intra block copy across a partition boundary, an intra prediction across a partition boundary, a predicted picture prediction across a partition boundary, and a bi-predicted picture prediction across a partition boundary; and the second prediction comprises a different one of: the intra block copy across a partition boundary, the intra prediction across a partition boundary, the predicted picture prediction across a partition boundary, and the bi-predicted picture prediction across a partition boundary.

[0019] According to an embodiment, there is provided a device for decoding a coded picture of a coded video sequence, the coded picture comprising a first partition and a second partition, the device comprising: at least one memory configured to store a computer program; and at least one processor configured to access the at least one memory and operate according to the computer program, to perform the above-mentioned method of decoding a coded picture of a coded video sequence.

[0020] According to embodiments, there is provided an apparatus for decoding a coded picture of a coded video sequence, wherein the coded picture comprises a first partition and a second partition, the apparatus comprising determining means and decoding means: the determining means are configured to determine a first decoding process for decoding the first partition based on at least a first syntax element, the first syntax element being configured to indicate that the first decoding process disables a first prediction; the determining means are further configured to determine a second decoding process for decoding the second partition based on at least a second syntax element, the second syntax element being configured to indicate that the second decoding process disables a second prediction, the second prediction being different from the first prediction; the decoding means are configured to decode the first partition based on the first decoding process disabling the first prediction; and the decoding means are further configured to decode the second partition based on the second decoding process disabling the second prediction; the decoding means are further configured to, if the first syntax element indicates that the first prediction is allowed and the second syntax element indicates that the second prediction is allowed, combine the first decoding process and the second decoding process to decode the first partition and the second partition, the first prediction comprising any one of: an intra block copy across a partition boundary, an intra prediction across a partition boundary, a prediction picture prediction across a partition boundary, and a bi-prediction picture prediction across a partition boundary; and the second prediction comprising a different one of: the intra block copy across a partition boundary, the intra prediction across a partition boundary, the prediction picture prediction across a partition boundary, and the bi-prediction picture prediction across a partition boundary.

[0021] According to embodiments, there is provided an apparatus for encoding a video, wherein pictures in the video comprise a first partition and a second partition, the apparatus comprising: at least one memory configured to store a computer program; and at least one processor configured to access the at least one memory and operate according to the computer program, to perform the above-mentioned method for encoding a video.

[0022] According to embodiments, there is provided a non-transitory computer-readable storage medium storing a program, the program causing a processor to perform the above-mentioned method.

[0023] According to embodiments, there is provided a computer device, the device comprising a processor and a memory: the memory is configured to store a program and transmit the program to the processor; the processor is configured to perform the above-mentioned method according to instructions in the program.

[0024] It is thus seen that by selection of the prediction mechanism used, e.g. the first decoding process determining to disable the first prediction and the second decoding process determining to disable the second prediction, it is possible to prevent one or more forms of cross tile and segment boundary prediction, which can have advantageous effects from an application and coding efficiency point of view. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a simplified block diagram of a communication system according to an embodiment;

[0026] Figure 2 is a diagram of placement of a video encoder and a video decoder in a streaming environment according to an embodiment;

[0027] Figure 3 is a functional block diagram of a video decoder according to an embodiment;

[0028] Figure 4 is a functional block diagram of a video encoder according to another embodiment;

[0029] Figure 5 is a diagram of a partitioned picture according to an embodiment;

[0030] Figure 6A and Figure 6B is a diagram of a parallel decoder system for partitioned pictures according to an embodiment;

[0031] Figure 7 is a diagram of syntax and semantics of IP slice syntax elements according to an embodiment;

[0032] Figure 8 is a diagram of syntax and semantics of PI slice syntax elements according to an embodiment;

[0033] Figure 9 is a diagram of syntax and semantics of B slice syntax elements according to an embodiment;

[0034] Figure 10 is a diagram of syntax and semantics of BI, PI and IP slice syntax elements according to an embodiment;

[0035] Figure 11 is a diagram of syntax and semantics of cross boundary prediction flags according to an embodiment;

[0036] Figure 12 is a diagram of a computer system suitable for implementing an embodiment;

[0037] Figure 13 is a flowchart showing a method of decoding an encoded picture of an encoded video sequence comprising a first divided segment and a second divided segment according to an embodiment;

[0038] Figure 14 is a simplified block diagram of a device for decoding a coded picture of a coded video sequence comprising a first partition and a second partition according to an embodiment. DETAILED DESCRIPTION

[0039] Embodiments relate to video encoding and decoding, and more specifically to partitioning of coded video pictures into segments such as slices and tiles that can not conform to known types of intra, inter, bi-predictive slices or tiles. For example, a slice or tile containing only intra coded blocks can still use prediction information from slices or tiles outside the slice or tile in reconstruction but belonging to the same picture in decoding order. In this case, the picture as a whole can be decoded independently of other pictures, while the slice or tile can need other slices or tiles of the same picture for successful decoding.

[0040] Figure 1 is a simplified block diagram of a communication system (100) according to an embodiment. The communication system (100) can include at least two terminals (1 10-120) interconnected via a network (150). For unidirectional transmission of data, a first terminal (1 10) can code video data at a local location to be transmitted to a second terminal (120) via the network (150). The second terminal (120) can receive the coded video data of the other terminal from the network (150), decode the coded data and display the recovered video data. Unidirectional data transmission can be common in media serving applications etc.

[0041] Figure 1 A second pair of terminals (130, 140) is shown that are provided to support bidirectional transmission of coded video that can occur during, for example, a videoconferencing session. For bidirectional transmission of data, each terminal (130, 140) can code video data captured at a local location to be transmitted to the other terminal via the network (150). Each terminal (130, 140) also can receive coded video data transmitted by the other terminal, can decode the coded data, and can display the recovered video data at a local display device.

[0042] In Figure 1In particular embodiments, terminals (110-140) can be illustrated as servers, personal computers and smartphones, but the principles of the embodiments are not limited thereto. Embodiments are applicable to laptop computers, tablets, media players and / or dedicated video conferencing equipment. Network (150) represents any number of networks that convey coded video data among the terminals (110-140), including for example wireline and / or wireless communication networks. Communication network (150) can exchange data in circuit- switched and / or packet-switched channels. Representative networks include telecommunications networks, local area and / or wide area networks, and / or the Internet. For the purposes of the present discussion, the architecture and topology of network (150) can not be critical, unless otherwise explained in the following.

[0043] Figure 2 FIG. 1 is a diagram of placement of video encoders and video decoders according to embodiments in a streaming environment. The disclosed subject matter can be equally applicable to other video enabled applications, including for example video conferencing, digital TV, storing of compressed video on digital media including CD, DVD, memory stick and the like.

[0044] A streaming system can include a capture subsystem (213) that can include a video source (201), for example a digital camera, to create, for example, an uncompressed video sample stream (202). The sample stream (202), depicted as a bold line to emphasize the high data volume when compared to encoded video bitstreams, can be processed by an encoder (203) coupled to the camera (201). The encoder (203) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video bitstream (204), depicted as a thin line to emphasize the lower data volume when compared to the sample stream, can be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) can access the streaming server (205) to retrieve copies (207, 209) of the encoded video bitstream (204). A client (206) can include a video decoder (210) that decodes the incoming copy of the encoded video bitstream (207) and creates an outgoing video sample stream (211) that can be rendered on a display (212) or other rendering device (not depicted). In some streaming systems, the video bitstreams (204, 207, 209) can be encoded according to certain video coding / compression standards. Examples of those standards include ITU-T

[0045] Figure 3 is a functional block diagram of a video decoder (210) according to an embodiment.

[0046] The receiver (310) can receive one or more codec video sequences to be decoded by the video decoder (210); in the same or another embodiment, one encoded video sequence at a time, where the decoding of each encoded video sequence is independent from the other encoded video sequences. The encoded video sequences can be received from a channel (312), which can be a hardware / software link into a storage device that stores the encoded video data. The receiver (310) can receive encoded video data along with other data such as encoded audio data and / or ancillary data streams, which can be forwarded to their respective consuming entities (not depicted). The receiver (310) can separate the encoded video sequences from the other data. To combat network jitter, a buffer memory (315) can be coupled between the receiver (310) and the entropy decoder / pars er (320) (hereafter “parser”). The buffer memory (315) can not be needed, or can be small, when the receiver (310) is receiving data from a store-and- forward device with sufficient bandwidth and controllability, or from an isochronous synchronous network. The buffer memory (315) can be relatively large, and can advantageously be of adaptive size, to handle use over a best-effort packet network such as the Internet.

[0047] The video decoder (210) can include a parser (320) to reconstruct symbols (321) from the entropy coded video sequence. The categories of these symbols include information for managing the operation of the decoder (210), and potentially information for controlling the rendering of pictures, e.g., of a display (212) that is not an integral part of the decoder, but can be coupled to it, like a smart phone's display that is operated independently of the decoder. Figure 3Control information for the rendering device(s) can be in the form of supplemental enhancement information (SEI messages) or video usability information (VUI) parameter set fragments (not depicted). The parser (320) can parse / entropy decode the received coded video sequence. The coding of the coded video sequence can be in accordance with video coding techniques or standards, and can follow principles of video coding in which the video sequence is partitioned into coded pictures and further partitioned into slices, tiles, or other such coded video sequence sub-divisions. The coded pictures can be in groups of pictures (GOPs) or other such groups. The coded pictures can be Intra coded or Inter coded pictures that can use various Inter coding techniques. The coded pictures can be further coded using prediction, transform, quantization, entropy coding and other such techniques. The coded pictures can be associated with parameters sets, including, but not limited to, picture parameter sets (PPSs) and other such parameter sets. The coded pictures can be associated with slice headers, tile headers, or other such coded video sequence sub-division headers. The coded pictures can be associated with supplemental enhancement information (SEI) messages, video usability information (VUI) metadata, or other such messages. The parser (320) can extract such information from the coded video sequence. The parser (320) can extract information, such as transform coefficients, quantizer parameter (QP) values, motion vectors, and other such information from the coded video sequence.

[0048] The parser (320) can perform entropy decoding / parsing operations on the video sequence received from the buffer (315), thereby creating symbols (321). The parser (320) can receive encoded data and selectively decode particular symbols (321). In addition, the parser (320) can determine whether to provide particular symbols (321) to the motion compensation prediction unit (353), the scaler / inverse transform unit (351), the intra prediction unit (352), or the loop filter unit (354).

[0049] The reconstruction of the symbols (321) can involve a number of different units depending on the type of coded video picture or portion of coded video picture (such as: inter and intra pictures, inter and intra blocks, etc.) and other factors. Which units are involved and how they are controlled can be controlled by subgroup control information that is parsed by the parser (320) from the coded video sequence. The flow of such subgroup control information between the parser (320) and the following units has not been depicted for reasons of clarity and brevity.

[0050] Beyond the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into a number of functional units as described infra. In practical implementations, many of these units interact closely with each other, and may, at least partly, be integrated into each other's functionality. For the purpose of describing the disclosed subject matter, however, the conceptual subdivision into the functional units below is appropriate.

[0051] The first unit is a scaler / inverse transform unit (351). The scaler / inverse transform unit (351) receives quantized transform coefficients as the symbol(s) (321) and control information, including which transform to use, block size, quantization factor, quantization scaling matrices, etc. from the parser (320). The scaler / inverse transform unit (351) can output blocks comprising sample values that can be input into the aggregator (355).

[0052] In some cases, the output samples of the scaler / inverse transform unit (351) can belong to an intra coded block; i.e., no predictive information from previously reconstructed pictures is used, although predictive information from previously reconstructed parts of the current picture can be used. Such predictive information can be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information taken from the current (partially reconstructed) picture (356). In some cases, the aggregator (355) adds the predictive information generated by the intra prediction unit (352) on a per sample basis to the output sample information provided by the scaler / inverse transform unit (351).

[0053] In other cases, the output samples of the scaler / inverse transform unit (351) can belong to an inter coded and potentially motion compensated block. In such cases, a motion compensated prediction unit (353) can access a reference picture memory (357) to fetch samples for prediction. After motion compensation of the fetched samples according to the symbols (321) pertaining to the block, these samples can be added by the aggregator (355) to the output of the scaler / inverse transform unit (in this case referred to as residual samples or residual signal) to thereby generate the output sample information. The address within the reference picture memory from which the motion compensated prediction unit fetches the prediction samples can be controlled by motion vectors, which are made available to the motion compensation unit in the form of symbols (321), which can have, e.g., X, Y, and reference picture components. Motion compensation can also include interpolation of sample values fetched from the reference picture memory when sub-sample precise motion vectors are used, motion vector prediction mechanisms, etc.

[0054] The output samples of the aggregator (355) can be subject to various loop filtering techniques in the loop filter unit (354). Video compression technologies can include in-loop filter technologies that are controlled by parameters included in the coded video bitstream and made available to the loop filter unit (354) as symbols (321) from the parser (320), however, video compression technologies can also be responsive to meta-information obtained during the decoding of previous (in decoding order) parts of the coded picture or coded video sequence, as well as responsive to previously reconstructed and loop-filtered sample values.

[0055] The output of the loop filter unit (354) can be a sample stream that can be output to the display device (212) and stored in the reference picture memory (356) for use in the inter prediction of future pictures.

[0056] Once fully reconstructed, certain coded pictures can be used as reference pictures for future prediction. Once a coded picture has been fully reconstructed and the coded picture is identified (by, for example, parser (320)) as a reference picture, the current reference picture (356) can become part of the reference picture buffer (357), and a fresh current picture memory can be reallocated before commencing the reconstruction of the subsequent coded picture.

[0057] The video decoder (210) can perform decoding operations according to a predetermined video compression technology that can be documented in a standard, such as ITU-T H.265. As specified in the video compression technology document or standard, and explicitly in the profile document therein, an coded video sequence can conform to the syntax specified by the video compression technology or standard being used in the sense that the coded video sequence follows the syntax of the video compression technology or standard. For conformance, the complexity of the coded video sequence is also required to be within bounds set by the level of the video compression technology or standard. In some cases, the bounds set by the level include maximum picture size, maximum frame rate, maximum reconstruction sample rate (measured in, for example megasamples per second), maximum reference picture size, and so on. In some cases, the limits set by the level can be further restricted by the hypothetical reference decoder (HRD) specifications and the HRD buffer management metadata signaled in the coded video sequence.

[0058] In an implementation, the receiver (310) can receive additional (redundant) data with the encoded video. The additional data can be included as part of the coded video sequence(s). The additional data can be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data can be in the form of, for example, a temporal, spatial, or signal-to-noise ratio (SNR) enhancement layer, redundant slices, redundant pictures, forward error correction codes, and so on.

[0059] Figure 4 is a functional block diagram of a video encoder (203) according to an implementation.

[0060] The encoder (203) can receive video samples from a video source (201) (that is not part of the encoder) that can capture video image(s) to be coded by the encoder (203).

[0061] The video source (201) can provide the source video sequence to be coded by the video encoder (203) in the form of a digital video sample stream that can be in any suitable bit depth (for example: 8 bit, 10 bit, 12 bit,...), any color space (for example, BT.601 Y CrCB, RGB,...) and any suitable sampling structure (for example Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (201) can be a storage device storing previously prepared video. In a videoconferencing system, the video source (201) can be a camera that captures local image information as a video sequence. Video data can be provided as a plurality of individual pictures that are imparted with motion when viewed in a sequence. The pictures themselves can be organized into groups of pictures (GOPs) that comprise the video stream. The GOP structure can be variable. Each picture can be further defined by an encoded macroblock, which can be a local spatial

[0062] According to an embodiment, the encoder (203) can code and compress the pictures of the source video sequence into a coded video bitstream (443), also called coded video sequence or coded video stream, in real time or under any other time constraints as required by the application. Enforcing appropriate coding speed is one function of controller (450). The controller controls other functional units as described below and is functionally coupled to these units. The coupling is not depicted for clarity. Parameters set by the controller can include rate control related parameters (picture skip, quantizer, lambda value of rate-distortion optimization techniques,...), picture size, group of pictures (GOP) layout, maximum motion vector search range, and others. A person skilled in the art can readily identify other functions of the controller (450) as they can pertain to the video encoder (203) optimized for a certain system design.

[0063] Some video encoders operate in a manner that is readily recognized by those skilled in the art as an "encoding loop." As an oversimplified description, the encoding loop can include an encoding portion of the encoder (430) (hereafter referred to as the "source encoder") that is responsible for creating symbols based on input pictures to be encoded and reference picture(s), and a (local) decoder (433) embedded in the encoder (203) that reconstructs the symbols to create sample data that a (remote) decoder would also create (as any compression between symbols and encoded video bitstream in the video compression technology considered in the disclosed subject matter is lossless). The reconstructed sample stream is input into the reference picture store (434). As the decoding of symbol streams results in bit-exact results independent of decoder location (local or remote), the reference picture buffer content is also bit-exact between the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" the same reference picture samples as would be "seen" by the decoder when using the predictions during decoding. This basic principle of reference picture synchronicity (and the drift that occurs if synchronicity cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0064] The operation of the "local" decoder (433) can be identical to the operation of the "remote" decoder (210) described in detail above. However, reference is also briefly made to the operation of the "remote" decoder (210) in the following. Figure 3 The operation of the "local" decoder (433) can be identical to the operation of the "remote" decoder (210) described in detail above. However, reference is also briefly made to the operation of the "remote" decoder (210) in the following. Figure 3 When symbols are available and can be losslessly encoded / decoded into encoded video sequences by the entropy encoder (445) and parser (320), the entropy decoding portion of the decoder (210) including the channel (312), receiver (310), buffer (315), and parser (320) can not be fully implemented in the local decoder (433).

[0065] At this point it can be observed that any decoder technology other than the parsing / entropy decoding present in the decoder must also necessarily be present in the corresponding encoder in substantially the same functional form. As the encoder technology is reciprocal to the fully described decoder technology, the description of the encoder technology can be simplified. More detailed description is only required in certain areas and is provided below.

[0066] As part of its operation, the source encoder (430) can perform motion- compensated predictive encoding that references one or more previously encoded frames from the video sequence (designated as "reference frames") to predictively encode an input frame. In this manner, the encoding engine (432) encodes differences between pixel blocks of an input frame and pixel blocks of reference frame(s) that can be selected as prediction references for the input frame.

[0067] The local video decoder (433) can decode encoded video data of frames that can be designated as reference frames based on symbols created by the source encoder (430). The operation of the encoding engine (432) can advantageously be lossy. When the encoded video data can be decoded by the video decoder (433), Figure 3 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process performed by the video decoder on the reference frame and can store the reconstructed reference frame in the reference picture cache (434). In this way, the video encoder (203) can locally store a copy of the reconstructed reference frame that shares the same content (no transmission errors) as the reconstructed reference frame that will be obtained by the remote video decoder.

[0068] The predictor (435) can perform a prediction search against the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) can search in the reference image memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. The predictor (435) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (435), the input image may have prediction references extracted from multiple reference images stored in the reference image memory (434).

[0069] The controller (450) can manage the encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0070] The outputs of all the above-described functional units can undergo entropy encoding in the entropy encoder (445). The entropy encoder converts the symbols generated by the various functional units into an encoded video sequence by lossless compression of the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, arithmetic coding, etc.).

[0071] The transmitter (440) can buffer (one or more) encoded video sequences created by the entropy encoder (445) in preparation for transmission via a communication channel (460), which may be a hardware / software link to a storage device capable of storing the encoded video data. The transmitter (440) can combine the encoded video data from the source encoder (430) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0072] The controller (450) can manage operation of the video encoder (203). During coding, the controller (450) can assign to each coded picture a certain coded picture type, which can affect the coding techniques that can be applied to the corresponding picture. For example, the pictures generally can be assigned as one of the following frame types:

[0073] An Intra Picture (I picture) that can be coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow for different types of Intra pictures, including, for example Independent Decoder Refresh (IDR) pictures. A person skilled in the art will recognize the variants of I pictures and their respective applications and features.

[0074] A predictive picture (P picture) that can be coded and decoded using either intra prediction or inter prediction that uses at most one motion vector and reference index to predict sample values for each block.

[0075] A bi-predictive picture (B picture) that can be coded and decoded using either intra prediction or inter prediction that uses at most two motion vectors and reference indices to predict sample values for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0076] Source pictures generally can be spatially subdivided into blocks of samples (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples per picture) and coded block by block. The blocks can be predictively coded with reference to other (already coded) blocks, determined by a coding assignment applied to the respective picture of the block. For example, blocks of an I picture can be non-predictively coded, or the blocks can be predictively coded with reference to coded blocks of the same picture (spatial or intra prediction). Blocks of a P picture can be non-predictively coded with reference to one previously coded reference picture, either via spatial prediction or via temporal prediction. Blocks of a B picture can be non-predictively coded with reference to one or two previously coded reference pictures, either via spatial prediction or via temporal prediction.

[0077] The video encoder (203) can perform coding operations in accordance with a predetermined video coding technology or standard, such as ITU-T H.265. In the operation of the video encoder (203), the video encoder (203) can perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. The coded video data, therefore, can conform to a syntax specified by the video coding technology or standard in use.

[0078] In an embodiment, the transmitter (440) can transmit additional data and the encoded video. The source coder (430) can include such data as part of the coded video sequence. Additional data can comprise temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Video Usability Information (VUI) parameter set fragments, and so on.

[0079] In the following, embodiments are described using tiles as an example of a coded picture segment. In at least some cases, embodiments can be equally applied to other segment types such as slices, GOBs, and so on.

[0080] Figure 5 A picture that is partitioned according to an embodiment.

[0081] Referring to Figure 5 In an embodiment, the coded picture (501) can be divided into three tile bitstreams (502, 504, 506) that represent three spatial regions (509, 510, 511) in the reconstructed picture (508), respectively. In this example, each tile can use a slice header (503, 505, 507). This example uses three tiles, but a person skilled in the art can easily generalize the example to more or less than three tiles. Each tile can contain, in addition to its header, one or more coding units CU that can be arranged in a scan order. That is, for consecutively ordered coding units in the bitstream (502, 504, or 506) of a tile, the regions covered by the coding units are arranged such that a subsequent coding unit is to the right and below a previous coding unit, in principle following a scan order established by cathode ray tube (CRT) beams. For uniform and non-uniform sized coding units, a person skilled in the art is well familiar with the scan order of coding units.

[0082] According to known video coding techniques or standards, only certain forms of prediction across slice boundaries are allowed, as shown in the spatial representation of the reconstructed picture (508). For example, H.265 tiles break all forms of prediction in the coded picture, whether the prediction is according to metadata (e.g., intra prediction modes, motion vectors, and so on) or sample prediction (e.g., prediction samples used in intra prediction or sample data used for IBC prediction). H.265 motion-constrained tile sets further break the import of sample values by motion compensation from spatial regions in reference pictures outside the current tile set. In this case, motion-constrained tile sets are comparable to H.263 rectangular slices with independent segment decoding mode enabled.

[0083] These sets of constraints were defined in the past after recognizing certain (limited) application scenarios in response to these scenarios and in response to certain hardware implementation related constraints. However, the introduction of new coding tools, the ever growing demand for additional coding efficiency and the recognition of additional application scenarios make it desirable to define the semantics of block boundaries more flexibly with respect to the interruption of different forms of prediction.

[0084] Figure 6A and Figure 6B is a diagram of a parallel decoder system for partitioning a picture according to embodiments.

[0085] With reference to Figure 5 and Figure 6A , a system (600a) is shown in which the three spatial regions (509, 510, 511) each cover a region that is too large to be decoded by a single decoder. In this case, the system (600a) can be used, which comprises multiple sub-decoders, each producing a video sample stream representative of the content of a single coded tile. The incoming coded video bitstream of the partition can be broken down into coded sub-bitstreams representative of the three spatial regions (509, 510, 511) by a parser (602) in the compressed domain. The breaking down can be a relatively lightweight process requiring little, if any, signal processing to bitstream parse when using a suitable syntax, such as the H.265 syntax that enables sets of tiles with restricted motion. The (relatively) low bandwidth nature of the compressed coded video bitstream (as compared to the reconstructed sample stream) is shown as thin lines. The three sub-decoders (603, 604, 605) can receive the bits of the coded sub-bitstream they are responsible for decoding over similar low bandwidth communication links (606, 607, 608), also depicted as thin lines, and each decoder decodes one region into a reconstructed tile. The samples of the reconstructed tiles generated by the three sub-decoders (603, 604, 605) can be passed to a stitcher (612) over high bandwidth links (609, 610, 611). The stitcher (612) can aggregate the tiles into a single sample stream representing all the reconstructed tiles in a single reconstructed picture stream (613). In this case, the sub-decoders (603, 604, 605) can have no communication relationship with each other, can only exchange (minimal) control information, and do not pass prediction information or sample information between each other. Therefore, no such communication relationship is depicted.

[0086] Based on the above observations, the person skilled in the art can easily design a corresponding encoding system, which is therefore not depicted or described in more detail.

[0087] As in the system (600a), by definition, there is no passing of prediction information between the sub-decoders (603, 604, 605) of the decoding system (600a), nor is there any such communication in a similar encoder system, such a system is suitable for independent sub-decoders or sub-encoders. However, it can also be suitable for application scenarios where the decoders and / or encoders are not assigned as sub-decoders / sub-encoders but based on application requirements. For example, if the (in this example three) spatial regions are known to have no semantic relationship between each other, there is little to no opportunity (if any) to achieve coding efficiency gains through prediction across region / tile boundaries. It should be avoided to introduce (in a contrived fashion) sample information from neighboring tiles into the tile being reconstructed. For example, if one spatial region contains content from one camera source, while the other spatial regions contain content from other camera sources, artificial content, other projections in a 360 scene, etc., the correlation between the content for various reasons can be small or non-existent. Therefore, even though it is technically feasible to use prediction in the system design, there can be no or little advantage to using prediction between these regions. This is one of the reasons why tile sets with motion restrictions were included in H.265.

[0088] Figure 6B A slightly different system design (600b) is depicted. Again, the system comprises a parser (602) that disassembles the incoming tiled encoded video bitstream into three sub-bitstreams (606, 607, 608) that are fed into three sub-decoders (603, 604, 605). The sub-decoders each create reconstructed tiles that are transferred (609, 610, 611) to a stitcher (612) that in turn creates an output reconstructed picture stream (613). However, in this design, a limited amount of information can be passed between the sub-decoders (603, 604, 605) over a suitable communication link (614) of intermediate bandwidth that can not be sufficient to pass a large amount of sample information but can pass more than minimal control information (to be described in more detail below). The nature of the communication link can be full connectivity, a bus, shared memory, or any other suitable communication technology. What is described here as the intermediate bandwidth communication link (614) is a bus that connects the sub-decoders (603, 604, 605).

[0089] A corresponding encoding system can be readily designed by those skilled in the art.

[0090] The decoding system (600b) can allow for the use of certain types of prediction across tile boundaries. Which types of prediction can be used depends largely on the available bandwidth of the intermediate bandwidth communication link (614).

[0091] In a first example, the link (614) can have sufficient bandwidth for a small amount of metadata and very limited sample data (e.g., a few sample values per coding unit (CU) to be decoded). In this case, certain intra-prediction mechanisms can be supportable, but intra-block copy and motion compensation can be non-supportable. The motion-constrained tile sets of H.265 contemplate this case.

[0092] In a second example, the link (614) can have sufficient bandwidth for metadata and sample data associated with motion compensation from a single reference picture in the order of previously decoded, but insufficient bandwidth or coordination capabilities to use intra-block copy. The non-motion-constrained (regular) tile sets of H.265 envision this case. Intra-block copy can be more burdensome in some implementations compared to (past picture) motion compensation because, in many implementations, intra-copy can require quasi-concurrent access to immediately adjacent sample data in the current picture memory, which can result in poor cache efficiency (especially if the cache design is not optimized for IBC). Coordination capabilities can present conceptual problems. If the reconstruction of a tile can require access to other tiles in the same reconstructed picture, the decoding pipeline for a given tile can need to stall until the IBC reference samples in the other tile become available. Although the discussion herein so far has focused on tiles, for this case, it should be noted that if the segments were not tiles but rather slices that are decoded linearly in scan order, there would be no coordination problem described above, although the memory access problem can still be an issue in some implementations.

[0093] In a third example, the link (614) can have sufficient bandwidth and coordination capabilities for both P-picture type motion compensation and IBC. Such a scenario is not currently contemplated in the context of H.265, but it is the basis for the "IP slice" concept described below.

[0094] Finally, in a fourth example, a moderate bit rate link (614) can have sufficient bit rate (and coordination capabilities) to support all forms of prediction contemplated in a video technology or standard, including, for example, intra-prediction, IBC, P-frame prediction, and B-frame prediction. Some shared memory and multi-processor architectures allow for this fourth scenario. Here, there can be no need for restrictions on the use of prediction tools across tile (or indeed segment) boundaries.

[0095] It must be noted that while the above description can be understood as implying a hierarchy of prediction techniques that can or can not be feasible based on link (614) bandwidth and / or coordination capabilities, this need not necessarily be the case. As a simple example, IBC's memory bandwidth requirements are at a similar level as P-prediction, while B-prediction can require twice the bandwidth (and multiple hypothesis prediction can require even more). However, even in cases where a system's design's memory bandwidth can allow for bi-prediction or multiple hypothesis prediction, IBC's coordination aspects can prevent its use across tile boundaries. Other architectural constraints can also exist.

[0096] It can be noted that the above example hardware architectures can be used in combination. For example, it can well be that hard-tiling according to system (600a) will be required to split very large pictures (8k and above) into manageable size units (e.g. 4k resolution can be achieved at write time, under commercially reasonable constraints for both software and hardware encoders and decoders). Within these hard-tiles (4k resolution in this example), further tiling can be advantageous and can be achieved by one or more of the schemes related to system (600b).

[0097] Likewise, it must be noted that the choice of prediction mechanism used across tile boundaries (or more generally, segment boundaries) can be driven not only by hardware implementation constraints such as the above, but also by application requirements. In some cases, preventing one or more forms of prediction across tile and segment boundaries can be advantageous from the perspective of application and coding efficiency.

[0098] In current video coding technology and standards, the forms of interrupted prediction are inflexible and in most cases associated with concepts and syntax elements that serve different purposes. As an example, H.265 (the following list is non-exhaustive):

[0099] IBC is allowed only when certain profiles are enabled, and IBC prediction across slice / tile boundaries is disabled regardless of profile;

[0100] Motion compensation and crossing slice / tile boundaries is allowed only for P and B slices (and independently of tiles) unless a motion-constrained tile set syntax element is set;

[0101] Intra prediction across slice boundaries is disabled, but can be across tile boundaries.

[0102] Some of these restrictions can be illustrated by the stepwise involvement of H.265. For example, H.265 was added after the first release of H.265, so it was necessary to place (and signal its use) by means of a profile. However, regardless of the history that made certain design choices necessary, the H.265 design disables certain combinations of prediction mechanisms across certain slice boundaries. This shortcoming of H.265 and other current video coding technologies and standards is now addressed.

[0103] Changing support for certain prediction mechanisms across slice boundaries in a video compression technology or standard can have two effects, both of which can need to be considered.

[0104] The first effect can be the need to specify the operation of decoders when using certain prediction mechanisms or when disabling the use of certain prediction mechanisms and thus not using them. In many video coding technologies and standards, this can be achieved by appropriately defining the "availability" of reference samples or reference metadata and using an inferred mechanism in the bitstream that is implied when disabling prediction across slice boundaries. These mechanisms are well known to those skilled in the art and are not elaborated on in detail herein.

[0105] The second effect can be the need to specify in the bitstream that a prediction tool across slice boundaries is allowed. The following describes options for this signaling mechanism in order from the least change compared to the signaling mechanisms available in H.265 to more generic mechanisms.

[0106] In the four examples of architectural constraints described above in connection with system (600b), the third example (which allows both IBC and P-prediction) can be implemented by introducing one or more additional slice types.

[0107] Figure 7 is a diagram of the syntax and semantics of the IP slice syntax element according to an embodiment.

[0108] In the same or another embodiment, a new slice type is introduced, referred to herein as an IP slice. The IP slice can be indicated using an appropriate value of the slice_type syntax element (701) in the slice_segment_header() syntax structure (702) as shown in Figure 7 with modified semantics of slice_type (703) relative to H.265. The modifications are shown using underlined text to indicate additions. The IP slice type can share all its properties with the I slice, except that the IP slice type allows IBC across its slice boundaries. In the same or another embodiment, an independently decodable refresh picture (IDR picture) can include I slices and IP slices (704).

[0109] Figure 8 is a diagram of the syntax and semantics of the PI slice syntax elements according to an embodiment.

[0110] In the same or another embodiment, a new slice type is introduced, referred to herein as the PI slice. The PI slice can be represented using an appropriate value of the slice_type syntax element (701) in the slice_segment_header() syntax structure (702) as shown in Figure 7 with modified semantics of slice_type (801) as shown in Figure 8 The PI slice type shares all its properties with the P slice, with the difference that the PI slice allows IBC across its slice boundaries.

[0111] Figure 9 is a diagram of the syntax and semantics of the B slice syntax elements according to an embodiment.

[0112] In the same or another embodiment, a new slice type is introduced, referred to herein as the BI slice. The BI slice can be represented using an appropriate value of the slice_type syntax element (701) in the slice_segment_header() syntax structure (702) as shown in Figure 7 with modified semantics of slice_type (901) as shown in Figure 9 The BI slice type shares all its properties with the B slice, with the difference that the BI slice allows IBC across its slice boundaries.

[0113] Figure 10 is a diagram of the syntax and semantics of the BI, PI and IP slice syntax elements according to an embodiment.

[0114] In the same or another embodiment, two or more of the aforementioned new slice types can be combined. As an example, Figure 10 shows semantics of the slice_type syntax element (1001) that includes all the three new slice types described above.

[0115] In the same or another embodiment, the use of the traditional I, B and P slices can imply that IBC prediction across slice boundaries is disabled regardless of the profile in use.

[0116] Figure 11 is a diagram of the syntax and semantics of the cross-boundary prediction flag according to an embodiment.

[0117] Reference is made to Figure 11In the same or another embodiment, a new syntax element ibc_accross_slice_boundaries_allowed_flag (1101) and / or ibc_across_tile_boundaries_allowed_flag can be introduced into a high-level syntax structure, such as a slice segment header (1102), a tile header, a picture parameter set, a sequence parameter set, a picture header, a GOP header, a sequence header, or any other appropriate high-level syntax structure. The semantics of such a flag can be as shown in Table 1, where ibc_accross_slice_boundaries_allowed_flag is used as an example. Figure 11

[0118] In the same or another embodiment, similar flags can be introduced for certain other prediction tools, including but not limited to intra prediction (intra_pred_accross_slice_boundaries_allowed_flag, 1104), P-frame prediction motion compensation (p-prediction-across-slice-boundaries-allowed-flag, 1105), bi-directional prediction motion compensation (b-prediction-across-slice-boundaries-allowed-flag, 1106), and so on. Those skilled in the art can readily come up with similar syntax elements related to segment types such as tiles, GOBs, and so on.

[0119] Many optimizations can be made to the encoding of one or more of the above flags. For example, since bi-directional prediction is only allowed for B slices by definition, the presence of b-prediction-across-slice-slice-boundaries-allowed-flag can be gated when the slice type is a B slice. Whether such parsing dependencies are worth the coding efficiency has been addressed in different (and sometimes inconsistent) ways in video coding technologies and standards. Both forms should be included.

[0120] The above-described techniques for prediction across segment boundaries can be implemented as computer software using computer-readable instructions, and physically stored in one or more computer-readable media.

[0121] Figure 12 is a diagram of a computer system (1200) suitable for implementing embodiments.

[0122] ​Computer software can be coded using any suitable machine code or computer language that can be subject to assembly, compilation, linking, or the like, and executed on a computer or multiple computers.

[0123] The instructions can be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, internet of things devices, and the like.

[0124] Figure 12 The components shown in the computer system (1200) are exemplary and not intended to suggest any limitation as to the scope of use or functionality of the employing computer software. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of a computer system (1200).

[0125] The computer system (1200) can include certain human interface input devices. Such a human interface input device can be responsive to user input data being generated anywhere within a particular vicinity of the computer system (1200). The user input

[0126] Input human interface devices can include one or more of (just one of each described): keyboard (1201), mouse (1202), touchpad (1203), touchscreen (1210), data g love (1204), joystick (1205), microphone (1206), scanner (1207), camera (1208).

[0127] The computer system (1200) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1210), data gloves (1204), or joystick (1205), but may also include tactile feedback devices that are not used as input devices, audio output devices (e.g., speakers (1209), headphones (not shown)), visual output devices (e.g., screens (1210), including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens, each screen having or not having touchscreen input capability, each having or not having tactile feedback capability—some of which may be able to output two-dimensional or more than three-dimensional visual outputs in a manner such as stereoscopic image output; virtual reality glasses (not depicted), holographic displays, and ashtrays (not depicted)) and printers (not depicted).

[0128] The computer system (1200) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1220) having media such as CD / DVD (1221), thumb drives (1222), removable hard disk drives or solid-state drives (1223), legacy magnetic media (e.g., magnetic tapes and floppy disks (not depicted)), devices based on dedicated ROM / ASIC / PLD (e.g., security dongles (not depicted)), etc.

[0129] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0130] The computer system (1200) may also include interfaces to one or more communication networks. These networks may be, for example, wireless networks, wired networks, or optical networks. Networks may also be local area networks, wide area networks (WANs), metropolitan area networks (MANs), vehicular and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of networks include: local area networks such as Ethernet and wireless LANs; cellular networks including Global System for Mobile Communications (GSM), 3G, 4G, 5G, and LTE; wired or wireless wide area digital network television including cable television, satellite television, and terrestrial broadcast television; and vehicular and industrial networks including CANBus. Some networks typically require external network interface adapters attached to certain general-purpose data ports or peripheral buses (1249) (e.g., the Universal Serial Bus (USB) port of the computer system (1200); others are typically integrated into the core of the computer system (1200) via attachment to the system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1200) can communicate with other entities. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., a CAN bus to some CAN bus device), or bidirectional, such as using a local area or wide area digital network to other computer systems. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0131] The human-machine interface devices, human-accessible storage devices and network interfaces mentioned above can be attached to the core (1240) of the computer system (1200).

[0132] The core (1240) may include one or more central processing units (CPUs) (1241), graphics processing units (GPUs) (1242), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (1243), hardware accelerators (1244) for certain tasks, etc. These devices, along with read-only memory (ROM) (1245), random access memory (RAM) (1246), and internal mass storage devices (1247) such as internal non-user-accessible hard disk drives (HDDs), SSDs, etc., can be connected via a system bus (1248). In some computer systems, the system bus (1248) may be accessed as one or more physical connectors to allow for expansion via additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (1249) to the core's system bus (1248). Peripheral bus architectures include peripheral component interconnect (PCI), USB, etc.

[0133] The CPU (1241), GPU (1242), FPGA (1243), and accelerator (1244) can execute certain instructions, which can be combined to form the aforementioned computer code. This computer code can be stored in ROM (1245) or RAM (1246). Transient data can also be stored in RAM (1246), while permanent data can be stored, for example, in an internal mass storage device (1247). Fast storage and retrieval of any storage device in the storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1241), GPUs (1242), mass storage devices (1247), ROMs (1245), RAMs (1246), etc.

[0134] Computer-readable media may have computer code thereon for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of implementation, or they may be of types known and available to those skilled in the art of computer software.

[0135] By way of example and not limitation, a computer system having an architecture (1200) – particularly the core (1240) – can provide functionality due to the execution of software embodied in one or more tangible computer-readable media by (one or more) processors (including CPUs, GPUs, FPGAs, accelerators, etc.). Such computer-readable media can be media associated with user-accessible mass storage devices as described above, as well as some non-transitory storage devices of the core (1240), such as internal mass storage devices (1247) or ROM (1245). Software implementing various embodiments can be stored in such devices and executed by the core (1240). Depending on specific needs, the computer-readable media may include one or more storage devices or chips. The software can cause the core (1240) – particularly the processors therein (including CPUs, GPUs, FPGAs, etc.) – to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1246) and modifying such data structures according to the processes defined by the software. Alternatively or concurrently, a computer system may provide functionality provided by hard-wired logic or otherwise embodied in circuitry (e.g., an accelerator (1244)) that may replace or operate with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, reference to software may include logic, and vice versa. Where appropriate, reference to computer-readable medium may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both.

[0136] Figure 13 This is a flowchart of a method (1300) for decoding an encoded image according to an embodiment. The encoded image includes an encoded video sequence divided into a first segment and a second segment. In some implementations, Figure 13 One or more processing blocks can be executed by the decoder (210). In some implementations, Figure 13 One or more processing blocks can be performed by another device or a group of devices (e.g., encoder 203) that are separate from or include the decoder (210).

[0137] Reference Figure 13 In the first block (1310), the method (1300) includes: determining a first decoding process for decoding the first segment based at least on a first syntax element of a high-level syntax structure applicable to the first segment and the second segment, the first syntax element being used to instruct the first decoding process to disable the first prediction.

[0138] In the second block (1320), the method (1300) includes: determining a second decoding process for decoding the second segment based at least on a second syntax element of the high-level syntax structure, the second prediction being different from the first prediction, the second syntax element being used to instruct the second decoding process to disable the second prediction.

[0139] In the third block (1330), the method (1300) includes: decoding the first segment based on a first decoding process that disables the first prediction.

[0140] In the fourth block (1340), the method (1300) includes: decoding the second segment based on a second decoding process that disables the second prediction.

[0141] At least one of the first and second partitions may include a slice.

[0142] At least one of the first and second division segments may include a tile.

[0143] At least one of the first and second partitions may include a block group.

[0144] The first prediction may include any one of the following: intra-block copying across segment boundaries, intra-frame prediction across segment boundaries, predicted image prediction across segment boundaries, and double-predicted image prediction across segment boundaries, and the second prediction may include any one of the following: intra-block copying across segment boundaries, intra-frame prediction across segment boundaries, predicted image prediction across segment boundaries, and double-predicted image prediction across segment boundaries.

[0145] Each of the first and second syntax elements can be a marker of a higher-level syntax structure.

[0146] Advanced syntax structures can be placed at the beginning of each segment in the first and second divisions.

[0147] Advanced syntax structures can be one of the image parameter set and the sequence parameter set.

[0148] Advanced syntax structures can be any of the following: image header, block header, and sequence header.

[0149] although Figure 13 An example block of method (1300) is shown, but in some implementations, it is different. Figure 13 Compared to the blocks depicted, method (1300) may include additional blocks, fewer blocks, different blocks, or blocks arranged differently. Additionally or alternatively, two or more blocks of method (1300) may be executed in parallel.

[0150] Furthermore, the proposed method can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In the example, one or more processors execute a program stored in a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0151] Figure 14 This is a simplified block diagram of a device (1400) for decoding an encoded image according to an embodiment, the encoded image comprising an encoded video sequence divided into a first segment and a second segment.

[0152] Reference Figure 14 The device (1400) includes a first determination code (1410), a second determination code (1420), a first decoding code (1430), and a second decoding code (1440).

[0153] Reference Figure 14 The first determining code (1410) is configured to determine a first decoding process for decoding the first segment based at least on a first syntax element of a high-level syntax structure applicable to the first segment and the second segment, the first syntax element being used to indicate that the first prediction is disabled.

[0154] The second determining code (1420) is configured to determine a second decoding process for decoding the second segment based at least on a second syntax element of a high-level syntax structure, the second prediction being different from the first prediction, the second syntax element being used to instruct the second decoding process to disable the second prediction.

[0155] The first decoding code (1430) is configured to decode the first segment based on the first decoding process that disables the first prediction.

[0156] The second decoding code (1440) is configured to decode the second segment based on the second decoding process that disables the second prediction.

[0157] At least one of the first and second partitions may include a slice.

[0158] At least one of the first and second division segments may include a tile.

[0159] At least one of the first and second partitions may include a block group.

[0160] The first prediction may include any one of the following: intra-block copying across segment boundaries, intra-frame prediction across segment boundaries, predicted image prediction across segment boundaries, and double-predicted image prediction across segment boundaries, and the second prediction may include any one of the following: intra-block copying across segment boundaries, intra-frame prediction across segment boundaries, predicted image prediction across segment boundaries, and double-predicted image prediction across segment boundaries.

[0161] Each of the first and second syntax elements can be a marker of a higher-level syntax structure.

[0162] Advanced syntax structures can be placed at the beginning of each segment in the first and second divisions.

[0163] Advanced syntax structures can be one of the image parameter set and the sequence parameter set.

[0164] Advanced syntax structures can be any of the following: image header, block header, and sequence header.

[0165] Therefore, by selecting the prediction mechanism used, such as determining the first decoding process that disables the first prediction and the second decoding process that disables the second prediction, it is possible to prevent one or more forms of prediction across tile and segment boundaries. From the perspective of application and coding efficiency, this approach can bring beneficial effects.

[0166] The above-mentioned technologies can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media.

[0167] Although several exemplary embodiments have been described in this disclosure, there are changes, substitutions, and various equivalent alternatives that fall within the scope of this disclosure. Therefore, it should be recognized that, although not expressly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that embody the principles of this disclosure and thus its spirit and scope.

Claims

1. A method of decoding a coded picture in a coded video sequence, the coded picture comprising a first partition and a second partition, the method comprising: The method comprises: determining a first decoding process for decoding the first partition based on at least a first syntax element, the first syntax element being used to indicate that the first decoding process disables a first prediction; determining a second decoding process for decoding the second partition based on at least a second syntax element, the second syntax element being used to indicate that the second decoding process disables a second prediction, the second prediction being different from the first prediction; wherein at least one of the first partition and the second partition comprises a slice; decoding the first partition based on the first decoding process disabling the first prediction; and decoding the second partition based on the second decoding process disabling the second prediction; the first prediction comprises any one of: an intra block copy across a segment boundary, an intra prediction across a segment boundary, a predicted picture prediction across a segment boundary, and a bi-predicted picture prediction across a segment boundary; and the second prediction is different from the first prediction and comprises one of: the intra block copy across a segment boundary, the intra prediction across a segment boundary, the predicted picture prediction across a segment boundary, and the bi-predicted picture prediction across a segment boundary.

2. The method of claim 1, wherein, Each of the first syntax element and the second syntax element is a signal flag.

3. The method of claim 1, wherein, The first syntax element and the second syntax element are located in a header of each of the first partition and the second partition, respectively.

4. The method of claim 1, wherein, The first syntax element and the second syntax element are one of a picture parameter set and a sequence parameter set.

5. The method of claim 1, wherein, The first syntax element and the second syntax element are located in any one of: a picture header, a block group header, and a sequence header.

6. A method of encoding a video, each picture in the video comprising a first partition and a second partition, characterized in that, The method comprises: determining a first encoding process for encoding the first partition based on at least a first syntax element, the first syntax element being used to indicate that the first encoding process disables a first prediction; determining a second encoding process for encoding the second partition based on at least a second syntax element, the second syntax element being used to indicate that the second encoding process disables a second prediction, the second prediction being different from the first prediction; wherein at least one of the first partition and the second partition comprises a slice; encoding the first partition based on the first encoding process disabling the first prediction; and encoding the second partition based on the second encoding process disabling the second prediction; the first prediction comprises any one of: an intra block copy across a segment boundary, an intra prediction across a segment boundary, a predicted picture prediction across a segment boundary, and a bi-predicted picture prediction across a segment boundary; and the second prediction is different from the first prediction and comprises one of: the intra block copy across a segment boundary, the intra prediction across a segment boundary, the predicted picture prediction across a segment boundary, and the bi-predicted picture prediction across a segment boundary.

7. The method of claim 6, wherein, Each of the first syntax element and the second syntax element is a signal flag.

8. The method of claim 6, wherein, The first syntax element and the second syntax element are located in a header of each of the first partition and the second partition.

9. The method of claim 6, wherein, The first syntax element and the second syntax element are one of a picture parameter set and a sequence parameter set.

10. The method of claim 6, wherein, The first syntax element and the second syntax element are located in any one of a picture header, a group of blocks header, and a sequence header.

11. An apparatus for decoding a coded picture of a coded video sequence, the apparatus comprising: The coded picture comprises a first partition and a second partition, the apparatus comprises a determining unit and a decoding unit: The determining unit is configured to determine a first decoding process for decoding the first partition based on at least a first syntax element, the first syntax element being used to indicate that the first decoding process disables a first prediction; The determining unit is further configured to determine a second decoding process for decoding the second partition based on at least a second syntax element, the second syntax element being used to indicate that the second decoding process disables a second prediction, the second prediction being different from the first prediction; wherein at least one of the first partition and the second partition comprises a slice; The decoding unit is configured to decode the first partition based on the first decoding process disabling the first prediction; and The decoding unit is further configured to decode the second partition based on the second decoding process disabling the second prediction; The first prediction comprises any one of an inter- segment boundary intra block copy, an inter- segment boundary intra prediction, an inter- segment boundary predicted picture prediction, and an inter- segment boundary bi-predicted picture prediction; and The second prediction is different from the first prediction and comprises one of the inter- segment boundary intra block copy, the inter- segment boundary intra prediction, the inter- segment boundary predicted picture prediction, and the inter- segment boundary bi-predicted picture prediction.

12. An apparatus for encoding a video, each picture in the video comprising a first partition and a second partition, characterized in that, The apparatus comprises: determine a first encoding process for encoding the first partition based on at least a first syntax element, the first syntax element being used to indicate that the first encoding process disables a first prediction; determine a second encoding process for encoding the second partition based on at least a second syntax element, the second syntax element being used to indicate that the second encoding process disables a second prediction, the second prediction being different from the first prediction; wherein at least one of the first partition and the second partition comprises a slice; encode the first partition based on the first encoding process disabling the first prediction; and encode the second partition based on the second encoding process disabling the second prediction; The first prediction comprises any one of an inter- segment boundary intra block copy, an inter- segment boundary intra prediction, an inter- segment boundary predicted picture prediction, and an inter- segment boundary bi-predicted picture prediction; and The second prediction is different from the first prediction and comprises one of the inter- segment boundary intra block copy, the inter- segment boundary intra prediction, the inter- segment boundary predicted picture prediction, and the inter- segment boundary bi-predicted picture prediction.

13. An apparatus for decoding a coded picture of a coded video sequence, the coded picture comprising a first partition and a second partition, the apparatus comprising: at least one memory configured to store computer programs; and at least one processor configured to access the at least one memory and operate as instructed by the computer programs to perform the method of any of claims 1-5.

14. An apparatus for encoding a video, the pictures in the video comprising a first partition and a second partition, the apparatus comprising: at least one memory configured to store computer programs; and at least one processor configured to access the at least one memory and operate as instructed by the computer programs to perform the method of any of claims 6-10.

15. A non-transitory computer readable storage medium storing a program which causes a processor to perform the method of any of claims 1-10.

16. A computer device, comprising: The apparatus comprises a processor and a memory: The memory is configured to store a program and transmit the program to the processor; The processor is configured to perform the method of any of claims 1-10 according to instructions in the program.

17. A method of storing a bitstream, the method comprising: The method of any of claims 6-10 is performed to generate a bitstream; and the bitstream is stored.

18. A method of transmitting a bitstream, the method comprising: The method of any of claims 6-10 is performed to generate a bitstream; and the bitstream is transmitted.

19. A computer readable storage medium having stored thereon computer programs / instructions and a bitstream, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method of any of claims 6-10 to generate the bitstream.

Images