Video bitstream encoding method, device, and program

Abstract

To provide a method and a device for controlling intra-inter prediction.SOLUTION: A method includes the steps of: determining whether or not a neighboring block of a current block is encoded according to an intra-inter prediction mode; based on determining that the neighboring block is encoded according to the intra-inter prediction mode, performing intra-mode encoding of the current block utilizing an intra prediction mode associated with the intra-inter prediction mode; setting a prediction mode flag associated with the neighboring block; acquiring a context value based on the prediction mode flag associated with the neighboring block; and performing entropy coding relative to the prediction mode flag associated with the current block that indicates that the current block is intra-coded utilizing the acquired context value.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 767.473, filed November 14, 2018, and U.S. Application No. 16 / 454.294, filed June 27, 2019, in the United States Patent and Trademark Office, the entire contents of which are incorporated herein by reference.

[0002] Methods and apparatus according to embodiments relate to video encoding, and more particularly to methods and apparatus for improved context design for entropy encoding of prediction modes and coding block flags (CBFs). [Background technology]

[0003] Figure 1A shows the intra prediction modes used in High Efficiency Video Coding (HEVC). In HEVC, there are a total of 35 intra prediction modes, among which mode 10 (101) is a horizontal mode, mode 26 (102) is a vertical mode, and mode 2 (103), mode 18 (104), and mode 34 (105) are diagonal modes. These intra prediction modes are signaled by three Most Probable Modes (MPMs) and the remaining 32 modes.

[0004] For versatile video coding (VVC), some coding unit syntax tables are shown below. If the slice type is not intra and skip mode is not selected, the flag pred_mode_flag is signaled and encoded using only one context (e.g., the variable pred_mode_flag). The syntax table of some coding units is as follows: [Table 1]

[0005] 1B, in VVC, there are a total of 87 intra-prediction modes, among which mode 18 (106) is a horizontal mode, mode 50 (107) is a vertical mode, and mode 2 (108), mode 34 (109), and mode 66 (110) are diagonal modes. Modes-1 to -10 (111), and modes 67 to 76 (112) are called Wide-Angle Intra Prediction (WAIP) modes.

[0006] For the chrominance component of an intra-coding block, the encoder selects the optimal chrominance prediction mode among five modes, including planar mode (mode index 0), DC mode (mode index 1), horizontal mode (mode index 18), vertical mode (mode index 50) and diagonal mode (mode index 66), and selects a direct copy of the intra-prediction mode of the associated luma component, i.e., DM mode. Table 1 below shows the mapping between the chrominance intra-prediction direction and the intra-prediction mode number. [Table 2] Summary of the Invention [Problem to be solved by the invention]

[0007] To avoid overlapping modes, the four modes other than the DM mode are assigned based on the intra prediction mode of the associated luma component. If the intra prediction mode number of the chroma component is 4, the intra prediction direction of the luma component is applied to generate the intra prediction samples of the chroma component. If the intra prediction mode number of the chroma component is not 4 but is the same as the intra prediction mode number of the luma component, the intra prediction direction 66 is applied to generate the intra prediction samples of the chroma component.

[0008] Multi-hypothesis intra-inter prediction combines one intra prediction and one merge indexed prediction, i.e., intra-inter prediction mode. In the merge coding unit (CU), one flag is signaled for the merge mode, and if the flag is true, an intra mode is selected from the intra candidate list. For the luma component, the intra candidate list is obtained from four intra prediction modes, including DC mode, planar mode, horizontal mode, and vertical mode, and the size of the intra candidate list may be 3 or 4 depending on the block shape. If the width of the CU is greater than twice the height of the CU, remove the horizontal mode from the intra candidate list, and if the height of the CU is greater than twice the width of the CU, remove the vertical mode from the intra candidate list. Use weighted average to combine one intra prediction mode selected by the intra mode index and one merge indexed prediction selected by the merge index. For the chroma component, no additional signaling is required, and always use DM.

[0009] The weights for combining predictions are explained as follows: If DC mode or planar mode is selected or the width or height of the coding block (CB) is less than 4, apply equal weights. For those CBs whose width or height is 4 or more, if horizontal / vertical mode is selected, first divide one CB vertically / horizontally into 4 equal area regions. For each region, assign the corresponding (w_intra i ,w_inter i), where i ranges from 1 to 4, and (w_intra1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra1,w_inter3)=(3,5), (w_intra4,w_inter4)=(2,6). (w_intra1,w_inter1) correspond to the region closest to the reference sample, and (w_intra4,w_inter4) correspond to the region furthest from the reference sample. The combined prediction is then calculated by summing the two weighted predictions and right shifting them by 3 bits. Also, the intra prediction mode of the predictor's intra hypothesis can be saved for subsequent intra mode coding for neighboring CBs if these CBs are intra coded. [Means for solving the problem]

[0010] According to an embodiment, a method for controlling intra-inter prediction for decoding or encoding a video sequence is executed by at least one processor, the method comprising: determining whether a neighboring block of a current block is coded by an intra-inter prediction mode, and performing the following operations based on determining that the neighboring block is coded by the intra-inter prediction mode: performing intra-mode coding of the current block using an intra-prediction mode associated with the intra-inter prediction mode, setting a prediction mode flag associated with the neighboring block, obtaining a context value based on the set prediction mode flag associated with the neighboring block, and performing entropy coding using the obtained context value for a prediction mode flag associated with the current block indicating that the current block is intra-coded.

[0011] According to an embodiment, an apparatus for controlling intra-inter prediction for decoding or encoding a video sequence includes at least one memory arranged to store computer program code, and at least one processor arranged to access the at least one memory and operate based on the computer program code. The computer program code includes a first decision code arranged to cause the at least one processor to determine whether a neighboring block of a current block is coded by an intra-inter prediction mode, an execution code arranged to cause the at least one processor to perform intra-mode coding of the current block using an intra-prediction mode associated with the intra-inter prediction mode based on the determination that the neighboring block is coded by the intra-inter prediction mode, and a setting code arranged to cause the at least one processor to perform the following operations based on the determination that the neighboring block is coded by the intra-inter prediction mode: set a prediction mode flag associated with the neighboring block, obtain a context value based on the set prediction mode flag associated with the neighboring block, and perform entropy coding using the obtained context value for a prediction mode flag associated with the current block indicating that the current block is intra-coded.

[0012] According to an embodiment, a non-transitory computer-readable storage medium having instructions stored thereon causes at least one processor to perform the following operations: determining whether a neighboring block of a current block is encoded by an intra-inter prediction mode; and, based on the determination that the neighboring block is encoded by the intra-inter prediction mode, performing intra-mode encoding of the current block using an intra-prediction mode associated with the intra-inter prediction mode, setting a prediction mode flag associated with the neighboring block, obtaining a context value based on the set prediction mode flag associated with the neighboring block, and performing entropy encoding using the obtained context value for the prediction mode flag associated with the current block indicating that the current block is intra-coded. [Brief description of the drawings]

[0013] [Figure 1A] FIG. 2 is a diagram of intra prediction modes in HEVC. [Figure 1B] FIG. 1 is a diagram showing intra prediction modes in VVC. [Diagram 2] FIG. 1 is a simplified block diagram of a communication system, according to an embodiment. [Diagram 3] FIG. 2 is a diagram of an arrangement of a video encoder and a video decoder in a streaming environment according to an embodiment. [Figure 4] FIG. 2 is a functional block diagram of a video decoder according to an embodiment. [Diagram 5] FIG. 2 is a functional block diagram of a video encoder according to an embodiment. [Figure 6] 2 is a diagram of a current block and neighboring blocks of the current block, according to an embodiment. [Figure 7] 1 is a flow chart illustrating a method for controlling intra-inter prediction for decoding or encoding a video sequence according to an embodiment. [Figure 8]1 is a simplified block diagram of an apparatus for controlling intra-inter prediction for decoding or encoding a video sequence according to an embodiment; [Figure 9] 1 is an illustration of a computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] FIG. 2 is a simplified block diagram of a communication system (200) according to an embodiment. The communication system (200) may include at least two terminals (210-220) interconnected via a network (250). In the case of one-way data transmission, a first terminal (210) can transmit video data at a local location to another terminal (220) via the network (250) by encoding the video data. The second terminal (220) can receive the encoded video data of the other terminal from the network (250), decode the encoded video data, and display the restored video data. One-way data transmission may be common in media service applications, etc.

[0015] 2 shows a second pair of terminals (230, 240) provided to support bidirectional transmission of encoded video, such as may occur during a video conference. For bidirectional data transmission, each terminal (230, 240) can transmit video data captured at a local location to the other terminal via the network (250) by encoding the video data. Each terminal (230, 240) can also receive encoded video data transmitted from the other terminal, decode the encoded video data, and display the reconstructed video data on a local display device.

[0016] In FIG. 2, the terminals (210-240) are shown as servers, personal computers, and smartphones, although the principles of the embodiments are not so limited. The embodiments apply to laptop computers, tablets, media players, and / or dedicated video conferencing equipment. The network (250) represents any number of networks, including, for example, wired and / or wireless communication networks, for transmitting encoded video data between the terminals (210-240). The communication network (250) may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of discussion of this application, the architecture and topology of the network (250) may not be important to the operation of the embodiments, unless otherwise described herein below.

[0017] 3 is a diagram of a configuration of video encoders and video decoders in a streaming environment, according to an embodiment. The disclosed subject matter may equally be applied to other applications for supporting video, including, for example, video conferencing, digital TV, storage of compressed video, etc. on digital media, including CDs, DVDs, memory sticks, etc.

[0018] The streaming system may include a capture subsystem (313), which may include a video source (301), such as a digital camera, for creating an uncompressed video sample stream (302). The sample stream (302) is depicted as a thick line to emphasize the amount of data compared to an encoded video bitstream, which may be processed by an encoder (303) coupled to the capture device (301). The encoder (303) may include hardware, software, or a combination thereof to realize or implement aspects of the disclosed subject matter, which are described in more detail below. The encoded video bitstream (304) is depicted as a thin line to emphasize the amount of data compared to the sample stream, which may be stored in a streaming server (305) for future use. One or more streaming clients (306, 308) can access the streaming server (305) to retrieve replicas (307, 309) of the encoded video bitstream (304). The client (306) can include a video decoder (310) for decoding the incoming replica (307) of the encoded video data and for creating an outgoing video sample stream (311) that is rendered on a display (312) or other rendering device (not shown). In some streaming systems, the video bitstreams (304, 307, 309) can be encoded according to a number of video encoding / compression standards. Examples of these standards include the ITU-T H.265 proposal. A video encoding standard informally referred to as VVC is under development. The disclosed subject matter may be applied in the context of VVC.

[0019] FIG. 4 is a functional block diagram of a video decoder (310) according to an embodiment.

[0020] The receiver (410) can receive one or more codec video sequences to be decoded by the decoder (310), in the same or other embodiments, one encoded video sequence at a time, with the decoding of each encoded video sequence being independent of the other encoded video sequences. The encoded video sequences can be received from a channel (412), which may be a hardware / software link to a storage device for storing the encoded video data. The receiver (410) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which can be forwarded to a respective using entity (not shown). The receiver (410) can separate the encoded video sequences from the other data. To deal with network jitter, a buffer memory (415) can be coupled between the receiver (410) and the entropy decoder / parser (hereinafter referred to as the "parser"). If the receiver 410 receives data from a store-and-forward device or an isochronous real-time network with sufficient bandwidth and controllability, it may not require a buffer memory 415, or the buffer 615 may be small. To favorably utilize packet networks such as the Internet, a buffer memory 415 may be required, which may be relatively large and advantageously of adaptive size.

[0021] The video decoder (310) may include a parser (420) to reconstruct symbols (421) based on the entropy coded video sequence. These symbol categories include information for managing the operation of the video decoder (310) and potentially information for controlling a rendering device, such as a display (312), which may not be part of the decoder but may be coupled to the decoder, as shown in FIG. 4. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (420) performs parsing / entropy decoding on the received coded video sequence. The coding of the coded video sequence may be based on a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length codes, Huffman coding, arithmetic coding with or without context-dependent coding, etc. The parser extracts a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on the at least one parameter corresponding to the group, including Groups of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The entropy decoder / parser may further extract information such as transform coefficients, quantizer parameter (QP) values, motion vectors, etc. from the coded video sequence.

[0022] The parser (420) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to produce symbols (421). The parser may receive the encoded data and selectively decode particular symbols (421). The parser may also determine whether a particular symbol (421) is to be provided to a motion compensated prediction unit (453), a scaler / inverse transform unit (451), an intra prediction unit (452), or a loop filter (454).

[0023] Depending on the coded video picture or some type of coded video picture (e.g., inter-picture vs. intra-picture, inter-block vs. intra-block) and other factors, the reconstruction of the symbol (421) may involve several different units. The units involved and the manner of involvement are controlled by subgroup control information parsed by the parser (420) from the coded video sequence. For the sake of brevity, the flow of such subgroup control information between the parser (420) and the following units is not described.

[0024] In addition to the functional blocks already mentioned, the decoder (310) may be conceptually subdivided into a number of functional units, as described below. In an actual implementation that operates under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of illustrating the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0025] The first unit is a scalar / inverse transform unit (451), which receives quantized transform coefficients as symbols (421) and control information from the parser (420), including the transform scheme to be used, the size of the block, the quantization factor, the quantization scaling matrix, etc. It can output blocks containing sample values ​​that can be input to an aggregator (455).

[0026] In some cases, the output samples of the scaler / inverse transform (451) may belong to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed part of the current picture. Such prediction information may be provided by an intra prediction unit (452). In some cases, the intra prediction unit (452) uses surrounding already reconstructed information obtained from the current (partially reconstructed) picture (456) to generate a block of the same size and shape as the block it is reconstructing. In some cases, the aggregator (455) adds the prediction information generated by the intra prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0027] In other cases, the output samples of the scalar / inverse transform unit (451) may belong to an inter-coded and potentially motion-compensated block. In such a case, the motion compensation prediction unit (453) may access the reference picture buffer (457) to obtain samples for prediction. After performing motion compensation on the obtained samples based on the symbols (421) belonging to the block, these samples may be added by the aggregator (455) to the output of the scalar / inverse transform unit (called residual samples or residual signals in this case) to generate output sample information. The addresses in the reference picture memory from which the motion compensation prediction unit extracts the prediction samples may be controlled by a motion vector, which may be used by the motion compensation prediction unit in the form of a symbol (421), which may have, for example, X, Y and reference picture components. Motion compensation may further include interpolation of sample values ​​obtained from the reference picture memory when a sub-sample accurate motion vector is used, a motion vector prediction mechanism, etc.

[0028] The output samples of the aggregator (455) may be processed by various loop filtering techniques in a loop filter unit (454). The video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream, which may be applied to the loop filter unit (454) as symbols (421) from the parser (420), however the video compression techniques may also be responsive to meta information obtained during decoding of a previous part (in decoding order) of the encoded picture or encoded video sequence, or to previously constructed loop filtered sample values.

[0029] The output of the loop filter unit (454) may be a sample stream, which may be output to the rendering device (312) and stored in a reference picture buffer (456) for use in future inter-picture prediction.

[0030] Once an encoded picture has been fully reconstructed, it is used as a reference picture for future prediction. Once an encoded picture has been fully reconstructed and the encoded picture has been recognized as a reference picture (e.g., by the parser (420)), the current reference picture (456) becomes part of the reference picture buffer (457) and a new current picture memory can be reallocated before starting reconstruction of a subsequent encoded picture.

[0031] The video decoder (310) may perform decoding operations based on a given video compression technique, e.g., documented in the ITU-T H.265 proposal standard. The encoded video sequence conforms to the syntax specified by the video compression technique or standard used, e.g., in a video compression technique document or standard, in the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard explicitly specified in its profile. Compliance also requires that the complexity of the encoded video sequence is within a range limited by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in mega samples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further limited by the specification of a Hypothetical Reference Decoder (HRD) and HRD buffer management metadata signaled in the encoded video sequence.

[0032] In an embodiment, the receiver (410) can receive additional (redundant) data along with the encoded video. The additional data is included as part of the encoded video sequence. The additional data is utilized by the video decoder (310) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0033] FIG. 5 may be a functional block diagram of a video encoder (303) according to an embodiment.

[0034] The encoder (303) can receive video samples from a video source (301) (not part of the encoder), which can capture video images that are encoded by the encoder (303).

[0035] The video source (301) may provide a source video sequence in the form of a digital video sample stream, which may include any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 Y CrCB, RGB, etc.), and any suitable sampling configuration (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4), to be encoded by the encoder (303). In a media serving system, the video source (301) may be a storage device for storing previously prepared video. In a video conferencing system, the video source (301) may be a camera for capturing local image information as a video sequence. The video data may be provided as a number of individual pictures that are given motion when viewed in sequence. The pictures themselves may be organized as a spatial pixel array, where each pixel may include one or more samples, depending on the sampling configuration, color space, etc. used. The relationship between pixels and samples is readily apparent to those skilled in the art. The following description focuses on samples.

[0036] According to an embodiment, the encoder (303) encodes pictures of a source video sequence in real time or with any other time constraint required by the application, and compresses them as an encoded video sequence (543). Running at an appropriate encoding rate is one of the functions of the controller (550). The controller controls and is functionally coupled to other functional units described below. For the sake of simplicity, the coupling is not shown. Parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, etc.), picture size, group of pictures (GOP) placement, search range for maximum motion vector, etc. One skilled in the art can easily recognize other functions of the controller (550) for a video encoder (303) optimized for a particular system design.

[0037] Some video encoders operate in a "coding loop" that is easily understood by those skilled in the art. As a very simplified explanation, the coding loop may include an encoding part (responsible for creating symbols based on the input picture to be encoded and reference pictures) of an encoder (530) (hereafter called the "source encoder"), and a (local) decoder (533) embedded in the encoder (303) that reconstructs the symbols to create sample data that is also intended to be created by the (remote) decoder (because in the video compression techniques considered in the subject matter of this disclosure, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream is input to a reference picture memory (534). The decoding of the symbol stream results in a bit-exact result that is independent of the decoder location (local or remote), so that the contents of the reference picture buffer are bit-exact between the local encoder and the remote encoder. That is, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​"seen" by the decoder when it uses the prediction during decoding. The basic principle of such reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, e.g., due to channel errors) is well known to those skilled in the art.

[0038] The operation of the "local" decoder (533) may be the same as the operation of the "remote" decoder (310), described in detail above with reference to Figure 4. However, with brief reference to Figure 4, if symbols are available and the entropy encoder (545) and parser (420) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the decoder (310), including the channel (412), receiver (410), buffer memory (415) and parser, need not be fully implemented in the decoder (533).

[0039] In this case, it can be observed that any decoder technique other than analysis / entropy decoding present in the decoder will necessarily be present in the corresponding encoder in essentially the same functional form. The description of the encoder technique can be simplified since the encoder technique and the fully described decoder technique are mutually inverse. A more detailed description is only necessary in a few places and is provided below.

[0040] As part of its operation, the source encoder (530) may perform motion-compensated predictive encoding, which performs predictive encoding on an input frame with reference to one or more previously encoded frames from the video sequence designated as “reference frames.” In this manner, the encoding engine (532) may encode differences between pixel blocks of the input frame and pixel blocks that may be selected as reference frames of the input frame's predicted reference.

[0041] The decoder (533) decodes previously encoded video data of frames designated as reference frames based on the codes created by the source encoder (530). The operation of the encoding engine (532) may advantageously be a lossy process. When the encoded video data is decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence may be a replica of the source video sequence, typically with some errors. The decoder (533) may copy the decoding process that may be performed on the reference frames by the video decoder and may store the reconstructed reference frames in a reference picture cache (534). In this way, the video encoder (303) may locally store replicas of reconstructed reference frames that have a common content with the reconstructed reference frames obtained by the remote video decoder (no transmission errors are present).

[0042] The predictor (535) performs a prediction search for the coding engine (532). That is, for a new frame to be coded, the predictor (535) may search the reference picture memory (534) for sample data (as candidate reference pixel blocks) that can be used as suitable prediction references for the new picture, or for specific metadata, such as the motion vectors, block shapes, etc., of the reference pictures. The predictor (535) can find suitable prediction references by operating on a pixel block by pixel block basis based on the sample blocks. In some cases, the input picture may have prediction references obtained from multiple reference pictures stored in the reference picture memory (534), as determined from the search results obtained by the predictor (535).

[0043] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters for encoding the video data and subgroup parameters.

[0044] The output of all these functional units may be subjected to entropy coding in an entropy coder (545) which performs lossless compression on the symbols produced by the various functional units, converting them into an encoded video sequence, using techniques known to those skilled in the art (e.g. Huffman coding, variable length coding, arithmetic coding, etc.).

[0045] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission over a communication channel (560), which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter (540) can merge the encoded video data from the source encoder (530) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0046] The controller (550) can manage the operation of the encoder (303). During encoding, the controller (550) may assign each encoded picture a particular encoded picture type, which may affect the encoding technique applied to the corresponding picture. For example, pictures are typically assigned as one of the following frame types:

[0047] An Intra picture (I-picture) may be a picture that is coded and decoded without using any other frame in a sequence as a source of prediction. Some video codecs allow different types of Intra pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art are aware of these variations of I-pictures and their corresponding uses and characteristics.

[0048] A predictive picture (P-picture) may be a picture that is encoded and decoded using intra- or inter-prediction to predict sample values ​​for each block using at most one motion vector and reference index.

[0049] A bidirectionally predictive picture (B-picture) may be a picture that is encoded and decoded using intra- or inter-prediction to predict the sample values ​​of each block using at most two motion vectors and reference indexes. Similarly, a multiple predicted picture may use more than two reference images and associated metadata to reconstruct a single block.

[0050] A source picture may generally be spatially subdivided into a number of sample blocks (e.g., blocks of 4x4, 8x8, 4x8 or 16x16 samples, respectively) and coded block by block. A block may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the block's corresponding picture. For example, a block of an I-picture may be non-predictively coded or predictively coded with reference to coded blocks of the same picture (spatial or intra prediction). A pixel block of a P-picture may be non-predictively coded via spatial or temporal prediction with reference to one previously coded reference picture. A block of a B-picture may be non-predictively coded via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0051] The video encoder (303) may perform encoding operations based on a given video encoding technique or standard, for example, the ITU-T H.265 proposal. During operation of the video encoder (303), the video encoder (303) may perform various compression operations, including predictive encoding operations that rely on temporal and spatial redundancies in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0052] In an embodiment, the transmitter (540) can transmit additional data along with the encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other types of redundant data such as redundant pictures and slices, supplemental enhancement information (SEI) messages, video usability information (VUI) parameter set segments, etc.

[0053] In the prior art, only one context is used to code the flag pred_mode_flag for indicating whether a block is intra-coded or inter-coded, instead of the value of the flag applied to the neighboring block. Also, if the neighboring block is coded by the intra-inter prediction mode, it may be more effective to predict the neighboring block using a combination of the intra prediction mode and the inter prediction mode, and therefore, for the context design of signaling the flag pred_mode_flag, to consider whether the neighboring block is coded by the intra-inter prediction mode.

[0054] The embodiments described herein may be used alone or in combination in any order. Below, the flag pred_mode_flag indicates whether the current block is intra-coded or inter-coded.

[0055] FIG. 6 is a diagram of a current block and neighboring blocks of the current block, according to an embodiment.

[0056] 6, there is shown a current block 610, a top neighboring block 620, and a left neighboring block 630 of the current block 610. The top neighboring block 620 and the left neighboring block 630 each have a width of 4 and a height of 4.

[0057] In an embodiment, information on whether the neighboring blocks (e.g., the top neighboring block (620) and the left neighboring block (630)) are coded using an intra prediction mode, an inter prediction mode, or an intra-inter prediction mode is used to obtain a context value for entropy coding the flag pred_mode_flag of the current block (e.g., the current block (610)). In particular, if the neighboring blocks are coded using an intra-inter prediction mode, the associated intra prediction mode is applied to the intra mode coding of the current block and / or the derivation of the MPM, but when deriving a context value for entropy coding the flag pred_mode_flag of the current block, the neighboring blocks are considered to be inter coding blocks, even though intra prediction was used for the neighboring blocks.

[0058] In one example, the associated intra prediction mode of an intra prediction mode is always a planar mode.

[0059] In another example, the associated intra prediction mode of an intra prediction mode is always DC mode.

[0060] In yet another example, the associated intra-prediction mode aligns with the intra-prediction mode applied in the intra-inter prediction mode.

[0061] In an embodiment, when encoding neighboring blocks (e.g., the top neighboring block (620) and the left neighboring block (630)) using an intra-inter prediction mode, the associated intra prediction mode is applied to the intra mode encoding of the current block (e.g., the current block (610)) and / or the derivation of the MPM, and the neighboring blocks are also considered to be intra-coded blocks when deriving context values ​​for entropy encoding the flag pred_mode_flag of the current block.

[0062] In one example, the associated intra prediction mode of an intra prediction mode is always a planar mode.

[0063] In another example, the associated intra prediction mode of an intra prediction mode is always DC mode.

[0064] In yet another example, the associated intra-prediction mode aligns with the intra-prediction mode applied in the intra-inter prediction mode.

[0065] In one embodiment, the context index or value is incremented by 2, 0 and 1 if the neighboring blocks are coded with intra prediction mode, inter prediction mode and inter-intra prediction mode respectively.

[0066] In another embodiment, if the neighboring blocks are coded using intra prediction, inter prediction and inter-intra prediction modes, respectively, the context index or value is incremented by 1, 0 and 0.5, respectively, and the final context index is rounded to the nearest integer.

[0067] After incrementing the context index or value for all neighboring blocks of the current block to determine a final context index, the average context index can be determined by dividing the determined final context index by the number of neighboring blocks and rounding to the nearest integer. Based on the determined average context index, a flag pred_mode_flag is set to indicate that the current block is intra-coded or inter-coded, and arithmetic coding is performed to code the pred_mode_flag of the current block.

[0068] In an embodiment, information on whether a current block (e.g., the current block (610)) is coded in an intra prediction mode, an inter prediction mode, or an inter-intra prediction mode is used to obtain one or more context values ​​for entropy coding the CBF of the current block.

[0069] In one embodiment, three separate contexts (e.g., variables) are used to entropy code the CBF: one context is used when the current block is coded by an intra prediction mode, one context is used when the current block is coded by an inter prediction mode, and one context is used when the current block is coded by an intra-inter prediction mode. The three separate contexts may be applied only for coding the luma CBF, only for coding the chroma CBF, or only for coding both the luma and chroma CBFs.

[0070] In other embodiments, two separate contexts (e.g., variables) are used to entropy code the CBF, one context used when the current block is coded by an intra prediction mode and one context used when the current block is coded by an inter prediction mode or an intra-inter prediction mode, and the two separate contexts may be applied only for coding the luma CBF, only for coding the chroma CBF, or only for coding both the luma and chroma CBFs.

[0071] In yet another embodiment, two separate contexts (e.g., variables) are used to entropy code the CBF, one context used when the current block is coded by an intra-prediction mode or an intra-inter-prediction mode, and one context used when the current block is coded by an inter-prediction mode, and the two separate contexts may be applied only for coding the luma CBF, only for coding the chroma CBF, or only for coding both the luma and chroma CBF.

[0072] 7 is a flow chart illustrating a method (700) for controlling intra-inter prediction for decoding or encoding a video sequence, according to an embodiment. In some implementations, one or more of the processing blocks in FIG. 7 may be performed by the decoder (310). In some implementations, one or more of the processing blocks in FIG. 7 may be performed by another device or group of devices (e.g., the encoder (303)) separate from or including the decoder (310).

[0073] 7, in a first block (710), the method (700) includes determining whether a neighboring block of the current block is coded by an intra-inter prediction mode. Based on determining (710-NO) that the neighboring block is not coded by an intra-inter prediction mode, the method (700) ends.

[0074] Based on a determination that the neighboring block is coded using an intra-inter prediction mode (710-YES), in a second block (720), the method (700) includes a step of performing intra-mode coding of the current block using an intra-prediction mode associated with the intra-inter prediction mode.

[0075] In a third block (730), the method (700) includes setting a prediction mode flag associated with the neighboring block.

[0076] In a fourth block (740), the method (700) includes obtaining a context value based on the set prediction mode flag associated with the neighboring block.

[0077] In a fifth block (750), the method (700) includes using the obtained context value to perform entropy coding of a prediction mode flag associated with the current block, the prediction mode flag indicating that the current block is intra-coded.

[0078] The method (700) further includes a step of performing derivation of an MPM for the current block using an intra prediction mode associated with the intra-inter prediction mode based on the determination (710-YES) that the neighboring block is coded using the intra-inter prediction mode.

[0079] The intra prediction mode associated with the intra-inter prediction mode may be a planar mode, a DC mode, or an intra prediction mode applied in the intra-inter prediction mode.

[0080] Setting a prediction mode flag associated with the neighboring block may include setting a prediction mode flag associated with the neighboring block to indicate that the neighboring block is intra-coded.

[0081] Setting a prediction mode flag associated with the neighboring block may include setting a prediction mode flag associated with the neighboring block to indicate that the neighboring block is inter-coded.

[0082] The method (700) may further include a step of determining whether the neighboring block is coded in an intra prediction mode, an inter prediction mode, or an intra-inter prediction mode; incrementing a context index of a prediction mode flag associated with the current block by 2 based on the determination that the neighboring block is coded in an intra prediction mode, incrementing the context index by 0 based on the determination that the neighboring block is coded in an inter prediction mode, and incrementing the context index by 1 based on the determination that the neighboring block is coded in an intra-inter prediction mode; determining an average context index based on the incremented context index and the number of neighboring blocks of the current block; and setting the prediction mode flag associated with the current block based on the determined average context index.

[0083] The method may further include the steps of determining whether the neighboring block is coded in an intra prediction mode, an inter prediction mode, or an intra-inter prediction mode; incrementing a context index of a prediction mode flag associated with the current block by 1 based on the determination that the neighboring block is coded in an intra prediction mode, incrementing the context index by 0 based on the determination that the neighboring block is coded in an inter prediction mode, and incrementing the context index by 0.5 based on the determination that the neighboring block is coded in an intra-inter prediction mode; determining an average context index based on the incremented context index and the number of neighboring blocks of the current block; and setting the prediction mode flag associated with the current block based on the determined average context index.

[0084] Although Figure 7 illustrates example blocks of the method (700), in some implementations, the method (700) may include additional blocks, fewer blocks, different blocks, or blocks in a different arrangement than those blocks depicted in Figure 7. Additionally or alternatively, two or more of the blocks of the method (700) may be performed in parallel.

[0085] The proposed methods may also be implemented by a processing circuit (e.g., one or more processors, or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0086] FIG. 8 is a simplified block diagram of an apparatus (800) for controlling intra-inter prediction for decoding or encoding a video sequence, according to an embodiment.

[0087] 8, the device (800) includes a first decision code (810), an execution code (820), and a setting code (830). The device (800) may further include an increment code (840) and a second decision code (850).

[0088] A first decision code (810) is arranged to cause the at least one processor to determine whether a neighboring block of a current block is coded by an intra-inter prediction mode.

[0089] Executable code (820) is arranged to cause at least one processor to perform intra-mode encoding of a current block using an intra-prediction mode associated with the intra-inter prediction mode based on determining that a neighboring block is encoded using the intra-inter prediction mode.

[0090] The setting code (830) is arranged to cause at least one processor to operate, based on determining that the neighboring block is coded in an intra-inter prediction mode, as follows: set a prediction mode flag associated with the neighboring block; obtain a context value based on the set prediction mode flag associated with the neighboring block; and perform entropy coding of a prediction mode flag associated with the current block using the obtained context value, where the prediction mode flag indicates that the current block is intra-coded.

[0091] The execution code (820) may further be configured to cause the at least one processor to perform derivation of a most probable mode (MPM) for the current block using an intra prediction mode associated with the intra-inter prediction mode based on determining that the neighboring block is coded using the intra-inter prediction mode.

[0092] The intra prediction mode associated with the intra-inter prediction mode may be a planar mode, a DC mode, or an intra prediction mode applied in the intra-inter prediction mode.

[0093] The setting code (830) may be further arranged to cause the at least one processor to set a prediction mode flag associated with the neighboring block to indicate that the neighboring block is intra-coded.

[0094] The setting code (830) may be further arranged to cause the at least one processor to set a prediction mode flag associated with the neighboring block to indicate that the neighboring block is inter-coded.

[0095] The first decision code (810) may be further arranged to cause the at least one processor to determine whether the neighboring block is coded by an intra prediction mode, an inter prediction mode, or an intra-inter prediction mode. The increment code (840) may be arranged to cause the at least one processor to perform the steps of incrementing a context index of a prediction mode flag associated with the current block based on the determination that the neighboring block is coded by an intra prediction mode by 2, incrementing the context index by 0 based on the determination that the neighboring block is coded by an inter prediction mode, and incrementing the context index by 1 based on the determination that the neighboring block is coded by an intra-inter prediction mode. The second decision code (850) may be further arranged to cause the at least one processor to determine an average context index based on the incremented context index and the number of neighboring blocks of the current block. The setting code (830) may be further arranged to cause the at least one processor to set a prediction mode flag associated with the current block based on the determined average context index.

[0096] The first decision code (810) may be further arranged to cause the at least one processor to determine whether the neighboring block is coded in an intra prediction mode, an inter prediction mode, or an intra-inter prediction mode. The increment code (840) may be arranged to cause the at least one processor to perform the steps of incrementing a context index of a prediction mode flag associated with the current block based on the determination that the neighboring block is coded in an intra prediction mode, incrementing the context index by 0 based on the determination that the neighboring block is coded in an inter prediction mode, and incrementing the context index by 0.5 based on the determination that the neighboring block is coded in an intra-inter prediction mode. The second decision code (850) may be arranged to cause the at least one processor to determine an average context index based on the incremented context index and the number of neighboring blocks of the current block. The setting code (830) may be further arranged to cause the at least one processor to set a prediction mode flag associated with the current block based on the determined average context index.

[0097] The above techniques may be implemented as computer software using computer readable instructions, which may be physically stored on one or more computer readable mediums.

[0098] FIG. 9 is a diagram of a computer system (900) suitable for implementing an embodiment.

[0099] Computer software can be encoded in any suitable machine code or computer language, which can be subjected to mechanisms such as assembling, compiling, linking, etc. to produce code containing instructions that can be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), etc., or interpreted, executed by microcode, etc.

[0100] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0101] The components for computer system (900) illustrated in Figure 9 are exemplary in nature and are not intended to imply any limitations on the scope or functionality of the computer software used to implement each embodiment, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system (900).

[0102] The computer system (900) may include a number of human interface input devices. Such human interface input devices may be responsive to input by one or more human users, such as tactile input (e.g., keystrokes, slides, data glove movements), audio input (e.g., voice, hand clapping), visual input (e.g., posture), olfactory input (not shown), etc. Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a static image capture device), and video (e.g., two-dimensional video, three-dimensional video including stereo video).

[0103] The human interface input devices may include one or more of a keyboard (901), a mouse (902), a touchpad (903), a touch panel (910), a data glove (904), a joystick (905), a microphone (906), a scanner (907), and an imaging device (908) (only one of each is shown).

[0104] The computer system (900) may further include several human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., touch screens (910), haptic feedback devices via data gloves (904) or joysticks (905), but also haptic feedback devices that are not used as input devices), audio output devices (e.g., speakers (909), headphones (not shown)), visual output devices (e.g., screens (910) including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each of which may or may not have touch screen input and haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three or more dimensional output, such as by means of stereoscopic graphics output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0105] The computer system (900) may further include human accessible storage devices and associated media, including optical media including CD / DVD ROM / RW (920) having media such as CDs / DVDs (921), thumb drives (922), removable hard drives or solid state drives (923), conventional magnetic media such as magnetic tape and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), and the like.

[0106] Additionally, those skilled in the art should understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0107] The computer system (900) may further include an interface to one or more communication networks. The networks may be, for example, wireless, wired, optical, etc. The networks may further be local area, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks (including Global Mobile Communications System (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), long term evolution (LTE), etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial television), vehicular and industrial (including CANBus), etc. Some networks typically require an external network interface adapter that connects to a specific general-purpose data port or peripheral bus (949) (e.g., a Universal Serial Bus (USB) port of the computer system (900)), while other networks are typically integrated into the core of the computer system (900) by connecting to a system bus described below (e.g., integrated into an Ethernet interface in a PC computer system, or a cellular network interface in a smartphone computer system). Using any of these networks, the computer system (900) can communicate with other entities. Such communication may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., a CANbus to a CANbus device), or two-way (e.g., over a local or wide area digital network to reach another computer system). Specific protocols and protocol stacks may be used for each of these networks and network interfaces as described above.

[0108] The above human interface devices, human-accessible storage devices, and network interfaces may be connected to a core (940) of the computer system (900).

[0109] The core (940) includes one or more central processing units (CPUs) (941), graphics processing units (GPUs) (942), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (943), hardware accelerators (944) for some tasks, etc. These devices are connected via a system bus (948), along with read only memory (ROM) (945), random access memory (RAM) (946), and internal mass storage (e.g., internal non-user accessible hard disk drives, solid state drives (SSDs), etc.) (947). In some computer systems, access to the system bus (948) may be in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected to the core's system bus (948) directly or via a peripheral bus (949). Peripheral bus architectures include peripheral component interconnect (PCI), USB, etc.

[0110] The CPU (941), GPU (942), FPGA (943) and accelerator (944) may execute a number of instructions, which, when combined, may constitute the computer code described above. The computer code may be stored in a ROM (945) or a RAM (946). Temporary data may also be stored in the RAM (946), while permanent data may be stored, for example, in an internal mass storage device (947). A cache memory may provide fast storage and retrieval from any of the storage devices, and the cache memory may be closely associated with one or more of the CPU (941), GPU (942), mass storage device (947), ROM (945), RAM (946), etc.

[0111] The computer-readable media may have computer code thereon for performing various computer implemented operations, and the media and computer code may be specially designed and constructed for the purposes of the embodiments, or they may be of the type well known and available to those skilled in the art of computer software.

[0112] By way of example and not limitation, the computer system (900) having the architecture, and in particular the core (940), may provide functionality by having a processor (including a CPU, GPU, FPGA, accelerator, etc.) execute software implemented in one or more tangible computer-readable media. Such computer-readable media may be the media related to mass storage accessible by a user as introduced above, as well as some storage of the core (940) that is non-transitory in nature, such as the core internal mass storage (947) or ROM (945). Software for implementing various embodiments is stored in such devices and executed by the core (940). Depending on the particular needs, the computer-readable media may include one or more memory devices or chips. The software causes the core (940), and in particular the processor (including a CPU, GPU, FPGA, etc.) therein to perform a particular process or a particular part of a particular process, including defining data structures stored in RAM (946) and modifying these data structures based on the process defined by the software, as described herein. Additionally or alternatively, the computer system may provide functionality through logic implemented in hardwired or otherwise circuitry (e.g., accelerator (944)) that may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. Where appropriate, references to software may include logic, and conversely, references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) on which software is stored for execution, circuitry embodying logic for execution, or both. Embodiments include any appropriate combination of hardware and software.

[0113] Although the present disclosure has previously described several exemplary embodiments, there are modifications, substitutions, and various substitute equivalents that fall within the scope of the present disclosure. It should thus be understood that those skilled in the art can devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. 1. A method for encoding a video bitstream executed by at least one processor, comprising: setting a first prediction mode flag pred_mode_flag associated with an upper or left neighboring block of a current block to indicate whether the neighboring block is in inter or intra prediction mode; encoding the first prediction mode flag pred_mode_flag into a video bitstream; encoding a decision flag into the video bitstream, the decision flag being used to finally decide whether the neighboring block above or to the left of the current block is coded in an inter prediction mode or an intra-inter prediction mode; deriving a context value for a second prediction mode flag pred_mode_flag associated with the current block based on the first prediction mode flag pred_mode_flag associated with the neighboring block; encoding the second prediction mode flag pred_mode_flag associated with the current block into the video bitstream using the derived context value; A method comprising:

2. performing intra-predictive coding of the current block using an intra-prediction mode associated with the intra-inter prediction mode based on a final determination that the neighboring block is coded using the intra-inter prediction mode; The method of claim 1 further comprising:

3. The method of claim 1 or 2, further comprising: based on a final determination that the neighboring block is coded by the intra-inter prediction mode, performing a derivation of a most probable mode (MPM) of the current block using an intra prediction mode associated with the intra-inter prediction mode.

4. The method according to claim 2 or 3, wherein an intra prediction mode associated with the intra-inter prediction mode is a planar mode.

5. The method according to claim 2 or 3, wherein an intra prediction mode associated with the intra-inter prediction mode is a direct current DC mode.

6. The method according to claim 2 or 3, wherein an intra prediction mode associated with the intra-inter prediction mode is an intra prediction mode applied in the intra-inter prediction mode.

7. The step of deriving the context value comprises: incrementing the context value by 2 based on determining that the neighboring block is coded using an intra prediction mode; incrementing the context value by 0 based on determining that the neighboring block is coded using an inter prediction mode; incrementing the context value by one based on determining that the neighboring block is coded using an intra-inter prediction mode; The method further comprises: determining an average context index based on the incremented context value and a number of neighboring blocks of the current block; setting the second prediction mode flag based on the determined average context index; The method of claim 1 , further comprising:

8. The step of deriving the context value related to the second prediction mode flag comprises: incrementing the context value by one based on determining that the neighboring block is coded using an intra prediction mode; incrementing the context value by 0 based on determining that the neighboring block is coded using an inter prediction mode; incrementing the context value by 0.5 based on determining that the neighboring block is coded using an intra-inter prediction mode; The method further comprises: determining an average context index based on the incremented context value and a number of neighboring blocks of the current block; setting the second prediction mode flag based on the determined average context index; The method of claim 1 , further comprising:

9. 9. A method according to claim 1, wherein when the decision flag is true, the prediction mode actually used for the neighboring block is the intra-inter prediction mode, which is different from the inter prediction mode indicated by the first prediction mode flag pred_mode_flag.

10. at least one memory storing a program; at least one processor coupled to said memory; Including, The program is configured to cause the at least one processor to execute a method according to any one of claims 1 to 9. Video bitstream coding device.

11. A program for causing a computer to execute the method according to any one of claims 1 to 9.

Images