Position-dependent spatial transformation for video coding

Abstract

This is a mechanism for position-dependent spatial variable transforms (SVTs) for video coding. [Solution] The prediction block and the corresponding transformation residual block are received in the decoder. The type of spatial transformation (SVT) used to generate the transformation residual block is determined. The position of the SVT relative to the transformation residual block is also determined. To reconstruct the reconstructed residual block, the inverse of the SVT is applied to the transformation residual block. Then, to reconstruct the image block, the reconstructed residual block is combined with the prediction block.

Description

Background Art

[0001] [Cross - Reference to Related Applications] This application claims the priority of U.S. Provisional Patent Application No. 62 / 634,613, titled "Position Dependent Spatial Varying Transform for Video Coding", filed on February 23, 2018 by Yin Zhao et al., and incorporates by reference the entire teachings and disclosures thereof.

[0002] [Description of Research or Development Sponsored by the Federal Government] Not applicable.

[0003] [Reference to Microfiche Appendix] Not applicable.

[0004] [Background Art] Video coding is a process of compressing video images into a smaller format. Video coding enables the encoded video to occupy less space when stored on a medium. Further, video coding supports streaming media. Specifically, content providers desire to provide media to end - users at even higher resolutions. Additionally, content providers desire to provide media on demand without making users wait for long periods while such media is transmitted to end - user devices such as televisions, computers, tablets, phones, etc. Advancements in video coding compression support the reduction of video file sizes and thus support both of the above - mentioned goals when applied with corresponding content delivery systems.

Summary of the Invention

[0005] The first aspect relates to a method implemented in a computing device. The method includes the steps of: analyzing a bitstream to obtain prediction blocks and corresponding transformation residual blocks by a processor in the computing device; determining the type of spatial transformation (SVT) used to generate the transformation residual blocks by the processor; determining the location of the SVT relative to the transformation residual blocks by the processor; determining the inverse of the SVT based on the location of the SVT by the processor; applying the inverse of the SVT to the transformation residual blocks to generate reconstructed residual blocks by the processor; and combining the reconstructed residual blocks with the prediction blocks by the processor to reconstruct image blocks.

[0006] This method facilitates an increase in the coding efficiency of SVT. In this regard, the transformation block is located at various candidate positions relative to the corresponding residual block. Therefore, the mechanism of disclosure uses different transformations for the transformation block based on the candidate position.

[0007] In the first implementation of the method according to the first embodiment itself, the type of SVT is either the SVT vertical (SVT-V) type or the SVT horizontal (SVT-H) type.

[0008] In the first embodiment itself or in the second embodiment of the method according to any of the above-described implementations of the first embodiment, the SVT-V type includes a height equal to the height of the conversion residual block and a width that is half the width of the conversion residual block, and the SVT-H type includes a height that is half the height of the conversion residual block and a width that is equal to the width of the conversion residual block.

[0009] In the first embodiment itself or in the form of a third implementation of the method in the form of the above implementation of either the first embodiment, svt_type_flag is parsed from the bitstream to determine the type of SVT.

[0010] In the first embodiment itself or in the fourth embodiment of the method according to any of the above-described implementations of the first embodiment, when only one type of SVT is permitted for the residual block, the type of SVT is determined by inference.

[0011] In the first embodiment itself or in the form of a fifth implementation of the method in the form of the above implementation of either the first embodiment, a position index is parsed from the bitstream to determine the position of the SVT.

[0012] In the first embodiment itself or in the form of a sixth implementation of the method in the form of any of the first embodiments, the position index includes a binary code indicating the position from a set of candidate positions determined according to a candidate position step size (CPSS).

[0013] In the first embodiment itself or in the seventh implementation of the method in the form of the above implementation of either the first embodiment, the most likely position of the SVT is allocated the minimum number of bits in the binary code indicating the position index.

[0014] In the first embodiment itself or in the form of an eighth implementation of the method in the form of any of the above-described implementations of the first embodiment, the SVT position is inferred by the processor when a single candidate position is available for SVT conversion.

[0015] In the first embodiment itself or in the form of a ninth implementation of the method in the form of any of the first embodiments, the position of the SVT is estimated by the processor when the residual block is generated by template matching in interprediction mode.

[0016] In the first embodiment itself or in the form of a tenth implementation of the method in the form of any of the first embodiments, the inverse discrete sinusoidal transform (DST) is used for an SVT perpendicular (SVT-V) type transform located at the left boundary of the residual block.

[0017] In the first embodiment itself or in the form of an eleventh implementation of the method according to any of the above implementations of the first embodiment, the inverse DST is used for SVT horizontal (SVT-H) type conversion located at the upper boundary of the residual block.

[0018] In the first embodiment itself or in the form of a twelfth implementation of the method in the form of any of the above implementations of the first embodiment, the inverse discrete cosine transform (DCT) is used for an SVT-V type transform located at the right boundary of the residual block.

[0019] In the first embodiment itself or in the form of a thirteenth implementation of the method in the form of the above implementation of either the first embodiment, the inverse DCT is used for SVT-H type conversion located at the lower boundary of the residual block.

[0020] In the first embodiment itself or in the form of a 14th implementation of the method in the form of any of the above implementations of the first embodiment, when the right-hand adjacent of a coding unit associated with a reconstructed residual block is reconstructed and the left-hand adjacent of a coding unit is not reconstructed, the samples in the reconstructed residual block are horizontally inverted before the reconstructed residual block is combined with the prediction block.

[0021] A second aspect relates to a method implemented in a computing device. The method includes the steps of: receiving a video signal from a video capture device, the video signal comprising an image block; a processor of the computing device generating a prediction block and a residual block to represent the image block; the processor selecting a transformation algorithm for a spatial transformation (SVT) based on the location of the SVT for the residual block; the processor transforming the residual block into a transformed residual block using the selected SVT; the processor encoding the type of the SVT into a bitstream; the processor encoding the location of the SVT into a bitstream; and the processor encoding the prediction block and the transformed residual block into a bitstream for transmission to a decoder.

[0022] This method facilitates an increase in the coding efficiency of SVT. In this regard, the transformation block is located at various candidate positions relative to the corresponding residual block. Therefore, the mechanism of disclosure uses different transformations for the transformation block based on the candidate position.

[0023] In the first implementation of the method according to the first embodiment itself, the type of SVT is either the SVT vertical (SVT-V) type or the SVT horizontal (SVT-H) type.

[0024] In the second embodiment itself or in the second embodiment of the method according to the above-described implementation method of either the second embodiment, the SVT-V type includes a height equal to the height of the residual block and a width that is half the width of the residual block.

[0025] In the second embodiment itself or in the third embodiment of the method according to any of the above-described implementations of the second embodiment, the SVT-H type includes a height that is half the height of the residual block and a width equal to the width of the residual block.

[0026] In the fourth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the position of the SVT is encoded in a position index.

[0027] In the fifth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the position index includes a binary code indicating a position from a set of candidate positions determined according to a candidate position step size (CPSS).

[0028] In the sixth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the most likely position of the SVT is assigned as the minimum number of bits in the binary code indicating the position index.

[0029] In the seventh implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the discrete sine transform (DST) algorithm is used by the processor for the SVT vertical (SVT-V) type transform located at the left boundary of the residual block.

[0030] In the eighth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the DST algorithm is selected by the processor for the SVT horizontal (SVT-H) type transform located at the upper boundary of the residual block.

[0031] In the ninth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the discrete cosine transform (DCT) algorithm is selected by the processor for the SVT-V type transform located at the right boundary of the residual block.

[0032] In the tenth implementation format of the method in the form of the second aspect itself or any of the above implementation formats of the second aspect, the DCT algorithm is selected by the processor for the SVT-H type transform located at the lower boundary of the residual block.

[0033] In the 11th implementation of the method according to the second embodiment itself or any of the above implementations of the second embodiment, the method further includes the step of the processor horizontally inverting the samples in the residual block before converting the residual block to a transformed residual block, when the right-hand adjacent of the coding unit relating to the residual block is encoded and the left-hand adjacent of the coding unit is not encoded.

[0034] A third aspect relates to a coding apparatus comprising: a receiver configured to receive a picture to be encoded or a bitstream to be decoded; a transmitter coupled to the receiver, configured to transmit a bitstream to a decoder or a decoded image to a display; a memory coupled to at least one of the receiver or transmitter, configured to store instructions; and a processor coupled to the memory, configured to execute instructions stored in the memory and to perform any of the above aspects or implementations.

[0035] The coding device facilitates an increase in the coding efficiency of the SVT. In this regard, the transformation block is located at various candidate positions relative to the corresponding residual block. Therefore, the mechanism of disclosure uses different transformations for the transformation block based on the candidate position.

[0036] In the form of the first implementation of the apparatus according to the third embodiment itself, the apparatus further includes a display configured to display an image.

[0037] A fourth aspect relates to a system including an encoder and a decoder that communicates with the encoder. The encoder or decoder includes a coding device in any of the above aspects or implementations.

[0038] The system facilitates increased coding efficiency for SVTs. In this regard, the transformation block is located at various candidate positions relative to the corresponding residual block. Therefore, the mechanism of disclosure uses different transformations for the transformation block based on the candidate position.

[0039] A fifth aspect relates to coding means including a receiving means configured to receive a picture to encode or a bitstream to decode; a transmitting means coupled to the receiving means, configured to transmit a bitstream to a decoder or a decoded image to a display means; a storage means coupled to at least one of the receiving means or the transmitting means, configured to store instructions; and a processing means coupled to the storage means, configured to execute instructions stored in the storage means and to perform a method in any of the above aspects or implementations.

[0040] The coding method facilitates an increase in the coding efficiency of SVT. In this regard, the transformation block is located at various candidate positions relative to the corresponding residual block. Therefore, the disclosure mechanism uses different transformations for the transformation block based on the candidate position.

[0041] For the purpose of clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of this disclosure.

[0042] These and other features will be better understood from the following detailed description made in relation to the attached drawings and claims. [Brief explanation of the drawing]

[0043] For a more complete understanding of this disclosure, references are made herein to the following brief description relating to the attached drawings and detailed description, where similar reference numerals represent similar parts. [Figure 1]A block diagram illustrating an exemplary coding system that can utilize spatial transformation (SVT) transformations. [Figure 2] A block diagram illustrating an example video encoder that can utilize spatial SVT transformation. [Figure 3] This is a block diagram showing an example of a video decoder that can utilize spatial SVT transformation. [Figure 4] This is a schematic diagram of the intra predictive mode used in video coding. [Figure 5] This shows an example of intra-prediction in video coding. [Figure 6] This is a schematic diagram of an exemplary video encoding mechanism. [Figure 7] An exemplary SVT conversion is shown. [Figure 8] An exemplary SVT conversion is shown. [Figure 9] This shows exemplary SVT transformation candidate locations for the residual block. [Figure 10] This shows an exemplary SVT transformation position for the residual block. [Figure 11] An example of horizontal inversion of the residual sample is shown. [Figure 12] This is a flowchart illustrating an exemplary method of video decoding using position-dependent SVT. [Figure 13] This is a flowchart illustrating an example method for video coding. [Figure 14] This is a flowchart illustrating an example method for video coding. [Figure 15] This is a schematic diagram of an exemplary computing device for video coding. [Figure 16] This is a schematic diagram of an embodiment of the coding means. [Modes for carrying out the invention]

[0044] Firstly, exemplary implementations of one or more embodiments are provided below, but it should be understood that the systems and / or methods of the disclosure may be carried out using any number of techniques, whether currently known or existing. The disclosure should not be limited in any way to the exemplary implementations, drawings and techniques shown below, including the exemplary designs and implementations described herein, and may be modified within the scope of the appended claims, along with the entire scope of their equivalents.

[0045] The standard now known as High Efficiency Video Coding (HEVC) is an advanced video coding system developed under the Joint Collaborative Team on Video Coding (JCT-VC) group of video coding experts from the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) research group. Details regarding the HEVC standard can be found in ITU-T Rec. H.265 and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 23008-2 (2013), High efficiency video coding, final draft approval Jan. 2013 (officially published by ITU-T in June 2013 and by ISO / IEC in November 2013), which are referenced here. An overview of HEVC can be found in GJ Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) Standard," IEEE Trans. Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649–1668, December 2012, and this is referenced here.

[0046] Figure 1 is a block diagram illustrating an exemplary coding system 10 that may utilize video coding techniques such as coding using an SVT mechanism. As shown in Figure 1, the coding system 10 includes a source device 12 that provides coded video data to be decoded by a destination device 14 at a later time. In particular, the source device 12 may provide the video data to the destination device 14 via a computer-readable medium 16. The source device 12 and destination device 14 may include any of a wide range of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephones such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication.

[0047] The destination device 14 may receive encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device that can move the encoded video data from the source device 12 to the destination device 14. For example, the computer-readable medium 16 may include a communication medium that enables the source device 12 to directly transmit the encoded video data to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that may be useful to facilitate communication from the source device 12 to the destination device 14.

[0048] In some examples, encoded data may be output to a storage device via an output interface 22. Similarly, encoded data may be accessed from the storage device via an input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, digital video disk (DVD), Compact Disc Read-Only Memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or other intermediate storage device capable of storing encoded video generated by the source device 12. The destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and sending that encoded video data to the destination device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. The destination device 14 may access the encoded video data through any standard data connection, including an internet connection. This may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of the encoded video data from the storage device may be via streaming transmission, download transmission, or a combination thereof.

[0049] The technology of this disclosure is not necessarily limited to wireless applications or settings. The technology may be applied to video coding that supports any of a variety of multimedia applications, such as wireless television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission such as dynamic adaptive streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the coding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video phone.

[0050] In the example shown in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, the video encoder 20 of the source device 12 and / or the video decoder 30 of the destination device 14 may be configured to apply techniques for video coding. In other examples, the source and destination devices may include other components or arrangements. For example, the source device 12 may receive video data from an external video source such as an external camera. Similarly, the destination device 14 may interface with an external display device rather than including an integrated display device.

[0051] The coding system 10 illustrated in Figure 1 is merely an example. The technique for video coding may be performed by any digital video coding and / or decoding device. While the technique of this disclosure is generally performed by a video coding device, the technique may also be performed by a video encoder / decoder, typically called a “codec”. Furthermore, the technique of this disclosure may also be performed by a video processor. The video encoder and / or decoder may be a graphics processing unit (GPU) or a similar device.

[0052] The source device 12 and destination device 14 are merely examples of coding devices such that the source device 12 generates coded video data for transmission to the destination device 14. In some examples, the source device 12 and destination device 14 may operate in a substantially symmetrical manner such that each of the source and destination devices 12 and 14 includes video coding and decoding components. Thus, the coding system 10 may support one-way or two-way video transmission between the video devices 12 and 14 for, for example, video streaming, video playback, video broadcasting, or video phone calls.

[0053] The video source 18 of the source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video supply interface for receiving video from a video content provider. As a further alternative, the video source 18 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video.

[0054] In some cases, when the video source 18 is a video camera, the source device 12 and destination device 14 may form a so-called camera phone or video phone. However, as stated above, the techniques described in this disclosure may be applicable to video coding in general and to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video information may then be output onto a computer-readable medium 16 via the output interface 22.

[0055] The computer-readable medium 16 may include a temporary medium such as a wireless broadcast or wired network transmission, or a storage medium (i.e., a non-temporary storage medium) such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, or other computer-readable medium. In some examples, a network server (not shown) may receive encoded video data from a source device 12 and provide the encoded video data to a destination device 14, for example, via network transmission. Similarly, a computing device in a media manufacturing facility, such as a disc stamping facility, may receive encoded video data from a source device 12 and manufacture a disc containing the encoded video data. Thus, the computer-readable medium 16 may be understood to include one or more computer-readable media in various forms in various examples.

[0056] The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information on the computer-readable medium 16 may include syntax information defined by the video encoder 20, which is also used by the video decoder 30 and includes syntax elements that describe the characteristics and / or processing of blocks and other coded units, such as a group of picture (GOP). The display device 32 displays the decoded video data to the user and may include any of a variety of display devices such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.

[0057] The video encoder 20 and video decoder 30 may operate in accordance with a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). Alternatively, the video encoder 20 and video decoder 30 may operate in accordance with other proprietary or industry standards, such as the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard, H.265 / HEVC, or extensions of such standards, as alternatively, Motion Picture Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC). However, the techniques of this disclosure are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although not shown in Figure 1, in some embodiments, the video encoder 20 and video decoder 30 may be integrated with the audio encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols such as the user datagram protocol (UDP).

[0058] The video encoder 20 and video decoder 30 may each be implemented as one or more suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or a combination thereof. When the technology is partially implemented in software, the device may store instructions for the software on a suitable non-temporary computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within each device. A device including the video encoder 20 and / or video decoder 30 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular phone.

[0059] Figure 2 is a block diagram showing an example of a video encoder 20 that can implement video coding technology. The video encoder 20 may perform intra and intercoding of video blocks within a video slice. Intra coding relies on spatial prediction to reduce or eliminate spatial redundancy in video within a given video frame or picture. Intercoding relies on temporal prediction to reduce or eliminate temporal redundancy in video within adjacent frames or pictures of a video sequence. The intra mode (I mode) may represent any of several spatial-based coding modes. Inter modes, such as uni-prediction (also known as uni-prediction) (P mode) or bi-prediction (also known as bi-prediction) (B mode), may represent any of several temporal-based coding modes.

[0060] As shown in Figure 2, the video encoder 20 receives the current video block in the video frame to be encoded. In the example in Figure 2, the video encoder 20 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a transformation unit 52, a quantization unit 54, and an entropy coding unit 56. Next, the mode selection unit 40 includes a motion compensation unit 44, a motion estimation unit 42, an intra-prediction unit (also known as intra-prediction) unit 46, and a segmentation unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transformation unit 60, and an adder 62. A deblocking filter (not shown in Figure 2) may also be included to filter block boundaries to remove blocky artifacts from the reconstructed video. If desired, the deblocking filter typically filters the output of the adder 62. Further filters (in-loop or post-loop) may also be used in addition to the deblocking filter. Such filters are not shown for brevity, but if desired, the output of adder 50 may be filtered (as an in-loop filter).

[0061] During the encoding process, the video encoder 20 receives video frames or slices to be coded. The frames or slices may be divided into multiple video blocks. The motion estimation unit 42 and the motion compensation unit 44 perform inter-predictive coding of the received video blocks for one or more blocks within one or more reference frames to provide temporal predictions. Alternatively, the intra-predictive unit 46 may perform intra-coding of the received video blocks for one or more adjacent blocks within the same frame or slice as the block to be coded to provide spatial predictions. The video encoder 20 may execute multiple coding paths, for example, to select an appropriate coding mode for each block of video data.

[0062] Furthermore, the partitioning unit 48 may partition blocks of video data into sub-blocks based on an evaluation of a previous partitioning method in a previous coding path. For example, the partitioning unit 48 may first partition a frame or slice into the largest coding unit (LCU), and then, based on rate distortion analysis (e.g., rate distortion optimization), partition each LCU into a sub-coding unit (sub-CU). The mode selection unit 40 may further generate a quadtree data structure that shows the partitioning of LCUs into sub-CUs. The leaf nodes CU of the quadtree may include one or more prediction units (PU) and one or more transform units (TU).

[0063] This disclosure uses the term “block” to refer to any CU, PU, ​​or TU in the context of HEVC, or similar data structures in the context of other standards (e.g., macroblocks and their subblocks in H.264 / AVC). A CU includes a coding node, a PU, and a TU associated with the coding node. The size of a CU corresponds to the size of the coding node and is square in shape. The size of a CU may range from 8x8 pixels to the size of a tree block having up to 64x64 pixels or more. Each CU may contain one or more PUs and one or more TUs. Syntax data associated with a CU may, for example, describe the division of the CU into one or more PUs. The division mode may differ depending on whether the CU is skipped or direct mode coded, intra-predictive mode coded, or inter-predictive (also known as inter-predictive) mode coded. A PU may be divided into non-square shapes. Syntax data associated with a CU may also describe the division of the CU into one or more TUs according to a quadtree, for example. A TU can be square or non-square (e.g., rectangular) in shape.

[0064] The mode selection unit 40 may, for example, select one of the coding modes, i.e., intra or intercoded, based on the error result, and provide the resulting intra or intercoded block to the adder 50 to generate residual block data and to the adder 62 to reconstruct the coded block for use as a reference frame. The mode selection unit 40 also provides syntax elements such as motion vectors, intra-mode indicators, segmentation information and other such syntax information to the entropy coding unit 56.

[0065] The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is a process that generates motion vectors that estimate the motion of a video block. The motion vectors may, for example, represent the displacement of the PU of a video block in the current video frame or picture relative to a predicted block in a reference frame (or other coded unit) relative to the current block (or other coded unit) coded in the current frame. The predicted block is a block that is found to closely match the block to be coded with respect to pixel differences that may be determined by a sum of absolute difference (SAD), sum of square difference (SSD), or other difference metric. In some examples, the video encoder 20 may calculate values ​​for sub-integer pixel positions of the reference picture stored in the reference frame memory 64. For example, the video encoder 20 may interpolate values ​​for 1 / 4 pixel positions, 1 / 8 pixel positions, or other fractional pixel positions of the reference picture. Therefore, the motion estimation unit 42 may perform motion search for all pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy.

[0066] The motion estimation unit 42 calculates a motion vector for the PU of a video block in an inter-coded slice by comparing the PU's position with the predicted block's position in a reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in the reference frame memory 64. The motion estimation unit 42 transmits the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0067] Motion compensation performed by the motion compensation unit 44 may include retrieving or generating a predicted block based on a motion vector determined by the motion estimation unit 42. Similarly, in some examples, the motion estimation unit 42 and the motion compensation unit 44 may be functionally integrated. Upon receiving a motion vector for the PU of the current video block, the motion compensation unit 44 may locate the predicted block pointed to by the motion vector in one of the reference picture lists. The adder 50 forms a residual video block and a pixel difference value by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the coded current video block, as described below. Generally, the motion estimation unit 42 performs motion estimation for the lumens component, and the motion compensation unit 44 uses a motion vector calculated based on the lumens component for both the chromens and lumens components. The mode selection unit 40 may also generate syntax elements related to video blocks and video slices for use by the video decoder 30 when decoding video blocks of video slices.

[0068] The intra-prediction unit 46 may intra-predict the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, as described above. In particular, the intra-prediction unit 46 may determine an intra-prediction mode to be used to encode the current block. In some examples, the intra-prediction unit 46 may encode the current block using various intra-prediction modes, for example, between separate encoding paths, and the intra-prediction unit 46 (or, in some examples, the mode selection unit 40) may select an appropriate intra-prediction mode to use from the tested modes.

[0069] For example, the intra-prediction unit 46 may calculate rate distortion values ​​using rate distortion analysis for various tested intra-prediction modes and select the intra-prediction mode with the best rate distortion characteristics among the tested modes. Rate distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to generate the encoded block, and the bit rate (i.e., number of bits) used to generate the encoded block. The intra-prediction unit 46 may calculate a ratio from the distortion and rate for various encoded blocks to determine which intra-prediction mode exhibits the best rate distortion value for the block.

[0070] Furthermore, the intra-prediction unit 46 may be configured to encode depth blocks of the depth map using a depth modeling mode (DMM). The mode selection unit 40 may, for example, use rate-distortion optimization (RDO) to determine whether an available DMM mode produces better coding results than the intra-prediction mode and other DMM modes. The texture image data corresponding to the depth map may be stored in the reference frame memory 64. The motion estimation unit 42 and the motion compensation unit 44 may also be configured to interpret depth blocks of the depth map.

[0071] After selecting an intra-prediction mode for a block (for example, one of the conventional intra-prediction mode or DMM mode), the intra-prediction unit 46 may provide the entropy coding unit 56 with information indicating the selected intra-prediction mode for the block. The entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. The video encoder 20 may include configuration data in the transmitted bitstream that may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also called codeword mapping tables), definitions of coding contexts for various blocks, and instructions for the most likely intra-prediction mode, intra-prediction mode index table, and modified intra-prediction mode index table to use for each context.

[0072] The video encoder 20 forms a residual video block by subtracting predicted data from the mode selection unit 40 from the original video block being coded. The adder 50 represents one or more components that perform this subtraction operation.

[0073] The transformation processing unit 52 applies a transformation such as a discrete cosine transform (DCT) or a conceptually similar transformation to the residual block to generate a video block containing residual transform coefficient values. The transformation processing unit 52 may also perform other transformations conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transformations may also be used.

[0074] The transformation processing unit 52 applies the transformation to the residual block to generate a block of residual transformation coefficients. The transformation may convert the residual information from the pixel value domain to a transformation domain such as the frequency domain. The transformation processing unit 52 may send the resulting transformation coefficients to the quantization unit 54. The quantization unit 54 quantizes the transformation coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of the matrix containing the quantized transformation coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0075] Following quantization, the entropy coding unit 56 entropy codes the quantized transformation coefficients. For example, the entropy coding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. In the case of context-based entropy coding, the context may be based on adjacent blocks. Following entropy coding by the entropy coding unit 56, the encoded bitstream may be transmitted to another device (e.g., video decoder 30) or archived for later transmission or retrieval.

[0076] The inverse quantization unit 58 and the inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual blocks in the pixel domain for later use as reference blocks. The motion compensation unit 44 may compute the reference block by adding the residual blocks to one of the predicted blocks in the reference frame memory 64. The motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual blocks to compute sub-integer pixel values ​​for use in motion estimation. The adder 62 adds the reconstructed residual blocks to the motion-compensated predicted blocks generated by the motion compensation unit 44 to generate reconstructed video blocks for storage in the reference frame memory 64. The reconstructed video blocks may be used by the motion estimation unit 42 and the motion compensation unit 44 as reference blocks for intercoding blocks in subsequent video frames.

[0077] Figure 3 is a block diagram showing an example of a video decoder 30 that can implement video coding technology. In the example in Figure 3, the video decoder 30 includes an entropy decoding unit 70, a motion compensation unit 72, an intra-prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. In some examples, the video decoder 30 may perform a decoding path that is generally the opposite of the coding path described with respect to the video encoder 20 (Figure 2). The motion compensation unit 72 may generate prediction data based on motion vectors received from the entropy decoding unit 70, while the intra-prediction unit 74 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 70.

[0078] During the decoding process, the video decoder 30 receives an encoded video bitstream from the video encoder 20, representing the video blocks and associated syntax elements of the encoded video slice. The entropy decoding unit 70 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-predictive mode indicators, and other syntax elements. The entropy decoding unit 70 transfers the motion vectors and other syntax elements to the motion compensation unit 72. The video decoder 30 may receive syntax elements at the video slice level and / or video block level.

[0079] When a video slice is coded as an intra-coding (I) slice, the intra-prediction unit 74 may generate prediction data for the video blocks of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an inter-coding (e.g., B, P, or GPB) slice, the motion compensation unit 72 generates prediction blocks for the video blocks of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 70. The prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct reference frame lists, i.e., List 0 and List 1, using default construction techniques based on the reference pictures stored in the reference frame memory 82.

[0080] The motion compensation unit 72 analyzes motion vectors and other syntax elements to determine prediction information about video blocks in the current video slice and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the motion compensation unit 72 uses some of the received syntax elements to determine the prediction mode used to encode the video blocks in the video slice (e.g., intra or interprediction), the interprediction slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more reference picture lists for the slice, motion vectors for each intercoded video block in the slice, the interprediction state for each intercoded video block in the slice, and other information for decoding the video blocks in the current video slice.

[0081] The motion compensation unit 72 may also perform interpolation based on an interpolation filter. The motion compensation unit 72 may use the interpolation filter used by the video encoder 20 during video block encoding to compute interpolation values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 72 may determine the interpolation filter used by the video encoder 20 from the received syntax elements and use the interpolation filter to generate the prediction block.

[0082] The texture image data corresponding to the depth map may be stored in the reference frame memory 82. The motion compensation unit 72 may also be configured to interpret depth blocks of the depth map.

[0083] Disclosed herein are various mechanisms for increasing the coding efficiency of SVT. As described above, the transformation block may be located at various candidate positions relative to the corresponding residual block. The disclosed mechanism uses different transformations on the transformation block based on the candidate position. For example, the inverse discrete sinusoidal transform (DST) can be applied to a candidate position covering the lower right corner of the residual block. Similarly, the inverse DCT can be applied to a candidate position covering the upper left corner of the residual block. This mechanism can be advantageous because DST is generally more efficient than DCT for transforming residual blocks that have more residual information distributed in the lower right corner, while DCT is generally more efficient than DST for transforming residual blocks that have more residual information distributed in the upper left corner. It should also be noted that the lower right corner of the residual block almost always contains statistically more residual information. The disclosed mechanism also supports horizontally inverting the residual samples of the transformation block in some cases. For example, the residual samples may be horizontally inverted / flipped after the inverse transformation block has been applied. This may occur when the adjacent block to the right of the current residual block has already been reconstructed, but the adjacent block to the left of the current residual block has not been reconstructed. This may also occur when the inverse DST is used as part of the corresponding transport block. This technique provides greater flexibility in encoding the information closest to the reconstructed block, which results in a reduction of the corresponding residual information. The disclosure mechanism also supports context coding of candidate location information for the transform block. Prediction information corresponding to the residual block may be used to encode the candidate location information. For example, in some cases the residual block may correspond to a prediction block generated by the template matching mode. Furthermore, the template used by the template matching mode may be selected based on the spatially adjacent reconstructed regions of the residual block. In such cases, the lower right portion of the residual block may contain more residual information than other portions of the residual block.Therefore, candidate positions covering the lower right portion of the residual block are most likely to be selected as the best position for the transformation. Thus, when the residual block is related to template matching-based interpretation, only one candidate position may be made available to the residual block, and / or other context coding techniques may be used to code the positional information for the transformation.

[0084] Figure 4 is a schematic diagram of an exemplary intra-predictive mode 400 used for video coding in the HEVC model. The video compression scheme utilizes data redundancy. For example, most images contain groups of pixels that have the same or similar color and / or light as adjacent pixels. As a specific example, an image of the night sky may contain large areas of black pixels and clusters of white pixels representing stars. The intra-predictive mode 400 utilizes these spatial relationships. Specifically, a frame can be divided into a series of blocks containing samples. Rather than transmitting each block, the light / color of a block can be predicted based on its spatial relationship with a reference sample in an adjacent block. For example, the encoder may indicate that the current block contains the same data as a reference sample in a previously coded block located in the upper-left corner of the current block. The encoder may then encode the predictive mode instead of the block's value. This significantly reduces the size of the encoding. As shown by the intra-predictive mode 400, the upper-left corner corresponds to the predictive mode 18 in HEVC. Therefore, instead of encoding pixel information about a block, the encoder can simply store the predictive mode 18 for the block. As shown in the figure, the intra-prediction mode 400 in HEVC includes 33 angle prediction modes, from prediction mode 2 to prediction mode 34. The intra-prediction mode 400 also includes intra-planar mode 0 and intra-direct current (DC) mode 1, which predict flat regions. Intra-planar mode 0 predicts a block as an amplitude plane with vertical and horizontal slopes derived from adjacent reference samples. Intra-DC mode 1 predicts a block as the average value of adjacent reference samples. The intra-prediction mode 400 may also be used to signal the luma (e.g., light) component of a block. Intra-prediction can also be applied to chroma (e.g., color) values. In HEVC, chroma values ​​are predicted by using the planar mode, angle 26 mode (e.g., vertical), angle 10 mode (e.g., horizontal), intra-DC, and derived modes, the derived mode predicting the correlation between the chroma and luma components encoded by the intra-prediction mode 400.

[0085] The intra-prediction mode 400 for an image block is stored by the encoder as prediction information. While the intra-prediction mode 400 is used for prediction in a single frame, it should be noted that inter-prediction may also be used. Inter-prediction leverages temporal redundancy across multiple frames. For example, a scene in a film may include a relatively static background, such as an immovable desk. Therefore, the desk is represented as substantially the same set of pixels across multiple frames. Inter-prediction uses the block in the first frame to predict the block in the second frame, thus avoiding the need to encode the desk in each frame in this example. Inter-prediction uses a block-matching algorithm to match and compare the block in the current frame with the block in the reference frame. The motion vector can then be encoded to indicate the best-matched block position in the reference frame and the position of the same location in the current / target block. Thus, a series of image frames can be represented as a series of blocks, which can then be represented as prediction blocks containing the prediction mode and / or motion vector.

[0086] Figure 5 shows an example of intra-prediction 500 in video coding using an intra-prediction mode such as intra-prediction mode 400. As shown in the figure, the current block 501 can be predicted by the samples in the adjacent block 510. Encoders may generally encode images from top left to bottom right. However, in some cases, encoders may encode from right to left, as described below. It should be noted that when used here, right indicates the right side of the encoded image, left indicates the left side of the encoded image, up indicates the top side of the encoded image, and down indicates the bottom side of the encoded image.

[0087] It should be noted that the current block 501 does not necessarily have an exact match with the sample from the adjacent block 510. In such cases, the prediction mode is encoded from the nearest matching adjacent block 510. The difference between the predicted value and the actual value is retained to allow the decoder to determine the appropriate value. This is called residual information. Residual information arises in both the intra-prediction 500 and the inter-prediction.

[0088] Figure 6 is a schematic diagram of an exemplary video coding mechanism 600 based on intra-prediction 500 and / or inter-prediction. An image block 601 can be obtained by an encoder from one or more frames. For example, an image may be divided into multiple rectangular image regions. Each region of the image corresponds to a coding tree unit (CTU). A CTU is divided into multiple blocks, such as coding units in HEVC. The block division information is then encoded into a bitstream 611. Thus, an image block 601 is a divided portion of an image and contains pixels representing the lumens and / or chroma components in the corresponding portion of the image. During coding, an image block 601 is coded as a prediction block 603 containing prediction information such as a prediction mode for intra-prediction (e.g., intra-prediction mode) and / or a motion vector for inter-prediction. Then, coding the image block 601 as a prediction block 603 may leave a residual block 605 containing residual information showing the difference between the prediction block 603 and the image block 601.

[0089] It should be noted that image block 601 may be divided into coding units containing one prediction block 603 and one residual block 605. The prediction block 603 may contain all the prediction samples of the coding unit, and the residual block 605 may contain all the residual samples of the coding unit. In this case, the prediction block 603 is the same size as the residual block 605. In another example, image block 601 may be divided into coding units containing two prediction blocks 603 and one residual block 605. In this case, each prediction block 603 contains a portion of the prediction samples of the coding unit, and the residual block 605 contains all of the residual samples of the coding unit. In yet another example, image block 601 is divided into coding units containing two prediction blocks 603 and four residual blocks 605. The division pattern of the residual blocks 605 within the coding units may be signaled within the bitstream 611. Such positional patterns may include a Residual Quad-Tree (RQT) in HEVC. Furthermore, image block 601 may include only the luma component (e.g., light) represented as the Y component of the image sample (or pixel). In other cases, image block 601 may include the Y, U, and V components of the image sample, where U and V represent the chrominance components (e.g., color) in the blue and red (UV) color spaces.

[0090] SVT may be used to further compress the information. Specifically, SVT uses transform block 607 to further compress the residual block 605. Transform block 607 includes a transform such as inverse DCT and / or inverse DST. The difference between the prediction block 603 and the image block 601 is fitted to the transform by using the transform coefficients. By indicating the transform mode (e.g., inverse DCT and / or inverse DST) and the corresponding transform coefficients of transform block 607, the decoder can reconstruct the residual block 605. When exact reproduction is not required, the transform coefficients can be further compressed by rounding certain values ​​to create a better fit through the transform. This process is known as quantization and is performed according to quantization parameters that describe acceptable quantization. Thus, the transform mode, transform coefficients, and quantization parameters of transform block 607 are stored as transform residual information within transform residual block 609, which may also, in some cases, simply be called the residual block.

[0091] Next, the prediction information from prediction block 603 and the transformation residual information from transformation residual block 609 can be encoded into bitstream 611. Bitstream 611 can be stored and / or sent to a decoder. The decoder can then reverse the process to recover image block 601. Specifically, the decoder can use the transformation residual information to determine transformation block 607. Transform block 607 can then be used together with transformation residual block 609 to determine residual block 605. Residual block 605 and prediction block 603 can then be used to reconstruct image block 601. Image block 601 can then be reconstructed to locate such frames and locate them relative to other decoded image blocks 601 to recover the encoded video.

[0092] SVT is described in more detail here. To perform SVT, the transform block 607 is selected to be smaller than the residual block 605. The transform block 607 is used to transform the corresponding portion of the residual block 605, leaving the rest of the residual block 605 without further coding / compression. This is because the residual information is not generally evenly distributed across the residual block 605. SVT uses a smaller transform block 607 with an adaptive position to capture most of the residual information within the residual block 605 without requiring the entire residual block 605 to be transformed. This technique can achieve better coding efficiency than transforming all the residual information within the residual block 605. Since the transform block 607 is smaller than the residual block 605, SVT uses a mechanism to signal the position of the transformation relative to the residual block 605. For example, when SVT is applied to a residual block 605 of size w × h (e.g., width × height), the size and position information of the transform block 607 may be encoded in the bitstream 611. This allows the decoder to reconstruct the conversion block 607 and configure it to be in the correct position relative to the conversion residual block 609 for the reconstruction of the residual block 605.

[0093] It should be noted that some prediction blocks 603 can be encoded without generating residual blocks 605. However, since this does not involve the use of SVT, it will not be further explained. As described above, SVT may be used for inter-prediction blocks or intra-prediction blocks. Furthermore, SVT may be used for residual blocks 605 generated by a specified inter-prediction mechanism (e.g., transformation model-based motion compensation), but not for residual blocks 605 generated by other specified inter-prediction mechanisms (e.g., affine model-based motion compensation).

[0094] Figure 7 shows an example of SVT 700, including a transformation block 707 and a residual block 705. The transformation block 707 and residual block 705 in Figure 7 are similar to the transformation block 607 and residual block 605 in Figure 6, respectively. For ease of reference, the SVT examples 700 are referred to as SVT-I, SVT-II, and SVT-III.

[0095] SVT-I is written as w_t=w / 2, h_t=h / 2, where w_t and h_t represent the width and height of the transformation block 707, respectively, and w and h represent the width and height of the residual block 705, respectively. For example, the width and height of the transformation block 707 are both half the width and height of the residual block 705. SVT-II is written as w_t=w / 4, h_t=h, with the variables as described above. For example, the width of the transformation block 707 is one-quarter of the width of the residual block 705, and the height of the transformation block 707 is equal to the height of the residual block 705. SVT-III is written as w_t=w, h_t=h / 4, with the variables as described above. For example, the width of the transformation block 707 is equal to the width of the residual block 705, and the height of the transformation block 707 is one-quarter of the height of the residual block 705. Type information indicating the type of SVT (e.g., SVT-I, SVT-II, or SVT-III) is coded into the bitstream to support reconstruction by the decoder.

[0096] As can be seen from Figure 7, each transformation block 707 can be located in various places relative to the residual block 705. The position of a transformation block 707 is represented by a position offset (x,y) relative to the upper left corner of the residual block 705, where x is the horizontal distance between the upper left corner of the transformation block 707 and the upper left corner of the residual block 705 in pixel units, and y is the vertical distance between the upper left corner of the transformation block 707 and the upper left corner of the residual block 705 in pixel units. Each potential position of a transformation block 707 within the residual block 705 is called a candidate position. For the residual block 705, the number of candidate positions is (w-w_t+1)×(h-h_t+1) for the type of SVT. More specifically, for a 16×16 residual block 705, there are 81 candidate positions when SVT-I is used. When SVT-II or SVT-III is used, there are 13 candidate positions. Once determined, the x and y values ​​of the position offset are coded into the bitstream along with the type of SVT block used. To reduce complexity for SVT-I, a subset of 32 positions can be selected from 81 possible candidate positions. This subset then acts as acceptable candidate positions for SVT-I.

[0097] One drawback of the SVT scheme using one of the SVT Examples 700 is that encoding SVT position information as residual information incurs considerable signaling overhead. Furthermore, encoder complexity can increase considerably as the number of positions tested by compression quality processes such as Rate-Distortion Optimization (RDO) increases. Since the number of candidate positions increases with the size of the residual block 705, the signaling overhead can be even greater for larger residual blocks 705, such as 32x32 or 64x128. Another drawback of using one of the SVT Examples 700 is that the size of the transform block 707 is one-quarter the size of the residual block 705. A transform block 707 of such size may often not be large enough to cover the main residual information within the residual block 705.

[0098] Figure 8 shows a further SVT example 800, including a conversion block 807 and a residual block 805. The conversion block 807 and residual block 805 in Figure 8 are similar to the conversion blocks 607 and 707 and residual blocks 605 and 705 in Figures 6-7, respectively. For ease of reference, SVT example 800 is referred to as SVT Vertical (SVT-V) and SVT Horizontal (SVT-H). SVT example 800 is similar to SVT example 700, but is designed to support reduced signaling overhead for the encoder and less complex processing requirements.

[0099] SVT-V is described as w_t=w / 2 and h_t=h, with the variables as described above. The width of the transformation block 807 is half the width of the residual block 805, and the height of the transformation block 807 is equal to the height of the residual block 805. SVT-H is described as w_t=w and h_t=h / 2, with the variables as described above. For example, the width of the transformation block 807 is equal to the width of the residual block 805, and the height of the transformation block 807 is half the height of the residual block 805. SVT-V is similar to SVT-II, and SVT-H is similar to SVT-III. Compared to SVT-II and SVT-III, the transformation block 807 in SVT-V and SVT-H is enlarged to half the size of the residual block 805 so that the transformation block 807 covers more residual information within the residual block 805.

[0100] Similar to SVT Example 700, SVT Example 800 may include several candidate positions, where the candidate positions are possible and acceptable positions of the transform block (e.g., transform block 807) relative to the residual block (e.g., residual block 805). The candidate positions are determined according to the Candidate Position Step Size (CPSS). The candidate positions may be separated at equal intervals specified by the CPSS. In such cases, the number of candidate positions is reduced to five or fewer. A reduced number of candidate positions reduces the signaling overhead associated with positional information, as the selected position for the transform can be signaled with fewer bits. Furthermore, reducing the number of candidate positions makes the selection of the transform position algorithmically simpler, which allows for a reduction in encoder complexity (e.g., resulting in fewer computational resources used for encoding).

[0101] Figure 9 shows an example of SVT 900 including a transformation block 907 and a residual block 905. The transformation block 907 and residual block 905 in Figure 9 are similar to the transformation blocks 607, 707, and 807 and residual blocks 605, 705, and 805 in Figures 6-8, respectively. Figure 9 shows various candidate positions, which are possible and acceptable positions of the transformation block (e.g., transformation block 907) relative to the residual block (e.g., residual block 905). Specifically, the SVT examples in Figures 9A-9E use SVT-V, and the SVT examples in Figures 9F-9J use SVT-H. The acceptable candidate positions for the transformation depend on the CPSS, which further depends on the portion of the residual block 905 that the transformation block 907 should cover and / or the step size between the candidate positions. For example, CPSS may be calculated as s = w / M1 for SVT-V, or as s = h / M2 for SVT-H, where w and h are the width and height of the residual block, respectively, and M1 and M2 are predetermined integers in the range of 2 to 8. More candidate positions are allowed by a larger value of M1 or M2. For example, both M1 and M2 may be set to 8. In this case, the value of the position index (P) describing the position of the transformation block 907 relative to the residual block 905 is between 0 and 4.

[0102] In other examples, CPSS is calculated as s=max(w / M1,Th1) for SVT-V or s=max(h / M2,Th2) for SVT-H, where Th1 and Th2 are predefined integers specifying the minimum step size. Th1 and Th2 may also be integers greater than or equal to 2. In this example, Th1 and Th2 are set to 4, and M1 and M2 are set to 8, and different block sizes may have a different number of candidate positions. For example, when the width of residual block 905 is 8, two candidate positions, particularly those in Figures 9A and 9E, are available for SVT-V. For example, when the step size indicated by Th1 is large, and the portion of residual block 905 covered by the transformation block 907, indicated by w / M1, is also large, only two candidate positions satisfy CPSS. However, when w is set to 16, the portion of residual block 905 covered by the transformation block 907 decreases with the change in w / M1. This results in more candidate positions, in this case, three candidate positions as shown in Figures 9A, 9C, and 9E. All five candidate positions shown in Figures 9A-9E are available when the width of residual block 905 is greater than 16, but the values ​​of Th1 and M1 are as described above.

[0103] Other examples may also be found when CPSS is calculated according to other mechanisms. Specifically, CPSS may be calculated as s=w / M1 for SVT-V or s=h / M2 for SVT-H. In this case, when M1 and M2 are set to 4, three candidate positions are allowed for SVT-V (e.g., candidate positions in Figures 9A, 9C, and 9E) and three candidate positions are allowed for SVT-H (e.g., candidate positions in Figures 9F, 9H, and 9J). Furthermore, when M1 and M2 are set to 4, the portion of residual block 905 covered by transformation block 907 increases, resulting in two acceptable candidate positions for SVT-V (e.g., candidate positions in Figures 9A and 9E) and two acceptable candidate positions for SVT-H (e.g., candidate positions in Figures 9F and 9J).

[0104] In another example, as described above, CPSS is calculated as s = max(w / M1, Th1) for SVT-V, or s = max(h / M2, Th2) for SVT-H. In this case, T1 and T2 are set as predefined integers, for example, 2, M1 is set as 8 when w ≥ h, or 4 when w < h, M2 is set as 8 when h ≥ w, or 4 when h < w. For example, the portion of the residual block 905 covered by the conversion block 907 depends on whether the height of the residual block 905 is greater than the width of the residual block 905 or vice versa. Therefore, the number of candidate positions for SVT-H or SVT-V further depends on the aspect ratio of the residual block 905.

[0105] In another example, as described above, CPSS is calculated as s = max(w / M1, Th1) for SVT-V, or s = max(h / M2, Th2) for SVT-H. In this case, the values of M1, M2, Th1, and Th2 are derived from the high-level syntax structure of the bitstream (e.g., the sequence parameter set). For example, the values used to derive CPSS can be signaled within the bitstream. M1 and M2 may share the same value parsed from the syntax element, and Th1 and Th2 may share the same value parsed from other syntax elements.

[0106] Figure 10 shows an exemplary SVT position 1000 indicating the position of the transform block 1007 relative to the residual block 1005. Six different positions (e.g., three vertical positions and three horizontal positions) are shown, but it should be recognized that a different number of positions may be used in actual applications. The SVT transform position 1000 is selected from candidate positions in the SVT example 900 in Figure 9. Specifically, the selected SVT transform position 1000 may be encoded into a position index (P). The position index P can be used to determine the position offset (Z) of the upper left corner of the transform block relative to the upper left corner of the residual block. For example, this position correlation can be determined according to Z = s × P, where s is the CPSS for the transform block based on the SVT type, and is calculated as described with respect to Figure 9. When the transform block is of type SVT-V, the value of P is

number

number

[0107] As will be explained in more detail below, the encoder may encode the SVT transformation type (e.g., SVT-H or SVT-T) and residual block size in the bitstream by using flags. The decoder may then determine the SVT transformation size based on the SVT transformation size and residual block size. Once the SVT transformation size is determined, the decoder can determine acceptable candidate positions for the SVT transformation, such as the candidate positions in SVT Example 900 in Figure 9, according to the CPSS function. Since the decoder can determine candidate positions for the SVT transformation, the encoder does not need to signal the coordinates of the position offset. Instead, a code can be used to indicate which candidate position is used for the corresponding transformation. For example, a position index P may be binarized into one or more bins using a truncated unary code for further compression. As a specific example, when the P value is in the range of 0 to 4, P values ​​0, 4, 2, 3 and 1 can be binarized as 0, 01, 001, 0001 and 0000, respectively. This binary code is more compressed than representing the decimal value of the position index. As another example, when the P value is in the range of 0 to 1, P values ​​0 and 1 can be binarized as 0 and 1, respectively. Thus, the position index can be increased or decreased by a desired size to signal a particular transformation block position, taking into account the possible candidate positions of the transformation block.

[0108] The position index P may be binarized into one or more bins by using the most likely position and the remaining less likely positions. For example, when the left and above adjacent blocks have already been decoded in the decoder and are therefore available for prediction, the most likely position may be set as the position covering the lower right corner of the residual block. In one example, when the P value is in the range of 0 to 4 and position 4 is set as the most likely position, the P values ​​4, 0, 1, 2, and 3 are binarized as 1, 000, 001, 010, and 011, respectively. Furthermore, when the P value is in the range of 0 to 2 and position 2 is set as the most likely position, the P values ​​2, 0, and 1 are binarized as 1, 01, and 00, respectively. Thus, the most likely position index of a candidate position is represented by the fewest bits to reduce the signaling overhead in the most common case. The probability can be determined based on the coding order of adjacent reconstructed blocks. Thus, the decoder can infer the codeword scheme that should be used for the corresponding block based on the decoding scheme used.

[0109] For example, in HEVC, the coding order of coding units is generally from top to bottom and left to right. In such cases, the right side of the coding unit currently being coded / decoded is unavailable, making the upper right corner the most likely transformed position. However, the motion vector predictor is derived from the left and upper spatial adjacencies. In such cases, residual information becomes statistically stronger towards the lower right corner. In this case, the candidate position covering the lower right portion is the most likely position. Furthermore, when an adaptive coding order of coding units is used, one node may be vertically split into two child nodes, and the right child node may be coded before the left child node. In this case, the right adjacency of the left child node is reconstructed before the decoding / coding of the left child node. Furthermore, in such cases, the left adjacency pixels are unavailable. When the right adjacency is available and the left adjacency is unavailable, the lower left portion of the residual block is likely to contain a large amount of residual information, and therefore, the candidate position covering the lower left portion of the residual block is the most likely position.

[0110] Therefore, the position index P may be binarized into one or more bins depending on whether the right-hand adjacent to the residual block has been reconstructed. In one example, the P values ​​are in the range of 0 to 2, as indicated by the SVT transformed position 1000. When the right-hand adjacent to the residual block has been reconstructed, P values ​​0, 2, and 1 are binarized as 0, 01, and 00. Otherwise, P values ​​2, 0, and 1 are binarized as 0, 01, and 00. In another example, when the right-hand adjacent to the residual block has been reconstructed but the left-hand adjacent to the residual block has not, P values ​​0, 2, and 1 are binarized as 0, 00, and 01. Otherwise, P values ​​2, 0, and 1 are binarized as 0, 00, and 01. In these examples, the position corresponding to a single bin is the most likely position, and the other two positions are the remaining positions. For example, the most likely position depends on the availability of the right-hand adjacent.

[0111] The probability distribution of the best position in terms of rate-distortion performance can be entirely different among interprediction modes. For example, when a residual block corresponds to a predicted block generated by template matching with a spatially adjacent reconstructed pixel as a template, the best position is position 2, which has the highest probability. In other interprediction modes, the probability that position 2 (or position 0 when the right neighbor is available and the left neighbor is not) is the best position is lower than the probability in the template matching mode. In light of this, the context model for the first bin of position index P may be determined according to the interprediction mode associated with the residual block. More specifically, when the residual block relates to template matching-based interprediction, the first bin of position index P uses the first context model. Otherwise, the second context model is used to encode / decode this bin.

[0112] In other examples, when residual blocks relate to template matching-based interpretation, the most likely position (e.g., position 2, or position 0 when a right neighbor is available but a left neighbor is not) is directly set as the transform block position, and the position information is not signaled in the bitstream. Otherwise, the position index is explicitly signaled in the bitstream.

[0113] It should also be noted that different transformations can be used depending on the position of the transformation block relative to the residual block. For example, the left side of the residual block is reconstructed, while the right side is not, which occurs for video coding with a fixed coding order of coding units from left to right and top to bottom (e.g., the coding order in HEVC). In this case, the candidate position covering the lower right corner of the residual block may use DST (e.g., DST version 7 (DST-7) or DST version 1 (DST-1)) for the transformation in the transformation block during encoding. Therefore, the inverse DST transformation is used in the decoder for the corresponding candidate position. Furthermore, the candidate position covering the upper left corner of the residual block may use DCT (e.g., DCT version 8 (DCT-8) or DCT version 2 (DCT-2)) for the transformation in the transformation block during encoding. Therefore, the inverse DCT transformation is used in the decoder for the corresponding candidate position. This is because, in this case, the lower right corner is the furthest from the spatially reconstructed region among the four corners. Furthermore, when the transformation block covers the lower right corner of the residual block, DST is more effective than DCT for transforming the residual information distribution. However, when the transformation block covers the upper left corner of the residual block, DCT is more effective than DST for transforming the residual information distribution. For the remainder of the candidate positions, the transformation type can be either inverse DST or DCT. For example, when the candidate position is closer to the lower right corner than the upper left corner, inverse DST is used as the transformation type. Otherwise, inverse DCT is used as the transformation type.

[0114] As a specific example, as shown in Figure 10, three candidate positions for the conversion block 1007 may be allowed. In this case, position 0 covers the upper left corner, and position 2 covers the lower right corner. Position 1 is in the center of the residual block 1005 and is equidistant from both the left and right corners. The conversion types can be selected in the encoder as DCT-8, DST-7, and DST-7 for positions 0, 1, and 2, respectively. Then, the inverse conversions DCT-8, DST-7, and DST-7 can be used in the decoder for positions 0, 1, and 2, respectively. In another example, the conversion types for positions 0, 1, and 2 are DCT-2, DCT-2, and DST-7 in the encoder, respectively. Then, the inverse conversions DCT-2, DCT-2, and DST-7 can be used in the decoder for positions 0, 1, and 2, respectively. Thus, the conversion types for the corresponding candidate positions can be predetermined.

[0115] In some cases, the position-dependent transformations described above may be applied only to the chroma transformation block. The corresponding chroma transformation block may always use inverse DCT-2 in the transformation / inverse transformation process.

[0116] Figure 11 shows an example of horizontal inversion of residual samples 1100. In some cases, favorable residual compression can be achieved by horizontally inverting the residual information within a residual block (e.g., residual block 605) before applying a transformation block (e.g., transformation block 607) in the encoder. Example 1100 illustrates such a horizontal inversion. In this context, the horizontal inversion indicates rotating the residual samples within the residual block around an intermediate axis between the left and right sides of the residual block. Such a horizontal inversion occurs before applying a transformation (e.g., transformation block) in the encoder and after applying an inverse transformation (e.g., transformation block) in the decoder. Such an inversion may be used when specified predefined conditions occur.

[0117] In one example, horizontal inversion occurs when the transformation block uses DST / inverse DST in the transformation process. In this case, the right-hand adjacent of the residual block is encoded / reconstructed before the current block, while the left-hand adjacent is not encoded / reconstructed before the current block. The horizontal inversion process swaps the residual sample of column i of the residual block with the residual sample of column w-1-i of the residual block. In this context, w is the width of the transformation block, and i = 0, 1, ..., (w / 2)-1. Horizontal inversion of residual samples can increase encoding efficiency by better fitting the residual distribution with the DST transformation.

[0118] Figure 12 is a flowchart of an exemplary method 1200 for video decoding using a position-dependent SVT with the mechanism described above. Method 1200 may be initiated in the decoder when a bitstream such as bitstream 611 is received. Method 1200 uses the bitstream to determine a prediction block and a transformation residual block such as prediction block 603 and transformation residual block 609. Method 1200 also determines a transformation block such as transformation block 607, which is used to determine a residual block such as residual block 605. The residual block 605 and prediction block 603 are then used to reconstruct an image block such as image block 601. Although Method 1200 is described from the perspective of the decoder, it should be noted that a similar method may be used to encode video by using an SVT (for example, in reverse).

[0119] In block 1201, a bitstream is acquired by the decoder. The bitstream may be received from memory or from a streaming source. The bitstream contains data that can be decoded into at least one image corresponding to video data from the encoder. Specifically, the bitstream contains block partitioning information that can be used to determine coding units, including prediction blocks and residual blocks, from the bitstream, as described in mechanism 600. Thus, coding information associated with coding units can be parsed from the bitstream, and the pixels of the coding units can be reconstructed based on the coding information as described below.

[0120] In block 1203, the prediction block and the corresponding transform residual block are obtained from the bitstream based on block partitioning information. In this example, the transform residual block is encoded according to SVT, as described with respect to mechanism 600 above. Method 1200 then reconstructs a residual block of size w × h from the transform residual block, as described below.

[0121] In block 1205, the use of SVT, the type of SVT, and the size of the transform block are determined. For example, the decoder first determines whether an SVT is used in the encoding, because some encodings use a transform that is the size of the residual block. The use of an SVT can be signaled by a syntax element in the bitstream. Specifically, when a residual block is allowed to use an SVT, a flag such as svt_flag is parsed from the bitstream. A residual block is allowed to use an SVT if it has non-zero transform coefficients (e.g., corresponding to any luma or chroma component). For example, a residual block may use an SVT if it contains any residual data. The SVT flag indicates whether the residual block is coded using a transform block of the same size as the residual block (e.g., svt_flag is set to 0) or whether the residual block is coded using a transform block smaller than the residual block (e.g., svt_flag is set to 1). The coded block flag (cbf) can be used to indicate whether the residual block contains non-zero conversion coefficients for color components, as used in HEVC. The root coded block (root cbf) flag can also indicate whether the residual block contains non-zero conversion coefficients for any color component, as used in HEVC. As a specific example, when an image block is predicted using interpretation and either the residual block width or residual block height falls within a predetermined range of [a1, a2], the residual block is permitted to use SVT, where a1=16 and a2=64, a1=8 and a2=64, or a1=16 and a2=128. The values ​​of a1 and a2 can be predetermined fixed values. These values ​​may also be derived from the sequence parameter set (SPS) or slice header in the bitstream. When the residual block does not use SVT, the conversion block size is set as the width and height of the residual block size.Otherwise, the conversion size is determined based on the SVT conversion type.

[0122] Once the decoder determines that an SVT is used for a residual block, it determines the type of SVT transformation block used and derives the transformation block size according to the SVT type. The acceptable SVT types for a residual block are determined based on the width and height of the residual block. An SVT-V transformation, as shown in Figure 8, is acceptable if the width of the residual block is within the range [a1, a2], and such values ​​are defined above. An SVT-H transformation, as shown in Figure 8, is acceptable if the height of the residual block is within the range [a1, a2], and such values ​​are defined above. The SVT may be used only for the luma component in the residual block, or the SVT may be used for both the luma and chroma components in the residual block. When the SVT is used only for the luma component, the residual information of the luma component is transformed by the SVT, and the chroma component is transformed by transforming the size of the residual block. When both SVT-V and SVT-H are acceptable, a flag such as svt_type_flag may be encoded in the bitstream. svt_type_flag indicates whether SVT-V is used for the residual block (e.g., svt_type_flag is set to 0) or whether SVT-H is used for the residual block (e.g., svt_type_flag is set to 1). Once the type of SVT transformation is determined, the transformation block size is set according to the signaled SVT type (e.g., w_t=w / 2 and h_t=h for SVT-V, and w_t=w and h_t=h / 2 for SVT-H). When only SVT-V is allowed, or only SVT-H is allowed, svt_type_flag does not need to be encoded in the bitstream. In such cases, the decoder can infer the transformation block size based on the allowed SVT type.

[0123] Once the SVT type and size are determined, the decoder proceeds to block 1207. In block 1207, the decoder determines the position of the transformation relative to the residual block and the type of transformation (e.g., either DST or DCT). The position of the transformation block can be determined according to the syntax elements in the bitstream. For example, the position index can be directly signaled and therefore parsed from the bitstream in some examples. In other examples, the position can be inferred as described with respect to Figures 8-10. Specifically, candidate positions for the transformation can be determined according to the CPSS function. The CPSS function can determine candidate positions by considering the width of the residual block, the height of the residual block, the SVT type determined by block 1205, the step size of the transformation, and / or the portion of the residual block covered by the transformation. The decoder can then determine the transformation block position from the candidate positions by obtaining a p-index containing a code that signals the correct candidate position according to the candidate position selection probability, as described with respect to Figure 10 above. Once the location of the transformation block is known, the decoder can infer the type of transformation used by the transformation block, as explained with respect to Figure 10 above. Therefore, the encoder can select the corresponding inverse transformation.

[0124] In block 1209, the decoder analyzes the conversion coefficients of the conversion block based on the conversion block size determined in block 1205. This process may be achieved according to the conversion coefficient analysis mechanism used in HEVC, H.264, and / or AVC. The conversion coefficients may be coded using run-length coding and / or as a set of conversion coefficient groups (CG). It should be noted that in some examples, block 1209 may be executed before block 1207.

[0125] In block 1211, the residual block is reconstructed based on the transformation position, transformation coefficients, and transformation type as determined above. Specifically, the inverse quantization and inverse transformation of size w_t × h_t are applied to the transformation coefficients to recover the residual samples of the residual block. The size of the residual block with residual samples is w_t × h_t. The inverse transformation may be inverse DCT or inverse DST, according to the position-dependent transformation type determined in block 1207. The residual samples are assigned to corresponding regions within the residual block according to the transformation block position. Residual samples inside the residual block and outside the transformation block may be set to zero. For example, when SVT-V is used, the number of candidate positions is 5, and the position index indicates the 5th transformation block position. Therefore, the reconstructed residual samples are assigned to a region of size (w / 2) × h to the left of the region with zero residual samples (e.g., the unshaded region in Figure 9) within the candidate transformation location of SVT example 900 in Figure 9 (e.g., the shaded region in Figure 9).

[0126] In the optional block 1213, the residual block information of the reconstructed block may be horizontally inverted, as described with respect to Figure 11. As mentioned above, this can occur when the transform block in the decoder uses inverse DST, and the right-hand adjacent block has already been reconstructed, while the left-hand adjacent block has not yet been reconstructed. Specifically, the encoder may horizontally invert the residual block before applying the DST transform in the above case in order to increase coding efficiency. Therefore, the optional block 1213 may be used to compensate for such horizontal inversion in the encoder in order to produce an accurate reconstructed block.

[0127] In block 1215, the reconstructed residual block may be configured together with a prediction block to generate a reconstructed image block containing the sample as part of the coding unit. A filtering process may also be applied to the reconstructed sample, such as the deblocking filter and sample adaptive offset (SAO) processing in HEVC. The reconstructed image block may then be combined with other image blocks decoded in a similar manner to generate frames for a media / video file. The reconstructed media file may then be displayed to the user on a monitor or other display device.

[0128] It should be noted that an equivalent implementation of Method 1200 can be used to generate reconstructed samples within residual blocks. Specifically, residual samples of a transformation block can be directly constructed with the predicted block at the location indicated by the transformation block location information, without first recovering the residual block.

[0129] Figure 13 shows a video coding method 1300. Method 1300 may be performed in a decoder (e.g., video decoder 30). In particular, method 1300 may be performed by the decoder's processor. Method 1300 may be performed when a bitstream is received directly or indirectly from an encoder (e.g., video encoder 20) or when it is retrieved from memory. In block 1301, the bitstream is parsed to obtain a prediction block (e.g., prediction block 603) and a conversion residual block corresponding to the prediction block (e.g., conversion residual block 609). In block 1303, the type of SVT used to generate the conversion residual block is determined. As described above, the type of SVT may be SVT-V or SVT-H. In embodiments, the SVT-V type includes a height equal to the height of the conversion residual block and a width that is half the width of the conversion residual block.

[0130] In the embodiment, the SVT-H type includes a height that is half the height of the transformation residual block and a width equal to the width of the transformation residual block. In the embodiment, svt_type_flag is parsed from the bitstream to determine the type of SVT. In the embodiment, when only one type of SVT is allowed for the residual block, the type of SVT is determined by inference.

[0131] In block 1305, the position of the SVT relative to the transformation residual block is determined. In embodiments, a position index is parsed from the bitstream to determine the position of the SVT. In embodiments, the position index includes a binary code indicating the position from a set of candidate positions determined according to CPSS. In embodiments, the most likely position of the SVT is assigned the fewest bits in the binary code indicating the position index. In embodiments, when a single candidate position is available for the SVT transformation, the position of the SVT is inferred by the processor. In embodiments, when the residual block is generated by template matching in interprediction mode, the position of the SVT is inferred by the processor.

[0132] In block 1307, the inverse of the SVT is determined based on the position of the SVT. In block 1309, the inverse of the SVT is applied to the transformed residual block to generate a reconstructed residual block (e.g., residual block 605). In the embodiment, the inverse DST is used for SVT-V type transformations located at the left boundary of the residual block. In the embodiment, the inverse DST is used for SVT-H type transformations located at the upper boundary of the residual block. In the embodiment, the inverse DCT is used for SVT-V type transformations located at the right boundary of the residual block. In the embodiment, the inverse DCT is used for SVT-H type transformations located at the lower boundary of the residual block.

[0133] In block 1311, the reconstructed residual block is combined with the prediction block to reconstruct the image block. In this embodiment, the image block is displayed on the display or monitor of an electronic device (e.g., a smartphone, tablet, laptop computer, personal computer, etc.).

[0134] Optionally, method 1300 may also include horizontally flipping the samples within the reconstructed residual block before combining the reconstructed residual block with the prediction block, when the right-hand adjacent of the coding unit associated with the reconstructed residual block has been reconstructed and the left-hand adjacent of the coding unit has not been reconstructed.

[0135] Figure 14 shows a video coding method 1400. Method 1400 may be implemented in an encoder (e.g., video encoder 20). In particular, method 1400 may be implemented by the encoder's processor. Method 1400 may be implemented to encode a video signal. In block 1401, a video signal is received from a video capture device (e.g., a camera). In an embodiment, the video signal includes an image block (e.g., image block 601).

[0136] In block 1403, prediction blocks (e.g., prediction block 603) and residual blocks (e.g., residual block 605) are generated to represent image blocks. In block 1405, a transformation algorithm is selected for the SVT based on the SVT's position relative to the residual block. In block 1407, the residual block is transformed into a transformed residual block using the selected SVT.

[0137] In block 1409, the SVT type is encoded into the bitstream. In embodiments, the SVT type is either SVT-V or SVT-H. In embodiments, the SVT-V type includes a height equal to the height of the residual block and a width that is half the width of the residual block. In embodiments, the SVT-H type includes a height that is half the height of the residual block and a width that is equal to the width of the residual block.

[0138] In block 1411, the SVT location is encoded into a bitstream. In an embodiment, the SVT location is encoded into a location index. In an embodiment, the location index includes a binary code indicating the location from a set of candidate locations determined according to CPSS. In an embodiment, the most likely location of the SVT is assigned as the minimum number of bits in the binary code indicating the location index.

[0139] In the embodiment, the DST algorithm is used by the processor for SVT-V type transformations located at the left boundary of the residual block. In the embodiment, the DST algorithm is selected by the processor for SVT-H type transformations located at the upper boundary of the residual block. In the embodiment, the DCT algorithm is selected by the processor for SVT-V type transformations located at the right boundary of the residual block. In the embodiment, the DCT algorithm is selected by the processor for SVT-H type transformations located at the lower boundary of the residual block.

[0140] Optionally, the processor may horizontally flip the samples within a residual block before converting it to a transformed residual block if the right-hand adjacent of a coding unit associated with the residual block is encoded and the left-hand adjacent of the coding unit is not encoded.

[0141] In block 1413, the prediction block and the transformation residual block are encoded into a bitstream. In the embodiment, the bitstream is configured to be sent to and / or sent to a decoder.

[0142] Figure 15 is a schematic diagram of an exemplary computing device 1500 for video coding according to an embodiment of the disclosure. The computing device 1500 is suitable for carrying out the embodiments of the disclosure described herein. The computing device 1500 includes an inlet port 1520 and a receiver unit (Rx) 1510 for receiving data, a processor, logic unit, or central processing unit (CPU) 1530 for processing data, a transmitter unit (Tx) 1540 and an exit port 1550 for transmitting data, and a memory 1560 for storing data. The computing device 1500 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the inlet port 1520, the receiver unit 1510, the transmitter unit 1540, and the exit port 1550 for the exit or input of optical or electrical signals. In some examples, the computing device 1500 may also include a wireless transmitter and / or receiver.

[0143] The processor 1530 is implemented by hardware and software. The processor 1530 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1530 communicates with an inlet port 1520, a receiver unit 1510, a transmitter unit 1540, an exit port 1550, and memory 1560. The processor 1530 includes an encoding / decoding module 1514. The encoding / decoding module 1514 implements embodiments of the above disclosure, such as methods 1300 and 1400, and other mechanisms for encoding / reconstructing residual blocks based on the transformed block position when using SVTs, and any other mechanisms described above. For example, the encoding / decoding module 1514 implements, processes, prepares, or provides various coding operations, such as encoding and / or decoding video data as described above. Therefore, the inclusion of the encoding / decoding module 1514 provides a substantial improvement to the functionality of the computing device 1500, resulting in the conversion of the computing device 1500 to different states. Alternatively, the encoding / decoding module 1514 is implemented as an instruction stored in memory 1560 and executed by the processor 1530 (for example, as a computer program product stored in a non-temporary medium).

[0144] Memory 1560 includes one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store such programs when they are selected for execution, and to store instructions and data read during program execution. Memory 1560 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM). The computing device 1500 may also include input / output (IO) devices for interacting with the end user. For example, the computing device 1500 may include a display such as a monitor for visual output, a speaker for audio output, and a keyboard / mouse / trackball, etc., for user input.

[0145] In summary, the above disclosure includes a mechanism for adaptively using multiple transformation types for transformation blocks at different locations. Furthermore, the disclosure enables horizontal inversion of residual samples within residual blocks to support coding efficiency. This occurs when transformation blocks use DST and inverse DST in the encoder and decoder, respectively, and when the right-adjacent block is available and the left-adjacent block is unavailable. Furthermore, the disclosure includes a mechanism for supporting coding position information in the bitstream based on the inter-prediction mode associated with the residual blocks.

[0146] Figure 16 is a schematic diagram of an embodiment of the coding means 1600. In this embodiment, the coding means 1600 is implemented in a video coding device 1602 (e.g., a video encoder 20 or a video decoder 30). The video coding device 1602 includes a receiving means 1601. The receiving means 1601 is configured to receive a picture to encode or a bitstream to decode. The video coding device 1602 includes a transmitting means 1607 coupled to the receiving means 1601. The transmitting means 1607 is configured to transmit a bitstream to a decoder or to transmit a decoded image to a display means (e.g., one of the I / O devices in a computing device 1500).

[0147] The video coding device 1602 includes a storage means 1603. The storage means 1603 is coupled to at least one of the receiving means 1601 or the transmitting means 1607. The storage means 1603 is configured to store instructions. The video coding device 1602 also includes a processing means 1605. The processing means 1605 is coupled to the storage means 1603. The processing means 1605 is configured to execute instructions stored in the storage means 1603 and to perform the method disclosed herein.

[0148] The first component is directly coupled to the second component when there are no intervening components between the first component and the second component other than lines, traces, or other media. The first component is indirectly coupled to the second component when there are intervening components other than lines, traces, or other media between the first component and the second component. The term "coupled" and its variations include both direct and indirect coupling. The use of the term "about" means a range including ±10% of the following number unless otherwise specified.

[0149] While several embodiments are provided in this disclosure, it should be understood that the systems and methods of the disclosure may be embodied in many other specific forms without departing from the true intent or scope of this disclosure. These examples should be considered illustrative and not limiting, and their intent is not limited to the details given herein. For example, various elements or components may be combined with or integrated into other systems, or certain features may be omitted or not implemented.

[0150] Furthermore, the technologies, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined with or integrated with other systems, components, technologies, or methods without departing from the scope of this disclosure. Other items illustrated or described as combined may be directly connected, or indirectly combined or communicated through several interfaces, devices, or intermediate components, whether electrical, mechanical, or otherwise. Other examples of modifications, substitutions, and alternatives are readily apparent to those skilled in the art and may be made without departing from the true intent and scope of this disclosure.

Claims

1. A decoding method, The steps include: analyzing the bitstream to obtain prediction blocks and transformation residual blocks corresponding to the prediction blocks; The steps include: analyzing the bitstream to obtain the type of spatial transformation (SVT) used in the transformation block to generate the transformation residual block, wherein the transformation block covers a portion of the residual block related to the prediction block, and the type of the SVT is either a vertical SVT (SVT-V) type or a horizontal SVT (SVT-H) type; The steps include: analyzing the bitstream to obtain the position of the transformation block relative to the residual block; The step of generating a reconstructed residual block by applying the inverse of the SVT to the transformed residual block based on the type of the SVT and the position of the transformed block relative to the residual block, wherein the transformed algorithm for the inverse of the SVT is based on the position of the transformed block, When the transformation block covers the lower right portion of the residual block, the inverse transformation algorithm of the SVT includes the inverse discrete sine transform (DST) version 7 (DST-7), or When the transformation block covers the upper left portion of the residual block, the inverse transformation algorithm of the SVT includes the steps of an inverse discrete cosine transform (DCT) version 8 (DCT-8), The steps include: obtaining a reconstructed image block by combining the reconstructed residual block with the prediction block; A method that includes this.

2. The method according to claim 1, wherein when the type of the SVT is the SVT-V type, the height of the conversion block is equal to the height of the residual block and the width of the conversion block is half the width of the residual block, and when the type of the SVT is the SVT-H type, the height of the conversion block is half the height of the residual block and the width of the conversion block is equal to the width of the residual block.

3. The method according to claim 1, further comprising the step of analyzing flags from the bitstream to determine the type of the SVT.

4. The method according to claim 1, further comprising the step of determining the type of SVT by inference when only one type of SVT is permitted for the residual block.

5. The step of analyzing the bitstream to obtain the position of the transformation block relative to the residual block is: The steps include determining the position of the conversion block according to the syntax elements in the bitstream, The steps include: analyzing the bitstream to obtain syntax elements, wherein the syntax elements indicate the position of the conversion block; and The method according to claim 1, including the method described in claim 1.

6. The method according to claim 5, wherein the value of the syntax element is in the range of 0 to 1.

7. An encoding method, A step of generating prediction blocks and residual blocks to represent image blocks, The step of generating a transformed residual block using the transformed block, based on the type of spatial transformation (SVT) and the position of the transformed block relative to the residual block, wherein the transformed block covers a portion of the residual block, the type of the SVT is a vertical SVT (SVT-V) type or a horizontal SVT (SVT-H) type, and the transformation algorithm for the SVT is based on the position of the transformed block relative to the residual block, When the transformation block covers the lower right portion of the residual block, the transformation algorithm of the SVT includes inverse discrete sine transform (DST) version 7 (DST-7), or When the transformation block covers the upper left portion of the residual block, the transformation algorithm of the SVT includes the steps of: The steps include specifying the type of the SVT in a bitstream, The steps include: indicating the position of the conversion block relative to the residual block using the bitstream; The steps include encoding the prediction information of the conversion residual block and the prediction block into the bitstream. A method that includes this.

8. The method according to claim 7, wherein, when the type of the SVT is the SVT-V type, the height of the conversion block is equal to the height of the residual block, and the width of the conversion block is half the width of the residual block.

9. The method according to claim 7, wherein, when the type of the SVT is the SVT-H type, the height of the conversion block is half the height of the residual block, and the width of the conversion block is equal to the width of the residual block.

10. The method according to claim 7, wherein the bitstream includes a syntax element indicating the position of the conversion block.

11. The method according to claim 10, wherein the value of the syntax element is in the range of 0 to 1.

12. The method according to claim 7, wherein the bitstream further includes a flag indicating the type of the SVT.

13. A decoding device, The invention includes a processing circuit for performing the method described in any one of claims 1 to 6, A decoding device comprising one or more processors and a memory coupled to the one or more processors and configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the decoding device becomes capable of carrying out the method according to any one of claims 1 to 6.

14. An encoding device, The invention includes a processing circuit for performing the method described in any one of claims 7 to 12, or An encoding device comprising one or more processors and a memory coupled to the one or more processors and configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the encoding device becomes capable of carrying out the method according to any one of claims 7 to 12.

15. A non-temporary storage medium comprising a computer program that causes a decryption device to perform the method described in any one of claims 1 to 6.

16. A non-temporary storage medium comprising a computer program that causes an encoding device to perform the method described in any one of claims 7 to 12.

17. A computer-readable storage medium comprising a bitstream obtained according to the method described in any one of claims 7 to 12.

Images