Method for position-dependent spatially varying transformation for video coding and decoding

Abstract

The present application discloses a position-dependent spatial varying transform (SVT) mechanism for video encoding and decoding. A decoder receives a prediction block and a corresponding transform residual block. The type of spatial varying transform (SVT) used to generate the transform residual block is determined. The position of the SVT relative to the transform residual block is also determined. The inverse of the SVT is applied to the transform residual block to reconstruct the reconstructed residual block. Then, the reconstructed residual block is combined with the prediction block to reconstruct an image block.

Description

[0001] This application is a divisional application. The application number of the original application is 201980015113.X, and the original application date is February 4, 2019. The entire contents of the original application are incorporated into this application by reference.

[0002] CROSS REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the U.S. Provisional Patent Application No. 62 / 634,613 filed by Yin Zhao et al. on February 23, 2018, entitled “PositionDependent Spatial Varying Transform for Video Coding”, the teachings and disclosures of which are hereby incorporated by reference in their entirety. Technical Field

[0004] The present application relates to the field of video coding and decoding, and in particular to a method and a related device for position-dependent spatial variation transformation for video coding and decoding. Background Art

[0005] Video codecs are the process of compressing video images into a smaller format. Video codecs allow the encoded video to take up less space when stored on a medium. In addition, video codecs support streaming media. Specifically, content providers want to provide media to end users with increasingly higher definition. Furthermore, content providers want to provide media on demand without forcing users to wait for a long period of time before such media is transmitted to end-user devices such as televisions, computers, tablets, phones, etc. Advances in video codec compression can reduce the size of video files. Therefore, when applied in conjunction with a corresponding content distribution system, both of the above goals are supported. Summary of the invention

[0006] A first aspect relates to a method implemented in a computing device. The method includes: a processor of the computing device parses a bitstream to obtain a prediction block and a transform residual block corresponding to the prediction block; the processor determines the type of spatially varying transform (SVT) used to generate the transform residual block; the processor determines the position of the SVT relative to the transform residual block; the processor determines the inverse of the SVT based on the position of the SVT; the processor applies the inverse of the SVT to the transform residual block to generate a reconstructed residual block; the processor combines the reconstructed residual block with the prediction block to reconstruct an image block.

[0007] The method is beneficial to improve the coding efficiency of SVT. In this regard, the transform block is positioned at each candidate position relative to the corresponding residual block. Therefore, the disclosed mechanism applies different transforms to the transform block based on the candidate position.

[0008] In a first implementation manner of the method according to the first aspect, the type of the SVT is a SVT vertical (SVT-V) type or a SVT horizontal (SVT-H) type.

[0009] In a second implementation of the method according to the first aspect or any preceding implementation of the first aspect, the SVT-V type includes a height equal to the height of the transform residual block and a width half of the width of the transform residual block; the SVT-H type includes a height half of the height of the transform residual block and a width equal to the width of the transform residual block.

[0010] In a third implementation of the method according to the first aspect or any preceding implementation of the first aspect, svt_type_flag in the bitstream is parsed to determine the type of the SVT.

[0011] In a fourth implementation of the method according to the first aspect or any preceding implementation of the first aspect, when the residual block is only allowed to use one type of SVT, the type of SVT is determined by inference.

[0012] In a fifth implementation of the method according to the first aspect or any preceding implementation of the first aspect, a position index in the bitstream is parsed to determine the position of the SVT.

[0013] In a sixth implementation of the method according to the first aspect or any preceding implementation of the first aspect, the position index comprises a binary code, wherein the binary code indicates a position in a candidate position set determined according to a candidate position step size (CPSS).

[0014] In a seventh implementation of the method according to the first aspect or any preceding implementation of the first aspect, the most likely position of the SVT is allocated the minimum number of bits in the binary code indicating the position index.

[0015] In an eighth implementation of the method according to the first aspect or any preceding implementation of the first aspect, the processor infers the position of the SVT when a single candidate position is available for the SVT transformation.

[0016] In a ninth implementation of the method according to the first aspect or any preceding implementation of the first aspect, when the residual block is generated by template matching in an inter-frame prediction mode, the processor infers the position of the SVT.

[0017] In a tenth implementation of the method according to the first aspect or any preceding implementation of the first aspect, an inverse discrete sine transform (DST) is used for an SVT vertical (SVT-V) type transform located at the left boundary of the residual block.

[0018] In an eleventh implementation of the method according to the first aspect or any preceding implementation of the first aspect, an inverse DST is adopted for an SVT-horizontal (SVT-H) type transform located at a top boundary of the residual block.

[0019] In a twelfth implementation of the method according to the first aspect or any of the aforementioned implementations of the first aspect, an inverse discrete cosine transform (DCT) is used for the SVT-V type transform located at the right boundary of the residual block.

[0020] In a thirteenth implementation of the method according to the first aspect or any of the preceding implementations of the first aspect, an inverse DCT is used for the SVT-H type transform located at the bottom boundary of the residual block.

[0021] In a fourteenth implementation of the method according to the first aspect or any preceding implementation of the first aspect, when a right adjacent coding unit of a coding unit associated with the reconstructed residual block has been reconstructed and a left adjacent coding unit of the coding unit has not been reconstructed, before the reconstructed residual block is combined with the prediction block, samples in the reconstructed residual block are horizontally flipped.

[0022] A second aspect relates to a method implemented in a computing device. The method comprises: receiving a video signal from a video capture device, wherein the video signal comprises an image block; a processor of the computing device generates a prediction block and a residual block to represent the image block; the processor selects a transform algorithm for a spatially varying transform (SVT) based on the position of the SVT relative to the residual block; the processor converts the residual block into a transformed residual block using the selected SVT; the processor encodes the type of the SVT into a bitstream; the processor encodes the position of the SVT into the bitstream; the processor encodes the prediction block and the transformed residual block into the bitstream for transmission to a decoder.

[0023] The method is beneficial to improve the coding efficiency of SVT. In this regard, the transform block is positioned at each candidate position relative to the corresponding residual block. Therefore, the disclosed mechanism applies different transforms to the transform block based on the candidate position.

[0024] In a first implementation manner of the method according to the second aspect, the type of the SVT is a SVT vertical (SVT-V) type or a SVT horizontal (SVT-H) type.

[0025] In a second implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the SVT-V type includes a height equal to the height of the transform residual block and a width that is half the width of the transform residual block.

[0026] In a third implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the SVT-H type includes a height that is half the height of the transform residual block and a width that is equal to the width of the transform residual block.

[0027] In a fourth implementation of the method according to the second aspect or any preceding implementation of the second aspect, the position of the SVT is encoded in a position index.

[0028] In a fifth implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the position index comprises a binary code, wherein the binary code indicates a position in a set of candidate positions determined according to a candidate position step size (CPSS).

[0029] In a sixth implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the most likely position of the SVT is allocated the minimum number of bits in the binary code indicating the position index.

[0030] In a seventh implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the processor uses a discrete sine transform (DST) algorithm for an SVT vertical (SVT-V) type transform at the left boundary of the residual block.

[0031] In an eighth implementation of the method according to the second aspect or any preceding implementation of the second aspect, the processor selects a DST algorithm for an SVT-horizontal (SVT-H) type transform located at a top boundary of the residual block.

[0032] In a ninth implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the processor selects a discrete cosine transform (DST) algorithm for an SVT-V type transform at a right boundary of the residual block.

[0033] In a tenth implementation of the method according to the second aspect or any preceding implementation of the second aspect, the processor selects a DCT algorithm for an SVT-H type transform located at a bottom boundary of the residual block.

[0034] In an eleventh implementation of the method according to the second aspect or any of the foregoing implementations of the second aspect, the method further includes: when the right adjacent coding unit of the coding unit associated with the residual block has been encoded and the left adjacent coding unit of the coding unit has not been encoded, before the processor converts the residual block into the transformed residual block, the processor horizontally flips the samples in the residual block.

[0035] The third aspect relates to a coding and decoding device, comprising: a receiver, configured to receive an image for encoding or a bit stream for decoding; a transmitter coupled to the receiver, wherein the transmitter is configured to transmit the bit stream to a decoder or transmit the decoded image to a display; a memory coupled to at least one of the receiver or the transmitter, wherein the memory is configured to store instructions; and a processor coupled to the memory, wherein the processor is configured to execute the instructions stored in the memory to execute a method of any one of the aforementioned aspects or implementations.

[0036] The encoding and decoding device is conducive to improving the coding efficiency of SVT. In this regard, the transform block is positioned at each candidate position relative to the corresponding residual block. Therefore, the disclosed mechanism applies different transforms to the transform block based on the candidate position.

[0037] In a first implementation manner of the device according to the third aspect, the device further includes a display for displaying an image.

[0038] A fourth aspect relates to a system, wherein the system comprises an encoder and a decoder communicating with the encoder. The encoder or the decoder comprises the encoding and decoding device of any one of the above aspects or implementations.

[0039] The system is useful for improving the coding efficiency of SVT. In this regard, the transform block is positioned at various candidate positions relative to the corresponding residual block. Thus, the disclosed mechanism applies different transforms to the transform block based on the candidate position.

[0040] A fifth aspect relates to a component for encoding and decoding, comprising: a receiving component for receiving an image for encoding or receiving a bit stream for decoding; a transmission component coupled to the receiving component, wherein the transmission component is used to transmit the bit stream to a decoder or transmit the decoded image to a display component; a storage component coupled to at least one of the receiving component or the transmission component, wherein the storage component is used to store instructions; a processing component coupled to the storage component, wherein the processing component is used to execute the instructions stored in the storage component to execute the method of any one of the aforementioned aspects or implementations.

[0041] The means for encoding and decoding is conducive to improving the coding efficiency of SVT. In this regard, the transform block is positioned at each candidate position relative to the corresponding residual block. Therefore, the disclosed mechanism applies different transforms to the transform block based on the candidate position.

[0042] For the sake of clarity, any of the above embodiments may be combined with any one or more of the other embodiments described above to create new embodiments within the scope of the present invention.

[0043] These and other features will become more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] For a more thorough understanding of the present invention, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0045] Figure 1 is a block diagram illustrating an exemplary encoding system that may utilize a spatially varying transform (SVT) transform.

[0046] Figure 2 is a block diagram illustrating an exemplary encoding system that may utilize a spatial SVT transform.

[0047] Figure 3 is a block diagram illustrating an example of a video decoder that may utilize a spatial SVT transform.

[0048] Figure 4 It is a schematic diagram of the intra-frame prediction mode used in video encoding and decoding.

[0049] Figure 5 An example of intra prediction in video codec is shown.

[0050] Figure 6 is a schematic diagram of an exemplary video encoding mechanism.

[0051] Figure 7 An exemplary SVT transform is shown.

[0052] Figure 8 An exemplary SVT transform is shown.

[0053] Fig. 9 Exemplary SVT transform candidate positions relative to the residual block are shown.

[0054] Fig.10 Exemplary SVT transform positions relative to the residual block are shown.

[0055] Fig.11 An example of horizontal flipping of residual samples is shown.

[0056] Fig.12 is a flow chart of an exemplary method for video decoding using position-dependent SVT.

[0057] Fig.13 is a flow chart of an exemplary method of video encoding and decoding.

[0058] Fig.14 is a flow chart of an exemplary method of video encoding and decoding.

[0059] Fig.15 is a schematic diagram of an exemplary computing device for video encoding and decoding.

[0060] Fig.16 is a schematic diagram of an embodiment of components for encoding and decoding. DETAILED DESCRIPTION

[0061] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The present invention should in no way be limited to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims and their full scope of equivalents.

[0062] The standard now known as High Efficiency Video Coding (HEVC) is an advanced video codec system developed by a Joint Collaborative Team on Video Coding (JCT-VC) of video codec experts from the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) study group. Details of the HEVC standard are found in ITU-T Rec. H.265 and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 23008-2 (2013), High Efficiency Video Coding, Final Approval Date: January 2013 (ITU-T officially released in June 2013, ISO / IEC released in November 2013), which are incorporated herein by reference. For an overview of HEVC, see GJ Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Trans. Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, December 2012, which is incorporated herein by reference.

[0063] Figure 1 1 is a block diagram showing an exemplary encoding system 10, wherein the exemplary encoding system 10 may utilize video coding techniques, such as using an SVT mechanism for encoding. Figure 1As shown, encoding system 10 includes a source device 12 that provides encoded video data that is subsequently decoded by a destination device 14. In particular, source device 12 may provide video data to destination device 14 via a computer-readable medium 16. Source device 12 and destination device 14 may include any of a variety of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones and "smart" tablets, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, source device 12 and destination device 14 may be equipped for wireless communication.

[0064] The destination device 14 may receive the encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 may include any type of medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 may include a communication medium so that the source device 12 can transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may constitute a portion of a packet network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other device that can be used to facilitate communication between the source device 12 and the destination device 14.

[0065] In some examples, the encoded data can be output from the output interface 22 to a storage device. Similarly, the encoded data can be accessed from the storage device through the input interface. The storage device may include any distributed or locally accessed data storage medium, such as a hard drive, a Blu-ray disc, a digital video disk (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device may correspond to a file server or another intermediate storage device that can store the encoded video data generated by the source device 12. The destination device 14 can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server capable of storing encoded video data and transmitting the encoded video data to the destination device 14. Exemplary file servers include a World Wide Web server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The destination device 14 may access the encoded video data through any standard data connection (including an Internet connection). The standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two suitable for accessing the encoded video data stored on the file server. The transmission of the encoded video data in the storage device may be streaming transmission, download transmission, or a combination thereof.

[0066] The techniques of the present invention are not necessarily limited to wireless applications or settings. These techniques can be applied to video encoding and decoding to support any multimedia application, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as HTTP dynamic adaptive streaming over HTTP, DASH), digital video encoded on a data storage medium, decoding of digital video stored in a data storage medium) or other applications. In some examples, system 10 can be used to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

[0067] exist Figure 1In the example of, source device 12 comprises video source 18, video encoder 20 and output interface 22. Destination device 14 comprises input interface 28, video decoder 30 and display device 32. According to the present invention, the video encoder 20 of source device 12 and / or the video decoder 30 of destination device 14 can be used for applying video coding and decoding technology. In other examples, source device and destination device can comprise other components or arrangement. For example, source device 12 can receive video data from external video source (such as external camera). Similarly, destination device 14 can be connected with external display device, rather than comprising integrated display device.

[0068] Figure 1 The coding system 10 shown is only an example. The video coding and decoding techniques can be performed by any digital video coding and / or decoding device. Although the techniques of the present invention are generally performed by video coding and decoding devices, these techniques can also be performed by a video encoder / decoder (commonly referred to as a "codec"). In addition, the techniques of the present invention can also be performed by a video preprocessor. The video encoder and / or decoder can be a graphics processing unit (GPU) or similar device.

[0069] Source device 12 and destination device 14 are merely examples of such codec devices, where source device 12 generates encoded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 can operate substantially symmetrically, such that both source device 12 and destination device 14 include video encoding and decoding components. Thus, encoding system 10 can support one-way or two-way video transmission between video devices 12, 14 for video streaming, video playback, video broadcasting, video telephony, etc.

[0070] Video source 18 of source device 12 may include a video capture device (e.g., a video camera), a video archive including previously captured video, and / or a video feed interface for receiving video from a video content provider. Optionally, video source 18 may generate computer graphics-based data as source video, or as a combination of live video, archived video, and computer-generated video.

[0071] In some cases, when the video source 18 is a video camera, the source device 12 and the destination device 14 may form a so-called camera phone or video phone. However, as described above, the techniques described in the present invention may be used generally for video encoding and decoding, and may also be used for wireless and / or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by a video encoder 20. The encoded video information may then be output to a computer-readable medium 16 via an output interface 22.

[0072] The computer-readable medium 16 may include transient media, such as wireless broadcast or wired network transmission, and may also include storage media (i.e., non-transient storage media), such as hard disks, flash drives, optical disks, digital video disks, Blu-ray disks, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from the source device 12 and provide the encoded video data to the destination device 14 via network transmission, etc. Similarly, a computing device of a media production facility (such as a disc stamping facility) may receive encoded video data from the source device 12 and generate a disc including the encoded video data. Therefore, in various examples, the computer-readable medium 16 may be understood to include one or more computer-readable media in various forms.

[0073] The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information of the computer-readable medium 16 may include syntax information defined by the video encoder 20. The syntax information is also used by the video decoder 30, including syntax elements that describe the characteristics and / or processing of blocks and other coding units (e.g., group of pictures (GOP)). The display device 32 displays the decoded video data to the user and may include any display device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0074] The video encoder 20 and the video decoder 30 may operate in accordance with a video codec standard, such as the High Efficiency Video Codec (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). Alternatively, the video encoder 20 and the video decoder 30 may operate in accordance with other proprietary or industry standards, such as the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard, otherwise known as the Motion Picture Expert Group (MPEG)-4, Part 10, Advanced Video Coding (AVC), H.265 / HEVC, and extensions of such standards. However, the technology of the present invention is not limited to any particular coding standard. Other examples of video codec standards include MPEG-2 and ITU-T H.263. Although in Figure 120 and 30 may be integrated with an audio encoder and an audio decoder, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to encode both audio and video in a common data stream or in separate data streams. If applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

[0075] The video encoder 20 and the video decoder 30 may each be implemented as any suitable encoder circuit, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are partially implemented in software, the device may store the instructions of the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of the present invention. Both the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and may be integrated into a corresponding device as part of a combined encoder / decoder (codec). Devices including the video encoder 20 and / or the video decoder 30 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular phone.

[0076] Figure 2 is a block diagram illustrating an example of a video encoder 20 that may implement video coding techniques. The video encoder 20 may perform intra-frame and inter-frame coding of video blocks within a video slice. Intra-frame coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or image. Inter-frame coding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or images of a video sequence. Intra-frame mode (I-mode) may refer to any of several spatial-based coding modes. Inter-frame modes, such as uni-directional (also known as uni-prediction) prediction (P-mode) or bi-directional prediction (also known as bi-prediction) (B-mode) may refer to any of several temporal-based coding modes.

[0077] like Figure 2 As shown, video encoder 20 receives a current video block within a video frame to be encoded. Figure 2In the example of FIG. 5 , the video encoder 20 includes a mode selection unit 40, a reference frame memory 64, an adder 50, a conversion processing unit 52, a quantization unit 54, and an entropy coding unit 56. The mode selection unit 40 includes a motion compensation unit 44, a motion estimation unit 42, an intra-frame prediction (also referred to as intra-frame prediction) unit 46, and a segmentation unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform unit 60, and an adder 62. A deblocking filter ( Figure 2 The output of adder 62 is filtered by a deblocking filter (not shown in the figure) to filter block boundaries and thereby remove blockiness artifacts from the reconstructed video. If desired, a deblocking filter will typically filter the output of adder 62. In addition to the deblocking filter, additional filters (in the loop or post-loop) may also be used. Such filters are not shown for simplicity, but if desired, the output of adder 50 may be filtered (as an in-loop filter).

[0078] During the encoding process, video encoder 20 receives a video frame or slice to be encoded. The frame or slice may be segmented into a plurality of video blocks. Motion estimation unit 42 and motion compensation unit 44 perform inter-frame prediction encoding on the received video block relative to one or more blocks in one or more reference frames to provide temporal prediction. Intra-frame prediction unit 46 may also perform intra-frame prediction encoding on the received video block relative to one or more neighboring blocks in the same frame or slice as the block to be encoded to provide spatial prediction. Video encoder 20 may perform multiple encoding passes, for example, selecting an appropriate encoding mode for each block of video data.

[0079] In addition, the segmentation unit 48 may segment the blocks of the video data into sub-blocks based on an evaluation of a previous segmentation scheme in a previous encoding pass. For example, the segmentation unit 48 may initially segment a frame or slice into a largest coding unit (LCU), and segment each LCU into sub-coding units (sub-CUs) based on a rate-distortion analysis (e.g., rate-distortion optimization). The mode selection unit 40 may further generate a quadtree data structure indicating the segmentation of the LCU into sub-CUs. The leaf node CU of the quadtree may include one or more prediction units (PU) and one or more transform units (TU).

[0080] The present invention uses the term "block" to refer to any of CU, PU or TU in the HEVC context, or similar data structures in other standard contexts (for example, macroblocks and their sub-blocks in H.264 / AVC). A CU includes a coding node, a PU associated with the coding node, and a TU. The size of the CU corresponds to the size of the coding node and is square. The size of the CU can range from 8×8 pixels to a tree block size of 64×64 pixels or larger. Each CU may include one or more PUs and one or more TUs. For example, the syntax data associated with the CU may describe the partitioning of the CU into one or more PUs. The partitioning mode may be different depending on whether the CU is skipped or the CU is encoded in direct mode, intra-frame prediction mode, or inter-frame prediction (also known as inter-frame prediction) mode. The PU may be partitioned into non-squares. For example, the syntax data associated with the CU may also describe the partitioning of the CU into one or more TUs according to a quadtree. A TU may be square or non-square (for example, rectangular).

[0081] The mode selection unit 40 may select one of the intra or inter coding modes based on the error results, etc., and provide the resulting intra or inter coded block to the adder 50 to generate residual block data, and to the adder 62 to reconstruct the coded block for use as a reference frame. The mode selection unit 40 also provides syntax elements, such as motion vectors, intra mode indicators, partition information, and other such syntax information to the entropy coding unit 56.

[0082] The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors, which are used to estimate the motion of video blocks. For example, a motion vector may indicate the displacement of a PU of a video block within a current video frame or image relative to a prediction block within a reference frame (or other coding unit), wherein the reference frame (or other coding unit) is related to the current block of the current frame (or other coding unit). A prediction block is a block that is found to be highly matched to a block to be encoded in terms of pixel differences, wherein the pixel difference may be determined by the sum of absolute difference (SAD), the sum of square difference (SSD), or other difference metrics. In some examples, the video encoder 20 may calculate the value of a sub-integer pixel position of a reference image stored in the reference frame memory 64. For example, the video encoder 20 may insert the value of a quarter pixel position, an eighth pixel position, or other fractional pixel position of the reference image. Therefore, the motion estimation unit 42 may perform a motion search relative to a full pixel position and a fractional pixel position, and output a motion vector with fractional pixel accuracy.

[0083] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-coded slice by comparing the position of the PU to the position of a prediction block of a reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each list identifying one or more reference pictures stored in reference frame memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy encoding unit 56 and motion compensation unit 44.

[0084] Motion compensation performed by motion compensation unit 44 may involve obtaining or generating a prediction block based on the motion vector determined by motion estimation unit 42. Additionally, in some examples, motion estimation unit 42 and motion compensation unit 44 may be functionally integrated. Upon receiving the motion vector for the PU of the current video block, motion compensation unit 44 may locate the prediction block to which the motion vector points in one of the reference picture lists. Adder 50 forms a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded, thereby forming pixel difference values, as discussed below. Typically, motion estimation unit 42 performs motion estimation with respect to a luma component, and motion compensation unit 44 uses motion vectors calculated based on the luma component for chroma components and luma components. Mode selection unit 40 may also generate syntax elements associated with video blocks and video slices for use by video decoder 30 in decoding video blocks of a video slice.

[0085] Intra-prediction unit 46 may perform intra-prediction on the current block as an alternative to the inter-prediction performed by motion estimation unit 42 and motion compensation unit 44, as described above. In particular, intra-prediction unit 46 may determine an intra-prediction mode for encoding the current block. In some examples, intra-prediction unit 46 may encode the current block using various intra-prediction modes in separate encoding passes, etc., and intra-prediction unit 46 (or, in some examples, mode selection unit 40) may select an appropriate intra-prediction mode from the test mode for use.

[0086] For example, intra-prediction unit 46 may calculate rate-distortion values ​​for various tested intra-prediction modes using rate-distortion analysis and select the intra-prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis typically determines the amount of distortion (or error) between a coded block and an original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-prediction unit 46 may calculate ratios based on the distortions and rates for the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0087] In addition, the intra prediction unit 46 can be used to encode the depth block of the depth image using a depth modeling mode (DMM). The mode selection unit 40 can determine whether the available DMM mode produces better encoding results than the intra prediction mode and other DMM modes (e.g., using rate-distortion optimization (RDO)). The data of the texture image corresponding to the depth image can be stored in the reference frame memory 64. The motion estimation unit 42 and the motion compensation unit 44 can also be used to inter-predict the depth block of the depth image.

[0088] After selecting an intra-prediction mode for a block (e.g., a conventional intra-prediction mode or one of the DMM modes), intra-prediction unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode. Video encoder 20 may include configuration data in the transmitted bitstream, wherein the configuration data includes a plurality of intra-prediction mode index tables and a plurality of modified intra-prediction mode index tables (also referred to as codeword mapping tables), definitions of encoding contexts for various blocks, and indications of the most probable intra-prediction mode, the intra-prediction mode index tables, and the modified intra-prediction mode index tables to use for each context.

[0089] Video encoder 20 forms a residual video block by subtracting the prediction data from mode select unit 40 from the encoded original video block. Adder 50 represents one or more components that perform this subtraction operation.

[0090] Transform processing unit 52 applies a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform, to the residual block to produce a video block comprising residual transform coefficient values. Transform processing unit 52 may perform other transforms conceptually similar to DCT. Wavelet transforms, integer transforms, sub-band transforms, or other types of transforms may also be used.

[0091] The transform processing unit 52 applies a transform to the residual block, thereby generating a block with residual transform coefficients. The transform can convert the residual information from the pixel value domain to a transform domain, such as the frequency domain. The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all coefficients. The degree of quantization can be modified by adjusting the quantization parameter. In some examples, the quantization unit 54 can then scan the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0092] After quantization, the entropy coding unit 56 performs entropy coding on the quantized transform coefficients. For example, the entropy coding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. In the case of context-based entropy coding, the context may be based on neighboring blocks. After the entropy coding unit 56 performs entropy coding, the encoded bitstream may be transmitted to another device (e.g., the video decoder 30) or archived for later transmission or retrieval.

[0093] Inverse quantization unit 58 and inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain, e.g., for later use as a reference block. Motion compensation unit 44 may calculate a reference block by adding the residual block to a prediction block of a frame in reference frame memory 64. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for use in motion estimation. Adder 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to produce a reconstructed video block for storage in reference frame memory 64. Motion estimation unit 42 and motion compensation unit 44 may use the reconstructed video block as a reference block to intra-code a block in a subsequent video frame.

[0094] Figure 3 is a block diagram illustrating an example of a video decoder 30 that may implement video encoding and decoding techniques. Figure 3 In the example of FIG. 1 , video decoder 30 includes entropy decoding unit 70, motion compensation unit 72, intra prediction unit 74, inverse quantization unit 76, inverse transform unit 78, reference frame memory 82, and adder 80. In some examples, video decoder 30 may perform operations generally related to video encoder 20 ( Figure 2 ) is a decoding pass that is the opposite of the encoding pass described in . The motion compensation unit 72 may generate prediction data based on the motion vector received from the entropy decoding unit 70, and the intra-prediction unit 74 may generate prediction data based on the intra-prediction mode indicator received from the entropy decoding unit 70.

[0095] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video slice and associated syntax elements from the video encoder 20. An entropy decoding unit 70 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 70 forwards the motion vectors and other syntax elements to the motion compensation unit 72. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0096] When the video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 74 may generate prediction data for the video blocks of the current video slice based on the signaled intra-prediction mode and data from the previously decoded blocks of the current frame or picture. When the video frame is encoded as an inter-coded (e.g., B, P, or GPB) slice, the motion compensation unit 72 generates prediction blocks for the video blocks of the current video slice based on the motion vectors and other syntax elements received from the entropy decoding unit 70. These prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. The video decoder 30 may construct the reference frame lists: List 0 and List 1 using a default construction technique based on the reference pictures stored in the reference frame memory 82.

[0097] The motion compensation unit 72 determines prediction information for the video blocks in the current video slice by parsing the motion vectors and other syntax elements, and uses the prediction information to generate prediction blocks for the decoded current video blocks. For example, the motion compensation unit 72 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) used to encode the video blocks of the video slice, the inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), the construction information of one or more reference picture lists for the slice, the motion vector for each inter-frame coded video block of the slice, the inter-frame prediction state for each inter-frame coded video block of the slice, and other information to decode the video blocks in the current video slice.

[0098] Motion compensation unit 72 may also perform interpolation based on an interpolation filter. Motion compensation unit 72 may calculate interpolated values ​​of sub-integer pixels of a reference block using an interpolation filter used by video encoder 20 during encoding of the video block. In this case, motion compensation unit 72 may determine the interpolation filter used by video encoder 20 from received syntax elements and use the interpolation filter to generate a prediction block.

[0099] Data of the texture image corresponding to the depth image may be stored in the reference frame memory 82. The motion compensation unit 72 may also be used to inter-predict a depth block of a depth image.

[0100] Various mechanisms for improving the coding efficiency of SVT are disclosed herein. As described above, the transform block can be positioned at various candidate positions relative to the corresponding residual block. The disclosed mechanism applies different transforms to the transform block based on the candidate position. For example, an inverse discrete sine transform (DST) can be applied to a candidate position covering the lower right corner of the residual block. In addition, an inverse DCT can be applied to a candidate position covering the upper left corner of the residual block. This mechanism can be beneficial because DST is generally more effective than DCT for transforming residual blocks that distribute more residual information in the lower right corner, while DCT is generally more effective than DST for transforming residual blocks that distribute more residual information in the upper left corner. It should also be noted that in most cases, the lower right corner of the residual block statistically includes more residual information. In some cases, the disclosed mechanism also supports horizontal inversion of the residual samples of the transform block. For example, after applying the inverse transform block, the residual samples can be horizontally inverted / flipped. This may occur when the neighboring blocks to the right of the current residual block have been reconstructed and the neighboring blocks to the left of the current residual block have not yet been reconstructed. This may also occur when inverse DST is employed as part of the corresponding transport block. The method provides greater flexibility for encoding information closest to the reconstructed block, which reduces the corresponding residual information. The disclosed mechanism also supports context encoding of candidate position information for the transform block. The prediction information corresponding to the residual block can be used to encode the candidate position information. For example, in some cases, the residual block can correspond to a prediction block generated by a template matching mode. In addition, the template adopted by the template matching mode can be selected based on the spatially adjacent reconstructed area of ​​the residual block. In this case, the lower right portion of the residual block may include more residual information than other portions of the residual block. Therefore, the candidate position covering the lower right portion of the residual block is most likely to be selected as the best position for the transform. Therefore, when the residual block is associated with inter-frame prediction based on template matching, only one candidate position can be provided for the residual block, and / or other context encoding methods can be used to encode the position information for the transform.

[0101] Figure 4is a schematic diagram of an exemplary intra-frame prediction mode 400 adopted in video coding and decoding in the HEVC model. Video compression schemes take advantage of data redundancy. For example, most images include groups of pixels that include the same or similar colors and / or lights as adjacent pixels. As a particular example, an image of the night sky may include large areas of black pixels and clusters of white pixels depicting stars. Intra-frame prediction mode 400 takes advantage of these spatial relationships. Specifically, a frame can be decomposed into a series of blocks including samples. Then, the light / color of the block can be predicted based on the spatial relationship with the reference samples in the adjacent blocks, instead of transmitting each block. For example, the encoder can indicate that the current block includes the same data as the reference sample in the previously encoded block located at the upper left corner of the current block. Then, the encoder can encode the prediction mode instead of the value of the block. This significantly reduces the size of the code. As shown in intra-frame prediction mode 400, the upper left corner corresponds to prediction mode 18 in HEVC. Therefore, the encoder can only store prediction mode 18 for the block instead of encoding the pixel information of the block. As shown in the figure, the intra prediction mode 400 in HEVC includes thirty-three angular prediction modes from prediction mode 2 to prediction mode 34. The intra prediction mode 400 also includes intra plane mode 0 and intra direct current (DC) mode 1 for predicting smooth areas. The intra plane mode 0 predicts the block as an amplitude surface with vertical and horizontal slopes derived from adjacent reference samples. The intra DC mode 1 predicts the block as the average of adjacent reference samples. The intra prediction mode 400 can be used to send a signal to the brightness (e.g., light) component of the block. Intra prediction can also be applied to chrominance (e.g., color) values. In HEVC, chrominance values ​​are predicted by adopting plane mode, angle twenty-six mode (e.g., vertical), angle ten mode (e.g., horizontal), intra DC and derived mode, wherein the derived mode predicts the correlation between the chrominance component and the brightness component encoded by the intra prediction mode 400.

[0102] The intra-frame prediction mode 400 of the image block is stored by the encoder as prediction information. It should be noted that although the intra-frame prediction mode 400 is used for prediction in a single frame, inter-frame prediction can also be used. Inter-frame prediction takes advantage of the temporal redundancy between multiple frames. For example, a scene in a movie may include a relatively static background, such as a motionless table. Therefore, the table is described as a substantially identical set of pixels in multiple frames. Inter-frame prediction uses a block in a first frame to predict a block in a second frame, which in this example avoids the need to encode the table in each frame. Inter-frame prediction uses a block matching algorithm to match and compare blocks in the current frame with blocks in a reference frame. Then, a motion vector can be encoded to indicate the best matching block position in the reference frame and the co-location position of the current / target block. Therefore, a series of image frames can be represented as a series of blocks, which can then be represented as prediction blocks including prediction modes and / or motion vectors.

[0103] Figure 5 An example of intra prediction 500 in a video codec using an intra prediction mode such as intra prediction mode 400 is shown. As shown, a current block 501 can be predicted by samples in an adjacent block 510. The encoder can typically encode an image from top left to bottom right. However, in some cases, the encoder can encode from right to left, as discussed below. It should be noted that as used herein, right refers to the right side of the encoded image, left refers to the left side of the encoded image, top refers to the top side of the encoded image, and bottom refers to the bottom side of the encoded image.

[0104] It should be noted that the current block 501 may not always exactly match the samples from the neighboring block 510. In this case, the prediction mode is encoded from the neighboring block 510 that best matches. In order to allow the decoder to determine the appropriate value, the difference between the predicted value and the actual value is retained. This is called residual information. Residual information is present in both intra-frame prediction 500 and inter-frame prediction.

[0105] Figure 6 Schematic diagram of an exemplary video coding mechanism 600 based on intra prediction 500 and / or inter prediction. An image block 601 may be obtained by an encoder from one or more frames. For example, an image may be partitioned into a plurality of rectangular image regions. Each region of the image corresponds to a coding tree unit (CTU). The CTU is partitioned into a plurality of blocks, such as coding units in HEVC. Then, the block partitioning information is encoded in a bitstream 611. Therefore, the image block 601 is a partitioned portion of the image, including pixels representing a luminance component and / or a chrominance component at a corresponding portion of the image. During encoding, the image block 601 is encoded as a prediction block 603, wherein the prediction block 603 includes prediction information, such as a prediction mode for intra prediction (e.g., intra prediction mode 400) and / or a motion vector for inter prediction. Then, encoding the image block 601 as the prediction block 603 may leave a residual block 605, wherein the residual block 605 includes residual information indicating the difference between the prediction block 603 and the image block 601.

[0106] It should be noted that the image block 601 can be partitioned into coding units including one prediction block 603 and one residual block 605. The prediction block 603 can include all prediction samples of the coding unit, and the residual block 605 can include all residual samples of the coding unit. In this case, the prediction block 603 has the same size as the residual block 605. In another example, the image block 601 can be partitioned into coding units including two prediction blocks 603 and one residual block 605. In this case, each prediction block 603 includes a portion of the prediction samples of the coding unit, and the residual block 605 includes all residual samples of the coding unit. In yet another example, the image block 601 is partitioned into coding units including two prediction blocks 603 and four residual blocks 605. The partitioning mode of the residual block 605 in the coding unit can be signaled in the bitstream 611. Such a position mode can include a residual quadtree (RQT) in HEVC. In addition, the image block 601 can include only the brightness component (e.g., light) of the image sample (or pixel), represented as the Y component. In other cases, image block 601 may include Y, U, and V components of image samples, where U and V indicate chrominance components (eg, colors) in a blue-luminance and red-luminance color space (UV).

[0107] SVT can be used to further compress information. Specifically, SVT uses a transform block 607 to further compress the residual block 605. The transform block 607 includes a transform, such as an inverse DCT and / or an inverse DST. The difference between the prediction block 603 and the image block 601 is adapted to the transform by using transform coefficients. By indicating the transform mode (e.g., inverse DCT and / or inverse DST) and the corresponding transform coefficients of the transform block 607, the decoder can reconstruct the residual block 605. When accurate reproduction is not required, the transform coefficients can be further compressed by rounding specific values ​​to create a better fit to the transform. This process is called quantization and is performed according to a quantization parameter that describes the quantization allowed. Therefore, the transform mode, transform coefficients, and quantization parameters of the transform block 607 are stored as transformed residual information in the transform residual block 609, which in some cases may also be simply referred to as a residual block.

[0108] The prediction information of the prediction block 603 and the transformed residual information of the transformed residual block 609 may then be encoded in a bitstream 611. The bitstream 611 may be stored and / or transmitted to a decoder. The decoder may then perform the reverse process to recover the image block 601. Specifically, the decoder may employ the transformed residual information to determine the transformed block 607. The transformed block 607 may then be used in conjunction with the transformed residual block 609 to determine the residual block 605. The residual block 605 and the prediction block 603 may then be employed to reconstruct the image block 601. The image block 601 may then be positioned relative to other decoded image blocks 601 to reconstruct a frame and to position such a frame to recover the encoded video.

[0109] SVT will now be described in further detail. To perform SVT, a transform block 607 that is smaller than the residual block 605 is selected. The transform block 607 is used to transform the corresponding portion of the residual block 605, and the remaining portion of the residual block 605 does not need to be additionally encoded / compressed. This is because the residual information is usually unevenly distributed in the residual block 605. SVT uses a smaller transform block 607 with an adaptive position to capture most of the residual information in the residual block 605 without transforming the entire residual block 605. This method can achieve higher coding efficiency than transforming all the residual information in the residual block 605. Since the transform block 607 is smaller than the residual block 605, SVT uses a mechanism for signaling the position of the transform relative to the residual block 605. For example, when SVT is applied to a residual block 605 of size w×h (e.g., width multiplied by height), the size and position information of the transform block 607 can be encoded into the bitstream 611. This allows the decoder to reconstruct the transform block 607 and combine the transform block 607 into the correct position relative to the transform residual block 609 to reconstruct the residual block 605.

[0110] It should be noted that some prediction blocks 603 may be encoded without generating residual blocks 605. However, this situation does not result in the use of SVT and is therefore not discussed further. As described above, SVT may be used for either inter-prediction blocks or intra-prediction blocks. In addition, SVT may be used for residual blocks 605 generated by a specified inter-prediction mechanism (e.g., motion compensation based on a translation model), but may not be used for residual blocks 605 generated by other specified inter-prediction mechanisms (e.g., affine model motion compensation).

[0111] Figure 7 An SVT example 700 including a transform block 707 and a residual block 705 is shown. Figure 7 The transform block 707 and the residual block 705 are similar to Figure 6 The transformed block 607 and the residual block 605 are shown in FIG. 7. For ease of reference, the SVT examples 700 are referred to as SVT-I, SVT-II, and SVT-III.

[0112] SVT-I is described as w_t=w / 2, h_t=h / 2, where w_t and h_t represent the width and height of the transform block 707, respectively, and w and h represent the width and height of the residual block 705, respectively. For example, the width and height of the transform block 707 are both half the width and height of the residual block 705. SVT-II is described as w_t=w / 4, h_t=h, where the variables are as described above. For example, the width of the transform block 707 is one-fourth the width of the residual block 705, and the height of the transform block 707 is equal to the height of the residual block 705. SVT-III is described as w_t=w, h_t=h / 4, where the variables are as described above. For example, the width of the transform block 707 is equal to the width of the residual block 705, and the height of the transform block 707 is equal to one-fourth the height of the residual block 705. Type information indicating the SVT type (e.g., SVT-I, SVT-II, or SVT-III) is encoded into the bitstream to support reconstruction by the decoder.

[0113] like Figure 7 As shown, each transform block 707 can be positioned at various positions relative to the residual block 705. The position of the transform block 707 is represented by a position offset (x, y) to the upper left corner of the residual block 705, where x indicates the horizontal distance between the upper left corner of the transform block 707 and the upper left corner of the residual block 705 in pixels, and y indicates the vertical distance between the upper left corner of the transform block 707 and the upper left corner of the residual block 705 in pixels. Each potential position of the transform block 707 within the residual block 705 is called a candidate position. For the residual block 705, the number of candidate positions for one type of SVT is (w–w_t+1)×(h–h_t+1). More specifically, for 16×16 residual blocks 705, when SVT-I is used, there are eighty-one candidate positions. When SVT-II or SVT-III is used, there are thirteen candidate positions. Once determined, the x and y values ​​of the position offset are encoded into the bitstream along with the adopted SVT block type. In order to reduce the complexity of SVT-I, a subset of thirty-two positions can be selected from the eighty-one possible candidate positions. This subset is then used as the allowed candidate positions for SVT-I.

[0114] One disadvantage of the SVT scheme using one of the SVT examples 700 is that encoding the SVT position information as residual information results in significant signaling overhead. In addition, the encoder complexity can increase significantly as the number of positions tested by compression quality processes such as rate-distortion optimization (RDO) increases. Since the number of candidate positions increases with the size of the residual block 705, the signaling overhead is even greater for larger residual blocks 705, such as 32×32 or 64×128. Another disadvantage of using one of the SVT examples 700 is that the size of the transform block 707 is one-fourth the size of the residual block 705. In many cases, a transform block 707 of this size may not be large enough to cover the main residual information in the residual block 705.

[0115] Figure 8 An additional SVT example 800 including a transform block 807 and a residual block 805 is shown. Figure 8 The transform block 807 and the residual block 805 are similar to Figure 6-Figure 7 707 and residual blocks 605, 705. For ease of reference, SVT example 800 is referred to as SVT vertical (SVT-V) and SVT horizontal (SVT-H). SVT example 800 is similar to SVT example 700, but is intended to support reduced signaling overhead and reduced complex processing requirements for the encoder.

[0116] SVT-V is described as w_t=w / 2 and h_t=h, where the variables are as described above. The width of the transform block 807 is half the width of the residual block 805, and the height of the transform block 807 is equal to the height of the residual block 805. SVT-H is described as w_t=w and h_t=h / 2, where the variables are as described above. For example, the width of the transform block 807 is equal to the width of the residual block 805, and the height of the transform block 807 is half the height of the residual block 805. SVT-V is similar to SVT-II, and SVT-H is similar to SVT-III. Compared with SVT-II and SVT-III, the transform block 807 in SVT-V and SVT-H is enlarged to half of the residual block 805, so that the transform block 807 covers more residual information in the residual block 805.

[0117] As with SVT example 700, SVT example 800 may include several candidate positions, where the candidate positions are possible allowed positions of a transform block (e.g., transform block 807) relative to a residual block (e.g., residual block 805). The candidate positions are determined based on a candidate position step size (CPSS). The candidate positions may be spaced at equal intervals specified by the CPSS. In this case, the number of candidate positions is reduced to no more than 5. The reduction in the number of candidate positions alleviates the signaling overhead associated with position information because fewer bits can be used to signal the selected position for the transform. In addition, reducing the number of candidate positions makes it algorithmically simpler to select the transform position, thereby allowing the complexity of the encoder to be reduced (e.g., resulting in fewer computational resources for encoding).

[0118] Fig. 9 An SVT example 900 including a transform block 907 and a residual block 905 is shown. Fig. 9 The transform block 907 and the residual block 905 are similar to Figure 6-Figure 8 The transformed blocks 607, 707, 807 and the residual blocks 605, 705, 805 are shown in FIG. Fig. 9 Various candidate positions are shown, where the candidate positions are possible allowed positions of a transform block (eg, transform block 907) relative to a residual block (eg, residual block 905). Specifically, Fig. 9 Middle (A)- Fig. 9 The SVT example in (E) uses SVT-V. Fig. 9 Middle (F)- Fig. 9 The SVT example in (J) adopts SVT-H. The allowed candidate positions of the transform depend on the CPSS, which further depends on the portion of the residual block 905 that the transform block 907 should cover and / or the step size between the candidate positions. For example, for SVT-V, the CPSS can be calculated as s=w / M1, or for SVT-H, the CPSS can be calculated as s=h / M2, where w and h are the width and height of the residual block, respectively, and M1 and M2 are predetermined integers in the range of 2 to 8. The larger the value of M1 or M2, the more candidate positions there are. For example, both M1 and M2 can be set to 8. In this case, the value of the position index (P) describing the position of the transform block 907 relative to the residual block 905 is between 0 and 4.

[0119] In another example, for SVT-V, CPSS is calculated as s=max(w / M1, Th1), and for SVT-H, CPSS is calculated as s=max(h / M2, Th2), where Th1 and Th2 are predefined integers specifying the minimum step size. Th1 and Th2 may be integers not less than 2. In this example, Th1 and Th2 are set to 4, M1 and M2 are set to 8, and different block sizes may have different numbers of candidate positions. For example, when the width of the residual block 905 is 8, two candidate positions may be used for SVT-V, specifically Fig. 9 Middle (A) and Fig. 9 For example, when the step size is larger as indicated by Th1 and the portion of the residual block 905 covered by the transform block 907 is also larger as indicated by w / M1, only two candidate positions satisfy the CPSS. However, when w is set to 16, the portion of the residual block 905 covered by the transform block 907 decreases due to the change in w / M1. This generates more candidate positions, in which case Fig. 9 Middle (A), Fig. 9 Middle (C) and Fig. 9 When the width of the residual block 905 is greater than 16 and the values ​​of Th1 and M1 are as discussed above, Fig. 9 Middle (A)- Fig. 9 All five candidate positions shown in (E) are available.

[0120] Other examples can be seen when CPSS is calculated according to other mechanisms. Specifically, for SVT-V, CPSS can be calculated as s=w / M1; or for SVT-H, CPSS can be calculated as s=h / M2. In this case, when M1 and M2 are set to 4, SVT-V allows 3 candidate positions (for example, Fig. 9 Middle (A), Fig. 9 Middle (C) and Fig. 9 ), SVT-H allows 3 candidate positions (e.g., Fig. 9 Middle (F), Fig. 9 Middle (H) and Fig. 9 In addition, when M1 and M2 are set to 4, the portion of the residual block 905 covered by the transform block 907 increases, so that SVT-V has two allowed candidate positions (e.g., Fig. 9 Middle (A) and Fig. 9 The candidate position in (E)) and SVT-H has two allowed candidate positions (e.g., Fig. 9 Middle (F) and Fig. 9 (j) in the candidate position.

[0121] In another example, as discussed above, for SVT-V, the CPSS is calculated as s = max(w / M1, Th1) or for SVT-H, the CPSS is calculated as s = max(h / M2, Th2). In this case, T1 and T2 are set to predefined integers (e.g., 2), M1 is set to 8 when w ≥ h; M1 is set to 4 when w < h. M2 is set to 8 when h ≥ w; or M2 is set to 4 when h < w. For example, the portion of the residual block 905 covered by the transform block 907 depends on whether the height of the residual block 905 is greater than the width of the residual block 905, or whether the width of the residual block 905 is greater than the height of the residual block 905. Therefore, the number of candidate positions for SVT-H or SVT-V also depends on the aspect ratio of the residual block 905.

[0122] In another example, as discussed above, for SVT-V, the CPSS is calculated as s = max(w / M1, Th1) or for SVT-H, the CPSS is calculated as s = max(h / M2, Th2). In this case, the values of M1, M2, Th1, and Th2 are derived from a high-level syntax structure (e.g., the sequence parameter set) in the bitstream. For example, the values used to derive the CPSS can be signaled in the bitstream. M1 and M2 may share the same value parsed from a syntax element, and Th1 and Th2 may share the same value parsed from another syntax element.

[0123] Fig.10 An exemplary SVT position 1000 describing the position of the transform block 1007 relative to the residual block 1005 is shown. Although six different positions are shown (e.g., three vertical positions and three horizontal positions), it should be understood that a different number of positions may be used in actual applications. The SVT transform position 1000 is selected from the candidate positions in the SVT example 900 of Fig. 9 . Specifically, the selected SVT transform position 1000 can be encoded into a position index (P). The position index P can be used to determine the position offset (Z) of the upper left corner of the transform block relative to the upper left corner of the residual block. For example, this position correlation can be determined according to Z = s × P, where s is the CPSS of the transform block based on the SVT type and is calculated as discussed with respect to Fig. 9 . When the transform block is of the SVT-V type, the value of P can be encoded as 0, 1,... When the transform block is of the SVT-H type, the value of P can be encoded as 0, 1,... More specifically, (0, 0) can represent the coordinates of the upper left corner of the residual block. In this case, the coordinates of the upper left corner of the transform block are (Z, 0) for SVT-V and (0, Z) for SVT-H.

[0124] As discussed in further detail below, the encoder can encode the SVT transform type (e.g., SVT-H or SVT-T) and the residual block size in the bitstream by using a flag. The decoder can then determine the SVT transform size based on the SVT transform size and the residual block size. Once the SVT transform size is determined, the decoder can determine the allowed candidate positions for the SVT transform based on the CPSS function, such as Fig. 9 The candidate positions in the SVT example 900 of FIG. 900. Since the decoder is able to determine the candidate positions for the SVT transform, the encoder may not signal the coordinates of the position offset. Instead, a code may be used to indicate which candidate positions are used for the corresponding transform. For example, the position index P can be binarized into one or more binary bits using a truncated unary code to increase compression. As a specific example, when the P value is in the range of 0 to 4, the P values ​​0, 4, 2, 3, and 1 can be binarized to 0, 01, 001, 0001, and 0000, respectively. This binary code is more compressed than the base ten values ​​representing the position index. As another example, when the P value is in the range of 0 to 1, the P values ​​0 and 1 can be binarized to 0 and 1, respectively. Therefore, the size of the position index can be increased or decreased as needed to signal a specific transform block position based on the possible candidate positions of the transform block.

[0125] The position index P can be binarized into one or more binary bits by adopting the most likely position and the remaining position with less possibility. For example, when the left adjacent block and the upper adjacent block have been decoded at the decoder and are therefore available for prediction, the most likely position can be set to the position covering the lower right corner of the residual block. In one example, when the P value is in the range of 0 to 4 and position 4 is set to the most likely position, the P values ​​4, 0, 1, 2 and 3 are binarized to 1, 000, 001, 010 and 011, respectively. In addition, when the P value is in the range of 0 to 2 and position 2 is set to the most likely position, the P values ​​2, 0 and 1 are binarized to 1, 01 and 00, respectively. Therefore, the most likely position index of the candidate position is represented by the least bits to reduce the signaling overhead in the most common case. The probability can be determined based on the coding order of the adjacent reconstructed blocks. Therefore, the decoder can infer the codeword scheme to be used for the corresponding block based on the adopted decoding scheme.

[0126] For example, in HEVC, the coding unit coding order is usually from top to bottom and from left to right. In this case, the right side of the current encoding / decoding coding unit is not available, making the upper right corner more likely to be the transformation position. However, the motion vector prediction value is derived from the left and upper spatial neighbors. In this case, the residual information is statistically more dense toward the lower right corner. In this case, the candidate position covering the lower right part is the most likely position. In addition, when the adaptive coding unit coding order is utilized, a node can be split vertically into two child nodes, and the right child node can be encoded before the left child node is encoded. In this case, the right neighbor of the left child node has been reconstructed before the left child node is decoded / encoded. In addition, in this case, the left neighboring pixels are not available. When the right neighboring pixels are available and the left neighboring pixels are not available, the lower left part of the residual block may include a large amount of residual information, and therefore, the candidate position covering the lower left part of the residual block becomes the most likely position.

[0127] Therefore, the position index P can be binarized into one or more binary bits depending on whether the right side of the adjacent residual block has been reconstructed. In one example, the P value is in the range of 0 to 2, as shown in SVT transform position 1000. When the right side of the adjacent residual block has been reconstructed, the P values ​​0, 2 and 1 are binarized into 0, 01 and 00. Otherwise, the P values ​​2, 0 and 1 are binarized into 0, 01 and 00. In another example, when the right side of the adjacent residual block has been reconstructed, but the left side of the adjacent residual block has not been reconstructed, the P values ​​0, 2 and 1 are binarized into 0, 00 and 01. Otherwise, the P values ​​2, 0 and 1 are binarized into 0, 00 and 01. In these examples, the position corresponding to a single binary bit is the most likely position, and the other two positions are the remaining positions. For example, the most likely position depends on the availability of the adjacent residual block on the right.

[0128] In terms of rate-distortion performance, the probability distribution of the best position may be very different in each inter-frame prediction mode. For example, when the residual block corresponds to a prediction block generated by template matching with spatially adjacent reconstructed pixels as a template, the best position is most likely position 2. For other inter-frame prediction modes, the probability that position 2 (or position 0 when the right neighbor is available and the left neighbor is not available) is the best position is lower than the template matching mode. In view of this, the context model of the first binary bit of the position index P can be determined according to the inter-frame prediction mode associated with the residual block. More specifically, when the residual block is associated with inter-frame prediction based on template matching, the first binary bit of the position index P uses the first context model. Otherwise, the second context model is used to encode / decode the binary bit.

[0129] In another example, when the residual block is associated with inter prediction based on template matching, the most likely position (e.g., position 2, or position 0 when the right neighbor is available but the left neighbor is not available) is directly set to the transform block position, and the position information is not signaled in the bitstream. Otherwise, the position index is explicitly signaled in the bitstream.

[0130] It should also be noted that different transforms can be used depending on the position of the transform block relative to the residual block. For example, the left side of the residual block is reconstructed without reconstructing the right side of the residual block, which occurs in video codecs with a fixed coding unit coding order from left to right and from top to bottom (e.g., the coding order in HEVC). In this case, the candidate position covering the lower right corner of the residual block can use DST (e.g., DST version 7 (DST-7) or DST version 1 (DST-1)) for the transform in the transform block when encoding. Therefore, an inverse DST transform is used for the corresponding candidate position at the decoder. In addition, the candidate position covering the upper left corner of the residual block can use DCT (e.g., DCT version 8 (DCT-8) or DCT version 2 (DCT-2)) for the transform in the transform block when encoding. Therefore, an inverse DCT transform is used for the corresponding candidate position at the decoder. This is because in this case, among the four corners, the lower right corner is farthest from the spatial reconstruction area. In addition, when the transform block covers the lower right corner of the residual block, DST transforms the residual information distribution more effectively than DCT. In addition, when the transform block covers the upper left corner of the residual block, DCT transforms the residual information distribution more effectively than DST. For the remaining candidate positions, the transform type can be inverse DST or inverse DCT. For example, when the candidate position is closer to the lower right corner than the upper left corner, inverse DST is used as the transform type. Otherwise, inverse DCT is used as the transform type.

[0131] As a specific example, three candidate positions of the transform block 1007 may be allowed, such as Fig.10 As shown. In this case, position 0 covers the upper left corner and position 2 covers the lower right corner. Position 1 is located in the middle of the residual block 1005, equidistant from the left and right corners. At the encoder, the transform types of position 0, position 1, and position 2 can be selected as DCT-8, DST-7, and DST-7, respectively. Then, inverse transforms DCT-8, DST-7, and DST-7 can be used for position 0, position 1, and position 2 at the decoder, respectively. In another example, at the encoder, the transform types of position 0, position 1, and position 2 are DCT-2, DCT-2, and DST-7, respectively. Then, inverse transforms DCT-2, DCT-2, and DST-7 can be used for position 0, position 1, and position 2 at the decoder, respectively. Therefore, the transform type corresponding to the candidate position can be predetermined.

[0132] In some cases, the above position-dependent multiple transforms may be applied only to the luma transform block. The corresponding chroma transform block may always use inverse DCT-2 during the transform / inverse transform process.

[0133] Fig.11 An example 1100 of horizontal flipping of residual samples is shown. In some cases, beneficial residual compression can be achieved by horizontally flipping the residual information in a residual block (e.g., residual block 605) before applying a transform block (e.g., transform block 607) at an encoder. Example 1100 shows such a horizontal flip. In this context, horizontal flipping refers to the rotation of the residual samples in the residual block by half around an axis between the left side of the residual block and the right side of the residual block. This horizontal flipping occurs before applying a transform (e.g., a transform block) at the encoder and after applying an inverse transform (e.g., a transform block) at the decoder. This flipping can be employed when specified predefined conditions occur.

[0134] In one example, horizontal flipping occurs when the transform block adopts DST / inverse DST during the transform process. In this case, the right adjacent residual block of the residual block is encoded / reconstructed before the current block, and the left adjacent residual block is not encoded / reconstructed before the current block. The horizontal flipping process exchanges the residual samples at the i-th column of the residual block with the residual samples at the w–1–i-th column of the residual block. In this context, w is the width of the transform block, i=0, 1,..., (w / 2)–1. Horizontal flipping of residual samples can improve coding efficiency by making the residual distribution better fit the DST transform.

[0135] Fig.12 1 is a flow chart of an exemplary method 1200 for video decoding using position-dependent SVT using the mechanisms discussed above. Method 1200 may be initiated at a decoder after receiving a bitstream (e.g., bitstream 611). Method 1200 uses the bitstream to determine prediction blocks and transform residual blocks, such as prediction block 603 and transform residual block 609. Method 1200 also determines a transform block (e.g., transform block 607), wherein transform block 607 is used to determine a residual block (e.g., residual block 605). Residual block 605 and prediction block 603 are then used to reconstruct an image block, such as image block 601. It should be noted that while method 1200 is described from the perspective of a decoder, a similar method (e.g., in reverse) may be used to encode video using SVT.

[0136] In block 1201, a bitstream is obtained at a decoder. The bitstream may be received from a memory or from a streaming source. The bitstream includes data that may be decoded into at least one image corresponding to video data from an encoder. Specifically, the bitstream includes block segmentation information, wherein the block segmentation information may be used to determine a coding unit, wherein the coding unit includes a prediction block and a residual block in the bitstream as described in mechanism 600. Thus, coding information associated with the coding unit may be parsed from the bitstream, and pixels of the coding unit may be reconstructed based on the coding information as discussed below.

[0137] In block 1203, a prediction block and a corresponding transformed residual block are obtained from the bitstream based on the block partition information. For this example, the transformed residual block has been encoded according to SVT as discussed above with respect to mechanism 600. The method 1200 then reconstructs a residual block of size w×h from the transformed residual block, as discussed below.

[0138] In block 1205, determine the use of SVT, SVT type, and transform block size. For example, the decoder first determines whether SVT has been used in the encoding. This is because the transform used by some encodings is the size of the residual block. The use of SVT can be signaled by a syntax element in the bitstream. Specifically, when the residual block is allowed to use SVT, a flag such as svt_flag is parsed from the bitstream. When the transformed residual block has non-zero transform coefficients (e.g., corresponding to any luminance component or chrominance component), the residual block is allowed to use SVT. For example, when the residual block includes any residual data, the residual block can use SVT. The SVT flag indicates whether the residual block is encoded using a transform block of the same size as the residual block (e.g., svt_flag is set to 0) or a transform block smaller than the residual block is used to encode the residual block (e.g., svt_flag is set to 1). A coded block flag (cbf) can be used to indicate whether the residual block includes non-zero transform coefficients of color components, such as used in HEVC. In addition, a root coded block (root cbf) flag may indicate whether the residual block includes non-zero transform coefficients for any color component, as used in HEVC. As a specific example, when an image block is predicted using inter-frame prediction and the residual block width or residual block height falls within a predetermined range of [a1, a2], the residual block is allowed to use SVT, where a1=16 and a2=64, or a1=8 and a2=64, or a1=16 and a2=128. The values ​​of a1 and a2 may be predetermined fixed values. These values ​​may also be derived from a sequence parameter set (SPS) or slice header in the bitstream. When the residual block does not use SVT, the transform block size is set to the width and height of the residual block size. Otherwise, the transform size is determined based on the SVT transform type.

[0139] Once the decoder determines that SVT has been used for the residual block, the decoder determines the type of SVT transform block used and derives the transform block size based on the SVT type. The allowed SVT types for the residual block are determined based on the width and height of the residual block. If the width of the residual block is within the range [a1, a2] as defined above, then the following values ​​are allowed: Figure 8 When the height of the residual block is within the range [a1, a2] defined above, then the following is allowed: Figure 8 The SVT-H transform shown. SVT can be used only for the luma component in the residual block, or SVT can be used for both the luma component and the chroma component in the residual block. When SVT is used only for the luma component, the luma component residual information is converted by SVT, and the chroma component is converted by converting the size of the residual block. When both SVT-V and SVT-H are allowed, a flag (such as svt_type_flag) can be encoded into the bitstream. svt_type_flag indicates whether SVT-V is used for the residual block (e.g., svt_type_flag is set to 0) or SVT-H is used for the residual block (e.g., svt_type_flag is set to 1). Once the type of SVT transform is determined, the transform block size is set according to the signaled SVT type (e.g., for SVT-V, w_t=w / 2 and h_t=h; for SVT-H, w_t=w and h_t=h / 2). When only SVT-V or only SVT-H is allowed, svt_type_flag may not be encoded into the bitstream. In this case, the decoder may infer the transform block size based on the allowed SVT types.

[0140] Once the SVT type and size are determined, the decoder proceeds to block 1207. In block 1207, the decoder determines the position of the transform relative to the residual block and the transform type (e.g., DST or DCT). The position of the transform block may be determined based on syntax elements in the bitstream. For example, in some examples, the position index may be directly signaled and thus parsed from the bitstream. In other examples, the position index may be directly signaled and thus parsed from the bitstream. Figure 8-Figure 10 Specifically, the candidate position of the transform can be determined according to the CPSS function. The CPSS function can determine the candidate position by considering the width of the residual block, the height of the residual block, the SVT type determined by block 1205, the step size of the transform, and / or the portion of the residual block covered by the transform. Then, the decoder can determine the transform block position according to the candidate position by obtaining the p index, wherein the p index includes the position of the transform block according to the above description of the residual block. Fig.10 The discussed candidate position selection probability signals the code for the correct candidate position. Once the transform block position is known, the decoder can infer the transform type used by the transform block, as described above with respect to Fig.10 Accordingly, the encoder can select the corresponding inverse transform.

[0141] In block 1209, the decoder parses the transform coefficients of the transform block based on the transform block size determined in block 1205. This process may be done according to the transform coefficient parsing mechanism employed in HEVC, H.264, and / or AVC. The transform coefficients may be encoded using run-length coding and / or as a set of transform coefficient groups (CGs). It should be noted that in some examples, block 1209 may be performed before block 1207.

[0142] In box 1211, the residual block is reconstructed based on the transform position, transform coefficient and transform type determined as above. Specifically, inverse quantization and inverse transform of size w_t×h_t are applied to the transform coefficients to recover the residual samples of the residual block. The size of the residual block with residual samples is w_t×h_t. According to the position-dependent transform type determined in box 1207, the inverse transform can be an inverse DCT or an inverse DST. According to the transform block position, the residual samples are assigned to corresponding areas within the residual block. Any residual samples within the residual block and outside the transform block can be set to zero. For example, when SVT-V is adopted, the number of candidate positions is 5, and the position index indicates the fifth transform block position. Therefore, the reconstructed residual samples are assigned to Fig. 9 The SVT example 900 transforms the region in the candidate position (eg, Fig. 9 The shaded area in the figure) and the area to the left of the area with a size of (w / 2)×h with zero residual samples ( Fig. 9 ).

[0143] In optional block 1213, the residual block information of the reconstructed block may be horizontally flipped, such as Fig.11 As discussed above, this may occur when the transform block at the decoder uses inverse DST, the right neighboring block has been reconstructed, and the left neighboring block has not yet been reconstructed. Specifically, in the above case, the encoder can horizontally flip the residual block before applying the DST transform to improve coding efficiency. Therefore, optional block 1213 can be used to correct this horizontal flip at the encoder to create an accurate reconstructed block.

[0144] In block 1215, the reconstructed residual block may be combined with the prediction block to generate a reconstructed image block including samples as part of a coding unit. A filtering process may also be applied to the reconstructed samples, such as deblocking filtering and sample adaptive offset (SAO) processing in HEVC. The reconstructed image block may then be combined with other image blocks decoded in a similar manner to generate a frame of a media / video file. The reconstructed media file may then be displayed to a user on a monitor or other display device.

[0145] It should be noted that the reconstructed samples in the residual block may be generated in an equivalent implementation of the method 1200. Specifically, the residual samples of the transform block may be directly combined with the prediction block at the position indicated by the transform block position information without first restoring the residual block.

[0146] Fig.13 It is a method 1300 for video coding and decoding. The method 1300 can be implemented in a decoder (e.g., the video decoder 30). In particular, the method 1300 can be implemented by a processor of the decoder. The method 1300 can be implemented when a bitstream has been directly or indirectly received from an encoder (e.g., the video encoder 20) or retrieved from a memory. In box 1301, the bitstream is parsed to obtain a prediction block (e.g., the prediction block 603) and a transform residual block corresponding to the prediction block (e.g., the transform residual block 609). In box 1303, the SVT type used to generate the transform residual block is determined. As described above, the SVT type can be SVT-V or SVT-H. In one embodiment, the SVT-V type includes a height equal to the height of the transform residual block and a width that is half the width of the transform residual block.

[0147] In one embodiment, the SVT-H type includes a height that is half the height of the transformed residual block and a width that is equal to the width of the transformed residual block. In one embodiment, the svt_type_flag is parsed from the bitstream to determine the SVT type. In one embodiment, when only one type of SVT is allowed for the residual block, the SVT type is determined by inference.

[0148] In box 1305, the position of the SVT relative to the transformed residual block is determined. In one embodiment, the position index is parsed from the bitstream to determine the SVT type. In one embodiment, the position index includes a binary code, wherein the binary code indicates a position in a set of candidate positions determined according to the CPSS. In one embodiment, the most likely position of the SVT is assigned a minimum number of bits indicating the position index in the binary code. In one embodiment, when a single candidate position is available for SVT transformation, the processor infers the position of the SVT. In one embodiment, when the residual block is generated by template matching in inter-frame prediction mode, the processor infers the position of the SVT.

[0149] In block 1307, an inverse of the SVT is determined based on the location of the SVT. In block 1309, the inverse of the SVT is applied to the transformed residual block to produce a reconstructed residual block (e.g., residual block 605). In one embodiment, an inverse DST is used for an SVT-V type transform located at the left boundary of the residual block. In one embodiment, an inverse DST is used for an SVT-H type transform located at the top boundary of the residual block. In one embodiment, an inverse DCT is used for an SVT-V type transform located at the right boundary of the residual block. In one embodiment, an inverse DCT is used for an SVT-H type transform located at the bottom boundary of the residual block.

[0150] In block 1311, the reconstructed residual block is combined with the prediction block to reconstruct the image block. In one embodiment, the image block is displayed on a display or monitor of an electronic device (eg, a smartphone, tablet, laptop, personal computer, etc.).

[0151] Optionally, method 1300 may further include: when a right neighboring coding unit of the coding unit associated with the reconstructed residual block has been reconstructed and a left neighboring coding unit of the coding unit has not been reconstructed, horizontally flipping samples in the reconstructed residual block before combining the reconstructed residual block with the prediction block.

[0152] Fig.14 1400 is a method of video coding and decoding. The method 1400 can be implemented in an encoder (e.g., video encoder 20). In particular, the method 1400 can be implemented by a processor of the encoder. The method 1400 can be implemented to encode a video signal. In block 1401, a video signal is received from a video capture device (e.g., a camera). In one embodiment, the video signal includes an image block (e.g., image block 601).

[0153] In block 1403, a prediction block (e.g., prediction block 603) and a residual block (e.g., residual block 605) are generated to represent the image block. In block 1405, a transform algorithm is selected for the SVT based on the position of the SVT relative to the residual block. In block 1407, the residual block is converted into a transformed residual block using the selected SVT.

[0154] In block 1409, the type of SVT is encoded into the bitstream. In one embodiment, the type of SVT is an SVT-V type or an SVT-H type. In one embodiment, the SVT-V type includes a height equal to the height of the transform residual block and a width equal to half the width of the transform residual block. In one embodiment, the SVT-H type includes a height equal to half the height of the transform residual block and a width equal to the width of the transform residual block.

[0155] In block 1411, the position of the SVT is encoded into the bitstream. In one embodiment, the position of the SVT is encoded in a position index. In one embodiment, the position index comprises a binary code, wherein the binary code indicates a position in a set of candidate positions determined according to the CPSS. In one embodiment, the most likely position of the SVT is assigned a minimum number of bits in the binary code indicating the position index.

[0156] In one embodiment, the processor uses the DST algorithm for the SVT-V type transform located at the left boundary of the residual block. In one embodiment, the processor selects the DST algorithm for the SVT-H type transform located at the top boundary of the residual block. In one embodiment, the processor selects the DCT algorithm for the SVT-V type transform located at the right boundary of the residual block. In one embodiment, the processor selects the DCT algorithm for the SVT-H type transform located at the bottom boundary of the residual block.

[0157] Optionally, when a right neighboring coding unit of a coding unit associated with the residual block has been encoded and a left neighboring coding unit of the coding unit has not been encoded, the processor horizontally flips samples in the residual block before converting the residual block into the transformed residual block.

[0158] The prediction block and the transformed residual block are encoded into a bitstream in block 1413. In one embodiment, the bitstream is configured to be transmitted to a decoder and / or transmitted to a decoder.

[0159] Fig.15 1 is a schematic diagram of an exemplary computing device 1500 for video encoding and decoding according to an embodiment of the present invention. The computing device 1500 is suitable for implementing the disclosed embodiments described herein. The computing device 1500 includes: an input port 1520 and a receiver unit (Rx) 1510 for receiving data; a processor, a logic unit or a central processing unit (CPU) 1530 for processing data; a transmitter unit (Tx) 1540 and an output port 1550 for transmitting data; and a memory 1560 for storing data. The computing device 1500 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the input port 1520, a receiver unit 1510, a transmitter unit 1540, and an output port 1550 for optical or electrical signals to or from. In some examples, the computing device 1500 may also include a wireless transmitter and / or receiver.

[0160] The processor 1530 is implemented by hardware and software. The processor 1530 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1530 communicates with the input port 1520, the receiver unit 1510, the transmitter unit 1540, the output port 1550, and the memory 1560. The processor 1530 includes a coding / decoding module 1514. The coding / decoding module 1514 implements the above-disclosed embodiments, such as the method 1300 and the method 1400, and other mechanisms for encoding / reconstructing the residual block based on the transform block position when the SVT is adopted, and any other mechanisms described above. For example, the coding / decoding module 1514 implements, processes, prepares, or provides various coding operations, such as the encoded video data and / or decoded video data discussed above. Thus, including the encoding / decoding module 1514 provides significant improvements to the functionality of the computing device 1500 and enables transitions of the computing device 1500 to different states. Alternatively, the encoding / decoding module 1514 is implemented as instructions stored in the memory 1560 and executed by the processor 1530 (e.g., a computer program product stored on a non-transitory medium).

[0161] Memory 1560 includes one or more disks, tape drives, and solid-state hard drives, and can be used as an overflow data storage device to store such programs when such programs are selected for execution, and to store instructions and data read during program execution. Memory 1560 can be volatile and / or non-volatile, and can be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM). Computing device 1500 may also include input / output (I / O) devices for interacting with a terminal user. For example, computing device 1500 may include a display (such as a monitor) for visual output, a speaker for audio output, and a keyboard / mouse / trackball for user input.

[0162] In summary, the above disclosure includes a mechanism for adaptively adopting multiple transform types for transform blocks at different positions. In addition, the present invention allows horizontal flipping of residual samples in residual blocks to improve coding efficiency. This occurs when the transform block uses DST and inverse DST at the encoder and decoder, respectively, and when the right neighboring block is available but the left neighboring block is not available. In addition, the present invention includes a mechanism for supporting encoding position information in the bitstream based on the inter-frame prediction mode associated with the residual block.

[0163] Fig.16 1 is a schematic diagram of an embodiment of an encoding component 1600. In an embodiment, the encoding component 1600 is implemented in a video codec device 1602 (e.g., video encoder 20 or video decoder 30). The video codec device 1602 includes a receiving component 1601. The receiving component 1601 is used to receive an image for encoding or receive a bitstream for decoding. The video codec device 1602 includes a transmission component 1607 coupled to the receiving component 1601. The transmission component 1607 is used to transmit the bitstream to a decoder or transmit the decoded image to a display component (e.g., one of the I / O devices in the computing device 1500).

[0164] The video codec device 1602 includes a storage component 1603. The storage component 1603 is coupled to at least one of the receiving component 1601 or the transmitting component 1607. The storage component 1603 is used to store instructions. The video codec device 1602 also includes a processing component 1605. The processing component 1605 is coupled to the storage component 1603. The processing component 1605 is used to execute the instructions stored in the storage component 1603 to perform the method disclosed herein.

[0165] A first component is directly coupled to a second component when there are no intermediate components between the first component and the second component other than a line, trace, or other medium. A first component is indirectly coupled to a second component when there are intermediate components between the first component and the second component other than a line, trace, or other medium. The term "coupled" and its variations include direct coupling and indirect coupling. Unless otherwise specified, the use of the term "about" is intended to include a range of ±10% of the subsequent number.

[0166] Although several embodiments have been provided in the present invention, it will be appreciated that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present invention. The present examples are to be considered illustrative rather than restrictive, and the present invention is not limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain features may be omitted or not implemented.

[0167] In addition, without departing from the scope of the present invention, the techniques, systems, subsystems and methods described and illustrated as discrete or separate in the various embodiments may be combined or integrated with other systems, components, techniques or methods. Other items shown or discussed as coupled may be directly connected or may be indirectly coupled or communicated through an interface, device or intermediate component by electrical, mechanical or other means. Other examples of variations, substitutions and alterations may be determined by those skilled in the art and may be exemplified without departing from the spirit and scope disclosed herein.

Claims

1. A method implemented in a decoding device, characterized in that The method comprises: Receiving an encoded video bitstream, and performing entropy decoding on the video bitstream to obtain a quantization coefficient of a current block and a syntax element associated with the current block; Dequantizing the quantized coefficients; Predicting the current block according to the syntax element to obtain a predicted block; Determine a type of a spatially varying transform SVT used to generate a transform residual block, the type of the SVT being an SVT vertical (SVT-V) type or an SVT horizontal (SVT-H) type; the height corresponding to the SVT-V type being equal to the height of the transform residual block, and the width being half of the width of the transform residual block; the height corresponding to the SVT-H type being half of the height of the transform residual block, and the width being equal to the width of the transform residual block; Determining a position of the SVT relative to the transform residual block; Determining an inverse transform of the SVT based on the position of the SVT; wherein, when the position of the SVT covers the lower right corner of the transform residual block, the inverse transform of the SVT includes an inverse discrete sine transform DST7 version; when the position of the SVT covers the upper left corner of the transform residual block, the inverse transform of the SVT includes an inverse discrete sine transform DCT8 version; Applying an inverse transform of the SVT to the transformed residual block to generate a reconstructed residual block, the transformed residual block comprising inversely quantized quantized coefficients; The reconstructed residual block is combined with the prediction block to reconstruct an image block.

2. The method according to claim 1, characterized in that Also includes: Parse the svt_type_flag in the bitstream to determine the type of the SVT.

3. The method according to claim 1 or 2, characterized in that: Also includes: When the residual block is only allowed to use one type of SVT, the type of SVT is determined by inference.

4. The method according to claim 1 or 2, characterized in that: Also includes: The position index in the bitstream is parsed to determine the location of the SVT.

5. The method according to claim 1 or 2, characterized in that: The position index includes a binary code, wherein the binary code indicates a position in the candidate position set determined according to the candidate position step size CPSS.

6. The method according to claim 1 or 2, characterized in that: The most probable position of the SVT is assigned the minimum number of bits in the binary code indicating the position index.

7. The method according to claim 1 or 2, characterized in that: When a single candidate position is available for the SVT transform, the processor infers the position of the SVT.

8. The method according to claim 1 or 2, characterized in that: When the residual block is generated by template matching in inter-prediction mode, the processor infers the position of the SVT.

9. The method according to claim 1 or 2, characterized in that: The inverse discrete sine transform DST is applied to the SVT vertical SVT-V type transform located at the left boundary of the residual block.

10. The method according to claim 1 or 2, characterized in that: Inverse DST is applied to the SVT-horizontal (SVT-H) type transform located at the top boundary of the residual block.

11. The method according to claim 1 or 2, characterized in that: The inverse discrete cosine transform DCT is applied to the SVT-V type transform located at the right boundary of the residual block.

12. The method according to claim 1 or 2, characterized in that: An inverse DCT is applied to the SVT-H type transform located at the bottom boundary of the residual block.

13. A method implemented in an encoding device, characterized in that The method comprises: receiving a video signal, wherein the video signal comprises image blocks; Performing prediction processing on the current block to obtain a syntax element associated with the current block and a prediction block; Obtaining a residual block according to the prediction block and the current block; Selecting a transform algorithm for a spatial varying transform (SVT) based on a position of the SVT relative to the residual block; Converting the residual block into a transformed residual block using the selected SVT, wherein when the position of the SVT covers the lower right corner of the transformed residual block, the SVT includes a discrete sine transform DST7 version; when the position of the SVT covers the upper left corner of the transformed residual block, the SVT includes a discrete sine transform DCT8 version; quantizing the transformed residual block to obtain a quantized transformed residual block; The syntax element, the type of the SVT, the position of the SVT, and the quantized transform residual block are encoded into a bitstream for transmission to a decoder; the type of the SVT is an SVT vertical (SVT-V) type or an SVT horizontal (SVT-H) type; the height corresponding to the SVT-V type is equal to the height of the transform residual block, and the width is half of the width of the transform residual block; the height corresponding to the SVT-H type is half of the height of the transform residual block, and the width is equal to the width of the transform residual block.

14. The method according to claim 13, characterized in that The position of the SVT is encoded in the position index.

15. The method according to claim 13 or 14, characterized in that The position index includes a binary code, wherein the binary code indicates a position in a set of candidate positions determined according to a candidate position step size (CPSS).

16. The method according to claim 13 or 14, characterized in that The most probable position of the SVT is assigned the minimum number of bits in the binary code indicating the position index.

17. The method according to claim 13 or 14, characterized in that The processor applies a discrete sine transform (DST) algorithm to a SVT vertical (SVT-V) type transform located at the left boundary of the residual block.

18. The method according to claim 13 or 14, characterized in that The processor selects the DST algorithm for an SVT-horizontal (SVT-H) type transform located at a top boundary of the residual block.

19. The method according to claim 13 or 14, characterized in that The processor selects a discrete cosine transform (DST) algorithm for an SVT-V type transform located at the right boundary of the residual block.

20. The method according to claim 13 or 14, characterized in that The processor selects a DCT algorithm for an SVT-H type transform located at a bottom boundary of the residual block.

21. A coding and decoding device, characterized in that: include: A receiver for receiving an image for encoding or a bit stream for decoding; a transmitter coupled to the receiver, wherein the transmitter is used to transmit the bitstream to a decoder or to transmit the decoded image to a display; a memory coupled to at least one of the receiver or the transmitter, wherein the memory is configured to store instructions; A processor coupled to the memory, wherein the processor is configured to execute the instructions stored in the memory to perform the method according to any one of claims 1 to 20.

22. The encoding and decoding device according to claim 21, characterized in that: A display for displaying images is also included.

23. A system of a decoding device, characterized in that: include: Encoder; A decoder communicating with the encoder, wherein the encoder or the decoder comprises a coding and decoding device according to claim 21 or 22.

24. A video encoding and decoding device, characterized in that: The video coding and decoding device comprises a receiving component, including: The receiving component is used to receive an image for encoding or receive a bit stream for decoding; a transmission component coupled to the receiving component, wherein the transmission component is used to transmit the bit stream to a decoder or transmit the decoded image to a display component; a storage component coupled to at least one of the receiving component or the transmitting component, wherein the storage component is used to store instructions; A processing means coupled to the storage means, wherein the processing means is configured to execute the instructions stored in the storage means to perform the method according to any one of claims 1 to 20.

25. A non-volatile computer readable medium, characterized in that The non-volatile computer-readable medium includes a computer program product for use by a video decoding device; the computer program product includes computer-executable instructions stored in the non-volatile computer-readable medium; when a processor executes the computer-executable instructions, the video decoding device performs the method according to any one of claims 1 to 20.

26. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on a computer, cause the computer to execute the method implemented in an encoding device as claimed in any one of claims 13 to 20.

27. A method for storing a coded video bitstream, characterized in that: include: Receiving the video bitstream, wherein the video bitstream comprises a video bitstream generated by the method of any one of claims 13 to 20; The video bitstream is stored in a storage medium.

28. A system for storing a coded video bitstream, characterized in that: include: A receiving unit, configured to receive the video bitstream, wherein the video bitstream comprises a video bitstream generated by the method of any one of claims 13 to 20; A storage unit is used to store the video bit stream.

29. A method for transmitting a coded video bitstream, characterized in that: include: Retrieve the video bitstream from a storage medium, wherein the video bitstream comprises a video bitstream generated by the method of any one of claims 13 to 20; The video bit stream is sent to a decoding device.

30. A system for transmitting a coded video bitstream, characterized in that: include: an acquisition unit, configured to acquire the video bitstream from a storage medium, wherein the video bitstream comprises a video bitstream generated by the method according to any one of claims 13 to 20; The sending unit is used to send the video bit stream to a decoding device.

Images