Simultaneous transform coding of multiple color components

Abstract

To provide a transform scheme for joint coding of residuals from multiple color components.SOLUTION: A method and apparatus comprise computer code configured to cause a processor to perform: receiving video data in an AOMedia Video 1 (AV1) format comprising data of at least two chroma prediction-residual signal blocks; a transformation between at least one signal block, having a size less than or equal to a combination of the chroma prediction-residual signal blocks, and the chroma prediction-residual signal blocks; and decoding the video data based on an output of the transformation comprising the at least one signal block having the size less than or equal to the combination of the chroma prediction-residual blocks.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] CROSS REFERENCE TO RELATED APPLICATIONS / PRIORITY CLAIM This application claims priority to U.S. Provisional Application No. 63 / 000109, filed March 26, 2020, and U.S. Provisional Application No. 17 / 083662, filed October 29, 2020, the entireties of which are expressly incorporated by reference herein.

[0002] The present disclosure relates generally to the field of data processing, and more specifically to video encoding and / or decoding. [Background technology]

[0003] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 that includes semiconductor companies, video-on-demand providers, video content producers, software developers, and web browser vendors. AV1 has a total of 56 orientation angles, of which 8 are nominal angles and the rest are specified as deltas from the nominal angles.

[0004] Many of the components of the AV1 project were contributed by previous research efforts by members of the Alliance. Individual contributors have started experimental technology platforms years ago: Daala from Xip / Mozilla already published code in 2010, VP10, an evolution project of Google's experimental VP9, ​​was announced on September 12, 2014, and Thor from Cisco on August 11, 2015. Based on the VP9 code base, AV1 incorporates additional technologies, some of which were developed in these experimental formats. The first version of the AV1 reference codec, 0.1.0, was published on April 7, 2016. The Alliance announced the release of the AV1 bitstream specification on March 28, 2018, together with reference software-based encoders and decoders. The validated version 1.0.0 of the specification was released on June 25, 2018. The validated version 1.0.0 of the standard, including Errata 1, was released on January 8, 2019. The AV1 bitstream specification includes reference video codecs.

[0005] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3) and 2016 (version 4). Since then, they have been studying the potential need for standardization of future video coding technologies that could significantly exceed HEVC in compression capabilities. In October 2017, they published a Joint Call for Proposals (CfP) for video compression with capabilities beyond HEVC. By February 15, 2018, a total of 22 CfP responses had been submitted for Standard Dynamic Range (SDR), 12 for High Dynamic Range (HDR) and 12 for 360 video categories. In April 2018, 122 MPEG / 10th JVET (Joint Video Exploration Team - Joint Video At a meeting of the Expert Team, all received CfP responses were evaluated. After careful evaluation, JVET officially launched the standardization of the next generation video coding beyond HEVC, namely the so-called Versatile Video Coding (VVC).

[0006] In AV1, prediction residual signals are generated for chroma channels, e.g., Cb and Cr, by various means and applied in intra- and inter-coding scenarios, respectively. In most cases, the same set of prediction methods are applied to C and Cr channels, so the resulting residual signals from different chroma channels tend to be correlated in a certain way that can be classified. However, AV1 does not have a coding tool to remove such redundancies for better compression efficiency. On the other hand, the current VVC design provides one way to exploit such tendencies, but it is limited in the sense that it exploits the correlation only between two co-located chroma samples. Summary of the Invention [Means for solving the problem]

[0007] This disclosure presents a series of advanced video coding techniques. More specifically, a transform scheme for joint coding of residuals from multiple color components, such as from two chroma components, is disclosed.

[0008] Apparatus and methods are included that include a memory configured to store computer program code and a processor configured to access the computer program code and operate as instructed by the computer program code, the computer program including receiving code configured to cause at least one processor to receive video data in AOMedia Video 1 (AV1) format including data of at least two chroma prediction residual signal blocks, executing code configured to cause the at least one processor to transform between at least one signal block having a size equal to or smaller than the combination of the chroma prediction residual signal blocks and the chroma prediction residual signal block, and decoding code configured to cause the at least one processor to decode video data based on an output of the transform including at least one signal block having a size equal to or smaller than the combination of the chroma prediction residual blocks.

[0009] According to an example embodiment, the chroma prediction residual signal blocks include N1xN2 Cb residual blocks and N1xN2 Cr residual blocks.

[0010] According to an exemplary embodiment, the transformation includes inverse quantizing transmitted indices of coefficients of at least one signal block having a size equal to or smaller than the combination of chroma prediction residual signal blocks, and transforming the inverse quantized transmitted indices by a 2xN1xN2×LxM matrix, where 2xN1xN2 is the size of the combination of chroma prediction residual blocks and LxM is the size of the combination of chroma prediction residual blocks.

[0011] According to an example embodiment, a result of transforming the dequantized transmitted index with a 2xN1xN2×LxM matrix includes a reconstructed 2xN1xN2 vector, and decoding the video data includes reconstructing an N1xN2 Cb residual block and an N1xN2 Cr residual block from the reconstructed 2xN1xN2 vector.

[0012] According to an example embodiment, the at least one signal block includes an interleaving of at least two chroma prediction residual signal blocks.

[0013] According to an example embodiment, at least one signal block includes a 2xN1xN2 three-dimensional (3D) cubic.

[0014] According to an example embodiment, the transformation includes an N1-point 1D transformation for each of the N1x1 vectors along the x-axis of the 3D cubic, an N2-point 1D transformation for each of the N2x1 vectors along the y-axis of the 3D cubic, and a 2-point transformation for each of the 2x1 vectors along the z-axis of the 3D cubic.

[0015] According to an example embodiment, the transformation includes a two-point 1D transformation for each of the 2x1 vectors along the z-axis of the 3D cubic, and an N1xN2 transformation for the xy plane of the 3D cubic.

[0016] According to an example embodiment, performing the transformation includes performing a primary transformation and a secondary transformation.

[0017] According to an example embodiment, at least one of the primary transform and the secondary transform includes setting a coefficient to zero in response to determining a predetermined condition in which a value of a coordinate associated with at least one coefficient of the chroma prediction residual signal block is greater than or equal to a threshold value. [Brief description of the drawings]

[0018] These and other objects, features and advantages will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings, in which various features of the drawings are not drawn to scale for clarity purposes in facilitating understanding by those skilled in the art in connection with the detailed description. [Figure 1] FIG. 1 illustrates a networked computing environment in accordance with at least one embodiment. [Diagram 2]FIG. 2 is a diagram of the nominal angle of AV1 according to at least one embodiment. [Diagram 3] FIG. 3 is an operational flow diagram illustrating steps performed by a program for encoding video data, according to at least one embodiment. [Figure 4] FIG. 4 is a block diagram of internal and external components of the computers and servers shown in FIG. 1, according to at least one embodiment. [Diagram 5] FIG. 5 is a block diagram of an exemplary cloud computing environment including the computer system shown in FIG. 1 according to at least one embodiment. [Figure 6] FIG. 6 is a block diagram of functional layers of the example cloud computing environment of FIG. 5, according to at least one embodiment. [Figure 7] FIG. 7 is a block diagram of features for a non-directional smooth intra predictor in AVI according to an example embodiment. [Figure 8] FIG. 8 is a block diagram of features related to a recursive intra-filtering mode, according to an example embodiment. [Figure 9] FIG. 9 is a block diagram of features relating to a multi-layer reference frame structure, according to an example embodiment. [Figure 10] FIG. 10 is a block diagram of features relating to candidate motion vector list construction according to an example embodiment. [Figure 11] FIG. 11 is a diagram of features related to motion field estimation, according to an example embodiment. [Figure 12] FIG. 12 is a block diagram of overlap region related features for overlapped block motion compensation (OMBC) according to an example embodiment. [Figure 13] FIG. 13 is a block diagram of features relating to warp motion compensation, according to an example embodiment. [Figure 14] FIG. 14 is a block diagram of features related to AdTest composite prediction, according to an example embodiment. [Figure 15]FIG. 15 is a flow diagram of an aspect according to an example embodiment. [Figure 16] FIG. 16 is a spatial diagram of residual features consolidated as a 3D cube according to an example embodiment. [Figure 17] FIG. 17 is a block diagram of system features in accordance with an example embodiment. [Figure 18] FIG. 18 is a block diagram of features related to a decoder according to an example embodiment. [Figure 19] FIG. 19 is a block diagram of features related to an encoder according to an example embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Although detailed embodiments of the structures and methods of the present invention are disclosed herein, it is understood that the disclosed embodiments are merely exemplary of the structures and methods of the present invention, which may be embodied in various forms. However, these structures and methods may be embodied in many different forms and should not be construed as being limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope to those skilled in the art. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0020] FIELD OF THE DISCLOSURE Embodiments relate generally to the field of data processing, and more specifically to video encoding and decoding. Exemplary embodiments described below provide, among other things, systems, methods and computer programs for encoding and / or decoding video data using delta angle values ​​derived from nominal angle values. Thus, some embodiments have the ability to improve the field of calculations by not requiring the transmission of all delta angles and by allowing on-the-fly calculation of delta angle values.

[0021] As mentioned above, AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 that includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors. There are a total of 56 orientation angles in AV1, of which 8 are nominal angles and the rest are specified as deltas from the nominal angles. However, both the nominal and delta angles for all orientation modes are conveyed for the chroma components, regardless of the co-located luma prediction mode. In addition, although delta angles are allowed in both luma and chroma intra prediction modes, no correlation of delta angles between luma and chroma components is used. Therefore, instead of conveying all 56 angle values, it may be advantageous to derive delta angle values ​​for the chroma components from the luma components based on the nominal angles.

[0022] Aspects are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer-readable media according to various embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0023] Referring now to Figure 1, a functional block diagram of a networked computing environment illustrates a video coding system 100 (hereinafter "system") for encoding and / or decoding video data using delta angles derived from a nominal angle. It should be understood that Figure 1 is only intended to provide an illustration of one implementation and is not intended to imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the illustrated environment may be made based on design and implementation requirements.

[0024] The system 100 may include a computer 102 and a server computer 114. The computer 102 may communicate with the server computer 114 via a communications network 110 (hereinafter, "network"). The computer 102 may include a processor 104 and software programs 108 stored in a data storage device 106 that enable user interaction and communication with the server computer 114. As described below with reference to FIG. 4, the computer 102 may include internal components 800A and external components 900A, respectively, and the server computer 114 may include internal components 800B and external components 900B, respectively. The computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device that can run programs, access a network, and access a database.

[0025] The server computer 114 may also operate in a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), as described below with respect to Figures 6 and 7. The server computer 114 may be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0026] A server computer 114, which may be used to encode video data, may execute a video encoding program 116 (hereinafter "program") that may interact with database 112. The video encoding program method is described in more detail below with respect to FIG. 3. In one embodiment, computer 102 may operate as an input device, including a user interface, while program 116 may operate primarily on server computer 114. In an alternative embodiment, program 116 may operate primarily on one or more computers 102, while server computer 114 may be used for processing and storage of data used by program 116. It is noted that program 116 may be a stand-alone program or may be integrated into a larger video encoding program.

[0027] However, it should be noted that processing for the program 116 may possibly be shared in any ratio between the computer 102 and the server computer 114. In another embodiment, the program 116 may run on more than one computer, server computer, or some combination of computers and server computers, such as multiple computers 102 communicating with a single server computer 114 over the network 110. In another embodiment, for example, the program 116 may run on multiple server computers 114 communicating with multiple client computers over the network 110. Alternatively, the program may run on a network server communicating with the server and multiple client computers over the network.

[0028] The network 110 may include wired connections, wireless connections, fiber optic connections, or some combination thereof. In general, the network 110 may be any combination of connections and protocols that support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a dedicated network, an ad-hoc network, an intranet, an optical fiber-based network, etc., and / or a combination of these or other types of networks.

[0029] The number and arrangement of devices and networks shown in Figure 1 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different arrangements of devices and / or networks than those shown in Figure 1. Furthermore, two or more devices shown in Figure 1 may be implemented within a single device, or a single device shown in Figure 1 may be implemented as multiple distributed devices. Additionally or alternatively, a set of devices (e.g., one or more devices) of system 100 may perform one or more functions that are described as being performed by another set of devices of system 100.

[0030] Referring now to FIG. 2, a diagram 200 showing nominal angles for AV1 is shown. VP9 supports eight directional modes corresponding to angles from 45 degrees to 207 degrees. To exploit more diverse spatial redundancy in directional textures, AV1 extends the directional intra mode to a finer granularity set of angles. In AV1, there are eight nominal angles from 45 to 207 degrees, including V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED. For each nominal angle, there can be seven finer angles, such that AV1 can have a total of 56 directional angles. The prediction angle is represented by an angle delta in addition to the nominal intra angle, which is a step size of 3 degrees multiplied by -3 to 3. The delta angle does not need to be conveyed by the chroma components, but can be derived according to the corresponding luma intra prediction mode.

[0031] In AV1, eight nominal modes along with five non-angular smooth modes are signaled first, and then, if the current mode is an angular mode, an index is further signaled to indicate the angle delta to the corresponding nominal angle. To implement directional prediction modes in AV1 via a common method, all of the 56 directional intra prediction modes in AV1 are implemented with a unified directional predictor that projects each pixel to a reference sub-pixel location and interpolates the reference pixel with a 2-tap bilinear filter.

[0032] In one or more embodiments, if the current chroma intra prediction mode is a directional intra prediction mode and the nominal angle of the chroma intra prediction mode is equal to the nominal angle of the corresponding luma intra prediction mode, the delta angle of the chroma component may be set equal to the delta angle of the luma intra prediction mode. Otherwise, the delta angle of the chroma component may be set equal to 0.

[0033] In one or more embodiments, the delta angle of the chroma component may be set equal to the delta angle of the luma component regardless of whether the nominal modes for luma and chroma are the same or different.

[0034] In one or more embodiments, if the current chroma intra prediction mode is a directional intra prediction mode and the nominal angle of the chroma intra prediction mode is equal to the nominal angle of the corresponding luma intra prediction mode, the delta angle of the chroma component may be set equal to the delta angle of the luma intra prediction mode. Otherwise, if the nominal angle of the chroma intra prediction mode is equal to the left / top neighboring mode of the corresponding luma block, the delta angle of the chroma component may be set equal to the delta angle of the left / top neighboring mode of the corresponding luma block. Otherwise, the delta angle of the chroma component may be set equal to 0.

[0035] In one or more embodiments, when semi-decoupling splitting is applied, there may be multiple luma blocks associated with one chroma block. Thus, multiple sample positions may be predefined, and delta angles and nominal angles associated with these positions may be identified to predict the co-located luma block. One or more of these identified nominal and delta angles may be used to derive the delta angle of the current chroma block. In one example, the predefined sample positions may include the top left and / or center / middle sample. In another example, the predefined sample positions may include the four corner samples and the center / middle sample. In another example, the delta angle that may be most frequently used in the identified prediction mode may be used to derive the delta angle of the chroma component. In another example, the predefined sample positions may include two selected positions of the four corner samples and one center / middle sample. In another example, the predefined sample positions may include three selected positions of the four corner samples and one center / middle sample.

[0036] In one or more embodiments, if the current chroma intra prediction mode is a directional intra prediction mode and the nominal angle of the chroma intra prediction mode is equal to the nominal angle of the corresponding luma intra prediction mode, the delta angle of the chroma component may be set equal to the delta angle of the luma intra prediction mode. Otherwise, if the nominal angle of the chroma intra prediction mode is equal to the left / top neighboring mode of the current chroma block, the delta angle of the chroma component may be set equal to the delta angle of the left / top neighboring mode of the current chroma block. Otherwise, the delta angle of the chroma component may be set equal to 0.

[0037] The delta angle of the corresponding luma block may be used for entropy coding of the delta angle of the chroma intra prediction mode. In one or more embodiments, the delta angle of the co-located luma block may be used as a context for entropy coding of the delta angle of the chroma component. In one or more embodiments, the delta angle of the neighboring chroma block is used as a context for entropy coding of the delta angle of the chroma component. In one or more embodiments, instead of signaling the delta angle of the current chroma block, the absolute difference between the delta angle of the chroma block and the corresponding luma block may be signaled for entropy coding of the chroma intra prediction mode. In one or more embodiments, the above embodiment is applied only when the nominal modes between luma and chroma may be the same or the absolute difference of the prediction angle between these two modes is within a given threshold.

[0038] According to one or more embodiments, for entropy coding of chroma intra prediction modes, a first flag may be communicated to indicate whether the current mode is chroma from luma (CfL). If the first flag is communicated as a value indicating that CfL is not used, a second flag may be communicated to indicate whether the current mode is equal to the nominal mode of the corresponding luma block. If the current mode is equal to the nominal mode of the corresponding luma block, a third flag may be communicated to indicate an index of the delta angle if the current mode is a directional mode and delta angles are allowed. Otherwise, the third flag is communicated to indicate which of the remaining nominal modes is the current mode. If the first flag is communicated as a value indicating that CfL is not used, parameters of the CfL mode may be further communicated.

[0039] According to one or more embodiments, for entropy coding of chroma intra prediction modes, a first flag may be communicated to indicate whether the current mode is equal to the nominal mode or the CfL mode of the corresponding luma block. If the first flag is communicated as a value indicating that the current mode is equal to the nominal mode or the CfL mode of the corresponding luma block, a second flag may be communicated to indicate which of the two modes is the current mode. If the current mode is equal to the nominal mode of the corresponding luma block, an index of the delta angle may be further communicated if the current mode is a directional mode and a delta angle is allowed. If the current mode is a CfL mode, a parameter of the CfL mode may be further communicated. If the first flag is communicated as a value indicating that the current mode is not equal to the nominal mode or the CfL mode of the corresponding luma block, a second flag may be communicated to indicate which of the remaining nominal modes applies to the current chroma block.

[0040] In one or more embodiments, the first delta angle value may be entropy coded based on using the delta angle of the co-located luma block as a context, corresponding to the nominal angle of the chroma component and the nominal angle of the luma component being the same or close to each other. The nominal angles may be close to each other if the difference between the nominal angles is 2 degrees or less. The delta angle value may be entropy coded based on using the delta angle of the co-located luma block as a context or based on using the delta angle of the adjacent chroma block as a context.

[0041] It will be appreciated that in one or more embodiments, the delta angles of the luma and chroma blocks may use separate contexts for entropy coding instead of sharing the same context among the delta angles of the luma and chroma blocks.

[0042] 3, an operational flow chart illustrating steps of a method 300 for encoding and / or decoding video data is shown. In some implementations, one or more process blocks of FIG. 3 may be performed by computer 102 (FIG. 1) and server computer 114 (FIG. 1). In some implementations, one or more process blocks of FIG. 3 may be performed by another device or group of devices separate from or including computer 102 and server computer 114.

[0043] At 302, the method 300 includes receiving video data including (1) a chroma component having a first nominal angle and a first delta angle, and (2) a luma component having a second nominal angle and a second delta angle, where the first delta angle is dependent on the second delta angle.

[0044] At 304, the method 300 includes communicating a first delta angle value for the first delta angle based at least on an intra prediction mode associated with the luma component.

[0045] At 306, the method 300 includes encoding and / or decoding the video data based on a first delta angle value corresponding to the first delta angle.

[0046] It will be appreciated that Figure 3 is only intended to provide an illustration of one implementation and is not intended to imply limitations on how different embodiments may be implemented Many modifications to the depicted environments may be made based on design and implementation requirements.

[0047] Figure 4 is a block diagram 400 of internal and external components of the computer shown in Figure 1 in accordance with an example embodiment. It will be appreciated that Figure 4 is only intended to provide an example of one implementation and is not intended to imply limitations on how different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0048] The computer 102 (FIG. 1) and the server computer 114 (FIG. 1) may include respective sets of internal components 800A,B and external components 900A,B shown in FIG. 4. Each set of internal components 800 includes one or more processors 820, one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824 on one or more buses 826, one or more operating systems 828, and one or more computer-readable tangible storage devices 830.

[0049] The processor 820 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 820 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSPP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor 820 includes one or more processors that are programmable to perform functions. The bus 826 includes components that enable communication between the internal components 800A,B.

[0050] One or more operating systems 828, software programs 108 (FIG. 1), and video encoding programs 116 (FIG. 1) on the server computer 114 (FIG. 1) are stored in one or more of the respective computer-readable tangible storage devices 830 for execution by one or more of the respective processors 820 via one or more of the respective RAMs 822 (which typically include cache memory). In the embodiment shown in FIG. 4, each of the computer-readable tangible storage devices 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 830 is a semiconductor storage device such as a ROM 824, an EPROM, a flash memory, an optical disk, a magneto-optical disk, a solid-state disk, a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or any other type of non-transitory computer-readable tangible storage device capable of storing computer programs and digital information.

[0051] Each set of internal components 800A,B also includes a R / W drive or interface 832 for reading from and writing to one or more portable computer readable tangible storage devices 936, such as a CD-ROM, a DVD, a memory stick, a magnetic tape, a magnetic disk, an optical disk, or a semiconductor storage device. Software programs, such as software program 108 (FIG. 1) and video encoding program 116 (FIG. 1), may be stored in one or more of the respective portable computer readable tangible storage devices 936 and read via the respective R / W drive or interface 832 and loaded onto the respective hard drives 830.

[0052] Each set of internal components 800A,B also includes a network adapter or interface 836, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G, 4G, or 5G wireless interface card, or other wired or wireless communication link. The software program 108 (FIG. 1) and the video encoding program 116 (FIG. 1) on the server computer 114 (FIG. 1) can be downloaded from an external computer to the computer 102 (FIG. 1) and the server computer 114 via a network (e.g., the Internet, a local area network, or other wide area network) and the respective network adapter or interface 836. From the network adapter or interface 836, the software program 108 and the video encoding program 116 on the server computer 114 are loaded onto the respective hard drives 830. The network may include copper wire, optical fiber, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers.

[0053] Each set of external components 900A,B may include a computer display monitor 920, a keyboard 930, and a computer mouse 934. The external components 900A,B may also include touch screens, virtual keyboards, touch pads, pointing devices, and other human interface devices. Each set of internal components 800A,B also includes a device driver 840 for interacting with the computer display monitor 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W drive or interface 832, and the network adapter or interface 836 include hardware and software (stored in the storage device 830 and / or the ROM 824).

[0054] Although this disclosure includes detailed descriptions of cloud computing, it is to be understood that implementation of the teachings described herein is not limited to a cloud computing environment, rather, some embodiments may be implemented in conjunction with any other type of computing environment now known or later developed.

[0055] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0056] Its features are as follows.

[0057] On-Demand Self-Service: Cloud customers can unilaterally set up computing capabilities such as server time and network storage as needed, without the need for human interaction with the service provider.

[0058] Broad network access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin and thick client platforms (eg, cell phones, laptops, and PDAs).

[0059] Resource Sharing: A provider's computing resources are pooled to serve multiple customers using a multi-tenant model, with physical and virtual resources dynamically allocated and reallocated on demand. Customers generally have no knowledge or control over the exact location of the resources provided, but there is a sense of location independence in that locations can be specified at a higher level of abstraction (e.g., country, state, data center).

[0060] Rapid Elasticity: Features can be delivered quickly and elastically, sometimes automatically, to scale out quickly, release quickly and scale in quickly. To the consumer, the features available for provisioning often appear unlimited, and any number can be purchased at any time.

[0061] Measurable Services: Cloud systems automatically control and optimize resource usage by utilizing metering at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of utilized services.

[0062] The service model is as follows:

[0063] Software as a Service (SaaS): The functionality offered to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or individual application functionality, with the possible exception of user-specific application configuration settings.

[0064] Platform as a Service (PaaS): The functionality offered to the customer is the deployment of customer-created or acquired applications created by using programming languages ​​and tools supported by the provider onto a cloud infrastructure. The customer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly the application hosting environment configuration.

[0065] Infrastructure as a Service (IaaS): The functionality provided to the customer is the provision of processing, storage, network, and other basic computing resources on which the customer can deploy and run any software, including operating systems and applications. The customer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and in some cases has limited control over the selection of networking components (e.g., host firewalls).

[0066] The deployment models are as follows: Private Cloud: The cloud infrastructure is operated exclusively for the organization. It is managed by the organization or a third party and can be on-site or off-site.

[0067] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policies and compliance considerations). It may be managed by the organization or a third party and may be on-site or off-site.

[0068] Public Cloud: The cloud infrastructure is available to the general public or large industry entities and is owned by organizations that sell cloud services.

[0069] Hybrid Cloud: A cloud infrastructure consists of two or more clouds (private, community or public) that remain as unique entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g. cloud bursting for load balancing between clouds).

[0070] A cloud computing environment is service-oriented with a focus on statelessness, low connectivity, modularity and semantic interoperability. At the heart of cloud computing is an infrastructure consisting of a network of interconnected nodes.

[0071] Referring to FIG. 5, an exemplary cloud computing environment 500 is shown. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10 with which local computing devices used by cloud users, such as, for example, a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N, may communicate. The cloud computing nodes 10 may communicate with each other. They may be physically or virtually grouped into one or more networks or combinations thereof, such as the private, community, public, or hybrid clouds described above. This allows the cloud computing environment 600 to provide infrastructure, platform, and / or software-as-a-services that do not require cloud users to maintain resources on local computing devices. It will be appreciated that the types of computing devices 54A-54N shown in FIG. 5 are merely exemplary, and that the cloud computing nodes 10 and the cloud computing environment 500 may communicate with any type of computing device over any type of network and / or network addressable connection (e.g., using a web browser).

[0072] Referring to Figure 6, a set of functional abstraction layers 600 provided by the cloud computing environment 500 (Figure 5) is shown. It should be understood in advance that the components, layers and functions shown in Figure 6 are merely exemplary and the embodiments are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0073] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, RISC (Minimum Instruction Set Computer) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0074] The virtualization layer 70 provides an abstraction layer over which virtual entities such as virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74 and virtual clients 75 may be provided.

[0075] In one example, management layer 80 may provide the functionality described below: Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing 82 tracks costs as resources are utilized within the cloud computing environment and provides billing or charging for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, and protection of data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides cloud computing resource allocation and management so that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides advance arrangement and procurement of cloud computing resources where future requirements are forecasted according to SLAs.

[0076] The workload tier 90 provides examples of functions for which a cloud computing environment may be utilized. Examples of workloads and functions provided from this tier include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instructional delivery 93, data analytics processing 94, transaction processing 95, and video encoding / decoding 96. Video encoding / decoding 96 may encode / decode video data using a delta angle derived from a nominal angle.

[0077] FIG. 7 is a block diagram 700 of features related to non-directional smooth intra predictor in AVI according to an example embodiment. For example, in AV1, there are five non-directional smooth intra prediction modes: DC, PAETH, SMOOTH, SMOOTH_V, SMOOTH_H. For DC prediction, the average of the left and top neighboring samples is used as the predictor of the block to be predicted. For PAETH predictor, the top, left, top left reference samples are taken first, and then the closest value to (top+left-top left) is set as the predictor of the pixel to be predicted. FIG. 7 shows the location of the top, left, and top left samples for one pixel in the current block. In SMOOTH, SMOOTH_V, SMOOTH_H modes, they predict the block using quadratic interpolation in the vertical or horizontal direction or the average in both directions.

[0078] FIG. 8 is a block diagram 800 of features related to recursive intra filtering modes according to an example embodiment. For example, to capture the attenuated spatial correlation with reference to edges, FILTER INTRA mode is designed for luma blocks. Five filter intra modes are pre-designed for AV1, each represented by a set of eight 7-tap filters reflecting the correlation between a pixel in a 4x2 patch and its seven neighboring pixels. That is, the weighting coefficients of the 7-tap filters are position-dependent. Take an 8x8 block as an example, it is divided into eight 4x2 patches as shown in FIG. 8. These patches are denoted by B0, B1, B1, B3, B4, B5, B6, and B7 in FIG. 8. For each patch, seven neighboring pixels denoted by R0-R7 are used to predict pixels in the current patch. For patch B0, all neighboring pixels have already been reconstructed. However, for other patches, some of the neighboring pixels have not been reconstructed, and the predicted values ​​of the immediately neighboring pixels are used as references. For example, all the neighboring pixels of patch B7 have not been reconstructed, so the predicted samples of the neighboring pixels are used instead.

[0079] Chroma-from-Luma (CfL) is a chroma-only intra predictor that models chroma pixels as linear functions of the co-reconstructed luma pixels. CfL prediction is expressed as follows (Equation 1): CfL(α)=α×L AC +DC formula (1) Here, L ACis the AC contribution of the luma component, α is a parameter of the linear model, and DC represents the DC contribution of the chroma components. Specifically, the reconstructed luma pixels are subsampled to the chroma resolution and then the average value is subtracted to form the AC contribution. Instead of requiring the decoder to calculate scaling parameters to approximate the chroma AC components from the AC contributions as in some prior art, AV1 CfL determines the parameters α based on the original chroma pixels and conveys them in the bitstream. This reduces the decoder complexity and results in more accurate predictions. As for the DC contributions of the chroma components, they are calculated using an intra DC mode, which is sufficient for most chroma content and has a mature and fast implementation.

[0080] 9 is a block diagram 900 of features related to a multi-layer reference frame structure, according to an example embodiment. For example, in addition to the three reference frames of VP9, ​​namely LAST (nearest past), GOLDEN (distant past), and ALTREF (temporally filtered future), there may be an extended reference frame where there are four types of reference frames: LAST2 and LAST3: The two most recent frames BWDREF, ALTREF2: 2 future frames BWDREF is a look-ahead frame coded without temporal filtering and is more useful as a relatively short distance backward reference. ·ALTREF2 is a filtered intermediate future reference between GOLDEN and ALTREF.

[0081] 10 is a block diagram 1000 of features related to candidate motion vector list construction according to an example embodiment. For example, in dynamic spatial and temporal motion vector referencing, AV1 incorporates a motion vector reference selection scheme to efficiently code motion vectors. To find temporal motion vector reference candidates, it uses a motion field estimation process in addition to a wider spatial neighborhood than its predecessor VP9.

[0082] 11 is a line diagram 1100 of features related to motion field estimation, according to an example embodiment. For example, the motion field estimation process searches for a collocated 128×128 region for all motion trajectories at 8×8 block resolution passing through each 64×64 processing unit. For example, in FIG. 11, the motion vectors MV Ref2 are searched following the process described above to derive predicted motion vectors such as MV0 and MV1. Once all candidate motion vectors are found, they are sorted, merged and ranked to build four final candidates. AV1 then communicates the index of the reference motion vector selected from the list and optionally codes the motion vector difference.

[0083] FIG. 12 is a block diagram 1200 of overlap region-related features for overlapped block motion compensation (OMBC) according to an example embodiment. For example, to reduce prediction error around block boundaries by combining predictions obtained from neighboring motion vectors, AV1 progressively combines block-based prediction with secondary predictors from the top and left edges by applying one-dimensional filters in the vertical and horizontal directions, respectively. As an example, in FIG. 12, the shaded region of prediction block 0 is predicted by recursively generating mixed prediction samples via a one-dimensional raised cosine filter. FIG. 12 shows the shaded region predicted using upper neighbor block 2 and left neighbor block 4, respectively.

[0084] FIG. 13 is a block diagram 1300 of features related to warped motion compensation according to an exemplary embodiment. For example, AV1 introduces two affine prediction models, namely global and local warped motion compensation. The former conveys a frame-level affine model between a frame and its reference, while the latter handles local motion changes implicitly with minimal overhead. Local motion parameters are derived at the block level using 2D motion vectors from causal neighbors. This affine model is realized through successive horizontal and vertical shearing operations based on 8-tap interpolation filters with 1 / 64 pixel accuracy. FIG. 13 shows a two-stage warping process in the horizontal shearing followed by the vertical direction.

[0085] 14 is a block diagram 1400 of features related to advanced compound prediction, according to an example embodiment. For example, according to an embodiment, AV1 may enable a more general prediction at each pixel location (i,j) as follows ("Definition 1").

[0086]

number

[0087] For compound wedge prediction, AV1 provides a predefined set of 16 possible wedge partitions and signals the selected wedge index. The orientations include horizontal, vertical, and diagonal with slopes of ±2 or ±0.5 for both square and rectangular blocks. A soft-cliff 2D wedge mask is used to reduce spurious artifacts.

[0088] In the case of differential modulated masked prediction, according to an embodiment, AV1 may allow some regions of the prediction to be larger from one prediction than the other. More specifically, a mask of the following form is used ("Definition 2"):

[0089]

number

[0090] For frame distance-based compound prediction, according to an embodiment, AV1 may include a modified weighting scheme that accounts for frame distance, defined as the absolute difference between the timestamps of two frames. To take into account not only temporal distance but also the reduction of quantization noise through averaging the effect of multiple references, AV1 uses a scheme based on frame distance with a slight weighting on distant predictors, as follows ("Definition 3" and "Definition 4").

[0091]

number

[0092]

number

[0093] In the case of compound inter-intra prediction, according to an embodiment, AV1 can combine intra and inter prediction, and four frequently used intra modes are supported for the intra part. The masks associated with the modes are (i) a smooth wedge mask similar to the one used for inter-inter modes, and (ii) a mode-dependent mask that falls off along the main direction of the intra mode.

[0094] According to an example embodiment, for each transform unit, AV1 coefficient coding begins with signaling a skip sign, followed by signaling the transform kernel type and end-of-block (eob) position if the skip sign is zero. Then, each coefficient value is mapped to a multi-level map and sign.

[0095] After the eob position is coded, the low level map and the mid level map are coded in reverse scan order, the former indicating whether the magnitude of the coefficient is between 0 and 2, and the latter indicating whether the range is between 3 and 14. The next step is to code, in forward scan order, the signs of the coefficients and the residual values ​​of the coefficients greater than 14 with an Exp-Golomb code.

[0096] Regarding the use of context modeling, the low-level map coding incorporates up to five neighboring coefficient information in addition to the transform size and direction, while the mid-level map coding takes a similar approach to the low-level map coding, except that the number of neighboring coefficients is reduced to two. In addition to the Exp-Golomb code for the residual level, the signs of the AC coefficients are coded without a context model, and the signs of the DC coefficients are coded using the DC signs of the neighboring transform units.

[0097] VVC draft 6 supports a mode in which the chroma residuals are jointly coded. The use (activation) of the joint chroma coding mode is indicated by a TU level flag tu_joint_cbcr_residual_flag, and the selected mode is implied by the chroma CBF. The flag tu_joint_cbcr_residual_flag is present if either or both of the chroma CBFs for the TU are equal to 1. In the PPS and slice header, chroma QP offset values ​​are signaled for the joint chroma residual coding mode to distinguish them from the chroma QP offset values ​​signaled for the regular chroma residual coding mode. These chroma QP offset values ​​are used to derive chroma QP values ​​for blocks coded using the joint chroma residual coding mode. If the corresponding joint chroma coding mode (mode 2 in Table 1) is active in the TU, this chroma QP offset is added to the applied luma derived chroma QP during quantization and decoding of that TU. For other modes (modes 1 and 3 in Table 1), the chroma QP is derived in the same way as for conventional Cb or Cr blocks. The reconstruction process of the chroma residuals (resCb and resCr) from the transmitted transform blocks is shown in Table 1. When this mode is activated, one single joint chroma residual block (resJointC[x][y] in Table 1) is transmitted, and the residual block for Cb (resCb) and the residual block for Cr (resCr) are denoted by tu_cbf_cb, tu_cbf_cr, and It is derived taking into account information such as Csign (the sign value specified in the slice header).

[0098] The three joint chroma coding modes mentioned above are only supported for intra-coded CUs. For inter-coded CUs, only mode 2 is supported. Therefore, for inter-coded CUs, the syntax element tu_joint_cbcr_residual_flag is only present if both chroma cbfs are 1. Table 1 below shows the reconstruction of the chroma residual, where the value Csign is the sign value (+1 or -1) specified in the slice header, and resJointC[][] is the transmitted residual.

[0099] [Table 1]

[0100] FIG. 15 is a flow diagram 1500 of an aspect of an example embodiment.

[0101] FIG. 16 is a spatial diagram of residual-related features integrated as a 3D cube according to an example embodiment. The features of FIG. 16 and the features described with respect to other figures herein may be used separately or in any order in combination. Also, each of the method (or embodiment), the encoder and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored in a non-transitory computer-readable medium. One or more N1xN2Cb prediction residual blocks and corresponding Cr prediction residual blocks shall be provided as input at S151.

[0102] In S151, the two N1xN2 Cb and Cr prediction residual signal blocks are transformed into another one or more signal blocks of size MxL, where MxL is less than or equal to 2xN1xN2. However, one or more modifications may be replaced or added in S151.

[0103] For example, improvements to the compression challenges discussed above can be affected by the transforms described herein that remove redundancies between chroma prediction residuals and similar signals.

[0104] For example, according to an exemplary embodiment, N1xN2Cb residual block and its corresponding Cr residual block are put together into one block, and one transform is performed on this one block. To achieve transform coefficients of size LxM, there may be 2D transforms that vectorize each input channel separately with various scan orders (herein "Type I"). In addition to the above scan orders, there may be 2D transforms in which there is vectorization of two input channels by interleaving (herein "Type II").

[0105] As a more specific embodiment of Type I and Type II, (2xN1xN2)×(2xN1xN2) KLT (Karhunen-Loeve transformation) ·Reduced size KLT (RD-KLT), ·Decoupling approximation of RD-KLT, Givens / Permutation matrix decomposition, and Combination of trigonometric transformations There may be separate coding of the two N1×N2 transform coefficient blocks according to an example embodiment for one or more of the N1×N2 transform coefficient blocks.

[0106] In one embodiment, when Cb and Cr are combined into one block in S151, the Cb and Cr residuals are vectorized separately following zigzag, raster or diagonal scan order, and then concatenated to form a (2xN1xN2)×1 vector. It is transformed by a (LxM)×(2xN1xN2) matrix to form a signal block of size (LxM)×1. Its coefficients are quantized and the quantized exponents are transmitted. The decoding process dequantizes the exponents and inverse transforms them by a (2xN1xN2)×(LxM) matrix to obtain a reconstructed 2xN1xN2 vector that can be inversely scanned into the reconstructed Cb and Cr residual blocks.

[0107] In one embodiment, when Cb and Cr are combined into one block in S151, the Cb residual block and its corresponding Cr residual block are vectorized in an interleaved manner, with each sample taken from the residual block following a zigzag, raster, or diagonal scan order (i.e., one sample from Cb and the next sample from Cr), to form a (2xN1xN2)×1 vector. Then, it is transformed by a (LxM)×(2xN1xN2) matrix to form a transformed signal block of size (LxM)×1. The coefficients are quantized, and the quantized exponents are transmitted. The decoding process dequantizes the exponents and inverse transforms them by a (2xN1xN2)×(LxM) matrix to obtain a reconstructed 2xN1xN2 vector that can be inversely scanned into the reconstructed Cb and Cr residual blocks.

[0108] In one embodiment, in S151, there is also a setting of LxM=2xN1xN2, and the output of the transform is further split into two N1xN2 blocks, which are further quantized separately in S153 and decoded or entropy coded into a bitstream.

[0109] In one embodiment, when LxM is equal to 2xN1xN2, in S151, the above transformation matrix has size (2xN1xN2)×(2xN1xN2) and can be found as KLT (Karhunen-Loeve Transform). Given a 2xN1xN2×1 input vector signal X (assuming it is mean removed and zero-mean), we have a (2xN1xN2)×(2xN1xN2) covariance matrix

[0110]

number

[0111] R xx I j = λ j I j Formula (2) Here, λ j and I j is the jth eigenvalue, and RXX are the corresponding eigenvectors of

[0112] The KLT matrix H has the above eigenvectors as its rows and transforms X to Y as follows (Equation 3):

[0113] Y=HX Equation (3)

[0114]

number

[0115] The elements of Y are uncorrelated and their variances are equal to the eigenvalues. In addition, H is orthogonally transformed such that the input signal vector X is recovered as follows (Equation 4): X=H T Y Equation (4) In one embodiment, in S151, the (LxM) x (2xN1xN2) matrix may be determined by selecting only the LxM eigenvectors corresponding to the LxM largest eigenvalues ​​from the (2xN1xN2) basis vectors of the KLT.

[0116] In one embodiment, in S151, the (LxM)x(2xN1xN2) KLT determined above can be approximated by its separable approximation. For example, the KLT matrix H can be approximated by solving the following optimization problem ("Problem 1"):

[0117]

number

[0118] In one embodiment, in S151, the (LxM) x (2xN1xN2) matrix can be constructed as multiple stages of Gibbins rotation and / or permutation matrices. For example, the KLT matrix H can be approximated by solving the following optimization problem ("Problem 2"):

[0119]

number

[0120]

number

[0121] In one embodiment, in S151, the (LxM) x (2xN1xN2) matrix can be adaptively constructed from a set of DCT / DST transforms.

[0122] In S154, it is determined whether the N1xN2 Cb residual block and its corresponding Cr residual block are joined together into a single 2xN1xN2 three-dimensional (3D) cube, as shown by cubic 1600 in FIG. 16, and if so, a 3D transform is performed on this single cubic in S155.

[0123] According to an example embodiment, there can be either fully separable 3D transforms and 1D+2D transforms used with these described features.

[0124] In one embodiment, at S155, this three-dimensional (3D) transformation may be performed separately along three axes: along the x-axis, an N1-point 1D transformation is performed for each of the N1x1 vectors, along the y-axis, an N2-point 1D transformation is performed for each of the N2x1 vectors, and along the z-axis, another 2-point 1D transformation is performed for each of the 2x1 vectors. The order of application of these transformations may be arbitrary.

[0125] In one embodiment, in S155, this three-dimensional (3D) transformation can be performed separately along the z-axis while any 2D transformation is performed to the xy-plane, i.e., a two-point 1D transformation is performed for each of the 2x1 vectors along the z-axis while an (N1xN2) transformation is performed to the xy-plane.

[0126] At S156, it may be determined whether, in the case, to further apply one or more of the transforms described with respect to S152 as a secondary transform whose inputs are the primary transform coefficients of the chroma prediction residual.

[0127] For example, for each N1xN2 chroma residual, it only requires a K1 transform coefficient, and these two K1 coefficients are concatenated to form the input to the secondary transform described with respect to S152, and a specific example embodiment is restrictions on the types of primary transformations to which secondary transformations may be applied; Zeroing out certain coefficients of the input to a quadratic transformation; Separate coding of coefficients of the linear transformation only and the remainder Contains one or more of the following:

[0128] If so, in one embodiment, in S152 from S156, the input of the forward secondary transform is the concatenation of two sets of the first K1 (K1<=N1xN2) coefficients along the forward scan order from the transformed Cb and Cr residuals, respectively, and the output of the forward secondary transform of size K2×(2×K1) is a set of K2 coefficients that replace the same two sets of K1 input coefficients. This set of K2 coefficients is entropy coded separately from the two sets of (N1×N2-K1) coefficients to which the secondary transform is not applied. The input of the inverse secondary transform of size (2×K1)×K2 is a quantized coefficient vector of size K2×1, and the output of the inverse secondary transform is a coefficient vector of size (2×K1), and then each K1 coefficient is concatenated with each of the (N1×N2-K1) coefficients that were entropy coded separately. In one embodiment, the coefficient is set as 0 whenever the coordinate (x, y) associated with the coefficient satisfies the condition that x+y is greater than or equal to a given threshold T. Exemplary values ​​of T include, but are not limited to, integers between 0 and 32. In one embodiment, coefficients that apply both a linear transform and a secondary transform are coded separately from coefficients that apply only a linear transform. That is, coefficients that apply both a linear and secondary transform may be coded first and coefficients that apply only a linear transform may be coded second, or vice versa.

[0129] Although flow diagram 1500 has been described with respect to chroma, it is also disclosed herein that flow diagram 1500 may include other embodiments. In one embodiment, the method is applied to a luma prediction residual with either one of the chroma prediction residuals. In one embodiment, the method is applied between selected two components of any multi-component signal, if applicable. In one embodiment, the method of S152 and any of the methods of S152-S156 may be applied to the entire M component signal by concatenating M input blocks. In one embodiment, the method of S155 may be applied to the entire M component signal by performing an M-dimensional transform.

[0130] That is, the method may be generalized to other types of multi-channel signals, where general multi-channel signals are also useful, such that the residual from luma can be correlated with the residual from chroma, and the methods of 2D transform (S152) and quadratic transform (S152-S156) can be applied by concatenating M input blocks to form an input in the case of an M channel signal, and an M dimensional transform can be applied.

[0131] Output 153 may be output for one or more of the coding and decoding features described herein and further below with respect to FIGS.

[0132] 17 shows an arrangement of video encoders and decoders in a streaming environment as an example application of the disclosed subject matter. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0133] The streaming system may include a video source 1701, a capture subsystem 1703 that may include, for example, a digital camera, generating an uncompressed video sample stream 1713. This sample stream 1713, which may be emphasized as being of high data volume when compared to an encoded video bitstream, may be processed by an encoder 1702 coupled to the camera 1701. The encoder 1702 may include hardware, software, or a combination thereof to effectuate or implement aspects of the disclosed subject matter, as described in more detail below. A coded video bitstream 1704, which may be emphasized as being of low data volume when compared to the sample stream, may be stored on a streaming server 1705 for future use. One or more streaming clients 1712 and 1707 may access the streaming server 1705 to retrieve copies 1708 and 1706 of the coded video bitstream 1704. The client 1712 may include a video decoder 1711 that decodes an input copy of the coded video bitstream 1708 and generates an output video sample stream 1710 that can be rendered on a display 1709 or other rendering device (not shown). In some streaming systems, the video bitstreams 1704, 1706, and 1708 may be coded according to a particular video coding / compression standard. Examples of these standards are described above and further herein.

[0134] FIG. 18 may be a functional block diagram of a video decoder 1800 according to one embodiment of the present invention.

[0135] The receiver 1802 receives one or more codec video sequences to be decoded by the decoder 1800, and in the same or another embodiment may receive one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel 1801, which may be a hardware / software link to a storage device that stores the coded video data. The receiver 1802 may receive the coded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to a respective using entity (not shown). The receiver 1802 may separate the coded video sequences from other data. To combat network jitter, a buffer memory 1803 may be coupled between the receiver 1802 and the entropy decoder / parser 1804 (hereinafter "parser"). If the receiver 1802 receives data from a storage / forwarding device with sufficient bandwidth and controllability or from an isochronous network, the buffer 1803 may not be required or may be small. For use in a best effort packet network such as the Internet, buffer 1803 will be required and may be relatively large and advantageously of adaptive size.

[0136] The video decoder 1800 may include a parser 1804 to reconstruct symbols 1813 from the entropy coded video sequence. These categories of symbols include information used to manage the operation of the decoder 1800 and information to control a rendering device, such as a display 1812, which may in some cases not be an integral part of the decoder but may be coupled to it. The rendering device control information may be in the form of an auxiliary enhancement information (SEI message) or a video usability information (VUI) parameter set fragment (not shown). The parser 1804 may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 1804 may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The entropy decoder / parser may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0137] Parser 1804 may perform an entropy decoding / parsing operation on the video sequence received from buffer 1803 to generate symbols 1813. Parser 1804 may receive the coded data and selectively decode particular symbols 1813. Parser 1804 may also determine whether a particular symbol 1813 should be provided to motion compensated prediction unit 1806, scaler / inverse transform unit 1805, intra prediction unit 1807, or loop filter 1811.

[0138] The reconstruction of symbols 1813 may involve several different units, depending on the type of coded video picture or part thereof (e.g., inter and intra pictures, inter and intra blocks) and other factors. Which units are involved and how can be controlled by subgroup control information parsed from the coded video sequence by parser 1804. The flow of such subgroup control information between parser 1804 and the following units is not shown for clarity.

[0139] In addition to the functional blocks already mentioned, the decoder 1800 can be conceptually subdivided into several functional units, which are described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of illustrating the subject matter of the disclosure, a conceptual subdivision into the following functional units is appropriate:

[0140] The first unit is a scalar / inverse transform unit 1805. The scalar / inverse transform unit 1805 receives quantized transform coefficients as well as control information from the parser 1804, including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. as symbols 1813. It can output blocks containing sample values ​​that can be input to an aggregator 1810.

[0141] In some cases, the output samples of the scaler / inverse transform 1805 may relate to intra-coded blocks, i.e. blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed part of the current picture. Such prediction information may be provided by an intra-picture prediction unit 1807. In some cases, the intra-picture prediction unit 1807 generates a block of the same size and shape as the block being reconstructed using already reconstructed surrounding information taken from the current (partially reconstructed) picture 1809. The aggregator 1810 adds, possibly on a sample-by-sample basis, the prediction information generated by the intra-prediction unit 1807 to the output sample information provided by the scaler / inverse transform unit 1805.

[0142] In other cases, the output samples of the scalar / inverse transform unit 1805 may relate to an inter-coded potential motion compensation block. In such a case, the motion compensation prediction unit 1806 may access the reference picture memory 1808 to obtain samples used for prediction. After motion compensating the obtained samples according to the symbols 1813 related to the block, these samples may be added by the aggregator 1810 to the output of the scalar / inverse transform unit (in this case called residual samples or residual signals) to generate output sample information. The address in the reference picture memory form from which the motion compensation unit obtains the prediction samples may be controlled by a motion vector available to the motion compensation unit in the form of the symbols 1813, which may have, for example, X, Y and reference picture components. Motion compensation may include interpolation of sample values ​​obtained from the reference picture memory, motion vector prediction mechanisms, etc., when sub-sample accurate motion vectors are used.

[0143] The output samples of aggregator 1810 may be subjected to various loop filtering techniques in loop filter unit 1811. Video compression techniques may include in-loop filter techniques controlled by parameters contained in the coded video bitstream and made available to loop filter unit 1811 as symbols 1813 from parser 1804, but which may also be responsive to previously reconstructed, loop filtered sample values ​​as well as to meta-information obtained during decoding of previous portions (in decoding order) of the coded pictures or coded video sequences.

[0144] The output of the loop filter unit 1811 may be a sample stream that is output to a rendering device 1812 and can be stored in a reference picture memory for use in future inter-picture prediction.

[0145] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. Once a coded picture is fully reconstructed and identified as a reference picture (e.g., by parser 1804), the current reference picture 1809 can become part of reference picture buffer 1808 and fresh current picture memory can be reallocated before beginning reconstruction of the next coded picture.

[0146] The video decoder 1800 may perform a decoding operation according to a given video compression technique. The coded video sequence may conform to the syntax defined by the video compression technique or standard in use, in the sense that it conforms to the syntax of the video compression technique or standard defined in a video compression technique document or standard, specifically a profile document therein. It is also necessary for compliance that the complexity of the coded video sequence is within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (e.g., measured in megasamples per second), the maximum reference picture size, etc. The limits set by the level can be further limited in some cases through the metadata for HRD buffer management and the hypothetical reference decoder (HRD) signaled in the coded video sequence.

[0147] In one embodiment, the receiver 1802 may receive additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder 1800 to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0148] FIG. 19 may be a functional block diagram of a video encoder 1900 according to one embodiment of the present disclosure.

[0149] The encoder 1900 may receive video samples from a video source 1901 (not part of the encoder), which may capture video images to be coded by the encoder 1900 .

[0150] The video source 1901 may provide a source video sequence that is coded by the encoder (1703) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source 1901 may be a storage device that stores pre-prepared video. In a video conferencing system, the video source 1901 may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that convey motion when viewed in succession. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can readily appreciate the relationship between pixels and samples. The following discussion focuses on samples.

[0151] According to an embodiment, the encoder 1900 may code and compress pictures of a source video sequence into a coded video sequence 1910 in real time or under any other time constraint required by the application. Enforcing an appropriate coding rate is one function of the controller 1902. The controller controls and is functionally coupled to other functional units described below. Coupling is not shown for clarity. Parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can easily identify other functions of the controller 1902 as they may be relevant to a video encoder 1900 optimized for a particular system design.

[0152] Some video encoders operate in a "coding loop" that is easily recognized by those skilled in the art. As an oversimplified explanation, the coding loop consists of an encoding part of the encoder 1902 (hereafter "source coder") (responsible for generating symbols based on the input picture to be coded and reference pictures) and a (local) decoder 1906 embedded in the encoder 1900 that reconstructs the symbols to generate sample data that the (remote) decoder may also generate (since any compression between the symbols and the coded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream is input to a reference picture memory 1905. Since the decoding of the symbol stream results in bit-exact results that are independent of the decoder location (local or remote), the reference picture buffer contents are also bit-exact between the local and remote encoders. That is, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder would "see" if it used prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift when synchrony cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0153] The operation of the "local" decoder 1906 may be the same as that of the "remote" decoder 1800, which has already been described in relation to Figure 18. Referring briefly to Figure 19 as well, however, the entropy decoding portion of the decoder 1800, including the channel 1801, the receiver 1802, the buffer 1803 and the parser 1804, may not be fully implemented in the local decoder 1906, since symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder 1908 and the parser 1904 may be lossless.

[0154] The view at this point is that any decoder techniques, other than parsing / entropy decoding, that are present in the decoder must also be present in substantially identical functional form in the corresponding encoder. Descriptions of the encoder techniques can be omitted since they are the inverse of the decoder techniques described generically. Only in certain areas are more detailed descriptions necessary and are provided below.

[0155] As part of its operation, source coder 1903 may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence designated as “reference frames.” In this manner, coding engine 1907 codes differences between pixel blocks of an input frame and pixel blocks of reference frames that may be selected as predictive references for the input frame.

[0156] The local video decoder 1906 may decode coded video data of frames that may be designated as reference frames based on symbols generated by the source coder 1903. The operation of the coding engine 1907 may advantageously be a lossy process. If the coded video data may be decoded in a video decoder (not shown in FIG. 19), the reconstructed video sequence may be a replica of the source video sequence, usually with some errors. The local video decoder 1906 may replicate the decoding process that may be performed by the video decoder on the reference frames, such that the reconstructed reference frames are stored in the reference picture cache 1905. In this way, the encoder 1900 may locally store copies of reconstructed reference frames that have common content with the reconstructed reference frames (lacking transmission errors) obtained by the far-end video decoder.

[0157] The predictor 1904 may perform a prediction search for the coding engine 1907. That is, for a new frame to be coded, the predictor 1904 may search the reference picture memory 1905 for sample data (as candidate reference pixel blocks) or specific metadata that may serve as suitable prediction references for the new picture, such as reference picture motion vectors, block shapes, etc. The predictor 1904 may operate on a sample block pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor 1904, the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory 1905.

[0158] Controller 1902 may manage the coding operations of video coder 1903, including, for example, setting parameters and subgroup parameters used to encode the video data.

[0159] All outputs of the aforementioned functional units may be subjected to entropy coding in an entropy coder 1908. The entropy coder converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as, for example, Huffman coding, variable length coding, arithmetic coding, etc.

[0160] Transmitter 1909 may buffer the coded video sequence generated by entropy coder 1908 and prepare it for transmission over a communication channel 1911, which may be a hardware / software link to a storage device that may store the coded video data. Transmitter 1909 may merge the coded video data from video coder 1903 with other data to be transmitted, such as coded audio data and / or an auxiliary data stream (source not shown).

[0161] A controller 1902 may manage the operation of the encoder 1900. During coding, a controller 1905 may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures are often assigned as one of the following frame types:

[0162] An intra picture (I-picture) may be one that can be coded and decoded without using other frames in a sequence as a source of prediction. Some video codecs allow different kinds of intra pictures, including, for example, independent decoder refresh pictures. Those skilled in the art will understand these variations of I-pictures and their respective uses and characteristics.

[0163] A predictive picture (P-picture) may be one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0164] Bidirectionally predicted pictures (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a block.

[0165] A source picture is usually spatially divided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8 or 16x16 samples, respectively) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the respective picture of the block. For example, blocks of an I-picture may be non-predictively coded or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be non-predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded via spatial prediction or via temporal prediction with reference to one or more previously coded reference pictures.

[0166] Video coder 1900 may perform coding operations according to a given video coding technique or standard, in which the video coder 1900 may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in an input video sequence, such that the coded video data may conform to a syntax defined by the video coding technique or standard being used.

[0167] In one embodiment, the transmitter 1909 may transmit additional data along with the coded video. The source coder 1903 may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, supplemental enhancement information (SEI) messages, visual usability information (VUI) parameter set fragments, etc.

[0168] Some embodiments may relate to systems, methods and / or computer-readable media at any possible level of technical detail integration. The computer-readable media may include a computer-readable non-transitory storage medium having computer-readable program instructions to cause a processor to perform an operation.

[0169] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read only memories (ROMs), erasable programmable read only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, punch cards, or mechanically encoded devices such as ridge structures in grooves having groove instructions recorded thereon, and any suitable combination thereof. A computer readable storage medium as used herein should not be construed as a transitory signal itself, such as an electric signal transmitted over a wire, or an electromagnetic wave or other freely propagating electromagnetic wave, or an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electromagnetic wave propagating through a wire or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or a wire.

[0170] The computer readable program instructions described herein can be downloaded from a computer readable storage medium or into each computer / processing device or can be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computer / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions to a computer readable storage medium in each computer / processing device for storage.

[0171] The computer readable program code / instructions for performing the operations may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer as a stand-alone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to perform a feature or operation.

[0172] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, generate means for performing the functions / acts specified in the blocks of the flowcharts and / or block diagrams. These computer readable program instructions may be stored on a computer readable storage medium that causes a computer, programmable data processing apparatus, and / or other apparatus to function in a particular manner, and a computer readable storage medium having instructions stored thereon includes an article of manufacture that includes instructions for performing aspects of the functions / acts specified in the blocks of the flowcharts and / or block diagrams.

[0173] The computer readable program instructions are loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / acts specified in the flowchart and / or block diagram blocks.

[0174] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, including one or more executable instructions for implementing a particular logical function. The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than those shown in the drawings. In some alternative implementations, the functions described in the blocks may deviate from the order described in the drawings. For example, two blocks shown in succession may in fact be executed simultaneously or substantially simultaneously, or the blocks may be executed in the reverse order, depending on the functionality involved. It should be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs a particular function or operation or a combination of dedicated hardware and computer instructions.

[0175] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to be limiting of implementation. Thus, the operation and behavior of the systems and / or methods have been described herein without reference to specific software code. It will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0176] No element, act, or instruction used herein should be construed as critical or essential unless expressly stated. Additionally, as used herein, the articles "a" and "an" are intended to include one or more items and may be used synonymously with "one or more." Additionally, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used synonymously with "one or more." When only one item is intended, the term "a" or similar term is used. As used herein, terms such as "having" are intended to be open-ended terms. Additionally, the term "based on" is intended to mean "based at least in part on" unless expressly stated otherwise.

[0177] The description of various aspects and embodiments has been presented for illustrative purposes, but is not intended to be limited or exhaustive of the disclosed embodiments. Although combinations of features are claimed and / or disclosed in the specification, these combinations are not intended to be limited to the disclosure of possible implementations. Indeed, many of these features may be combined in ways not specifically claimed and / or disclosed in the specification. Although each dependent claim described below may directly depend on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the set of claims. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used in this specification have been selected to best describe the principles of the embodiments, practical applications, or technical improvements to the technology found in the marketplace, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A method of video encoding performed by at least one processor, the method comprising: receiving video data including at least two chroma prediction residual signal blocks of data; converting the at least two chroma prediction residual signal blocks into at least one signal block having a size equal to or smaller than a combination of the at least two chroma prediction residual signal blocks; generating video data in AOMedia Video 1 (AV1) format based on an output of the transform, the video data including the at least one signal block having a size equal to or smaller than a combination of the at least two chroma prediction residual signal blocks; A method comprising:

2. The method of claim 1, wherein the at least two chroma prediction residual signal blocks include an N1xN2 Cb residual block and an N1xN2 Cr residual block.

3. The conversion of the at least two chroma prediction residual signal blocks into the at least one signal block is performed by converting a 2xN1xN2 vector obtained from the at least two chroma prediction residual signal blocks by a (LxM)x(2xN1xN2) matrix, and quantizing the coefficients obtained thereby to obtain a quantization index; 2×N1×N2 is a size of the combination of the at least two chroma prediction residual signal blocks; L×M is the size less than or equal to a combination of the at least two chroma prediction residual signal blocks. The method of claim 2.

4. The result of transforming the 2xN1xN2 vector by a (LxM)x(2xN1xN2) matrix contains an LxM vector. The method according to claim 3.

5. A method according to claim 1, wherein the at least one signal block comprises an interleaving of the at least two chroma prediction residual signal blocks.

6. The method described in claim 1 or 2, wherein at least one signal block includes a 2xN1xN2 three-dimensional (3D) cubic.

7. The method of claim 6, wherein performing a transformation of the at least two chroma prediction residual signal blocks into the at least one signal block includes performing a one-point 1D transformation between the at least one signal block for each of N1x1 vectors along an x-axis of the 3D cubic and the at least two chroma prediction residual signal blocks, an N2-point 1D transformation between the at least one signal block for each of N2x1 vectors along a y-axis of the 3D cubic and the at least two chroma prediction residual signal blocks, and a two-point transformation between the at least one signal block for each of 2x1 vectors along a z-axis of the 3D cubic and the at least two chroma prediction residual signal blocks.

8. The method of claim 6, wherein performing a transformation of the at least two chroma prediction residual signal blocks into the at least one signal block includes performing a two-point 1D transformation between the at least one signal block and the at least two chroma prediction residual signal blocks for each of 2x1 vectors along the z-axis of the 3D cubic, and an N1xN2 transformation between the at least one signal block and the at least two chroma prediction residual signal blocks for an xy plane of the 3D cubic.

9. A method according to any one of claims 1 to 8, wherein converting the at least two chroma prediction residual signal blocks into the at least one signal block includes performing a primary transformation and a secondary transformation between the at least one signal block and the at least two chroma prediction residual signal blocks.

10. The method of claim 9, wherein at least one of the primary transform and the secondary transform includes setting a coefficient to zero in response to determining a predetermined condition in which a value of a coordinate associated with at least one coefficient of the at least two chroma prediction residual signal blocks is greater than or equal to a threshold value.

11. An apparatus for video encoding, comprising: at least one memory configured to store computer program code; at least one processor configured to access said computer program code and to operate as instructed by said computer program code to perform the method of any one of claims 1 to 10; Including, Device.

12. A computer program for causing a computer to carry out the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable medium storing the computer program of claim 12.

Images