Modal Dependent Joint Component Transformations

Abstract

To provide a method for coding video data, a computer program, and a computer system.SOLUTION: Video data is received and entropy-parsed into one or more components. The one or more entropy-parsed components are de-quantized. A joint component secondary transformation (JCST) is performed on the one or more components. The video data is decoded on the basis of one or more residual components corresponding to the joint component secondary transformed components.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Application No. 17 / 061,829, filed October 2, 2020, and claims priority to U.S. Provisional Application No. 63 / 030,623, filed in the U.S. Patent and Trademark Office on May 27, 2020, which is incorporated by reference in its entirety herein.

[0002] FIELD OF THE DISCLOSURE The present disclosure relates generally to the field of data processing, and more particularly, to video encoding and decoding. [Background technology]

[0003] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. It was developed by the Alliance for Open Media (AOMedia), founded in 2015, as a successor to VP9. The consortium includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors. Many of the components of the AV1 project were sourced from previous research efforts by members of the alliance. Individual contributors started experimental technology platforms several years ago, with Xiph and Mozilla's Daala, whose code was already publicly available in 2010, and Google's experimental VP9 evolution projects VP10, announced on September 12, 2014, and Cisco's Thor, published on August 11, 2015. Based on the VP9 code base, AV1 incorporates additional technologies, some of which were developed in these experimental formats. The first version, 0.1.0, of the AV1 reference codec was published on April 7, 2016. The Alliance announced the release of the AV1 Bitstream Specification on March 28, 2018, along with reference software-based encoders and decoders. Effective version 1.0.0 of the specification was released on June 25, 2018. Effective version 1.0.0 with Errata 1 of the specification was released on January 8, 2019. The AV1 Bitstream Specification contains the reference video codecs. Summary of the Invention [Problem to be solved by the invention]

[0004] Embodiments relate to methods, systems, and computer-readable media for encoding video data. According to one aspect, a video data encoding method is provided. The method may include receiving video data and entropy analyzing the received video data into one or more components. The one or more entropy analyzed components are inverse quantized. A joint component quadratic transform (JCST) is performed on the one or more components. The video data is decoded based on one or more residual components corresponding to the joint component quadratic transformed components.

[0005] According to another aspect, a computer system for encoding video data is provided. The computer system may include one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored in at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, thereby enabling the computer system to perform a method. The method may include receiving video data and entropy analyzing the received video data into one or more components. The one or more entropy analyzed components are inverse quantized. A joint component quadratic transform (JCST) is performed on the one or more components. The video data is decoded based on one or more residual components corresponding to the joint component quadratic transformed components.

[0006] According to yet another aspect, a computer-readable medium for encoding video data is provided. The computer-readable medium may include one or more computer-readable storage devices and program instructions stored in at least one of the one or more tangible storage devices, the program instructions being executable by a processor. The program instructions are executable by the processor to perform a method that may include receiving video data and entropy analyzing the received video data into one or more components. The one or more entropy analyzed components are inverse quantized. A joint component quadratic transform (JCST) is performed on the one or more components. The video data is decoded based on one or more residual components corresponding to the joint component quadratic transformed components.

[0007] These and other objects, features and advantages will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings, in which various features are not to scale since the illustrations are for clarity in facilitating understanding by those skilled in the art in conjunction with the detailed description. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a networked computing environment in accordance with at least one embodiment. [Figure 2A] 1 is an example JCST encoder according to at least one embodiment. [Figure 2B] 1 is an exemplary JCST decoder according to at least one embodiment. [Figure 3A] 1 is an example JCCT encoder according to at least one embodiment. [Figure 3B] 1 is an exemplary JCCT decoder according to at least one embodiment. [Figure 4]1 is an operational flowchart illustrating steps performed by a program for encoding video data based on encoding residuals from multiple color components, according to at least one embodiment. [Diagram 5] FIG. 2 is a block diagram of internal and external components of the computer and server depicted in FIG. 1 according to at least one embodiment. [Figure 6] FIG. 2 is a block diagram illustrating an exemplary cloud computing environment that includes the computer system shown in FIG. 1 in accordance with at least one embodiment. [Figure 7] FIG. 7 is a block diagram illustrating functional layers of the example cloud computing environment of FIG. 6 in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Although detailed embodiments of the claimed structures and methods are disclosed herein, it can be understood that the disclosed embodiments are merely examples of the claimed structures and methods that can be embodied in various forms. These structures and methods, however, may be embodied in many different forms and should not be construed as being limited to the exemplary embodiments described herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey its scope to those skilled in the art. In this description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0010] FIELD OF THE DISCLOSURE The embodiments relate generally to the field of data processing, and more particularly to video encoding and decoding. The exemplary embodiments described below provide, among other things, a system, method, and computer program for encoding and decoding video data based on coding residuals from multiple color components. Thus, some embodiments have the ability to improve computational fields by enabling improved encoding and decoding of video data by a computer through the reduction of redundancy between multiple color components of the video data.

[0011] As mentioned above, AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. It was developed by the Alliance for Open Media (AOMedia), founded in 2015, as a successor to VP9. The consortium includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors. Many of the components of the AV1 project were sourced from previous research efforts by members of the alliance. Individual contributors started experimental technology platforms several years ago, with Xiph and Mozilla's Daala, whose code was already publicly available in 2010, followed by Google's experimental VP9 evolution project VP10, announced on September 12, 2014, and Cisco's Thor, published on August 11, 2015. Based on the VP9 code base, AV1 incorporates additional technologies, some of which were developed in these experimental formats. The first version 0.1.0 of the AV1 reference codec was published on April 7, 2016. The Alliance announced the release of the AV1 Bitstream Specification on March 28, 2018, along with reference software-based encoders and decoders. Effective version 1.0.0 of the specification was released on June 25, 2018. Effective version 1.0.0 with Errata 1 of the specification was released on January 8, 2019. The AV1 Bitstream Specification contains the reference video codecs.

[0012] In AV1, it is observed that the prediction residual signals generated for the chroma channels (e.g., Cb and Cr) are highly correlated with each other, indicating that residual coding can be further improved by reducing the statistical redundancy between the Cb and Cr prediction residuals. Furthermore, the correlation between the Cb and Cr prediction residuals may depend on the prediction mode. A fixed joint chroma component transform may be suboptimal to handle all prediction modes. Therefore, it may be advantageous to apply a joint mode-dependent transform to multiple colors to improve coding efficiency.

[0013] For each transform unit, the AV1 coefficient coder starts with coding a skip code, followed by the transform kernel type and the end of block (EOB) location of all non-zero coefficients if the transform coding is not skipped. Then, each coefficient value is mapped to multiple level maps and codes, where the code plane covers the code of the coefficient, and the three level planes correspond to different ranges of coefficient magnitudes, namely the lower level, the middle level, and the upper level planes. The lower level planes correspond to the range 0 to 2, the middle level planes handle the range 3 to 14, and the upper level planes cover the range 15 and above.

[0014] After the EOB position is coded, the lower level plane and the mid level plane are coded together in a reverse scan order, the former indicating whether the coefficient magnitude is between 0 and 2, and the latter indicating whether the range is between 3 and 14. Then, the code plane and the upper level plane are coded together in a forward scan order, the upper level plane indicating the residual values ​​having a magnitude greater than 14, and the residuals are entropy coded using an Exp-Golomb code. AV1 adopts the conventional zigzag scan order.

[0015] Such separation allows us to assign rich context models to the lower-level planes, which consider the transform direction (bidirectional, horizontal, and vertical), transform size, and up to five neighboring coefficients to improve compression efficiency with moderate context model sizes. The mid-level planes use context models like the lower-level planes, with the number of context neighboring coefficients reduced from five to two. The upper planes are coded by exponential-Golomb codes without using a context model. In the code planes, except for the DC code, which is coded using the DC code of its neighboring transform unit as context information, the other code bits are coded directly without using a context model.

[0016] VVC Draft 6 supports modes in which the chroma residuals are jointly coded. The use (activation) of a joint chroma coding mode is indicated by a TU-level flag tu_joint_cbcr_residual_flag, and the selected mode is implicitly indicated by the chroma CBFs. The flag tu_joint_cbcr_residual_flag is present if any or both of the TU's chroma CBFs are equal to 1. In the PPS and slice headers, chroma QP offset values ​​are signaled for the joint chroma residual coding modes to distinguish them from the chroma QP offset values ​​signaled for the regular chroma residual coding modes. These chroma QP offset values ​​are used to derive the chroma QP values ​​for those blocks coded using the joint chroma residual coding modes. When the corresponding joint chroma coding mode (e.g., mode 2) is active in a TU, this chroma QP offset is added to the luma-derived chroma QP applied during quantization and decoding of that TU. For other modes (e.g., modes 1 and 3), the chroma QP is derived in the same manner as for conventional Cb or Cr blocks. The chroma residuals (resCb and resCr) can be reconstructed from the transmitted transform blocks. When this mode is activated, one joint chroma residual block (e.g., resJointC[x][y]) is signaled, and the residual block for Cb (resCb) and the residual block for Cr (resCr) are derived taking into account information such as the sign values ​​tu_cbf_cb, tu_cbf_cr, and CSign specified in the slice header.

[0017] The three joint chroma coding modes mentioned above are only supported for intra-coded CUs. For inter-coded CUs, only mode 2 is supported. Therefore, for inter-coded CUs, the syntax element tu_joint_cbcr_residual_flag is present only if both chroma cbfs are 1.

[0018] Aspects are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer-readable media according to various embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0019] Referring now to Figure 1, a functional block diagram of a networked computing environment is shown illustrating a video encoding system 100 (hereinafter "system") for encoding and decoding video data based on encoding residuals from multiple color components. It should be understood that Figure 1 provides only an illustration of one implementation and does not imply limitations with respect to the environments in which different embodiments may be implemented. Many modifications to the illustrated environment may be made based on design and implementation requirements.

[0020] The system 100 may include a computer 102 and a server computer 114. The computer 102 may communicate with the server computer 114 via a communication network 110 (hereinafter "network"). The computer 102 may include a processor 104 and software programs 108 stored in a data storage device 106 and capable of interfacing with a user and communicating with the server computer 114. As described below with reference to FIG. 5, the computer 102 may include internal components 800A and external components 900A, respectively, and the server computer 114 may include internal components 800B and external components 900B, respectively. The computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database.

[0021] The server computer 114 may also operate in a cloud computing service model, such as software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (laaS), as described below with respect to Figures 6 and 7. The server computer 114 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0022] The server computer 114, which can be used to encode video data based on encoded residuals from multiple color components, can execute a video encoding program 116 (hereinafter, the "program") that can interact with the database 112. The video encoding program method is described in more detail below with respect to FIG. 4. In one embodiment, the computer 102 can operate as an input device including a user interface, and the program 116 can execute primarily on the server computer 114. In an alternative embodiment, the program 116 can execute primarily on one or more computers 102, and the server computer 114 can be used to process and store data used by the program 116. It should be noted that the program 116 can be a stand-alone program or can be integrated into a larger video encoding program.

[0023] However, it should be noted that the processing of the program 116 may, in some cases, be shared between the computer 102 and the server computer 114 in any ratio. In another embodiment, the program 116 may run on multiple computers, server computers, or some combination of computers and server computers, e.g., multiple computers 102 communicating with a single server computer 114 over the network 110. In another embodiment, for example, the program 116 may run on multiple server computers 114 communicating with multiple client computers over the network 110. Alternatively, the program may run on a network server that communicates with the server and multiple client computers over the network.

[0024] The network 110 may include wired connections, wireless connections, fiber optic connections, or some combination thereof. In general, the network 110 may be any combination of connections and protocols that support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as the public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad-hoc network, an intranet, an optical fiber-based network, etc., and / or a combination of these or other types of networks.

[0025] The number and arrangement of devices and networks shown in Figure 1 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks in a different arrangement than those shown in Figure 1. Furthermore, two or more devices shown in Figure 1 may be implemented within a single device, or a single device shown in Figure 1 may be implemented as multiple distributed devices. Additionally or alternatively, a set of devices (e.g., one or more devices) of system 100 may perform one or more functions that are described as being performed by another set of devices of system 100.

[0026] 2A and 2B, a mode-dependent joint component quadratic transform (MD-JCST) encoder 200A and an MD-JCST decoder 200B according to one or more embodiments may be illustrated, respectively. The MD-JCST encoder 200A may include, among other things, forward transform modules 202A and 202B, a JCST module 204, a quantization module 206A and 206B, and an entropy coding module 208A and 208B. The MD-JCST encoder 200A may perform a mode-dependent joint component quadratic transform on transform coefficients of multiple color components. The JCST may be performed after the forward transform and before quantization. The MD-JCST decoder 200B may include, among other things, entropy analysis modules 210A and 210B, inverse quantization modules 212A and 212B, a JCST module 214, and an inverse transform module 216A and 216B. The MD-JCST decoder 200B may perform JCST after inverse quantization transform and before inverse (inverted) transform.

[0027] In one embodiment, the kernel used in JCST may be selected depending on the prediction mode. In one embodiment, different JCST kernels may be applied for different intra prediction modes (e.g., DC, SMOOTH_H, SMOOTH_V, directional mode, Paeth Predictor, filtering mode). In one embodiment, different JCST kernels may be applied for different nominal intra prediction modes. In one embodiment, for some intra prediction modes, JCST may be disabled. Examples of these intra prediction modes include, but are not limited to, CfL, Paeth Predictor.

[0028] In one embodiment, different JCST kernels may be applied for different nominal intra prediction modes. In one embodiment, different JCST kernels may be applied for intra prediction modes and inter prediction modes. In one embodiment, different JCST kernels may be applied for intra block copy (IBC or IntraBC) modes. In one embodiment, different JCST kernels may be applied for Palette mode. In one embodiment, different JCST kernels may be applied for different inter prediction modes (e.g., local warped motion, overlapped block motion compensation / OBMC).

[0029] In one embodiment, different JCST kernels can be applied for different motion information. The JCST kernel may depend on whether the motion vector predictor comes from above, from the left, or from a temporal motion vector prediction (MVP) candidate. The JCST kernel may depend on the magnitude of the motion vector. The JCST kernel may depend on the direction of the motion vector. In one embodiment, the kernel used in JCST depends on the primary transform type selected for the current block. In one embodiment, different JCST kernels can be applied for different primary transforms (e.g., discrete cosine transform / DCT, identity transform / IDT, asymmetric discrete sine transform / ADST, line graph transform / LGT) applied to color components that also apply JCST. In one embodiment, the kernel used in JCST depends on the secondary transform kernel selected for the current block.

[0030] 3A and 3B, proposals for applying a mode-dependent joint color component transform (MD-JCCT) encoder 300A and a MD-JCCT decoder 300B according to one or more embodiments can be illustrated, respectively. The MD-JCCT encoder 300A can include, among other things, a joint color component transform (JCCT) module 302, forward transform modules 304A and 304B, quantization modules 306A and 306B, and entropy coding modules 308A and 308B. C T Encoder 3 The MD-JCCT encoder 300A may perform a mode-dependent joint color component transform (MD-JCCT) on transform coefficients of multiple color components. The MD-JCCT encoder 300A may perform the JCCT before the forward transform and quantization. The MD-JCCT decoder 300B may include, among others, entropy analysis modules 310A and 310B, inverse quantization modules 312A and 312B, inverse transform modules 314A and 314B, and a JCCT module 316. The MD-JCCT decoder 300B may perform the JCCT after the inverse quantization and inverse (inverse) transform.

[0031] In one embodiment, the kernel used in the JCCT may be selected depending on the prediction mode. In one embodiment, different JCCT kernels are used for different intra prediction modes (e.g., DC, SMOOTH_H, SMOOTH_V, directional mode, filtering mode). C In one embodiment, different JCCT kernels may be applied for different nominal intra prediction modes. In one embodiment, for some intra prediction modes, JCCT may be disabled. Examples of these intra prediction modes include, but are not limited to, CfL. In one embodiment, different JCCT kernels may be applied for different nominal intra prediction modes. C In one embodiment, different JCCT kernels may be applied to intra and inter prediction modes. In one embodiment, different JCCT kernels may be applied to intra block copy (IBC or IntraBC) modes. In one embodiment, different JCCT kernels may be applied to palette modes. In one embodiment, different JCCT kernels may be applied to different inter prediction modes (e.g., locally warped motion, OBMC).

[0032] In one embodiment, different JCCT kernels can be applied to different motion information. The JCCT kernel may depend on whether the motion vector predictor comes from above or left or a temporal MVP candidate. The JCCT kernel may depend on the magnitude of the motion vector. The JCCT kernel may depend on the direction of the motion vector.

[0033] In one embodiment, the kernel used in the JCCT depends on the primary transform type selected for the current block. In one embodiment, different JCCT kernels can be applied for different primary transforms (e.g., DCT, IDT, ADST, LGT) applied to color components that also apply the JCCT. In one embodiment, the kernel used in the JCCT depends on the secondary transform kernel selected for the current block.

[0034]

[0036] Referring now to Figure 4, an operational flow chart illustrating steps of a method 400 for encoding video data is shown. In some implementations, one or more processing blocks of Figure 4 may be performed by computer 102 (Figure 1) and server computer 114 (Figure 1). In some implementations, one or more processing blocks of Figure 4 may be performed by another device or group of devices separate from or including computer 102 and server computer 114.

[0035] At 402, the method 400 includes receiving video data.

[0036] At 404, the method 400 includes entropy analyzing the received video data into one or more components.

[0037] At 406, the method 400 includes dequantizing one or more entropy analyzed components.

[0038] At 408, the method 400 includes performing a joint component secondary transformation on one or more components.

[0039] At 410, the method 400 includes decoding the video data based on one or more residual components corresponding to the joint component quadrature transformed component.

[0040] It will be appreciated that Figure 4 is only intended to provide an illustration of one implementation and is not intended to imply any limitations as to how different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0041] Figure 5 is a block diagram 500 of the internal and external components of the computer shown in Figure 1, according to an exemplary embodiment. It should be understood that Figure 5 provides only an example of one implementation and does not imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0042] The computer 102 (FIG. 1) and the server computer 114 (FIG. 1) may include respective sets of internal components 800A, 800B and external components 900A, 900B shown in FIG. 4. Each of the set of internal components 800 includes one or more processors 820, one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824 on one or more buses 826, one or more operating systems 828, and one or more computer-readable tangible storage devices 830.

[0043] The processor 820 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 820 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor 820 includes one or more processors that can be programmed to perform functions. The bus 826 includes components that enable communication between the internal components 800A, 800B.

[0044] One or more operating systems 828, software programs 108 (FIG. 1), and video encoding programs 116 (FIG. 1) on the server computer 114 (FIG. 1) are stored in one or more of the respective computer-readable tangible storage devices 830 for execution by one or more of the respective processors 820 via one or more of the respective RAMs 822 (which typically include cache memory). In the embodiment shown in FIG. 5, each of the computer-readable tangible storage devices 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 830 is a semiconductor storage device such as ROM 824, EPROM, flash memory, optical disk, magneto-optical disk, solid state disk, compact disk (CD), digital versatile disk (DVD), floppy disk, cartridge, magnetic tape, and / or another type of non-transitory computer-readable tangible storage device capable of storing computer programs and digital information.

[0045] Each set of internal components 800A, 800B also includes a R / W drive or interface 832 for reading from and writing to one or more portable computer-readable tangible storage devices 936, such as a CD-ROM, a DVD, a memory stick, a magnetic tape, a magnetic disk, an optical disk, or a semiconductor storage device. Software programs, such as software program 108 (FIG. 1) and video encoding program 116 (FIG. 1), may be stored in one or more of the respective portable computer-readable tangible storage devices 936 and read via the respective R / W drive or interface 832 and loaded onto the respective hard drive 830.

[0046] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G, 4G, or 5G wireless interface card or other wired or wireless communication link. The software program 108 (FIG. 1) and the video encoding program 116 (FIG. 1) on the server computer 114 (FIG. 1) can be downloaded from an external computer to the computer 102 (FIG. 1) and the server computer 114 via a network (e.g., the Internet, a local area network, or other wide area network) and the respective network adapter or interface 836. From the network adapter or interface 836, the software program 108 and the video encoding program 116 on the server computer 114 are loaded onto the respective hard drives 830. The network can include copper wire, optical fiber, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers.

[0047] Each of the sets of external components 900A, 900B may include a computer display monitor 920, a keyboard 930, and a computer mouse 934. The external components 900A, 900B may also include touch screens, virtual keyboards, touch pads, pointing devices, and other human interface devices. Each of the sets of internal components 800A, 800B may also include a device driver 840 for interfacing to the computer display monitor 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W drive or interface 832, and the network adapter or interface 836 include hardware and software (stored in the storage device 830 and / or the ROM 824).

[0048] Although this disclosure includes detailed descriptions of cloud computing, it is understood in advance that implementation of the teachings described herein is not limited to a cloud computing environment. Rather, some embodiments may be practiced in conjunction with any other type of computing environment now known or later developed.

[0049] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administration or interaction with a service provider. The cloud model includes at least five characteristics, at least three service models, and at least four deployment models.

[0050] The characteristics are as follows: On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time or network storage, automatically as needed, without the need for human interaction with the provider of the service. Pervasive Network Access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., cell phones, laptops, and PDAs). Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated depending on demand. Consumers generally have no control or knowledge over the exact location of the resources provided, although there is a sense of location independence in that they can specify location at a higher level of abstraction (e.g., country, state, or data center). Rapid scalability: It can be allocated and provisioned quickly, elastically, and sometimes automatically, to instantly scale out and quickly release to instantly scale in. To the consumer, the capacity available for provisioning often appears unlimited, and can be purchased in any quantity at any time. Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of the services utilized.

[0051] The service model is as follows: Software as a Service (SaaS): The functionality provided to the consumer is the use of a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage devices, or even individual application functions, except possibly for limited user-specific application configuration settings. Platform as a Service (PaaS): The functionality offered to the consumer is the deployment of consumer-created or acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does control the deployed applications and possibly the application hosting environment configuration. Infrastructure as a Service (laaS): The functionality provided to the consumer is to provision processing, storage, network, and other basic computing resources, on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0052] The deployment model is as follows: Private Cloud: The cloud infrastructure operates solely for the organization. It may be managed by the organization or a third party and may exist on-premise or off-premise. Community Cloud: Cloud infrastructure is shared by several organizations to support a particular community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). The cloud infrastructure may be managed by the organizations or by a third party and may exist on-premise or off-premise. Public cloud: Cloud infrastructure is offered to the general public or large industry groups and is owned by organizations that sell cloud services. Hybrid Cloud: A cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are tied together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0053] Cloud computing environments are service-oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0054] Referring to FIG. 6, an exemplary cloud computing environment 600 is illustrated. As illustrated, the cloud computing environment 600 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N, can communicate. The cloud computing nodes 10 may communicate with each other. These cloud computing nodes 10 may be physically or virtually grouped (not shown) in one or more networks, such as the previously described private cloud, community cloud, public cloud, hybrid cloud, or combinations thereof. This allows the cloud computing environment 600 to provide infrastructure, platform, and / or software as a service without the cloud consumer having to maintain resources on the local computing device. It will be understood that the types of computing devices 54A-N shown in FIG. 5 are intended as examples only, and that the cloud computing node 10 and cloud computing environment 600 may communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0055] Referring to Figure 7, a set of functional abstraction layers 700 provided by the cloud computing environment 600 (Figure 6) is illustrated. It should be understood in advance that the components, layers, and functions illustrated in Figure 7 are for illustrative purposes only, and some embodiments are not limited thereto. As illustrated, the following layers and corresponding functions are provided:

[0056] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, RISC (reduced instruction set computer) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0057] The virtualization layer 70 provides an abstraction layer within which the following examples of virtual entities can be provided: virtual servers 71, virtual storage devices 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0058] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides allocation and management of cloud computing resources such that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-positioning and procurement of cloud computing resources whose future requirements are anticipated according to SLAs.

[0059] Workload tier 90 provides examples of functionality that may utilize a cloud computing environment. Examples of workloads and functionality that may be provided from this tier include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instructional delivery 93, data analytics processing 94, transaction processing 95, and video encoding 96. Video encoding 96 can encode and decode video data based on coding residuals from multiple color components.

[0060] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail integration. The computer-readable media may include a computer-readable non-transitory storage medium having computer-readable program instructions for causing a processor to perform operations.

[0061] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridges in grooves with instructions recorded thereon, and any suitable combinations of the above. A computer-readable storage medium as used herein should not be construed as being a transitory signal itself, such as an electric signal transmitted over a wire, or an electromagnetic wave or other freely propagating electromagnetic wave, or an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable).

[0062] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0063] The computer readable program code / instructions for performing the operations may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or object oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform an aspect or operation.

[0064] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to manufacture a machine such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that a computer readable storage medium having instructions stored thereon includes a product including instructions implementing aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0065] The computer readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to cause the computer, other programmable device, or other device to perform a series of operational steps to generate a computer-implemented process, such that the instructions executing on the computer, other programmable device, or other device perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0066] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, including one or more executable instructions for implementing a specified logical function. The method, the computer system, and the computer-readable media may include more blocks, fewer blocks, different blocks, or blocks in different arrangements than those shown in the figures. In some alternative implementations, the functions described in the blocks may be performed in a different order than that described in the figures. For example, two blocks shown in succession may actually be executed simultaneously or substantially simultaneously, or the blocks may sometimes be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, as well as combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0067] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation form. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0068] No element, act, or instruction used herein should be construed as critical or essential unless expressly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Additionally, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with "one or more." When only one item is intended, the term "one" or similar terms are used. Also, as used herein, terms such as "has," "have," and "having" are intended to be open-ended terms. Additionally, the phrase "based on" is intended to mean "based, at least in part, on," unless otherwise specified.

[0069] The description of various aspects and embodiments is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Although combinations of features are recited in the claims and / or disclosed herein, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features may be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements to the technology found in the marketplace, or to enable those skilled in the art to understand the embodiments disclosed herein. [Explanation of symbols]

[0070] 10 cloud computing nodes 54 Computing Devices 54A Mobile Phone 54B Desktop Computer 54C Laptop Computer 54N Automotive Computer System 60 Hardware and Software Layers 61 Mainframe 62,63 Server 64 Blade Server 65 Storage device 66 Network Components 67 Network Application Server Software 68 Database Software 70 Virtualization Layer 71 Virtual Server 72 Virtual Storage Devices 73 Virtual Network 74 Operating Systems 75 Virtual Clients 80 Management layer 81 Resource Provisioning 82 Pricing 83 User Portal 84 Service Level Management 85 Planning and Implementing Service Level Agreements (SLAs) 90 Workload Tier 91 Mapping and Navigation 92 Software Development and Lifecycle Management 93 Virtual Classroom Education Delivery 94 Data Analysis Processing 95 Transaction Processing 96 Video Coding 100 Video Encoding System 102 Computer 104 processors 106 Data storage device 108 Software Programs 110 Communication Network 112 Database 114 Server Computer 116 Video Encoding Program 200A MD-JCST Encoder 200B MD-JCST Decoder 202A, 202B forward conversion module 204 JCST Module 206A, 206B Quantization Module 208A, 208B Entropy coding module 210A, 210B Entropy Analysis Module 212A, 212B Inverse quantization module 214 JCST Module 216A, 216B Reverse conversion module 300A MD-JCCT Encoder 300B MD-JCCT Decoder 302 JCCT Module 304A, 304B forward conversion module 306A, 306B Quantization Module 308A, 308B Entropy coding module 310A, 310B Entropy Analysis Module 312A, 312B Inverse quantization module 314A, 314B Reverse conversion module 316 JCCT Module 400 ways 500 Block Diagram 600 Cloud Computing Environment 700 Functional Abstraction Layer 800, 800A, 800B Internal components 820 Processor 822 RAM 824 ROM 826 Bus 828 Operating Systems 830 Hard Drive 830 Storage device 830 Computer-readable tangible storage device 832 R / W drive or interface 836 Network Adapter or Interface 840 Device Drivers 900A, 900B External Components 920 Computer Display Monitor 930 Keyboard 934 Computer Mouse 936 Portable computer-readable tangible storage device

Claims

1. A method for encoding video data executable by a processor, comprising: obtaining one or more residual components corresponding to one or more components of the video data; performing a joint component quadratic transform (JCST) operation on the one or more residual components; performing a quantization on a result of the JCST operation; performing entropy coding on the result of the quantization; encoding the video data based on a result of the entropy encoding to generate an encoded video stream corresponding to the video data; 13. A method for encoding video data, comprising:

2. The method of claim 1, wherein a JCST kernel is selected based on a prediction mode corresponding to the video data.

3. The method of claim 2, wherein the JCST kernel is applied based on an intra-prediction mode associated with the video data.

4. The method described in claim 3, wherein the intra prediction modes include one or more of DC, SMOOTH_H, SMOOTH_V, a directional mode, a Paes predictor, and a filtering mode.

5. The method of claim 3, wherein the JCST kernel is applied based on a nominal intra-prediction mode.

6. The method of claim 3, wherein JCST is disabled based on the intra prediction mode.

7. The method of claim 6, wherein the intra prediction modes include one or more of a chroma from luma mode and a Paes predictor.

8. The method of claim 2, wherein different JCST kernels are applied to intra prediction and inter prediction modes.

9. The method of claim 2, wherein the JCST kernel is applied based on an intra block copy mode.

10. The method of claim 2, wherein the JCST kernel is applied based on a palette mode.

11. The method of claim 2, wherein the JCST kernels are applied based on different inter-prediction modes.

12. The method of claim 2, wherein the JCST kernel is applied based on motion information.

13. The method of claim 12, wherein the JCST kernel depends on whether the motion vector predictor comes from above, to the left, or from a temporal motion vector predictor candidate.

14. The method of claim 12, wherein the JCST kernel depends on the magnitude of the motion vector.

15. The method of claim 12, wherein the JCST kernel depends on a direction of a motion vector.

16. The method of claim 1, wherein the JCST kernel depends on a primary transform type selected for a current block associated with the video data.

17. The method of claim 16, wherein the JCST kernel is applied based on a primary transform applied to color components associated with the video data to which the JCST is applied.

18. The method of claim 1, wherein the JCST kernel depends on a secondary transform kernel selected for a current block associated with the video data.

19. A computer system for encoding video data, comprising: one or more computer processors; one or more computer-readable non-transitory storage media storing a computer program for causing said one or more computer processors to carry out the method according to any one of claims 1 to 18; 2. A computer system comprising:

20. A computer program product for causing one or more computer processors to carry out the method of any one of claims 1 to 18.

Images