Method, system and medium for encoding video data

Abstract

A method, computer program, and computer system for encoding video data are provided. Video data is received, the video data including a reference view and a current view. For a current block in the current view, a collocated block in the reference view is identified. A prediction block vector is calculated based on an offset vector between the current block and the collocated block and a disparity vector between the reference block and the collocated block in the reference view. The video data is then encoded / decoded based on the calculated prediction block vector.

Description

[0001] Priority information

[0002] This application claims priority to U.S. application No. 17 / 026,886, filed on September 21, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to the field of data processing, and more particularly to video encoding and decoding. Background Art

[0004] Block compensation based on different images is well-known motion compensation. Similarly, block compensation can also be performed based on previously reconstructed areas within the same image. This is called intra-frame image block compensation, current picture referencing (CPR) or intra-frame block copy (IBC). The displacement vector representing the offset between the current block and the reference block is called a block vector (Block Vector, or BV for short). Unlike the motion vector in motion compensation, which can be any value (positive or negative in the x-direction or y-direction), the block vector has some constraints to ensure that the reference block it points to is available and has been reconstructed. In addition, in some codec designs, such as HEVC SCC, some reference areas at tile boundaries or wavefront trapezoid boundaries are also excluded for parallel processing considerations. In addition to the above constraints, in HEVC SCC, reconstructed areas of the same image can be used as references for intra-frame block copy prediction.

[0005] The coding of block vectors can be explicit or implicit. In explicit mode (or AMVP mode in inter-frame coding), the difference between the block vector and its prediction is signaled; in implicit mode, the block vector is recovered entirely from the prediction of the block vector in a manner similar to the motion vector in merge mode. In some implementations, the resolution of the block vector is limited to integer bits; in other systems, the resolution of the block vector may also be allowed to point to fractional bits.

[0006] A block-level flag called the intra block copy flag can be used to signal the use of intra block copy at the block level. In one embodiment, this flag is issued when the current block is not encoded in merge mode. Alternatively, this flag can be issued via a reference indexing approach. This is achieved by using the current decoded picture as a reference picture. In HEVC SCC, such a reference picture will be placed at the last position in the list. This special reference picture is also managed along with other temporal reference pictures in the DPB.

[0007] There are also some changes for intra block copy, such as treating intra block copy as a third mode different from intra prediction mode or inter prediction mode. In this way, block vector prediction in merge mode and AMVP mode is separated from the regular inter mode. For example, a separate merge candidate list is defined for intra block copy mode, where all entries in the list are block vectors. Similarly, the block vector prediction list in intra block copy AMVP mode also consists of only block vectors. The general rule applied to these two lists is that in terms of the candidate derivation process, these two lists can follow the same logic as the inter merge candidate list or AMVP prediction list. For example, 5 spatial neighbors in HEVC or VVC inter merge mode are visited for intra block copy to derive its own merge candidate list.

[0008] Currently in VVC, the search range of the Intra block copy mode is limited to the current coding tree unit. For the Intra block copy mode, the effective memory requirement for storing reference samples is the size of the largest coding tree unit of samples. Considering that the existing reference sample memory is used to store the reconstructed samples of the current 64×64 area, three more 64×64 reference sample memories are needed. Based on this fact, the proposed method extends the effective search range of the Intra block copy mode to a part of the left coding tree unit while keeping the total memory requirement for storing reference pixels unchanged (one coding tree unit size, four 64×64 reference sample memories). Summary of the Invention

[0009] Embodiments relate to a method, system, and medium for encoding video data. According to one aspect, a method for encoding video data is provided. The method may include receiving video data, the video data including a reference view and a current view. For a current block in the current view, identifying a co-located block in the reference view. Calculating a prediction block vector based on an offset vector between the current block and the co-located block and a disparity vector between the reference block and the co-located block in the reference view. Encoding / decoding the video data based on the prediction block vector.

[0010] According to another aspect, a computer system for encoding video data is provided. The computer system may include one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, whereby the computer system is capable of performing a method. The method may include: receiving video data, the video data including a reference view and a current view. For a current block in the current view, identifying a co-located block in the reference view. Based on an offset vector between the current block and the co-located block and a disparity vector between the reference block in the reference view and the co-located block, a prediction block vector is calculated. Based on the prediction block vector, the video data is encoded / decoded.

[0011] According to another aspect, a computer-readable medium for encoding video data is provided. The computer-readable medium may include one or more computer-readable storage devices and program instructions stored on at least one of the one or more tangible storage devices, the program instructions being executable by a processor. The program instructions are executable by the processor to perform a method, which may accordingly include receiving video data including a reference view and a current view. For a current block in the current view, a co-located block in the reference view is identified. Based on an offset vector between the current block and the co-located block and a disparity vector between the reference block in the reference view and the co-located block, a prediction block vector is calculated. Based on the prediction block vector, the video data is encoded / decoded. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] These and other objects, features and advantages will become apparent from the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings. The various features of the drawings are not drawn to scale because the illustrations are intended to be clear and readily understood by those skilled in the art in conjunction with the detailed description. In the drawings:

[0013] Figure 1 illustrates a networked computer environment according to at least one embodiment;

[0014] Figure 2 is a schematic diagram of stitched video data with multiple views according to at least one embodiment;

[0015] Figure 3 an operational flow diagram illustrating steps performed by a program for encoding video data in accordance with at least one embodiment;

[0016] Figure 4 According to at least one embodiment Figure 1 block diagrams of the internal and external components of computers and servers depicted in;

[0017] Figure 5 According to at least one embodiment, Figure 1 A block diagram of an illustrative cloud computing environment for a computer system depicted in ; and

[0018] Figure 6 According to at least one embodiment Figure 5 A block diagram of the functional layers of an illustrative cloud computing environment. DETAILED DESCRIPTION

[0019] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it will be understood that the disclosed embodiments are merely illustrative of the claimed structures and methods, which may be implemented in various forms. However, these structures and methods may be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete and fully convey the scope to those skilled in the art. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0020] Embodiments generally relate to the field of data processing, and more particularly to video encoding and decoding. The exemplary embodiments described below provide a system, method, and computer program that include encoding and decoding video data based on prediction block vectors between multiple views of the video data. Thus, some embodiments have the potential to improve the computing field by enabling improved encoding of video data based on intra-frame prediction.

[0021] As mentioned above, the search range of the intra block copy mode is limited to the current coding tree unit. For the intra block copy mode, the effective memory requirement for storing reference samples is the size of the largest coding tree unit of samples. Considering that the existing reference sample memory is used to store the reconstructed samples of the current 64×64 area, three more 64×64 reference sample memories are needed. Based on this fact, the proposed method extends the effective search range of the intra block copy mode to a part of the left coding tree unit while keeping the total memory requirement for storing reference pixels unchanged (one coding tree unit size, four 64×64 reference sample memories).

[0022] In traditional methods, multiple views of the same scene are spliced ​​into one signal video frame. The spliced ​​video can be encoded using traditional 2-D video codecs (such as AVC, HEVC, etc.). However, in such methods, the encoding of very large images requires a significant increase in computational complexity. Typically, for the intra-frame block copy mode, the best matching block of the current coding block will be located around the current block in the same image. For some unique content (such as content with repeated patterns), the best matching block can be found farther away. Similarly, for inter-frame motion estimation, it is generally assumed that the object moves smoothly from the reference image to the current image in a continuous motion. Therefore, the best matching block in the reference image will appear around the projection center, which may have similar coordinates to the current block in the current image. Therefore, it may be advantageous to exploit the similarity between spliced ​​views within the same image to improve coding efficiency and coding speed. In this way, in the case of multi-view video coding, the intra-frame block copy mode can be used to efficiently find the best matching block

[0023] Various aspects are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer-readable media according to various embodiments. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0024] Now refer to Figure 1 , shows a functional block diagram of a networked computer environment for a video encoding system 100 (hereinafter referred to as "the system") for encoding video data based on predicting block vectors between multiple views within the video data. It should be understood that Figure 1 This is merely an illustration of one embodiment and does not imply any limitations with respect to the environments in which different embodiments may be implemented. Many modifications may be made to the depicted environments based on design and implementation requirements.

[0025] System 100 may include a computer 102 and a server computer 114. Computer 102 may communicate with server computer 114 via a communication network 110 (hereinafter referred to as "network"). Computer 102 may include a processor 104 and a software program 108 stored on a data storage device 106, and computer 102 may be connected to a user and communicate with server computer 114. Figure 4 As discussed, computer 102 may include internal components 800A and external components 900A, and server computer 114 may include internal components 800B and external components 900B, respectively. Computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop, a tablet, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database.

[0026] As referenced below Figure 5 and Figure 6 As discussed, server computer 114 may also operate in a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). Server computer 114 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0027] The server computer 114 can be used to encode video data and can run a video encoding program 116 (hereinafter referred to as "program") that can interact with the database 112. Figure 3 The video encoding program method is described in more detail. In one embodiment, computer 102 can operate as an input device including a user interface, while program 116 can be primarily executed on server computer 114. In an alternative embodiment, program 116 can be primarily executed on one or more computers 102, while server computer 114 can be used to process and store data used by program 116. It should be noted that program 116 can be a stand-alone program or can be integrated into a larger video encoding program.

[0028] However, it should be noted that in some cases, the processing of program 116 may be shared in any proportion between computer 102 and server computer 114. In another embodiment, program 116 may be executed on more than one computer, server computer, or some combination of computers and server computers, for example, on multiple computers 102 communicating with a single server computer 114 via network 110. In another embodiment, for example, program 116 may be executed on multiple server computers 114 communicating with multiple client computers via network 110. Alternatively, the program may be executed on a network server that communicates with the server and multiple client computers via the network.

[0029] The network 110 may include a wired connection, a wireless connection, a fiber optic connection, or some combination of these. In general, the network 110 may be any combination of connections and protocols that will support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long-term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, a fiber-based network, etc., and / or a combination of these or other types of networks.

[0030] Figure 1 The number and arrangement of devices and networks shown are provided as examples. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different Figure 1 The devices and / or networks shown may be arranged differently. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of system 100 may perform one or more functions described as being performed by another set of devices of system 100.

[0031] Now refer to Figure 2 , shows a schematic diagram of stitched video data 200 with multiple views. The stitched video data 200 may include several views (view 0 to view 1) that are spatially stitched together to form a two-dimensional image. Figure 5 ). By way of example and not limitation, Figure 2Six views are depicted in FIG. However, it will be appreciated that any number of views may be stitched together. It will be further appreciated that there may be several ways to spatially stitch the views. For example, for six views, one, two, or three views may be stitched per row, which may result in a 1x6 stitching, a 2x3 stitching, and a 3x2 stitching, respectively. For multi-view video, the views may be stitched in a horizontal or vertical raster scan order, a zigzag scan order, or other scan order (e.g., a serpentine scan order).

[0032] The stitching can be designed so that the resulting super-sized image has the desired image size. For example, the super-sized image can be close to a square shape, a 4:3 aspect ratio rectangular shape, a 16:9 aspect ratio rectangular shape, etc. The frame resolution of each view can be W×H, where W and H can be the width and height of the view frame resolution, respectively. In the case where the super-sized image is close to a square, a factorization pair (L, M) of N can be searched so that |W*LH*M| can be minimized, where N=L*M is the number of views. The minimizer can be denoted as (L0, M0), and L0 views can be stitched per row. This produces an L0*M0 stitching with a super-sized image of size width W*L0 and height H*M0.

[0033] After spatial stitching, intra block copying can be used to explore correlations between views in the same image. To speed up block matching in intra block copying mode, the intra block copy search can be restricted to the same previously coded view. When encoding the first view in an image, intra block copying can be disabled. Block vectors can point to reference blocks in previously coded views.

[0034] Before block matching, two views (e.g., view 0 and view 1) can be estimated. Figure 5 ). A perspective transform can create a bijective mapping between two views. A perspective transform can be applied to a reference view (i.e., view 0), which maps coordinates from the reference view to the current view being coded (i.e., view 1). Figure 5 ). The "transformed" reference view can be used as a reference for block matching.

[0035] For a block in the current view (e.g., block 202A), a co-located block of the block (e.g., block 204) can be found at the same position in the "transformed" reference view, and block matching can be started with the co-located block as the starting point (e.g., matching with block 202B). Since the perspective transform is very close to the conversion between the two views, block matching can be limited to a small neighborhood of the co-located block in the "transformed" reference view. In another embodiment, the current block and its co-located block have the same position offset relative to the upper left corner position of their corresponding views.

[0036] The perspective transform can be estimated by analyzing the reference view and the current view. Techniques from computational photography, such as keypoint detection and matching, can be used to derive the perspective transform. Alternatively, the perspective transform can be computed directly from the camera parameters and depth map. When the depth map and camera parameters are available, forward projection can be applied to project the depth map of the reference view onto the coordinates of the current view. The depth map of the current view can be located within the projected depth map of the reference view. Backward projection can be applied to derive the color information of each pixel in the current view from the color information of the reference view. This can create a projected reference view that will be used for block matching.

[0037] Assuming that the samples in the block share the same disparity, the disparity vector DV(Dx, Dy) can be used to indicate the disparity between the same-position block in the reference view and the reference block in the reference view. The disparity vector can be derived using the method discussed above. It should be noted that for each view pair, the block vectors of blocks at different positions in the view may be different. The block vector pointing from the current block to its reference block in the reference view consists of two parts: the view position offset plus the disparity vector. The equation for the current block (with coordinates (Xcurr, Ycurr)) to find its same-position block (with coordinates (Xcol, Ycol)) in the reference view (assuming the reference view is at the top and left of the current view) can be given as follows:

[0038] Xcol = Xcurr - N * view_width + Dx (Equation 1)

[0039] Ycol = Ycurr - M * view_height + Dy (Equation 2)

[0040] Where M and N may be the number of views between the current view and the reference view in the oversized image in the X and Y directions, respectively; view_width and view_height may be the size of each view.

[0041] Block vector prediction can be used to improve block vector difference coding. Similar to the above, the coordinates (Xcol, Ycol) of the same position block can be derived. The block vector prediction can be given by the following equation:

[0042] BVP = (Xcol, Ycol) - (Xcurr, Ycurr) (Equation 3)

[0043] After the encoder finds the actual block vector BV for encoding, the difference BVD = BV - BVP can be encoded. It should be noted that such a block vector prediction can be used as one of the possible block vector prediction candidates. On the decoder side, the block vector prediction described above can be selected as BVP. The block vector prediction can be decoded and added to the block vector difference BVD to form the block vector of the current block.

[0044] In one or more embodiments, the derived block vector prediction candidate BVP may be added to a merge candidate list for intra block copy merge or skip mode. In one example, it may be placed after spatial neighboring candidates.

[0045] In one or more embodiments, there may be multiple such derived block vector prediction candidates available (e.g., several block vector prediction candidates from different views). The block vector prediction candidates may be added together to a merge candidate list for intra block copy merge or skip mode. In one example, the block vector prediction candidates may be placed in a list where candidates from closer views (compared to the current view) may be placed before candidates from more distant views.

[0046] In one or more embodiments, the derived block vector prediction candidate may be added to a block vector prediction candidate list for a block vector coding mode. In one example, the derived block vector prediction candidate may be added after spatial neighboring candidates.

[0047] In one or more embodiments, the encoder may perform block vector estimation using the derived reference position indicated by the block vector prediction as a starting point (or search center) and continue searching around that position in a manner similar to conventional methods, where more computational resources (more search points) may be used at positions near the center and fewer computational resources (fewer search points) may be used at positions farther away.

[0048] In one or more embodiments, the encoder may perform block vector estimation using the derived reference position indicated by the block vector prediction as one of the possible best matching positions. If the match result returned from that position has a sufficiently low cost, the block vector estimation process may be terminated early without further searching or examining more candidates within the search window.

[0049] In one or more embodiments, the encoder performs tile-based encoding on the view. Each view can be encoded as a tile and can also be decoded as a tile. The method may include receiving video data, the video data including a reference tile and a current tile. For a current block in a current tile, a co-located block in a reference tile is identified. Based on the offset vector between the current block and the co-located block and the disparity vector between the co-located block and the reference block in the reference tile, a prediction block vector is calculated. The video data is encoded / decoded based on the calculated prediction block vector.

[0050] In one or more embodiments, the encoder performs candidate block vector index signaling, such as motion vector index signaling in HEVC.

[0051] In one or more embodiments, the encoder performs an ordered search among block vector candidates, wherein when multiple candidate vectors are added to a candidate vector list, the candidate vector list is sorted based on a proximity between a current view and multiple views corresponding to the multiple candidate vectors, or based on a quality ranking among multiple reference views.

[0052] In one or more embodiments, a disparity vector between a collocated block in a reference view and a reference block is derived based on a geometric relationship between the current view and the reference view.

[0053] Now refer to Figure 3 , which shows an operational flow diagram of the steps of a method 300 for encoding video data. In some implementations, Figure 3 One or more flow blocks of the process may be executed by the computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) is executed. In some implementations, Figure 3 One or more of the flow blocks may be performed by another device or group of devices separate from or including the computer 102 and the server computer 114 .

[0054] At 302 , method 300 includes receiving video data, the video data including a reference view and a current view.

[0055] At 304 , method 300 includes, for a current block in a current view, identifying a co-located block in a reference view.

[0056] At 306 , method 300 includes calculating a prediction block vector based on an offset vector between the current block and the collocated block and a disparity vector between the collocated block and a reference block in a reference view.

[0057] At 308 , method 300 includes encoding / decoding video data based on the prediction block vector.

[0058] I understand. Figure 3 This merely provides an illustration of one implementation and does not imply any limitations regarding how different embodiments may be implemented. Many modifications may be made to the depicted environments based on design and implementation requirements.

[0059] An embodiment of the present application provides a computer system for encoding video data. The computer system includes one or more computer-readable non-transitory storage media and one or more computer processors. The one or more computer-readable non-transitory storage media are configured to store computer program code, and the one or more computer processors are configured to access the computer program code and operate according to the instructions of the computer program code. The computer program code includes: a receiving code configured to enable one or more computer processors to receive video data, the video data including a reference view and a current view; an identification code configured to enable one or more computer processors to identify a co-located block in a reference view for a current block in a current view; a calculation code configured to enable one or more computer processors to calculate a prediction block vector based on an offset vector between the current block and the co-located block and a disparity vector between the reference block in the reference view and the co-located block; and an encoding code configured to enable one or more computer processors to encode / decode the video data of the current block based on the calculated prediction block vector.

[0060] Figure 4 According to an illustrative embodiment Figure 1 400. It should be understood that Figure 4 This is merely an illustration of one implementation and does not imply any limitations with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

[0061] like Figure 4 As shown, the computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) may include a set of internal components 800A, 800B and external components 900A, 900B, respectively. Each set of internal components 800 includes one or more processors 820, one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824, one or more operating systems 828, and one or more computer-readable tangible storage devices 830, located on one or more buses 826.

[0062] The processor 820 is implemented in hardware, firmware, or a combination of hardware and software. The processor 820 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or other types of processing components. In some implementations, the processor 820 includes one or more processors that can be programmed to perform functions. The bus 826 includes components that allow communication between internal components 800A and 800B.

[0063] Located on server computer 114 ( Figure 1 ) on one or more operating systems 828, software programs 108 ( Figure 1 ) and video encoding program 116 ( Figure 1 ) are stored on one or more of the respective computer-readable tangible storage devices 830 for execution by one or more of the respective processors 820 via one or more of the respective RAMs 822 (typically including cache). Figure 4 In the illustrated embodiment, each computer-readable tangible storage device 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 830 is a semiconductor memory device such as ROM 824, EPROM, flash memory, optical disk, magneto-optical disk, solid-state disk, compact disk (CD), digital versatile disk (DVD), floppy disk, magnetic cassette, magnetic tape, and / or other types of non-transitory computer-readable tangible storage devices that can store computer programs and digital information.

[0064] Each set of internal components 800A, 800B also includes a R / W drive or interface 832 to read from and write to one or more portable computer readable tangible storage devices 936 (e.g., CD-ROM, DVD, memory stick, tape, magnetic disk, optical disk, or semiconductor storage device). Figure 1 ) and video encoding program 116 ( Figure 1) can be stored on one or more of the corresponding portable computer-readable tangible storage devices 936, read via the corresponding R / W drive or interface 832 and loaded into the corresponding hard disk drive 830.

[0065] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card; a wireless Wi-Fi interface card; or a 3G, 4G or 5G wireless interface card or other wired or wireless communication link. Server computer 114 ( Figure 1 ) on the software program 108 ( Figure 1 ) and video encoding program 116 ( Figure 1 ) can be downloaded from an external computer to the computer 102 via a network (eg, the Internet, a local area network, or other wide area network) and a corresponding network adapter or interface 836 ( Figure 1 ) and the server computer 114. The software program 108 and the video encoding program 116 on the server computer 114 are loaded onto respective hard drives 830 via a network adapter or interface 836. The network may include copper wire, fiber optics, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.

[0066] Each set of external components 900A, 900B may include a computer display 920, a keyboard 930, and a computer mouse 934. External components 900A, 900B may also include a touch screen, a virtual keyboard, a touchpad, a pointing device, and other human-computer interface devices. Each set of internal components 800A, 800B also includes a device driver 840 to interface with the computer display 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W driver or interface 832, and the network adapter or interface 836 comprise hardware and software (stored in the storage device 830 and / or ROM 824).

[0067] It should be understood in advance that although the present disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Instead, some embodiments can be implemented in conjunction with any other type of computing environment now known or later developed.

[0068] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with a service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0069] Features are as follows:

[0070] On-demand self-service: Cloud consumers can automatically and unilaterally provision computing capacity (such as server time and network storage) as needed without manual interaction with the service provider.

[0071] Broad Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0072] Resource pooling: Pooling a provider's computing resources to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence because consumers generally cannot control or know the exact location of the provided resources, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0073] Rapid elasticity: Capacity can be provisioned quickly and elastically, in some cases automatically, to scale up quickly and release quickly to scale down quickly. To the consumer, the available capacity often appears unlimited and can be purchased at any time and in any quantity.

[0074] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0075] The service model is as follows:

[0076] Software as a Service (SaaS): The ability provided to consumers is to use the provider's applications running on a cloud infrastructure. Applications are accessed from a variety of client devices through a thin client interface, such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0077] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications they create or acquire onto cloud infrastructure. Applications are built using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage. Instead, they control the deployed applications and the configuration of the application hosting environment.

[0078] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources, where consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather control the operating system, storage, deployed applications, and possibly limited control of selected networking components (e.g., host firewalls).

[0079] The deployment model is as follows:

[0080] Private cloud: Cloud infrastructure is run for the sole use of a single organization. The cloud infrastructure can be managed by that organization or a third party and can exist on-premises or off-premises.

[0081] Community cloud: Cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policy, and compliance considerations). Community clouds can be managed by these organizations or a third party and can exist on-premises or off-premises.

[0082] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0083] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct entities but are tied together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0084] The cloud computing environment is service-oriented, with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure consisting of a network of interconnected nodes.

[0085] refer to Figure 5 , depicts an illustrative cloud computing environment 500. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10, and local computing devices used by cloud consumers can communicate with the cloud computing nodes 10, such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automobile computer systems 54N. The cloud computing nodes 10 are communicative with each other. The cloud computing nodes 10 can be physically or virtually grouped in one or more networks (not shown), such as private clouds, community clouds, public clouds, or hybrid clouds as described above, or a combination thereof. This allows the cloud computing environment 500 to provide infrastructure, platforms, and / or software as services that do not require cloud consumers to maintain resources on local computing devices. It should be understood that Figure 5The types of computing devices 54A-N shown are for illustration only, and cloud computing node 10 and cloud computing environment 500 may communicate with any type of computerized device over any type of network and / or network-addressable connection (eg, using a web browser).

[0086] Reference Figure 6 , showing a cloud computing environment 500 ( Figure 5 ) provides a set of functional abstraction layers 600. It should be understood in advance that Figure 6 The components, layers, and functions shown in the figure are for illustration only and the embodiments are not limited thereto. As depicted in the figure, the following layers and corresponding functions are provided:

[0087] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on Reduced Instruction Set Computer (RISC) architecture; server 63; blade server 64; storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0088] Virtualization layer 70 provides an abstraction layer from which examples of the following virtual entities may be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0089] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic acquisition of computing resources and other resources for performing tasks in a cloud computing environment. Metering and pricing 82 provides cost traceability when resources are used in a cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment to consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service Level Agreement (SLA) planning and implementation 85 provides pre-arrangement and acquisition of cloud computing resources, for which future demand is anticipated based on the SLA.

[0090] Workload layer 90 provides examples of functionality that can utilize a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and video encoding 96. Video encoding 96 can encode / decode video data based on predicting block vectors between multiple views in the video data.

[0091] Some embodiments may involve integrating systems, methods, and / or computer-readable media at any possible level of technical detail. The computer-readable medium may include a computer-readable non-transitory storage medium (or medium) having computer-readable program instructions thereon that cause a processor to perform operations.

[0092] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device (e.g., a raised structure in a groove or a punched card with instructions recorded thereon), and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be construed as themselves transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through a wire.

[0093] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0094] The computer readable program code / instructions for performing the operations may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., SmallTalk, C++, etc.) and procedural programming languages ​​(e.g., the "C" programming language or similar programming languages). The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, which may be partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or a connection may be established with an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits to perform various aspects or operations.

[0095] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device create a means for implementing the functions / actions specified in the flowcharts and / or block diagram blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium and may direct the computer, programmable data processing device, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in the flowcharts and / or block diagram blocks.

[0096] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device, so that a series of operational steps are executed on the computer, other programmable device, or other device to produce a computer-implemented process, whereby the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0097] The flowcharts and block diagrams in the figures illustrate the architecture, functions and operations of possible implementations of the systems, methods and computer-readable media according to various embodiments. In this regard, each block in the flowchart or block diagram may represent a module, section or portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. Compared to the blocks depicted in the figures, the methods, computer systems and computer-readable media may include additional blocks, fewer blocks, different blocks or blocks arranged in different ways. In some optional implementations, the functions noted in the blocks may not occur in the order noted in the figures. For example, two blocks shown in succession may actually be executed simultaneously or substantially simultaneously, or sometimes the blocks may be executed in the opposite order, depending on the functions involved. It should also be noted that each block illustrated in the block diagram and / or flowchart and the combination of the blocks illustrated in the block diagram and / or flowchart can be implemented by a system based on dedicated hardware that performs a specified function or action or implements a combination of dedicated hardware and computer instructions.

[0098] Obviously, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it is understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0099] Unless otherwise clearly described, the elements, actions or instructions used herein should not be understood as critical or necessary. In addition, as used herein, the articles "one" and "an" are intended to include one or more projects and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more projects (e.g., related projects, unrelated projects, a combination of related projects and unrelated projects, etc.), and can be used interchangeably with "one or more". In the case of only one project being meant, the term "one" or similar language is used. In addition, as used herein, the terms "have", "have", "contain" or similar terms are intended to be open terms. In addition, the phrase "based on" means "based at least in part on", unless otherwise clearly stated.

[0100] Descriptions of various aspects and embodiments have been given for purposes of illustration, but are not intended to be exhaustive or limiting to the disclosed embodiments. Even though combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features may be combined in ways that are not specifically stated in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly subordinate to only one claim, the disclosure of possible implementations includes the combination of each dependent claim with every other claim in the claim set. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein have been chosen to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the marketplace, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for encoding video data, characterized in that Executed by a processor, the method includes: receiving video data, the video data comprising a reference view and a current view; for a current block in the current view, identifying a collocated block in the reference view; after performing a corresponding perspective transform on the reference view to map coordinates from the reference view to the current view, a top left corner position of the current block relative to the current view and a top left corner position of the collocated block relative to the reference view have the same position offset; a prediction block vector being calculated based on an offset vector between the current block and the collocated block and a disparity vector between a reference block in the reference view and the collocated block; the reference block being a block in the reference view that starts block matching with the collocated block as a starting point for block matching; the disparity vector being used to indicate the disparity between the reference block and the collocated block in the reference view; the prediction block vector being a block vector pointing from the current block to the reference block; the offset vector pointing from the current block to the collocated block, and the disparity vector pointing from the collocated block to the reference block; the prediction block vector being added to a candidate vector list for a block vector coding mode; and the candidate vector list being sorted based on proximity between the current view and a plurality of views corresponding to the plurality of candidate vectors, or based on a quality sort in the reference view; and The video data of the current block is encoded based on the calculated prediction block vector.

2. The method according to claim 1, characterized in that The video data is encoded based on predicting a position of the reference block in the reference view by applying the calculated prediction block vector to the current block in the current view.

3. The method according to claim 2, characterized in that The encoder performs block vector estimation based on using the position of the collocated block and the position of the reference block as one or more possible matching positions.

4. The method according to claim 1, wherein The prediction block vector is added to a candidate vector list for intra block copy merge mode or intra skip mode.

5. The method according to claim 4, characterized in that The candidate vector list is sorted based on proximity between the current view and a plurality of views corresponding to a plurality of candidate vectors, or based on a quality ranking in the reference view.

6. The method according to any one of claims 1 to 5, characterized in that The encoder performs block vector estimation based on using the endpoints of the predicted block vector as search centers.

7. The method according to any one of claims 1 to 5, characterized in that A disparity vector between the collocated block in the reference view and the reference block is derived based on a geometric relationship between the current view and the reference view.

8. A computer system for encoding video data, characterized in that The computer system comprises: one or more computer-readable non-transitory storage media configured to store computer program code; and, One or more computer processors are configured to access the computer program code and operate according to instructions of the computer program code to implement the method according to any one of claims 1 to 7.

9. A computer system for encoding video data, characterized in that The computer system comprises: a memory for storing computer program instructions; A processor, configured to call the computer program instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable medium having stored thereon a computer program for encoding video data, the computer program being configured to cause one or more computer processors to execute the method according to any one of claims 1 to 7.

11. A method for processing a video stream, characterized in that: The video code stream is generated according to the method according to any one of claims 1-7.

Images