Video decoding method and device, computer system and storage medium

Abstract

The invention provides a video decoding method and device, a computer system and a storage medium. Video data including one or more chunks is received. A displacement vector of a current block of the one or more blocks encoded in an intra block copy mode or a string matching mode is predicted based on an encoded block vector or a string offset vector corresponding to one or more spatially adjacent blocks or spatially non-adjacent blocks of the one or more blocks. And decoding the video data based on the predicted displacement vector of the current block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Application No. 17 / 240,567, filed on April 26, 2021, which claims priority to U.S. Provisional Application No. 63 / 038,020, filed on June 11, 2020. The entire disclosure of the prior application is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to the field of data processing, and more particularly, to video encoding and decoding. Background Art

[0004] Block-based compensation from different pictures is also called motion compensation. Similarly, block compensation can also be done from previously reconstructed areas within the same picture. This is called intra-picture block compensation, current picture reference (CPR for short), or intra-block copy (IBC for short). Summary of the invention

[0005] Embodiments relate to methods, systems, and computer-readable media for video decoding. According to one aspect, a method for video decoding is provided. The method may include receiving video data including one or more blocks. Based on a coded block vector or a string offset vector corresponding to one or more spatially adjacent blocks or spatially non-adjacent blocks in the one or more blocks, predicting a displacement vector of a current block encoded in an intra-block copy mode or a string matching mode in the one or more blocks. Decoding the video data based on the predicted displacement vector of the current block.

[0006] According to another aspect, a video decoding device is provided. The device may include: a receiving module configured to receive video data including one or more blocks; a prediction module configured to predict a displacement vector of a current block encoded in an intra-block copy mode or a string matching mode in the one or more blocks based on a coding block vector or a string offset vector corresponding to one or more spatially adjacent blocks or spatially non-adjacent blocks in the one or more blocks; and a decoding module configured to decode the video data based on the predicted displacement vector of the current block.

[0007] According to another aspect, a computer system for video decoding is provided. The computer system may include one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for at least one of the one or more processors to execute via at least one of the one or more memories, whereby the computer system is capable of performing a method. The method may include receiving video data including one or more blocks. Based on a coded block vector or a string offset vector corresponding to one or more spatially adjacent blocks or spatially non-adjacent blocks in the one or more blocks, a displacement vector of a current block encoded in an intra-block copy mode or a string matching mode in the one or more blocks is predicted. The video data is decoded based on the predicted displacement vector of the current block.

[0008] According to another aspect, a computer-readable medium for video decoding is provided. The computer-readable medium may include one or more computer-readable storage devices and program instructions stored on at least one of the one or more tangible storage devices, and the program instructions may be executed by a processor. The program instructions may be executed by a processor to perform a method, which may accordingly include receiving video data including one or more blocks. Based on the coded block vectors or string offset vectors corresponding to one or more spatially adjacent blocks or spatially non-adjacent blocks in the one or more blocks, a displacement vector of a current block encoded in an intra-block copy mode or a string matching mode in the one or more blocks is predicted. The video data is decoded based on the predicted displacement vector of the current block. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other objects, features and advantages will become apparent from the following detailed description of exemplary embodiments read in conjunction with the accompanying drawings. The various features of the drawings are not drawn to scale because the illustrations are for clarity in order to facilitate understanding by those skilled in the art in conjunction with the detailed description. In the drawings:

[0010] Figure 1 illustrates a networked computer environment according to at least one embodiment;

[0011] Figure 2A is a block diagram depicting intra block copying in a picture according to at least one embodiment;

[0012] Figure 2B is a block diagram of intra block compensation with one coding tree unit (CTU) search range according to at least one embodiment;

[0013] Figure 2C is a block diagram of HEVC / VVC spatial merging candidates according to at least one embodiment;

[0014] Figure 2D is a block diagram of a spatial block according to at least one embodiment;

[0015] Figure 3 is an operational flow diagram of steps performed by a program for encoding and decoding video data based on spatial displacement vectors in accordance with at least one embodiment;

[0016] Figure 4 According to at least one embodiment Figure 1 block diagrams of internal and external components of computers and servers as shown in;

[0017] Figure 5 According to at least one embodiment, Figure 1 A block diagram of an exemplary cloud computing environment of a computer system is shown; and

[0018] Figure 6 According to at least one embodiment Figure 5 A block diagram of the functional layers of an exemplary cloud computing environment. DETAILED DESCRIPTION

[0019] Specific embodiments of the claimed structures and methods are disclosed herein. However, it is to be understood that the disclosed embodiments are merely examples of the claimed structures and methods that may be embodied in various forms. However, these structures and methods may be embodied in many different forms and should not be construed as being limited to the exemplary embodiments described herein. On the contrary, these exemplary embodiments are provided so that this disclosure is comprehensive and complete and fully conveys the scope to those skilled in the art. In the specification, details of well-known features and technologies may be omitted to avoid unnecessary confusion of the presented embodiments.

[0020] Embodiments generally relate to the field of data processing, and more specifically, to video encoding and decoding. The exemplary embodiments described below provide a system, method, and computer program to encode and decode video data based on spatial displacement vectors. Thus, some embodiments have the ability to improve the computing field by allowing improved video encoding and decoding based on using spatially adjacent or non-adjacent blocks for a current block in the video data.

[0021] As mentioned earlier, block-based compensation from different pictures is also called motion compensation. Similarly, block compensation can also be done from a previously reconstructed area within the same picture. This is called intra-picture block compensation, current picture reference (CPR for short) or intra-block copy (IBC for short). However, currently in VVC, the search range of the CPR mode is limited to the current CTU. The effective memory requirement for storing reference samples for the CPR mode is 1 CTU sized sample. Considering that the existing reference sample memory stores the reconstructed samples in the current 64×64 area, an additional 3 64×64 sized reference sample memories are required. Therefore, it may be advantageous to extend the effective search range of the CPR mode to a part of the left CTU while the total memory requirement for storing reference pixels remains unchanged.

[0022] Various aspects are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer-readable media of various embodiments. It should be understood that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0023] The exemplary embodiments described below provide a system, method and computer program for encoding and decoding video data based on spatial displacement vectors. Figure 1 , a functional block diagram of a networked computer environment shows a video codec system 100 (hereinafter referred to as "system") for encoding and decoding video data based on spatial displacement vectors. It should be understood that Figure 1 This only provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Various modifications may be made to the described environment based on design and implementation requirements.

[0024] System 100 may include computer 102 and server computer 114. Computer 102 may communicate with server computer 114 via communication network 110 (hereinafter referred to as "network"). Computer 102 may include processor 104 and software program 108 stored in data storage device 106 and capable of interacting with a user and communicating with server computer 114. Figure 4 As discussed, computer 102 may include internal components 800A and external components 900A, respectively, and server computer 114 may include internal components 800B and external components 900B, respectively. Computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database.

[0025] The server computer 114 may also operate in a cloud computing service model, such as software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS), as described below with reference to Figure 5 and Figure 6 The server computer 114 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0026] The server computer 114 capable of video encoding can run a video encoding program 116 (hereinafter referred to as "program"), which can interact with the database 112. Figure 3 The video encoding program method is explained in more detail. In one embodiment, the computer 102 can operate as an input device including a user interface, and the program 116 can be primarily operated on the server computer 114. In an alternative embodiment, the program 116 can be primarily operated on one or more computers 102, and the server computer 114 can be used to process and store data used by the program 116. It should be noted that the program 116 can be a stand-alone program, or can be integrated into a larger video encoding program.

[0027] However, it should be noted that in some cases, the processing of program 116 may be shared at any ratio between computer 102 and server computer 114. In another embodiment, program 116 may be executed on more than one computer, server computer, or some combination of computers and server computers, such as multiple computers 102 communicating with a single server computer 114 via network 110. In another embodiment, for example, program 116 may be executed on multiple server computers 114 communicating with multiple client computers via network 110. Alternatively, the program may be executed on a network server that communicates with the server and multiple client computers via the network.

[0028] The network 110 may include a wired connection, a wireless connection, a fiber optic connection, or some combination thereof. In general, the network 110 may be any combination of connections and protocols that support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, a fiber-based network, etc., and / or a combination of these or other types of networks.

[0029] Figure 1 The number and arrangement of devices and networks shown are provided as examples. Figure 1 There may be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently than those shown. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of system 100 may perform one or more functions described as being performed by another set of devices of system 100.

[0030] Reference now Figure 2A , which shows a block diagram depicting intra-block copying in a picture 200A. The picture 200A may include a current block 202A, a reference block 202B, and a block vector BV. The block vector BV may be a displacement vector that may indicate the offset between the current block 202A and the reference block 202B. Unlike a motion vector in motion compensation that may be any value (positive or negative in the x-direction or the y-direction), this block vector BV has some constraints to ensure that the reference block 202B pointed to is available and has been reconstructed. Moreover, for the sake of parallel processing, some reference areas that are tile boundaries or wavefront trapezoid boundaries are also excluded.

[0031] The coding of block vectors can be either explicit or implicit. In explicit mode (or Advanced Motion Vector Prediction (AMVP) mode in inter-coding), the difference between the block vector and its prediction is signaled; in implicit mode, the block vector is fully recovered from its prediction, in a similar way to motion vectors in merge mode. In some embodiments, the resolution of block vectors is restricted to integer positions; in other systems, it may be allowed to point to fractional positions.

[0032] A block level flag (called IBC flag) can be used to signal the use of intra block copy at the block level. In one embodiment, the flag is signaled when the current block 202A is not encoded in merge mode. Alternatively, the flag can be signaled by means of a reference index. This is done by treating the current decoded picture as a reference picture. In HEVC SCC (Screen Content Coding), such reference pictures are placed at the last position in the list. This special reference picture is also managed with other temporal reference pictures in the decoder picture buffer (DPB).

[0033] There are also some changes for intra block copy, such as treating intra block copy as a third mode different from intra or inter prediction modes. By doing so, block vector prediction in merge mode and AMVP mode is separated from regular inter mode. For example, a separate merge candidate list is defined for intra block copy mode, where all entries in this list are block vectors. Similarly, the block vector prediction list in intra block copy AMVP mode consists only of block vectors. The general rule that applies to these two lists is that they can follow the same logic as the inter merge candidate list or AMVP prediction value list in terms of the candidate derivation process. For example, 5 spatial neighbors in HEVC or VVC inter merge mode are accessed for intra block copy to derive its own merge candidate list.

[0034] Reference now Figure 2B , which depicts a block diagram 200B of intra block compensation with a one coding tree unit (CTU) search range. The search may be performed through stages 204A-204D. Currently in VVC, the search range of the CPR mode is limited to the current CTU. The effective memory requirement for storing reference samples for the CPR mode is 1 CTU sized sample. Considering that the existing reference sample memory stores the reconstructed samples in the current 64×64 region C, it is possible to use an additional 3 64×64 sized reference sample memories. Therefore, the effective search range of the CPR mode can be extended to a portion of the left CTU, while the total memory requirement for storing reference pixels remains unchanged (1 CTU size, a total of 4 64×64 reference sample memories). The bitstream consistency conditions that the valid block vector (mvL, 1 / 16 pixel resolution) should follow are listed as follows. For example, the luminance motion vector mvL may be subject to one or more constraints.

[0035] A1: When the block availability derivation process specified in the neighboring block availability check process is called with the current luma position (xCurr, yCurr) set equal to (xCb, yCb) and the neighboring luma position (xCb+(mvL[0]>>4), yCb+(mvL[1]>>4)) as input, and the output should be equal to true (TRUE).

[0036] A2: A2: When the block availability derivation process specified in the neighboring block availability check process is called with the current luminance position (xCurr, yCurr) set equal to (xCb, yCb) and the neighboring luminance position (xCb+(mvL[0]>>4)+cbWidth-1, yCb+(mvL[1]>>4)+cbHeight-1) as input, and the output should be equal to true (TRUE).

[0037] B1: One or both of the following conditions should be true. The value of (mvL[0]>>4)+cbWidth is less than or equal to 0. The value of (mvL[1]>>4)+cbHeight is less than or equal to 0.

[0038] C1: The following conditions should be true:

[0039] (yCb+(mvL[1]>>4))>>CtbLog2SizeY=yCb>>CtbLog2SizeY

[0040] (yCb+(mvL[1]>>4)+cbHeight-1)>>CtbLog2SizeY=yCb>>CtbLog2SizeY(xCb+(mvL[0]>>4))>>CtbLog2SizeY>=(xCb>>CtbLog2SizeY)-1

[0041] (xCb+(mvL[0]>>4)+cbWidth-1)>>CtbLog2SizeY<=(xCb>>CtbLog2SizeY)

[0042] C2: When (xCb+(mvL[0]>>4))>>CtbLog2SizeY is equal to (xCb>>CtbLog2SizeY)-1, the block availability derivation process specified in the neighboring block availability check process is called with the current luminance position (xCurr,yCurr) set equal to (xCb,yCb) and the neighboring luminance positions (((xCb+(mvL[0]>>4)+CtbSizeY)>>(CtbLog2SizeY-1))<<(CtbLog2SizeY-1),((yCb+(mvL[1]>>4))>>(CtbLog2SizeY-1))<<(CtbLog2SizeY-1)) as input and the output shall be equal to false (FALSE).

[0043] Reference now Figure 2C , which depicts a block diagram 200C of HEVC / VVC spatial merge candidates. The five spatial merge candidates for HEVC and VVC may include A0, A1, B0, B1, and B2. The order of forming a candidate list from these positions may be A0->B0->B1->A1->B2.

[0044] The history-based motion vector prediction (HMVP) merge candidate is added to the merge list after the spatial MVP (motion vector prediction) and the temporal MVP (TMVP). In this method, the motion information of the previously coded blocks is stored in a table and used as the MVP of the current CU. A table containing multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the relevant motion information is added as a new HMVP candidate to the last entry of the table.

[0045] In VTM3, the HMVP table size S is set to 6, which means that up to 6 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a constrained first-in-first-out (FIFO) rule is used, where a redundancy check is first applied to find if the same HMVP exists in the table. If found, the same HMVP is deleted from the table, and then all HMVP candidates are moved forward.

[0046] HMVP candidates can be used in the construction process of the merge candidate list. The latest HMVP candidates in the table are checked in turn and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates.

[0047] In order to reduce the number of redundant check operations, one or more simplifications are introduced. The number of HMVP candidates for merge list generation is set to (N<=4)?M:(8–N), where N represents the number of existing candidates in the merge list and M represents the number of HMVP candidates available in the table. Once the total number of available merge candidates reaches the maximum allowed merge candidate minus 1, the construction process of the merge candidate list from HMVP is terminated.

[0048] When Intra-BC is run as a mode separate from Inter-mode, a separate history buffer (called HBVP) is used to store previously encoded Intra-BC block vectors.

[0049] As a mode separated from inter prediction, it is desirable to have a simplified block vector derivation process for intra block copy mode. A similar history-based block vector prediction value buffer can be used to perform BV prediction. In the following, some information is provided for some specific uses of such HBVP.

[0050] A HBVP buffer is established to record the BV information of the previous IBC coded block, including some other auxiliary information, such as block size, block position, etc.

[0051] Based on the recorded information, for each current block, the BVs in the HBVP that meet the following conditions are divided into corresponding categories:

[0052] Category 0: The area (width * height) of the coding block is greater than or equal to the threshold (64 pixels);

[0053] Category 1: The frequency of BV is greater than or equal to 2;

[0054] Category 2: The coding block coordinates (upper left corner) are to the left of the current block;

[0055] Category 3: The coding block coordinates (upper left corner) are above the current block;

[0056] Category 4: The coding block coordinates (upper left corner) are on the upper left side of the current block;

[0057] Category 5: The coding block coordinates (upper left corner) are on the upper right side of the current block;

[0058] Category 6: The coding block coordinates (upper left corner) are at the lower left side of the current block.

[0059] For each class, the BV of the most recently coded block is derived as the BV predictor candidate. A CBVP list is constructed by appending the BV predictors of each class in order from 0 to 6.

[0060] A coding block can be divided into several consecutive strings, each followed by the next string in the scan order. The scan order can be raster scan or traverse scan. For each string, the string offset vector (SV) and the string length are signaled. The SV is used to indicate the position of the reference string in the reference area. The length is used to indicate the length of the current / reference string. If a sample in the current block cannot find a matching sample in the reference area, an escape sample is signaled and its value is encoded directly.

[0061] Reference now Figure 2D , which depicts a block diagram 200D of a spatial block. The spatial block may include spatially adjacent blocks A0-E0 and spatially non-adjacent blocks A1-E1, A2-E2, and A3-E3. Vector prediction includes block vector prediction for IBC mode and SV prediction for string matching mode, which may refer to skip mode, direct / merge mode, or vector prediction with differential coding. Spatially adjacent blocks A0-E0 may refer to encoded blocks next to the current block. Other positions along the top row or left column of the current block may also be considered as spatially adjacent positions. In contrast, spatially non-adjacent blocks A1-E1, A2-E2, and A3-E3 refer to previously encoded blocks that are not spatially adjacent blocks (these blocks may be found along the top row or left column of the current block).

[0062] According to one or more embodiments, the encoded BV or SV in the spatially adjacent blocks or the spatially non-adjacent blocks may be used to predict the current block encoded in the IBC mode. In one embodiment, only the spatially non-adjacent blocks encoded in the IBC mode or the string matching mode are considered as candidates for predicting the BV of the current block encoded in the IBC mode. In another embodiment, only the spatially non-adjacent blocks encoded in the IBC mode are considered as candidates for predicting the BV of the current block encoded in the IBC mode. Both the spatially adjacent blocks and the spatially non-adjacent blocks encoded in the string matching mode are considered as candidates for predicting the BV of the current block encoded in the IBC mode.

[0063] According to one or more embodiments, the encoded BV or SV in the spatially adjacent blocks or the spatially non-adjacent blocks may be used to predict the current block encoded in the string matching mode. In one embodiment, only the spatially non-adjacent blocks encoded in the IBC mode or the string matching mode are considered as candidates for predicting the SV of the current block encoded in the string matching mode. In another embodiment, the spatially adjacent blocks encoded in the IBC mode or the string matching mode are considered as candidates for predicting the SV of the current block encoded in the string matching mode. In another embodiment, both the BV and SV in the spatially adjacent blocks and the spatially non-adjacent blocks encoded in the IBC mode or the string matching mode are considered as candidates for predicting the SV of the current block encoded in the string matching mode.

[0064] According to one or more embodiments, the spatial candidates for BV or SV prediction discussed above may be placed before the historically based BV or SV candidates in the BV or SV prediction value list. Alternatively, the spatial candidates for BV or SV prediction discussed above may be placed after the historically based BV or SV candidates in the BV or SV prediction value list. When category-based prediction is used, as described in the introduction, placing spatially adjacent blocks or non-adjacent blocks in the list may require position and size information associated with the candidate. Using this information, the new spatial candidate can be placed in the correct prediction value category. Therefore, in addition to the BV or SV information, this information is also required to store each spatially adjacent block or spatially non-adjacent block.

[0065] According to one or more embodiments, when multiple spatial candidates are placed in a prediction list, a certain order is followed to access the specified spatial positions. For example, to predict SV in string matching mode, only spatial non-adjacent blocks are considered. Positions A1, B1, C1, D1, and E1 can then be accessed in sequence. If one of the blocks is encoded in IBC mode or string matching mode, the vector (BV or SV) associated with the block can be used to predict the SV in the current block. Similar examples can be derived to predict the BV in the current block encoded with IBC.

[0066] Reference now Figure 3, which depicts an operational flow diagram of the steps of a method 300 performed by a program for encoding and decoding video data based on spatial displacement vectors.

[0067] At 302 , method 300 may include receiving video data including one or more blocks.

[0068] At 304, method 300 may include predicting a displacement vector of a current block encoded in an intra block copy mode or a string matching mode in the one or more blocks based on a coded block vector or a string offset vector corresponding to one or more spatially adjacent or non-adjacent blocks in the one or more blocks.

[0069] At 306 , method 300 may include decoding the video data based on the displacement vector of the predicted current block.

[0070] It should be understood that Figure 3 This only provides an illustration of one implementation and does not imply any limitations on how to implement different embodiments. Various modifications may be made to the described environment based on design and implementation requirements.

[0071] Figure 4 According to an exemplary embodiment Figure 1 400 is a block diagram of the internal and external components of a computer shown in FIG. Figure 4 This only provides an illustration of one implementation and does not imply any limitation to the environments in which different embodiments may be implemented. Various modifications may be made to the described environment based on design and implementation requirements.

[0072] Computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) can include Figure 4 Respective sets of internal components 800A, 800B and external components 900A, 900B are shown. Each set of internal components 800 includes one or more processors 820 connected to one or more buses 826, one or more computer-readable random access memories (RAM) 822 and one or more computer-readable read-only memories (ROM) 824, including one or more operating systems 828, and one or more computer-readable tangible storage devices 830.

[0073] Processor 820 is implemented in hardware, firmware or a combination of hardware and software. Processor 820 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC) or other types of processing components. In some embodiments, processor 820 includes one or more processors that can be programmed to perform functions. Bus 826 includes components that allow communication between internal components 800A and 800B.

[0074] One or more operating systems 828, and server computer 114 ( Figure 1 ) on the software program 108 ( Figure 1 ) and video encoding program 116 ( Figure 1 ) are stored on one or more corresponding computer-readable tangible storage devices 830 for execution by one or more corresponding processors 820 via one or more corresponding RAMs 822 (which typically include cache memories). Figure 4 In the illustrated embodiment, each computer-readable tangible storage device 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 830 is a semiconductor storage device such as ROM 824, an erasable programmable read-only memory (EPROM), flash memory, an optical disk, a magneto-optical disk, a solid-state disk, a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or other types of non-volatile computer-readable tangible storage devices capable of storing computer programs and digital information.

[0075] Each set of internal components 800A, 800B also includes a read / write (R / W) drive or interface 832 for reading from or writing to one or more portable computer readable tangible storage devices 936 (e.g., CD-ROMs, DVDs, memory sticks, tapes, disks, optical disks, or semiconductor storage devices). Figure 1 ) and video encoding program 116 ( Figure 1 ) can be stored on one or more corresponding portable computer-readable tangible storage devices 936, read via a corresponding R / W drive or interface 832 and loaded into a corresponding hard disk drive 830.

[0076] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G, 4G or 5G wireless interface card or other wired or wireless communication link. Figure 1 ) on the software program 108 ( Figure 1) and video encoding program 116 ( Figure 1 ) can be downloaded from an external computer to the computer 102 via a network (eg, the Internet, a local area network or other network, a wide area network) and a corresponding network adapter or interface 836 ( Figure 1 ) and the server computer 114. The software program 108 and the video encoding program 116 on the server computer 114 are loaded into the corresponding hard disk drive 830 from the network adapter or interface 836. The network may include copper wire, optical fiber, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers.

[0077] Each set of external components 900A, 900B may include a computer display 920, a keyboard 930, and a computer mouse 934. External components 900A, 900B may also include a touch screen, a virtual keyboard, a touch pad, a pointing device, and other human-computer interface devices. Each set of internal components 800A, 800B also includes a device driver 840 to interface with the computer display 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W driver or interface 832, and the network adapter or interface 836 include hardware and software (stored in the storage device 830 and / or ROM 824).

[0078] It should be understood that although the present application includes a detailed description of cloud computing, the embodiments listed herein are not limited to cloud computing environments. Instead, certain embodiments can be implemented in conjunction with any other type of computing environment now known or later developed.

[0079] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. Cloud models can include at least five characteristics, at least three service models, and at least four deployment models.

[0080] Features are as follows:

[0081] On-demand self-service: Cloud users can automatically and unilaterally provision computing capabilities, such as server time and network storage, as needed without manual interaction with the service provider.

[0082] Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use on heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and personal digital assistants).

[0083] Resource pooling: Pooling a provider's computing resources to serve multiple users using a multi-tenant model, dynamically allocating and reallocating different physical and virtual resources based on demand. Location independence means that users typically have no control or knowledge of the exact location of the resources provided, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0084] Rapid elasticity: Capabilities that can be provisioned quickly and elastically, and in some cases automatically provisioned to scale out quickly, and released quickly to scale in quickly. To the user, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.

[0085] Measured service: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to providers and users of the services being used.

[0086] The service model is as follows:

[0087] Software as a Service (SaaS): The functionality provided to users is to use the provider's applications running on a cloud infrastructure. Applications can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). Users do not manage or control the underlying cloud infrastructure including networks, servers, operating systems, storage, or even individual application functions, but may be limited to user-specific application configuration settings.

[0088] Platform as a Service (PaaS): The functionality provided to users is to deploy user-created or acquired applications on cloud infrastructure. The user-created or acquired applications are created using programming languages ​​and tools supported by the provider. Users do not manage or control the underlying cloud infrastructure including networks, servers, operating systems or storage, but control the deployed applications and possible application hosting environment configuration.

[0089] Infrastructure as a Service (IaaS): The functionality provided to users is to provide processing, storage, network and other basic computing resources, where users can deploy and run arbitrary software including operating systems and applications. Users do not manage or control the underlying cloud infrastructure, but control the operating system, storage, deployed applications, and may have limited control over selected network components (e.g., host firewalls).

[0090] The deployment model is as follows:

[0091] Private cloud: Cloud infrastructure is run only for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0092] Community cloud: The cloud infrastructure is shared by multiple organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0093] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0094] Hybrid cloud: A cloud infrastructure composed of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies to enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0095] The cloud computing environment is service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure consisting of a network of interconnected nodes.

[0096] Reference Figure 5 , which shows an exemplary cloud computing environment 500. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10, and local computing devices used by cloud users (such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C and / or automobile computer systems 54N) can communicate with these cloud computing nodes 10. The cloud computing nodes 10 can communicate with each other. They can be physically or virtually grouped in one or more networks, such as the private clouds, community clouds, public clouds, hybrid clouds, or combinations thereof described above (not shown). This allows the cloud computing environment 500 to provide infrastructure, platforms and / or software as services without the cloud users needing to maintain resources for these services on local computing devices. It should be understood that Figure 5The types of computing devices 54A-N shown are exemplary only, and cloud computing node 10 and cloud computing environment 500 may communicate with any type of computer device over any type of network and / or network-addressable connection (eg, using a web browser).

[0097] Reference Figure 6 , which shows a cloud computing environment 500 ( Figure 5 ) provides a set of functional abstraction layers 600. It should be understood that Figure 6 The components, layers and functions shown are exemplary only, and the embodiments are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0098] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a host 61, a server 62 based on a RISC (Reduced Instruction Set Computer) architecture, a server 63, a blade server 64, a storage device 65, and a network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0099] Virtualization layer 70 provides an abstraction layer from which examples of virtual entities may be provided: virtual servers 71 , virtual storage 72 , virtual networks 73 including virtual private networks, virtual applications and operating systems 74 , and virtual clients 75 .

[0100] In one example, the management layer 80 may provide the following functions. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks in a cloud computing environment. Metering and pricing 82 provides cost monitoring when resources are utilized in a cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud users and tasks and provides protection for data and other resources. User portal 83 provides access to the cloud computing environment for users and system administrators. Service level management 84 provides allocation and management of cloud computing resources to meet the required service levels. Service level agreement (SLA) planning and implementation 85 provides pre-arrangement and acquisition of cloud computing resources in anticipation of future needs according to the SLA.

[0101] The workload layer 90 provides examples of functions that can utilize the cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91, software development and lifecycle management 92, virtual classroom teaching implementation 93, data analysis processing 94, transaction processing 95, and video encoding 96. Video encoding 96 can encode and decode video data based on spatial displacement vectors.

[0102] Some embodiments may be directed to systems, methods and / or computer-readable media at any possible level of technical detail integration.Computer-readable media may include non-volatile computer-readable storage media (or media) having stored thereon computer-readable program instructions that cause a processor to perform operations.

[0103] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or a structure protruding in a groove on which instructions are recorded), and any suitable combination of the foregoing. The computer-readable storage medium used herein should not be interpreted as being a volatile signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0104] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to store them in a computer-readable storage medium in the corresponding computing / processing device.

[0105] The computer readable program code / instruction for performing the operation can be an assembly instruction, an instruction set architecture (ISA, instruction-set-architecture) instruction, a machine instruction, a machine-dependent instruction, a microcode, a firmware instruction, a state setting data, a configuration data for an integrated circuit, or a source code or object code written in any combination of one or more programming languages, the programming language includes object-oriented programming languages ​​such as Smalltalk, C++, and a procedural programming language (e.g., "C" programming language) or similar programming languages. The computer readable program instruction can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits to perform aspects or operations.

[0106] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions are executed via the processor of the computer or other programmable data processing device to create a device for implementing the functions / actions specified by one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can instruct a computer, a programmable data processing device, and / or other equipment to operate in a specific manner, so that the computer-readable storage medium in which the instructions are stored contains an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified by one or more blocks in the flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operating steps to be performed on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified by one or more boxes of the flowchart and / or block diagram.

[0108] The flow charts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of the systems, methods and computer-readable media according to various embodiments. In this regard, each box in the flow chart or block diagram may represent a module, a fragment or a portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. Compared to what is depicted in the figure, the method, computer system and computer-readable medium may include more blocks, fewer blocks, different blocks or blocks of different arrangements. In some optional embodiments, the functions marked in the box may not occur in the order marked in the accompanying drawings. For example, depending on the functions involved, the two boxes shown in succession may actually be executed simultaneously or substantially simultaneously, or the boxes may sometimes be executed in reverse order. It should also be noted that each box of the block diagram and / or flow chart and the combination of the boxes of the block diagram and / or flow chart may be implemented by a system based on dedicated hardware that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0109] It is apparent that the systems and / or methods described herein may be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited by these embodiments. Therefore, the operation and behavior of these systems and / or methods are described herein without reference to specific software code - it should be understood that software and hardware can be designed based on the description herein to implement these systems and / or methods.

[0110] Unless explicitly stated, the elements, actions or instructions used in this article shall not be interpreted as key or necessary. In addition, as used herein, the articles "a" and "an" are intended to include one or more items, and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related items and unrelated items, etc.), and can be used interchangeably with "one or more". In the case of only wishing one item, the term "one" or similar language is used. In addition, as used herein, the terms "has", "have", "having" etc. are intended to be open terms. Further, the phrase "based on" is intended to mean "based at least in part on", unless explicitly stated otherwise.

[0111] Descriptions of various aspects and embodiments have been given for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Even if combinations of features are described in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways that are not specifically described in the claims and / or not disclosed in the specification. Although each dependent claim listed below may be directly subordinate to only one claim, the disclosure of possible implementations includes the combination of each dependent claim with each other claim in the claim set. Without departing from the scope of the described embodiments, many modifications and variations will be apparent to those of ordinary skill in the art. The terms used herein are selected to best explain the principles of the embodiments of the present application, the practical application or technical improvement of the technology found on the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A video decoding method, characterized in that: include: Receiving video data in a video stream including one or more blocks; Predicting a displacement vector of a current block encoded in an intra block copy mode among the one or more blocks based on coded block vectors corresponding to one or more spatially adjacent blocks and one or more spatially non-adjacent blocks among the one or more blocks, wherein the one or more spatially non-adjacent blocks are encoded in a string matching mode or the intra block copy mode, the one or more spatially adjacent blocks are encoded in the string matching model, and the intra block copy mode is not the string matching mode; as well as The video data is decoded based on the displacement vector of the predicted current block.

2. The method according to claim 1, characterized in that Only spatially non-neighboring blocks coded in the Intra block copy mode or the string match mode are considered as candidates for predicting the displacement vector of the current block coded in the Intra block copy mode.

3. The method according to claim 1, characterized in that The coded block vectors or string offset vectors in the one or more spatially adjacent blocks and the one or more spatially non-adjacent blocks are used to predict the current block encoded in the string matching mode in the one or more blocks.

4. The method according to claim 3, characterized in that Only one or more spatially non-neighboring blocks encoded in the intra block copy mode or the string matching mode are considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

5. The method according to claim 3, characterized in that: The one or more spatial neighboring blocks encoded in the intra block copy mode or the string matching mode are considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

6. The method according to claim 3, characterized in that The coded block vectors and string offset vectors in the one or more spatially adjacent blocks and the one or more spatially non-adjacent blocks encoded in the intra-block copy mode or the string matching mode are all considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

7. The method according to claim 3, characterized in that The spatial candidates for predicting the coding block vector of the current block or the string offset vector are history-based block vector or string offset vector candidates in a block vector prediction value list or a string offset vector prediction value list.

8. The method according to claim 7, characterized in that Based on using class-based prediction, spatially adjacent or non-adjacent blocks, position information and size information are added to the prediction value list.

9. A video encoding method, characterized in that: Used to generate encoded video streams, including: Obtain one or more blocks of video data in a video stream to be encoded; Determining a displacement vector of a current block encoded in an intra block copy mode among the one or more blocks based on coded block vectors corresponding to one or more spatially neighboring blocks and one or more spatially non-neighboring blocks among the one or more blocks, wherein the one or more spatially non-neighboring blocks are encoded in a string matching mode or the intra block copy mode, the one or more spatially neighboring blocks are encoded in the string matching model, and the intra block copy mode is not the string matching mode; and The video data is encoded based on the determined displacement vector of the current block.

10. The method according to claim 9, characterized in that Only spatially non-neighboring blocks coded in the Intra block copy mode or the string match mode are considered as candidates for predicting the displacement vector of the current block coded in the Intra block copy mode.

11. The method according to claim 9, characterized in that The coded block vectors or string offset vectors in the one or more spatially adjacent blocks and the one or more spatially non-adjacent blocks are used to predict the current block encoded in the string matching mode in the one or more blocks.

12. The method according to claim 11, characterized in that Only one or more spatially non-neighboring blocks encoded in the intra block copy mode or the string matching mode are considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

13. The method according to claim 11, characterized in that The one or more spatial neighboring blocks encoded in the intra block copy mode or the string matching mode are considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

14. The method according to claim 11, characterized in that The coded block vectors and string offset vectors in the one or more spatially adjacent blocks and the one or more spatially non-adjacent blocks encoded in the intra-block copy mode or the string matching mode are all considered as candidates for predicting the string offset vector of the current block encoded in the string matching mode.

15. The method according to claim 11, characterized in that The spatial candidates for predicting the coding block vector of the current block or the string offset vector are history-based block vector or string offset vector candidates in a block vector prediction value list or a string offset vector prediction value list.

16. The method according to claim 15, characterized in that Based on using class-based prediction, spatially adjacent or non-adjacent blocks, position information and size information are added to the prediction value list.

17. A method for processing a video code stream, characterized in that: The video code stream is generated according to the video encoding method described in any one of claims 9-16, or is decoded according to the video decoding method described in any one of claims 1-8.

18. A video encoding and decoding device, characterized in that: The device comprises: at least one memory configured to store program code; and At least one processor is configured to read the program code and operate according to the instructions of the program code to execute the video decoding method described in any one of claims 1-8, or the video encoding method described in any one of claims 9-16.

19. A non-volatile computer-readable medium having a computer program for video encoding and decoding stored thereon, wherein the computer program is configured to cause one or more computer processors to execute the method according to any one of claims 1 to 17.

Images