Video compression method and apparatus adjusting elements of an inter-frame nearest motion vector array

Abstract

The present application provides a video compression method and device for dynamically adjusting the position of elements of a recent motion vector array in inter-frame prediction coding. The method and device effectively reduce the number of bits consumed in coding motion vectors and greatly improve coding efficiency by dynamically adjusting the elements of the motion vector array with high occurrence frequency to a specific part of the motion vector array and using a short code word to represent the storage address of the specific part.

Description

[0001] This application is a divisional application of the following original application:

[0002] The original application was filed on January 15, 2017.

[0003] Application number of the original application: 2017100271539

[0004] Original invention application title: Data compression method and apparatus for dynamically adjusting elements of the most recently encoded parameter array Technical Field

[0005] This invention relates to an encoding and decoding system for lossy or lossless compression of data, particularly a method and apparatus for encoding and decoding image and video data. Background Technology

[0006] As human society enters the era of big data, cloud computing, mobile computing, cloud-mobile computing, ultra-high definition (4K) and ultra-high definition (8K) video image resolution, 4G / 5G communication, and virtual reality, ultra-high compression ratio and extremely high quality data compression for various types of data, including big data, image data, and video data, has become an indispensable technology.

[0007] A dataset is a finite collection of data (e.g., a one-dimensional data queue, a two-dimensional data file, a single image frame, a video sequence, a transform domain, a transform block, multiple transform blocks, a three-dimensional scene, or a sequence of continuously changing three-dimensional scenes) arranged in a certain spatial (one-dimensional, two-dimensional, or multi-dimensional) shape, consisting of sample values ​​(e.g., bytes, bits, pixels, pixel components, spatial sampling points, transform domain coefficients). When encoding (and corresponding decoding) a dataset, especially a two-dimensional or higher-dimensional dataset, for data compression, the dataset is typically divided into several subsets with predetermined shapes, called encoding blocks (or decoding blocks from a decoding perspective, collectively referred to as encoding / decoding blocks). Encoding or decoding is performed block by block in a predetermined time sequence. At any given moment, the encoding block being encoded is called the current encoding block. At any given moment, the decoding block being decoded is called the current decoding block. The current encoding block or current decoding block is collectively referred to as the current encoding / decoding block or simply the current block. The sample value being encoded or decoded is called the current encoded sample value or the current decoded sample value, or simply the current sample value.

[0008] For an encoding / decoding block with a certain shape (not necessarily limited to a square or rectangle, but can be any other reasonable shape), in many cases it is necessary to divide it into finer primitives (basic units), and encode or decode one primitive at a time according to a predetermined time sequence. For all samples within a primitive, the same type of encoding or decoding operation is usually performed. At any given moment, the primitive being encoded or decoded is called the current primitive. Encoding a primitive results in one or more encoding parameters, ultimately producing a compressed data bitstream containing these encoding parameters. Decoding a primitive involves parsing the compressed data bitstream to obtain one or more encoding parameters, and then reconstructing the reconstructed data samples from these one or more encoding parameters.

[0009] Examples of primitives include codec blocks (the entire block is considered a primitive), sub-blocks, micro-blocks, strings, pixel strings, sample strings, index strings, and lines.

[0010] A significant characteristic of many common datasets is the presence of numerous matching (i.e., similar or even identical) patterns. For example, image and video sequences typically contain many matching pixel patterns. Therefore, existing data compression techniques often employ a matching (also known as prediction or compensation) approach. This involves using "predicted values" (also known as "compensation values" or "reference samples," such as "reference pixels") to match (also known as prediction, representation, representation, compensation, approximation, or approximation of) the samples currently being encoded or decoded (simply referred to as "current samples") to achieve lossless or lossy data compression. Simply put, the basic operation of the matching method is to copy the reference sample, i.e., the sample at the reference position, to the position of the current sample. Therefore, the matching or prediction method is also called the copying method. In the matching method, the reconstructed (also known as reconstruction or restoration) samples that have undergone at least some encoding and at least some decoding operations form a reference set (also known as the reference set space or reference buffer). The reconstructed samples and their positions in the reference set correspond one-to-one with the original samples and their positions in the original dataset.

[0011] During the encoding and decoding of a current block, the matching method divides the current block into several matching (also called prediction) primitives. Each matching primitive has one or more matching (encoding) parameters (also called matching relations, replication parameters, or reference relations) to represent its characteristics. One of the most important matching parameters is the displacement vector (also called motion vector, position offset, offset, relative position, location, relative address, address, relative coordinates, coordinates, relative index, index, etc.). The displacement vector represents the relative displacement between the sample value of the current primitive and its reference sample value, equivalent to the one-dimensional offset of the data samples arranged in one-dimensional data. Clearly, the reference position of the reference sample value can be derived from the displacement vector. The displacement vector of the current primitive is called the current displacement vector. Other examples of matching parameters include: matching type, matching length, unmatchable (predictable) samples, etc.

[0012] Examples of matching primitives include codec blocks, sub-blocks, micro-blocks, strings, pixel strings, sample strings, index strings, and lines.

[0013] Examples of matching methods (sometimes called matching types or matching patterns) include: block matching, sub-block matching, micro-block matching, string matching, pixel string matching, sample string matching, index string matching, and line matching.

[0014] Different matching patterns typically use at least one of the same encoding parameters. For example, block matching, sub-block matching, micro-block matching, string matching, pixel string matching, sample string matching, index string matching, and line matching all use the "displacement vector" encoding parameter.

[0015] In data compression, especially in matching methods, and particularly when the matching primitives are fine-grained, it is often necessary to encode numerous identical encoding parameters. To improve the encoding efficiency of a single encoding parameter, the values ​​of the most recently occurring encoding parameters are typically stored in a "recently occurring encoding parameter array." If the value of a new current encoding parameter equals an element in the recently occurring encoding parameter array, then only the storage address of that element in the array needs to be encoded and written to the compressed data stream; it is not necessary to directly encode the value of the current encoding parameter itself and write it to the compressed data stream. This method is simply called the recent parameter method.

[0016] On the other hand, in order to increase the frequency of the selection of the most recently encoded parameter array elements, the number of elements in the most recently encoded parameter array cannot be too small. In this way, a considerable number of bits are still needed to encode the storage address of the most recently encoded parameter array elements, which greatly affects the encoding efficiency of the most recently encoded parameter method. Summary of the Invention

[0017] To address this issue in data compression, particularly in the nearest-nearest-parameter (RNS) approach, this invention provides a data compression method and apparatus that dynamically adjusts the positions of elements in the nearest-nearest-parameter (RNS) array. This method and apparatus effectively reduce the number of bits consumed by the RNS approach and significantly improve encoding efficiency by dynamically adjusting frequently occurring RNS array elements to a specific location within the RNS array and using short codewords to represent the storage address of that specific location.

[0018] The primary technical feature of this invention is that the positions of the elements in the encoding parameter array are adjusted according to predetermined rules.

[0019] The most fundamental and unique technical feature of the encoding method or apparatus of the present invention is that, in the encoder, after encoding a new and valid current encoding parameter, the position of the most recent encoding parameter array element in the encoding parameter array is adjusted as necessary according to predetermined rules, such as the frequency of occurrence of the most recent encoding parameter array element and the time interval between occurrences. For example, the positions of two elements with different frequencies of occurrence are swapped, or elements with predetermined frequency of occurrence characteristics are moved to a specific part of the most recent encoding parameter array. Figure 1 This is a schematic diagram of the encoding method or apparatus of the present invention.

[0020] The most fundamental and unique technical feature of the decoding method or apparatus of the present invention is that, after decoding a new and valid current encoding parameter in the decoder, the position of the most recent encoding parameter array element in the encoding parameter array is adjusted according to predetermined rules, such as the frequency of occurrence of the most recent encoding parameter array element and the time interval between occurrences. For example, the positions of two elements with different frequencies of occurrence are swapped, or elements with predetermined frequency of occurrence characteristics are moved to a specific part of the most recent encoding parameter array. Figure 1 This is also a schematic diagram of the encoding method or apparatus of the present invention.

[0021] According to one aspect of the present invention, an encoding method or apparatus for compressing data, or a decoding method or apparatus, is provided, comprising at least the steps or modules that perform the following functions and operations:

[0022] For a given encoding parameter, an update operation is performed on the array of the most recent encoding parameters of that encoding parameter, following a predetermined rule.

[0023] From a first perspective, the present invention provides an encoding method for compressing data, characterized by comprising at least the following steps:

[0024] For a given encoding parameter, an update operation is performed on the array of the most recent encoding parameters of that encoding parameter, following a predetermined rule.

[0025] From a second perspective, the present invention provides an encoding apparatus for compressing data, characterized by comprising at least the following modules:

[0026] For a given encoding parameter, an update operation is performed on the array of the most recent encoding parameters of that encoding parameter, following a predetermined rule.

[0027] From a third perspective, the present invention provides a decoding method for compressing data, characterized by comprising at least the following steps:

[0028] For a given encoding parameter, an update operation is performed on the array of the most recent encoding parameters of that encoding parameter, following a predetermined rule.

[0029] From a fourth perspective, the present invention provides a decoding apparatus for compressing data, characterized by comprising at least the following modules:

[0030] For a given encoding parameter, an update operation is performed on the array of the most recent encoding parameters of that encoding parameter, following a predetermined rule.

[0031] This invention is applicable to the encoding and decoding of lossy data compression, and it is also applicable to the encoding and decoding of lossless data compression. This invention is applicable to the encoding and decoding of image data, and it is also applicable to the encoding and decoding of any one-dimensional, two-dimensional, or multi-dimensional data.

[0032] In this invention, the data compression involves one or a combination of the following types of data.

[0033] 1) One-dimensional data;

[0034] 2) Two-dimensional data;

[0035] 3) Multidimensional data;

[0036] 4) Images;

[0037] 5) Image sequences;

[0038] 6) Video;

[0039] 7) Three-dimensional scene;

[0040] 8) A sequence of continuously changing 3D scenes;

[0041] 9) Virtual reality scenarios;

[0042] 10) A sequence of continuously changing virtual reality scenes

[0043] 11) Images in pixel format;

[0044] 12) Transform domain data of the image;

[0045] 13) A collection of bytes with two or more dimensions;

[0046] 14) A set of two or more bits;

[0047] 15) A collection of pixels;

[0048] 16) A set of pixel components.

[0049] In this invention, when the data is an image, a sequence of images, a video, etc., the coding block or decoding block is a coding region or a decoding region of the image, including at least one of the following: the entire image, a sub-image of the image, a macroblock, a maximum coding unit (LCU), a coding tree unit (CTU), a coding unit (CU), a sub-region of a CU, a prediction unit (PU), and a transform unit (TU).

[0050] In this invention, the primitives include one or a combination of the following: codec block, sub-block, micro-block, string, pixel string, sample string, index string, and line.

[0051] In this invention, the encoding parameters include one or a combination of the following:

[0052] 1) Displacement vector;

[0053] 2) Motion vector;

[0054] 3) Position offset;

[0055] 4) Offset;

[0056] 5) Relative position;

[0057] 6) Location;

[0058] 7) Relative address;

[0059] 8) Address;

[0060] 9) Relative coordinates;

[0061] 10) Coordinates;

[0062] 11) Relative index;

[0063] 12) Index;

[0064] 13) String displacement vector;

[0065] 14) Primitive size;

[0066] 15) Element size;

[0067] 16) String length. Attached Figure Description

[0068] Figure 1This is a schematic diagram of the encoding method or apparatus or the decoding method or apparatus of the present invention. Implementation

[0069] The technical features of the present invention have been illustrated above through several specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0070] The following are further implementation details or variations of the present invention.

[0071] Implementation or variant example 1

[0072] In the encoding method or apparatus, or the decoding method or apparatus, the update operation includes:

[0073] The position of the nearest encoding parameter array element in the nearest encoding parameter array is adjusted;

[0074] and / or

[0075] The values ​​of new, qualified encoding parameters that appear during the encoding and decoding process will be placed into the most recently encoded parameter array;

[0076] and / or

[0077] Remove one or more elements from the most recently encoded parameter array.

[0078] Implementation or variant example 2

[0079] In the encoding method or apparatus or the decoding method or apparatus, the predetermined rule includes adjusting the position of the most recent encoding parameter array element in the encoding parameter array based on the frequency of occurrence of the most recent encoding parameter array element and / or the length of the interval between occurrences of the most recent encoding parameter array element.

[0080] Implementation or variant example 3

[0081] In the encoding method or apparatus or the decoding method or apparatus, each most recent encoding parameter array element has an associated recorder that records the occurrence of that element, that is, the situation where the value of a qualified encoding parameter during the encoding or decoding process is equal to the value of that element.

[0082] Implementation or variant example 4

[0083] In the encoding method or apparatus or decoding method or apparatus described in Embodiment 1 or Embodiment 3, the qualified encoding parameters include encoding parameters that fall under one or a combination of the following categories:

[0084] 1) Any encoding parameter (i.e., all encoding parameters are valid encoding parameters);

[0085] 2) Encoding parameters that meet a predetermined set of conditions;

[0086] 3) Encoding parameters that do not meet a certain set of predetermined conditions;

[0087] 4) Encoding parameters for encoding and decoding using predetermined methods, including the nearest encoding parameter array method and / or the encoding parameters of adjacent primitives and / or the direct method;

[0088] 5) Encoding parameters that are not encoded or decoded using predetermined methods, including the method of the nearest encoding parameter array and / or the method of encoding parameters of adjacent primitives and / or the direct method;

[0089] 6) Encoding parameters whose values ​​fall within a predetermined range;

[0090] 7) Encoded parameters whose values ​​fall outside a predetermined range.

[0091] Implementation or variant example 5

[0092] In the implementation or variant of Example 3, the encoding method or apparatus or decoding method or apparatus, the occurrence situation includes the frequency of occurrence and / or the length of the occurrence interval.

[0093] Implementation or variant example 6

[0094] In implementing or in a variation of Example 1, the encoding method or apparatus, or the decoding method or apparatus, the adjustment includes one or a combination of the following operations:

[0095] 1) Swap the positions of two elements;

[0096] 2) Move one or more elements to a specific location in the nearest encoded parameter array;

[0097] 3) Move one or more elements to the front of the nearest encoded parameter array;

[0098] 4) Move one or more elements to the end of the most recently encoded parameter array.

[0099] Implementation or variant example 7

[0100] In the encoding method or apparatus, or the decoding method or apparatus, when a qualified current encoding parameter is found, the following operations are performed to update the most recent encoding parameter array:

[0101] If the value of the current encoding parameter is equal to an element in the most recent encoding parameter array, called the hit element, the position of the element in the most recent encoding parameter array of the current encoding parameter is adjusted.

[0102] Otherwise (i.e., the value of the current encoding parameter is not equal to the value of any element in the most recent encoding parameter array), the value of the current encoding parameter is placed into the most recent encoding parameter array.

[0103] Implementation or variant example 8

[0104] In the encoding method or apparatus, or the decoding method or apparatus, when a qualified current encoding parameter is found, the following operations are performed to update the most recent encoding parameter array:

[0105] If the value of the current encoding parameter is equal to an element in the most recent encoding parameter array, called the hit element, then the hit element is moved to the beginning of the most recent encoding parameter array.

[0106] Otherwise (i.e., the value of the current encoding parameter is not equal to the value of any element in the most recent encoding parameter array), the current encoding parameter is inserted at the beginning of the most recent encoding parameter array, and the last element of the most recent encoding parameter array is removed from the most recent encoding parameter array.

[0107] Implementation or variant example 9

[0108] In the encoding method or apparatus, or the decoding method or apparatus, the encoding parameter is a displacement vector, and the nearest encoding parameter array is a nearest displacement vector array with N elements, labeled as...

[0109] ROA[N][2] = {offset_in_roa[k] = (offset_in_roa[k][0], offset_in_roa[k][1]): 0 ≤ k < N};

[0110] When a valid displacement vector offset = (offsetX, offsetY) is found, offset is compared one by one with the N elements offset_in_roa[(RoaStartPos + k)%N] of ROA[N][2], where 0 ≤ k < N, where % is the modulo operation and RoaStartPos is the current starting position of the nearest displacement vector array; then, the following operations are performed to update ROA[N][2] in sequence:

[0111] 1) If offset equals the k0th element among these N elements, offset_in_roa [(RoaStartPos +k0)%N], perform the following operation:

[0112] If k0 ≥ 1, let k take values ​​from k0 to 1 in descending order and perform the following assignment operations k0 times in sequence:

[0113] offset_in_roa[(RoaStartPos + k0)%N] = offset_in_roa[(RoaStartPos + k0- 1)%N]

[0114] 2) Otherwise (i.e., offset is not equal to any of these N elements), perform the following update operation on the current starting position RoaStartPos of the nearest displacement vector array: RoaStartPos = (RoaStartPos + 1)%N;

[0115] 3) ROA[RoaStartPos] = offset.

Claims

1. A method for compressing video, characterized in that, It should include at least the steps to complete the following functions or operations: In inter-frame predictive coding, the motion vectors are encoded using at least the nearest motion vector array; At least one of the following methods, or a combination thereof, will be used to update the most recent motion vector array with a new, qualified current motion vector that appears during the encoding process: Method 1: Update the most recent motion vector array by performing at least the following operations: If the value of the current motion vector is equal to the value of an existing motion vector found after comparing it one by one with the existing motion vectors in the most recent motion vector array, then at least some or all of the existing motion vectors in the most recent motion vector array are adjusted in the position of the most recent motion vector array. Otherwise, if the value of the current motion vector is not equal to the value of any existing motion vector in the most recent motion vector array, the value of the current motion vector shall at least be placed in the most recent motion vector array; Method 2: Update the most recent motion vector array by performing at least the following operations: If the value of the current motion vector is equal to the value of an existing motion vector found after comparing it one by one with the existing motion vectors in the nearest motion vector array, then at least that existing motion vector with the same value is replaced at the beginning of the nearest motion vector array; Otherwise, if the value of the current motion vector is not equal to the value of any existing motion vector in the most recent motion vector array, the value of the current motion vector shall be inserted at least at the beginning of the most recent motion vector array, and existing motion vectors at the end of the most recent motion vector array shall be removed from the most recent motion vector array. Method 3: Update the most recent motion vector array by performing at least the following operations: The nearest motion vector array has N motion vectors, labeled as ROA[N] = {offset_in_roa[k]: 0 ≤k < N}; The current motion vector marked as offset is compared one by one with the N elements offset_in_roa[(RoaStartPos + k)%N] of ROA[N], where 0 ≤ k < N, where % is the modulo operation and RoaStartPos is the current starting position of the nearest motion vector array; then, the following operations are performed to update ROA[N]: 1) If offset equals the k0th element among these N elements, offset_in_roa [(RoaStartPos + k0)%N], perform the following operation: If k0 ≥ 1, let k take values ​​from k0 to 1 in descending order and perform the following assignment operations k0 times in sequence: offset_in_roa[(RoaStartPos + k0)%N] = offset_in_roa[(RoaStartPos + k0 -1)%N] 2) Otherwise, if offset is not equal to any of these N elements, perform the following update operation on the current starting position RoaStartPos of the nearest motion vector array: RoaStartPos = (RoaStartPos + 1)%N; 3) ROA[RoaStartPos] = offset; The qualified motion vector includes motion vectors that fall under one or a combination of the following conditions: Motion vectors are encoded using predetermined methods, including the nearest motion vector array method and / or the motion vector of adjacent primitives method and / or the direct method; Motion vectors that are not encoded using predetermined methods, including the nearest motion vector array method and / or the motion vector method of adjacent primitives and / or the direct method; A motion vector whose value falls within the first predetermined range; A motion vector whose value falls outside the second predetermined range.

2. An encoding apparatus for compressing video, characterized in that, Includes a module that implements the encoding method as described in claim 1.

3. A decoding method for compressing video, characterized in that, It should include at least the steps to complete the following functions or operations: In inter-frame prediction decoding, at least the most recent motion vector array is used to decode the motion vectors; At least one of the following methods or a combination thereof will be used to update the most recent motion vector array with a new, qualified current motion vector that appears during the decoding process; Method 1: Update the most recent motion vector array by performing at least the following operations: If the value of the current motion vector is equal to the value of an existing motion vector found after comparing it one by one with the existing motion vectors in the most recent motion vector array, then at least some or all of the existing motion vectors in the most recent motion vector array are adjusted in the position of the most recent motion vector array. Otherwise, if the value of the current motion vector is not equal to the value of any existing motion vector in the most recent motion vector array, the value of the current motion vector shall at least be placed in the most recent motion vector array; Method 2: Update the most recent motion vector array by performing at least the following operations: If the value of the current motion vector is equal to the value of an existing motion vector found after comparing it one by one with the existing motion vectors in the nearest motion vector array, then at least that existing motion vector with the same value is replaced at the beginning of the nearest motion vector array; Otherwise, if the value of the current motion vector is not equal to the value of any existing motion vector in the most recent motion vector array, the value of the current motion vector shall be inserted at least at the beginning of the most recent motion vector array, and existing motion vectors at the end of the most recent motion vector array shall be removed from the most recent motion vector array. Method 3: Update the most recent motion vector array by performing at least the following operations: The nearest motion vector array has N motion vectors, labeled as ROA[N] = {offset_in_roa[k]: 0 ≤k < N}; The current motion vector marked as offset is compared one by one with the N elements offset_in_roa[(RoaStartPos + k)%N] of ROA[N], where 0 ≤ k < N, where % is the modulo operation and RoaStartPos is the current starting position of the nearest motion vector array; then, the following operations are performed to update ROA[N]: 1) If offset equals the k0th element among these N elements, offset_in_roa [(RoaStartPos + k0)%N], perform the following operation: If k0 ≥ 1, let k take values ​​from k0 to 1 in descending order and perform the following assignment operations k0 times in sequence: offset_in_roa[(RoaStartPos + k0)%N] = offset_in_roa[(RoaStartPos + k0 -1)%N] 2) Otherwise, if offset is not equal to any of these N elements, perform the following update operation on the current starting position RoaStartPos of the nearest motion vector array: RoaStartPos = (RoaStartPos + 1)%N; 3) ROA[RoaStartPos] = offset; The qualified motion vector includes motion vectors that fall under one or a combination of the following conditions: The motion vectors are decoded using predetermined methods, including the nearest motion vector array method and / or the motion vector of adjacent primitives method and / or the direct method; Motion vectors that are decoded without using predetermined methods, including the nearest motion vector array method and / or the motion vector method of adjacent primitives and / or the direct method; A motion vector whose value falls within the first predetermined range; A motion vector whose value falls outside the second predetermined range.

4. The decoding method according to claim 3, characterized in that, The motion vector is the motion vector of the decoding block; the decoding block is a coding region or a decoding region of an image, including at least one of the following: the entire image, a sub-image of the image, a macroblock, a maximum coding unit (LCU), a coding tree unit (CTU), a coding unit (CU), a sub-region of a CU, a prediction unit (PU), and a transform unit (TU).

5. The decoding method according to claim 3, characterized in that, A motion vector is the motion vector of a primitive; the primitive includes one or a combination of the following: decoded block, sub-block, micro-block, string, pixel string, sample string, index string, line.

6. The decoding method according to claim 3, characterized in that, The operation of updating the most recent motion vector array also includes: The position of the nearest motion vector array element of the motion vector is adjusted within the nearest motion vector array; and / or Remove one or more elements from the most recent motion vector array.

7. The decoding method according to claim 6, characterized in that, The adjustment includes one or a combination of the following operations: Swap the positions of two elements; Move one or more elements to a specific location in the nearest motion vector array; Move one or more elements to the front of the nearest motion vector array; Move one or more elements to the end of the nearest motion vector array.

8. The decoding method according to claim 3, characterized in that, The position of the most recent motion vector array element in the motion vector array is adjusted based on the frequency of occurrence of the most recent motion vector array element and / or the length of the interval between occurrences of the most recent motion vector array element.

9. The decoding method according to claim 3, characterized in that, Each element of the recent motion vector array has an associated logger that records the occurrence of that element, that is, the cases where the value of a valid motion vector encountered during the decoding process is equal to the value of that element.

10. A decoding apparatus for compressing video, characterized in that, Includes a module that implements the decoding method as described in any one of claims 3-9.

Images