Offline prediction method and system for motion vectors

Abstract

This invention belongs to the field of image and video coding technology, and particularly relates to a method and system for offline motion vector prediction. It includes: acquiring the coding information of the current block and its five neighboring blocks; determining the classification feature types of the five neighboring blocks and calculating their numerical values, classifying the coding information of the five neighboring blocks to obtain multiple categories; training to obtain the relationship matrix between the motion vector of the current block and the motion vectors of the five neighboring blocks corresponding to each category; determining the five neighboring blocks of the target block whose motion vector is to be predicted, classifying the coding information of the target block's five neighboring blocks, and based on the classification results, indexing to obtain the relationship matrix between the target block and its five neighboring motion vectors, thus obtaining the motion vector of the target block. This invention provides more motion vector selections for the merge list candidates, and these selections fully integrate the motion information of the surrounding five neighboring blocks, using more neighborhood information than a simple average motion vector, achieving a more accurate approximation.

Description

Technical Field

[0001] This invention belongs to the field of image and video coding technology, and particularly relates to a method and system for offline prediction of motion vectors. Background Technology

[0002] With the exponential growth of video data volume, the transmission bandwidth and storage space of video data face enormous challenges. In order to further improve compression efficiency while maintaining high-quality video reconstruction results, the Joint Video Experts Group (JVET) launched research on the next-generation video coding standard, namely Enhanced Compression Model (ECM), after releasing the H.266 / VVC standard. ECM aims to further reduce bitrate and eliminate redundant information in images by introducing more sophisticated and complex coding tools.

[0003] The ECM framework still adopts a hybrid video coding framework. Inter-frame prediction captures temporal redundancy in neighboring frames through motion estimation techniques. It also acquires motion information synchronously at the encoder and decoder by transmitting residuals (the difference between the current and predicted values) and motion information (including motion vectors, reference frame numbers, etc.). Inter-frame prediction is obtained through techniques such as motion compensation, and finally reconstructed together with the reconstruction residuals.

[0004] However, the motion information of adjacent blocks is often quite similar, resulting in significant redundancy. The current approach uses a merge mode, constructing a merge list for the current block based on the motion information of related blocks. The motion information of the current block is then obtained by transmitting the merge list index. The core improvement in the current merge list construction is to add more existing motion vectors, including HMVP and CMVP. However, due to the insufficient richness of existing motion information, it cannot fully cover motion in complex real-world scenarios. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides an offline motion vector prediction method and system. It collects the motion vectors of the five neighboring blocks in the spatial domain and the motion vector of the current block, estimates the relationship between them using least squares to obtain a relation matrix, and then predicts the motion vector of the current block in the ECM using the motion vectors of the five neighboring blocks in the spatial domain and the relation matrix. This prediction is inserted before the motion vector in the merge list 0, avoiding the limitation of searching only among existing motion vectors. It provides more motion vector choices for the merge list candidates, and these choices fully integrate the motion information of the surrounding five neighboring blocks, using more neighborhood information than a simple average motion vector, thus achieving a more accurate approximation.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a method for offline prediction of motion vectors.

[0007] The offline motion vector prediction method includes the following steps: Obtain the encoding information of the current block and its five neighboring blocks, the encoding information including motion vectors and reference frame numbers; The classification feature types of the five neighboring blocks are determined and the classification feature values ​​are calculated. The encoded information of the five neighboring blocks is classified to obtain multiple categories. Using the least squares method, the relationship matrix between the current block motion vector and the five neighboring block motion vectors corresponding to each category is obtained through fitting training, resulting in a relationship matrix table; The five neighboring blocks of the target block whose motion vector is to be predicted are determined. The encoding information of the five neighboring blocks of the target block is classified. Based on the classification results, the relationship matrix between the motion vectors of the target block and its five neighboring blocks is obtained by indexing from the relationship matrix table. Based on the relationship matrix between the target block and the motion vectors of its five neighboring blocks, the motion vector of the target block is obtained.

[0008] As an alternative technical solution, the five neighboring blocks are the pixel blocks to the left, top, upper left, upper right, and lower left of the current block or the target block of the motion vector to be predicted.

[0009] As an alternative technical solution, the classification feature types of the five neighboring blocks are determined and the classification feature values ​​are calculated. The encoded information of the five neighboring blocks is then classified to obtain multiple categories, specifically including a first classification method and a second classification method, wherein: The first classification method is: Calculate the average motion intensity of the five neighboring blocks using the first calculation method; Using only the average motion intensity of the five neighboring blocks as the classification feature, the encoded information of the five neighboring blocks is divided into three categories; The second classification method is: Define the reference frame that is used most frequently in the five neighboring blocks as the primary reference frame, and calculate the absolute value of the distance between the primary reference frame and the current frame; Based on the magnitude of the absolute distance between the primary reference frame and the current frame, the absolute distance values ​​are divided into 5 categories; According to the second calculation method, the average motion intensity of the five neighboring blocks is calculated, and the average motion intensity of the five neighboring blocks is also divided into three categories. By combining the absolute distance values ​​of the five classes with the average motion intensity of the five neighboring blocks of the three classes, the coding information of the five neighboring blocks is divided into 15 classes.

[0010] As an alternative technical solution, in the first classification method: First, using the first calculation method, calculate the motion vector of each of the five neighboring blocks: ; in, This represents the motion vector after adjustment of the five neighboring blocks; Indicates the reference frame number of the neighboring block; Indicates the current frame number; Then, based on the motion vectors of each neighboring block, the average motion vector of multiple neighboring blocks among the five neighboring blocks is calculated; Finally, based on the relationship between the average motion vector and the preset upper and lower limit motion amplitude thresholds, the motion intensity is divided into three categories: slow, medium, and fast.

[0011] As an alternative technical solution, in the second classification method: Based on the magnitude of the absolute distance between the primary reference frame and the current frame, the absolute distance is divided into 5 categories, as follows: When TD=1, it is divided into the first frame; When TD=2, it is divided into the first 2 frames; When TD=4, it is divided into the first 3-4 frames; When TD=8, it is divided into the first 5-8 frames; When TD=0, it is divided into other time distances, where TD represents the absolute value of the distance.

[0012] As an alternative technical solution, in the second classification method: According to the second calculation method, the motion vector of each of the five neighboring blocks is first calculated: ; in, This represents the motion vector after adjustment of the five neighboring blocks; Indicates the reference frame number of the neighboring block; Indicates the primary reference frame number. Indicates the current frame number; Then, based on the motion vectors of each neighboring block, the average motion vector of multiple neighboring blocks among the five neighboring blocks is calculated; Finally, based on the relationship between the average motion vector and the preset upper and lower limit motion amplitude thresholds, the motion intensity is divided into three categories: slow, medium, and fast.

[0013] As an alternative technical solution, in the second classification method: If two reference frames among the five neighboring blocks have the same number of uses and are used the most, then the reference frame that is closest in time to the current block is selected as the primary reference frame.

[0014] As an alternative technical solution, the least squares method is used to obtain the relationship matrix between the current block motion vector and the five neighboring block motion vectors for each category through fitting training. Specifically, this includes: The relationship matrix between the current block's motion vector and the motion vectors of its five neighboring blocks is represented as follows: ; Where Y is the motion vector of the current block, X is the vector composed of the motion vectors of the five neighboring blocks, and A is the relation matrix; Using MATLAB, the least squares method is used to iteratively solve the relation matrix A; By combining the relation matrices A of multiple categories, a relation matrix table is obtained.

[0015] As an optional technical solution, it also includes inserting the obtained motion vector of the target block into the merge list: Insert the motion vector of the target block before the 0 motion vector, and do not exceed the maximum number of candidates.

[0016] A second aspect of the present invention provides an offline motion vector prediction system.

[0017] An offline motion vector prediction system includes: The training data acquisition module is configured to: acquire the encoding information of the current block and its five neighboring blocks, wherein the encoding information includes motion vectors and reference frame numbers; The neighboring block coding information classification module is configured to: determine the classification feature types of five neighboring blocks and calculate the classification feature values; and classify the coding information of the five neighboring blocks based on the classification feature values ​​to obtain multiple categories. The relation matrix fitting training module is configured to: use the least squares method to obtain the relation matrix between the current block motion vector and the five neighboring block motion vectors corresponding to each category through fitting training, and obtain the relation matrix table; The indexing module is configured to: determine the five neighboring blocks of the target block whose motion vector is to be predicted, classify the encoding information of the five neighboring blocks of the target block, and, based on the classification results, index the relationship matrix between the target block and the motion vectors of the five neighboring blocks from the relationship matrix table; The prediction module is configured to obtain the motion vector of the target block based on the relationship matrix between the motion vectors of the target block and its five neighboring blocks.

[0018] The above one or more technical solutions have the following beneficial effects: This invention discloses an offline motion vector prediction method and system. It collects the motion vectors of the five neighboring blocks in the spatial domain and the motion vector of the current block. The relationship between the two is estimated using least squares to obtain a relation matrix. Then, in the ECM (Elastic Computation Model), the motion vector of the current block is predicted using the motion vectors of the five neighboring blocks in the spatial domain and the relation matrix. This prediction is inserted before the motion vector in the merge list 0. This avoids the limitation of searching only among existing motion vectors, providing more motion vector choices for the merge list candidates. Furthermore, these choices fully integrate the motion information of the surrounding five neighboring blocks, using more neighborhood information than a simple average motion vector, thus achieving a more accurate approximation.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a flowchart of the method in Example 1.

[0022] Figure 2 This is a location diagram of the current block and its five neighboring blocks A0\A1\B0\B1\B2 in Example 1.

[0023] Figure 3 The figure shows the experimental results under the second prediction method LDB in Example 1. Detailed Implementation

[0024] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0026] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] Example 1 In existing technologies, the construction of the merge list typically includes spatial motion vector prediction for neighboring and non-neighboring blocks, temporal motion vector prediction, HMVP historical motion vector prediction, CMVP chained motion vector prediction, pairwise averaged motion vector prediction, and 0 motion vector prediction. A brief explanation of these six motion vector prediction methods is as follows: 1. Spatial neighbor block movement information.

[0028] Motion information for spatial neighboring blocks is obtained from the spatially adjacent positions of the current coding block. Typically, five fixed positions are checked: left (A0, A1) and above (B0, B1, B2). The system scans these positions in a preset order, and once usable motion information (such as motion vectors and reference frame indices) is found, it is added to the candidate list. This method is based on the principle of spatial correlation; neighboring regions often have similar motion characteristics, making it simple and direct to implement.

[0029] 2. Temporal neighbor block movement information.

[0030] Temporal neighbor block motion information is derived from reference frames that have been encoded in the temporal dimension to identify motion candidates. Specifically, the current block's position within the reference frame is located, and motion information from that position or its neighboring blocks is obtained. A key step is motion vector scaling to compensate for the temporal distance difference between the current and reference frames. This method effectively captures motion continuity between frames and is particularly suitable for scenes containing uniformly moving or stationary objects.

[0031] 3. Non-neighboring block movement information.

[0032] Non-neighboring block motion information breaks through the traditional limitations of adjacent locations, collecting motion candidates from spatially non-directly adjacent but potentially related locations. In implementation, it may select blocks with specific relative coordinates (such as diagonal directions) or use template matching techniques to find motion parameters of regions with similar texture features to the current block. This approach expands the candidate source and helps to handle complex or irregular motion patterns.

[0033] 4. HMVP.

[0034] HMVP (Historical Motion Vector Prediction) accumulates motion information by maintaining a first-in, first-out (FIFO) history buffer. Whenever a block is encoded, its motion parameters are stored in the buffer. When building the candidate list, the system retrieves available historical records from the buffer in reverse chronological order as supplementary candidates. This approach breaks the limitations of spatial and temporal locality, enabling the reuse of various motion patterns occurring throughout the frame.

[0035] 5. CMVP.

[0036] CMVP (Chained Motion Vector Prediction) generates new motion candidates using a recursive tracing approach. Starting with pre-derived fusion candidates, it forms new candidates by progressively tracing and accumulating motion vectors or block vectors from subsequent blocks. Specifically, the algorithm checks the target block and its surrounding locations in a preset order, and performs vector superposition if usable motion information exists. Chained tracing can span multiple reference blocks, constructing motion paths pointing to higher-quality reference images. This method expands the reference range of candidates, particularly helpful in improving prediction accuracy under low bitrate conditions.

[0037] 6. Paired average.

[0038] Pairwise averaging generates new motion candidates by combining existing candidates. Specifically, the system selects two candidates from the constructed candidate list (usually in a preset order, such as the first two), takes the arithmetic mean of the horizontal and vertical components of their motion vectors, and appropriately handles related parameters such as the reference frame index. This method generates intermediate motion states with extremely low computational cost, effectively enriching the diversity of the candidate set.

[0039] Therefore, only pairwise averaged motion vectors are used to obtain new motion vectors by averaging existing motion vectors and inserting them into the merge list. However, simple averaging cannot fully utilize the relationship between related blocks and the current block to obtain a better approximation of complex real-world motion.

[0040] The core improvement in current merge list construction is to add more existing motion vectors, including HMVP and CMVP. However, the existing motion information is not rich enough to fully cover motion in real-world complex situations. Therefore, this invention discloses an offline motion vector prediction method. This method collects the motion vectors of the five neighboring blocks in the spatial domain and the motion vector of the current block. The relationship between the two is estimated using least squares to obtain a relationship matrix. In the ECM (Elastic Compute Manager), the motion vector of the current block is predicted using the motion vectors of the five neighboring blocks in the spatial domain and the relationship matrix, and then inserted before the motion vector in merge list 0.

[0041] This embodiment utilizes a neighboring block motion vector for offline motion vector prediction, which can be used to construct a motion vector candidate list.

[0042] like Figure 1 As shown, the offline motion vector prediction method includes the following steps: Obtain the encoding information of the current block and its five neighboring blocks, the encoding information including motion vectors and reference frame numbers; The classification feature types of the five neighboring blocks are determined and the classification feature values ​​are calculated. The encoded information of the five neighboring blocks is classified to obtain multiple categories. Using the least squares method, the relationship matrix between the current block motion vector and the five neighboring block motion vectors corresponding to each category is obtained through fitting training, resulting in a relationship matrix table; The five neighboring blocks of the target block whose motion vector is to be predicted are determined. The encoding information of the five neighboring blocks of the target block is classified. Based on the classification results, the relationship matrix between the motion vectors of the target block and its five neighboring blocks is obtained by indexing from the relationship matrix table. Based on the relationship matrix between the target block and the motion vectors of its five neighboring blocks, the motion vector of the target block is obtained.

[0043] The technical solution of this invention generally includes two parts: The first part is the process of obtaining the relation matrix. The motion vector of the current block and its five spatial neighboring blocks are counted, and the relation matrix is ​​obtained through streaming least squares.

[0044] The second part describes the prediction process using the relation matrix. Motion vectors are predicted using five neighboring blocks in the spatial domain and then applied to the ECM merge list construction.

[0045] Specifically, the relation matrix in the first part is obtained through two different methods, and the relation matrices obtained through the two different methods are combined to obtain the first relation matrix table and the second relation matrix table.

[0046] The technical solution of this embodiment will be explained in detail below.

[0047] (a) The process of obtaining the relation matrix.

[0048] In ECM, when encoding the CTU, the rate-distortion optimization mode is determined through the `compressCtu` function. This function obtains the encoding mode for the current CTU, including motion vectors, reference frame index, residuals, and prediction modes. After obtaining the encoding mode, subsequent operations, such as entropy coding, are performed. Specifically, the `coding_tree_unit` function obtains the final bitrate. Therefore, to obtain the final encoding mode, the `coding_tree_unit` function, specifically the `coding_tree` function, outputs the motion vector of the current CTU block and the motion vectors of its five spatial neighboring blocks during motion vector entropy coding. Furthermore, not all five spatial neighboring blocks of each CTU are available; for example, edge blocks may lack their left or top neighboring blocks. Therefore, for missing neighboring blocks, all are set to 0.

[0049] (1) The specific process and functions for collecting statistical data are as follows.

[0050] This function collects detailed coding information of coding units (CUs) in video coding, primarily for statistical analysis of inter-frame prediction CUs (MODE_INTER). The information collected includes basic CU attributes (such as POC, location information, etc.), motion information, segmentation depth, and neighbor information. It retrieves the reference frame index and motion vector from the `pu` function. For example, to retrieve the above information in the direction of the reference frame list `list0`, the reference frame index is `pu.refIdx[REF_PIC_LIST_0]`, and the motion vectors are `pu.mv[REF_PIC_LIST_0].getHor()` and `pu.mv[REF_PIC_LIST_0].getVer()`. Based on the current CU's position information, obtain the information of the five surrounding neighboring blocks. For example, if the current block position is lumaPos.x and lumaPos.y (the coordinates of the top left corner of the current block), then the position of the left neighboring block is (as shown in Figure A0) lumaPos.x - 1, lumaPos.y + lumaSize.height - 1. Obtain the PU object based on the position information, and use the same method to obtain the reference frame index, motion vector, and motion precision of the left neighboring block from the PU.

[0051] like Figure 2 As shown, with the top-left pixel coordinates of the current block as the origin (0,0), the width of the current block is W, and the height is H, the relative coordinates of these five positions are defined as follows: Position L (Left) A0: coordinates (-1, H-1), which is the bottommost pixel block immediately adjacent to the left boundary of the current block; Position A (Above) B0: coordinates (W-1, -1), which is the rightmost pixel block immediately adjacent to the upper boundary of the current block; Position AL (Above-Left) B2: coordinates (-1, -1), which is the diagonal pixel block opposite the top left corner of the current block; Position AR (Above-Right) B1: Coordinates (W, -1), which is the diagonal pixel block opposite the upper right corner of the current block; Position BL (Below-Left) A1: Coordinates (-1, H), which is the diagonal pixel block opposite the bottom left corner of the current block.

[0052] After analyzing the information of the current block and its five neighboring blocks, this information needs to be normalized and classified. As mentioned earlier, the classification features of the five neighboring blocks are determined and their values ​​are calculated. The encoded information of the five neighboring blocks is then classified into multiple categories, specifically including a first classification method and a second classification method. The first classification method divides the encoded information of the five neighboring blocks into 3 categories, while the second classification method divides it into 15 categories.

[0053] The relation matrix obtained by the first classification method is used to predict motion vectors and reference frame numbers; The relation matrix obtained by the second classification method is used to predict motion vectors (the reference frame number uses the main reference frame).

[0054] The two classification methods will be described in detail below. For ease of description, this embodiment refers to the process of classifying the coding information of five neighboring blocks and predicting motion vectors and reference frame numbers as the first prediction method; and the process of classifying the coding information of five neighboring blocks and predicting motion vectors and reference frame numbers as the second prediction method.

[0055] It can be understood that the two prediction methods provided in this embodiment can be used selectively: When only motion vectors need to be predicted (the reference frame number is relatively easy to obtain: for example, when the reference frames of five neighboring blocks are the same or similar), the second prediction method can be used for prediction. When it is necessary to predict both motion vectors and reference frame numbers simultaneously (reference frame numbers are difficult to obtain: for example, the reference frames of five neighboring blocks are quite different, and it is not easy to find the main reference frame (the most frequently used reference frame)), the first prediction method can be used for prediction.

[0056] (2) The execution process of the first classification method.

[0057] The first classification method can be summarized as follows: The average motion intensity of the five neighboring blocks is used for classification. Using the data of each category, the relationship matrix between the motion information of the five neighboring blocks (including motion vector and reference frame number) and the motion information of the current block is obtained by least squares. Finally, the relationship matrix of all categories is obtained.

[0058] Specifically, the first classification method is as follows: Calculate the average motion intensity of the five neighboring blocks using the first calculation method; Using only the average motion intensity of the five neighboring blocks as the classification feature, the encoded information of the five neighboring blocks is divided into three categories.

[0059] Specifically, in the first classification method: First, using the first calculation method, calculate the motion vector of each neighboring block in the normalized five-neighbor block: ; in, This represents the motion vector after adjustment of the five neighboring blocks; Indicates the reference frame number of the neighboring block; Indicates the current frame number; Then, based on the motion vectors of each neighboring block, the average motion vector of multiple neighboring blocks among the five neighboring blocks is calculated; Finally, based on the relationship between the average motion vector and the preset upper and lower limit motion amplitude thresholds, the motion intensity is divided into three categories: slow, medium, and fast.

[0060] More detailed: After obtaining the adjusted motion vector MV = (MvX, MvY), the amplitude is calculated as follows: ; Then, the MV amplitude of each of the five neighboring blocks is calculated, and the average value of the MV amplitude of the five neighboring blocks is taken as the average motion intensity.

[0061] Set two motion amplitude thresholds: slowMotionThreshold = 32 (2 whole pixels = 8 quarter pixel units) and mediumMotionThreshold = 80 (5 whole pixels = 20 quarter pixel units), with the thresholds set at 1 / 16 precision; Based on the current block MV amplitude intensity, the motion information is divided into three categories, as follows: If the motion is less than slowMotionThreshold, then motionCategory=0, classifying it as slow. Between slowMotionThreshold and mediumMotionThreshold, motionCategory=1, classifying it as medium speed; If it is greater than mediumMotionThreshold, then motionCategory=2, and it is classified as fast.

[0062] (3) The second classification method is: Define the reference frame that is used most frequently in the five neighboring blocks as the primary reference frame, and calculate the absolute value of the distance between the primary reference frame and the current frame; Based on the magnitude of the absolute distance between the primary reference frame and the current frame, the absolute distance values ​​are divided into 5 categories; According to the second calculation method, the average motion intensity of the five neighboring blocks is calculated, and the average motion intensity of the five neighboring blocks is also divided into three categories. By combining the absolute distance values ​​of the five classes with the average motion intensity of the five neighboring blocks of the three classes, the coding information of the five neighboring blocks is divided into 15 classes.

[0063] Furthermore, in the second classification method: Based on the magnitude of the absolute distance between the primary reference frame and the current frame, the absolute distance is divided into 5 categories, as follows: When TD=1, it is divided into the first frame; When TD=2, it is divided into the first 2 frames; When TD=4, it is divided into the first 3-4 frames; When TD=8, it is divided into the first 5-8 frames; When TD=0, it is divided into other time distances, where TD represents the absolute value of the distance.

[0064] Further: According to the second calculation method, the motion vector of each of the five neighboring blocks is first calculated: ; in, This represents the motion vector after adjustment of the five neighboring blocks; Indicates the reference frame number of the neighboring block; Indicates the primary reference frame number. Indicates the current frame number; Then, based on the motion vectors of each neighboring block, the average motion vector of multiple neighboring blocks among the five neighboring blocks is calculated; Finally, based on the relationship between the average motion vector and the preset upper and lower limit motion amplitude thresholds, the motion intensity is divided into three categories: slow, medium, and fast.

[0065] More detailed: The most frequently used reference frame in the five neighboring blocks is identified (if two reference frames are used the same number of times, the one with the smallest time distance is used), and this frame is defined as the primary reference frame. The primary reference frame is used as the prediction reference frame for the current block, and these data are divided into five categories based on the absolute value of the distance between the primary reference frame and the current frame.

[0066] Based on the current block movement intensity, they are divided into three categories: Generally, the motion of an object is linear in a short time. Therefore, when the reference frame of the neighboring block is adjusted to the main reference frame, the motion vectors of the five neighboring blocks are adjusted to the main reference frame according to the second calculation method to obtain the adjusted motion vectors of the neighboring blocks. After obtaining the adjusted neighboring block motion vector MV = (MvX, MvY), the amplitude calculation formula is as follows: .

[0067] Set two motion amplitude thresholds: slowMotionThreshold = 32 (2 whole pixels = 8 quarter pixel units) and mediumMotionThreshold = 80 (5 whole pixels = 20 quarter pixel units), with the thresholds set at 1 / 16 precision; Based on the current block MV amplitude intensity, the motion information is divided into three categories, as follows: If the motion is less than slowMotionThreshold, then motionCategory=0, classifying it as slow. Between slowMotionThreshold and mediumMotionThreshold, motionCategory=1, classifying it as medium speed; If it is greater than mediumMotionThreshold, then motionCategory=2, and it is classified as fast.

[0068] By combining the absolute distance values ​​of the five classes with the average motion intensity of the five neighboring blocks of the three classes, the coding information of the five neighboring blocks is divided into 5×3=15 classes, for a total of 15 classes.

[0069] (4) Use MATLAB to train the prediction matrix A.

[0070] 1) Use the relation matrix corresponding to the first classification method in step (2): The motion information of the current block is predicted based on the motion information of its five neighboring blocks, as shown below: ; Where Y1 is the motion information of the current block (including motion vector and reference frame number), which is a 3x1 vector; X1 is a vector composed of the motion information of the five neighboring blocks (including motion vector and reference frame number), which is a 15x1 vector; and A1 is the prediction matrix, which is a 3x15 matrix.

[0071] 2) Use the relation matrix corresponding to the second classification method in step (3): The motion vector of the current block is Y, a 1x2 vector. The motion vectors of its five neighboring blocks form vector X, a 1x10 vector. There is a strong spatial correlation between the current block and its five neighboring blocks; therefore, the motion vector of the current block can be predicted using the motion vectors of its five neighboring blocks, as shown below: ; Where Y2 is the motion vector of the current block, which is a 2x1 vector; X2 is a vector composed of the motion vectors of the five neighboring blocks, which is a 10x1 vector; A2 is the prediction matrix, which has a dimension of 2x10.

[0072] The aforementioned statistical data section has already adjusted and normalized the motion vectors of the current block and its five neighboring blocks and output them. Now, given X (i.e., X1 or X2 mentioned above) and Y (i.e., Y1 or Y2), the prediction matrix A (i.e., ...) can be obtained using the least squares method. or First, process the data in batches, for example, each batch contains 50,000 rows, denoted as X_batch and Y_batch, and obtain the cumulative statistics, i.e.: XTX = XTX + X_batch' X_batch XTY = XTY + X_batch' Y_batch Where XTX is the covariance matrix of the features, which measures the correlation between features, and has a dimension of 10x10 (or 15x15). XTY is the covariance matrix of the features and the label, which measures the predictive ability of the features to the label, and has a dimension of 10x2 (or 15x3).

[0073] The initial solution is obtained using the least squares method, i.e.: initial_A = (XTX + lambda eye(10)) \ XTY; Where lambda = 1e-6 is the regularization coefficient to prevent overfitting. In MATLAB, \ is the left division operator used to solve linear equations, and eye is the identity matrix generator.

[0074] Define the objective function for SAD (more robust, insensitive to outliers).

[0075] Optimization was performed using the quasi-Newton method, specifically the `fminunc` function from the MATLAB optimization toolbox, with the optimizer set to... options = optimoptions('fminunc', ... 'Algorithm', 'quasi-newton', ... 'Display', 'iter', ... 'MaxIter', 50, ...% Maximum number of iterations per batch 'OptimalityTolerance', 1e-6); Where 'Algorithm' and 'quasi-newton' represent the quasi-Newton method; 'Display' and 'iter' represent displaying detailed information for each iteration; 'MaxIter' and 50 represent a maximum of 50 iterations; and 'OptimalityTolerance' and 1e-6 represent that when the norm of the gradient vector is less than this value, it is considered to have reached a local optimum, thus controlling the convergence accuracy.

[0076] Reread X_batch and Y_batch according to the batch size, and combine them into the SAD target, that is: objective = @(params) sum(abs(Y_batch - X_batch reshape(params, 10,2)), 'all'); Using the initial value `initial_A`, the optimizer options, and the target objective for the current batch, `fminunc` is called to refine the initial matrix `A`, obtaining a new matrix `A`. This new result is then used as the initial value for the next batch, and optimization is performed batch by batch to finally obtain the prediction matrix `A`. Furthermore, there are 15 classes in total, each receiving a matrix `A`, differentiated by TD (Transformation Time) and motion intensity (see Appendix). The statistical data for motion vectors were obtained by running the ECM's CTC dataset and the common coding domain dataset BVI-DVC.

[0077] (ii) The process of prediction using the relation matrix.

[0078] In the specific applications described below, this embodiment will only describe the second prediction method in detail. The first prediction method will not be described in detail.

[0079] Specifically, first, collect the information of the five neighboring blocks of the current block, including motion vectors, reference frame numbers, and whether the neighboring blocks are available. Note that at this time, it is necessary to collect the five neighboring blocks of list0 and list1. First, obtain the position of the neighboring block, such as PositionposL(pu.Y().x - 1, pu.Y().y + pu.Y().height - 1), and then obtain the nbPU = pu.cs->getPU(posL, pu.chType) that stores the neighboring block information based on the position of the neighboring block. Obtain the motion vector, reference frame number refPOC, and motion vector precision Imv from nbPU, such as neiMvX[i] = nbPU->mv[REF_PIC_LIST_0].getHor(); neiMvY[i] = nbPU->mv[REF_PIC_LIST_0].getVer().

[0080] Process the five neighboring blocks in the directions of list0 and list1 separately. Here, we take the five neighboring blocks in the direction of list0 as an example: First, find the reference frame used most frequently among the five neighboring blocks. If two reference frames have the same number of uses and are the most frequently used, select the reference frame that is closest in time to the current frame and define this reference frame as the main reference frame (MainRefPOC). At this point, TD = abs(CurrPOC - MainRefPOC). Adjust the motion vector onto the main reference frame, and then calculate its motion amplitude. Based on two motion amplitude thresholds, determine its motion intensity category. Since there are five neighboring blocks, there may be five motion intensities. Take the average value to determine the category, i.e., motionCategory. Based on the TD value and motionCategory, the matrix A to be used can be determined.

[0081] Adjust the motion vectors of the five neighboring blocks to the main reference frame to obtain X, according to the formula The motion vector Y of the current block is obtained. The primary reference frame is used as the reference frame for predicting the motion vector.

[0082] Finally, in the ECM19.0 function `getInterMergeCandidates`, the motion vector of the current block is predicted and inserted into the merge list. The function `setMvField` is used to put the motion vector and reference frame number into the merge lists `list0` and `list1`. When only one direction's prediction is valid, the motion vector in the other direction is set to 0, i.e., `setMvField(Mv(0,0),0)`.

[0083] Furthermore, the order of candidates in the merge list is as follows: Spatial motion vector prediction for neighboring and non-neighboring blocks, temporal motion vector prediction, HMVP historical motion vector, CMVP chained motion vector prediction, pairwise averaged motion vector prediction, and 0 motion vector.

[0084] The motion vectors predicted by this method are inserted before the zero motion vectors and do not exceed the maximum number of candidates (i.e., the maximum number of candidates maxNumMergeCand).

[0085] like Figure 3 The figure shows the experimental results for the second prediction method, LDB, in this embodiment. The figure compares the baseline method and the second prediction method for this application scenario horizontally, and presents four indicators vertically.

[0086] right Figure 3 The various indicators are explained as follows: kbps stands for kilobits per second, used to represent the speed of data transmission / playback. k = kilo, b = bit (the smallest unit of data), ps = per second. Therefore: 1 kbps = 1000 bits / s.

[0087] Y-PSNR, or Peak Signal-to-Noise Ratio of the Luminance Component, is the most commonly used image quality evaluation metric in video / image compression. In video encoding, images are typically divided into three components: Y and UV, where Y represents luminance and UV represents chrominance.

[0088] Figure 3 The percentage in the rightmost column represents the BDrate. That is, the percentage of bitrate saved by the two methods under the same decoding and reconstruction quality. A negative value indicates that our method saves bitrate compared to the original anchor method. For example, in the BQSquare sequence, our method saves 0.46% bitrate compared to the original method in the Y component.

[0089] from Figure 3 As can be seen, for sequences with intense motion, such as Racehorses, or sequences with special amplification or reduction motions, such as BQSquare, the method in this embodiment has a significant advantage over the baseline method in the Y component. However, for sequences with gentle motion, the advantage is not as obvious.

[0090] In ECM 19.0, the motion vectors in the merge list all use existing motion vectors, such as the MV of spatially adjacent blocks, the MV of temporal blocks, motion vectors of non-adjacent blocks, historical motion vectors, chained derivation motion vectors, and 0. Only motion vectors with pairwise averages are obtained by averaging existing motion vectors to obtain new motion vectors, which are then inserted into the merge list. However, simple averaging cannot fully utilize the relationship between neighboring blocks and the current block to obtain a better approximation of complex actual motions.

[0091] This method fully explores the relationship between spatial motion vectors and the current block's motion vector through extensive statistical data, obtaining a relationship matrix A. In the actual encoding process, the predicted motion vector Y=AX of the current block is obtained using matrix A and the motion vectors X of neighboring blocks. This avoids the limitation of searching only among existing motion vectors, providing more motion vector choices for the merge list. These choices fully integrate the motion information of the five surrounding blocks, utilizing more neighborhood information than a simple average motion vector. The fusion method is also based on a large amount of statistical data and has certain statistical significance.

[0092] Example 2 This embodiment discloses an offline motion vector prediction system.

[0093] An offline motion vector prediction system includes: The training data acquisition module is configured to: acquire the encoding information of the current block and its five neighboring blocks, wherein the encoding information includes motion vectors and reference frame numbers; The neighboring block coding information classification module is configured to: determine the classification feature types of five neighboring blocks and calculate the classification feature values; and classify the coding information of the five neighboring blocks based on the classification feature values ​​to obtain multiple categories. The relation matrix fitting training module is configured to: use the least squares method to obtain the relation matrix between the current block motion vector and the five neighboring block motion vectors corresponding to each category through fitting training, and obtain the relation matrix table; The indexing module is configured to: determine the five neighboring blocks of the target block whose motion vector is to be predicted, classify the encoding information of the five neighboring blocks of the target block, and, based on the classification results, index the relationship matrix between the target block and the motion vectors of the five neighboring blocks from the relationship matrix table; The prediction module is configured to obtain the motion vector of the target block based on the relationship matrix between the motion vectors of the target block and its five neighboring blocks.

[0094] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0095] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method of motion vector offline prediction, characterized in that, The method comprises the following steps: obtaining the encoding information of the current block and the five-neighbor blocks, the encoding information comprising motion vectors and reference frame numbers; determining the classification feature types of the five-neighbor blocks and calculating the classification feature values, classifying the encoding information of the five-neighbor blocks to obtain multiple categories; using the least square method to obtain the relationship matrix between the motion vectors of the current block and the five-neighbor blocks for each category through fitting training, and obtaining a relationship matrix table; determining the five-neighbor blocks of the target block of the motion vector to be predicted, classifying the encoding information of the five-neighbor blocks of the target block, and based on the classification result, indexing the relationship matrix table to obtain the relationship matrix between the motion vectors of the target block and the five-neighbor blocks; obtaining the motion vector of the target block based on the relationship matrix between the motion vectors of the target block and the five-neighbor blocks.

2. The motion vector offline prediction method of claim 1, wherein, The five-neighbor blocks are the pixel blocks on the left, top, top-left, top-right and bottom-left of the current block or the target block of the motion vector to be predicted.

3. The motion vector offline prediction method of claim 1, wherein, The method for determining the classification feature types of the five-neighbor blocks and calculating the classification feature values, classifying the encoding information of the five-neighbor blocks to obtain multiple categories comprises a first classification mode and a second classification mode, wherein: the first classification mode is as follows: calculating the average motion intensity of the five-neighbor blocks according to a first calculation mode; only using the average motion intensity of the five-neighbor blocks as the classification feature type to divide the encoding information of the five-neighbor blocks into three categories; the second classification mode is as follows: defining the reference frame with the highest usage frequency in the five-neighbor blocks as the main reference frame, calculating the distance absolute value between the main reference frame and the current frame; dividing the distance absolute value into five categories based on the size of the distance absolute value between the main reference frame and the current frame; calculating the average motion intensity of the five-neighbor blocks according to a second calculation mode, and dividing the average motion intensity of the five-neighbor blocks into three categories; combining the five categories of distance absolute values and the three categories of average motion intensity of the five-neighbor blocks to divide the encoding information of the five-neighbor blocks into fifteen categories.

4. The motion vector offline prediction method of claim 3, wherein, In the first classification mode: firstly, the motion vectors of each neighbor block in the five-neighbor blocks are calculated according to the first calculation mode; ； wherein, represents the adjusted motion vector of the pentagonal block; represents the reference frame number of the neighboring block; represents the current frame number; then, the average motion vectors of multiple neighbor blocks in the five-neighbor blocks are calculated based on the motion vectors of each neighbor block; finally, the motion intensity is divided into three categories, i.e., slow, medium and fast, based on the relationship between the average motion vectors and the pre-set upper and lower motion amplitude thresholds.

5. The motion vector offline prediction method of claim 3, wherein, In the second classification mode: the distance absolute value is divided into five categories based on the size of the distance absolute value between the main reference frame and the present frame, which is as follows: when TD=1, it is divided into the first frame; when TD=2, it is divided into the first two frames; when TD=4, it is divided into the third to fourth frames; when TD=8, it is divided into the fifth to eighth frames; when TD=0, it is divided into other time distances, wherein TD represents the distance absolute value.

6. The motion vector offline prediction method of claim 3, wherein, In the second classification mode: firstly, the motion vectors of each neighbor block in the five-neighbor blocks is calculated according to the second calculation mode; ； wherein, represents the adjusted motion vector of the pentagonal block; represents the reference frame index of the neighboring block; represents the primary reference frame index, represents the current frame index; then, the average motion vectors of multiple neighbor blocks in the five-neighbor blocks is calculated based on the motion vectors of each neighbor block; finally, the motion intensity is divided into slow, medium and fast based on the relationship between the average motion vectors and the pre-set upper and lower motion amplitude thresholds.

7. The motion vector offline prediction method of claim 3, wherein, In the second classification manner: If two reference frames in the pentagon block have the same and maximum number of usage, the reference frame closest to the current block in time is selected as the main reference frame.

8. The motion vector offline prediction method of claim 1, wherein, The relationship matrix between the current block motion vector and the pentagon block motion vector corresponding to each category is obtained through fitting training by using the least square method, and specifically includes: The relationship matrix between the current block motion vector and the pentagon block motion vector is expressed as follows: ； Wherein, Y is the motion vector of the current block, X is the vector composed of the motion vector of the pentagon block, and A is the relationship matrix. The relationship matrix A is solved by using matlab and the least square method. The relationship matrices A of multiple categories are collected to obtain a relationship matrix table.

9. The motion vector offline prediction method of claim 1, wherein, Further, the obtained motion vector of the target block is inserted into the merge list: The motion vector of the target block is inserted before the 0 motion vector and does not exceed the maximum number of candidates.

10. A motion vector offline prediction system characterized by, It includes: The training data acquisition module is configured to: acquire the encoding information of the current block and the pentagon block, the encoding information including the motion vector and the reference frame sequence number; The neighbor block encoding information classification module is configured to: determine the classification feature category of the pentagon block and calculate the classification feature value, classify the encoding information of the pentagon block based on the classification feature value, and obtain multiple categories; The relationship matrix fitting training module is configured to: obtain the relationship matrix between the current block motion vector and the pentagon block motion vector corresponding to each category through fitting training by using the least square method, and obtain a relationship matrix table; The index module is configured to: determine the pentagon block of the target block to be predicted, classify the encoding information of the pentagon block of the target block, and index the relationship matrix between the target block and the pentagon block motion vector from the relationship matrix table based on the classification result; The prediction module is configured to: obtain the motion vector of the target block based on the relationship matrix between the target block and the pentagon block motion vector.

Images