Fast motion and disparity estimation method for multi-view video coding

Abstract

The application discloses a fast motion and disparity estimation method for multi-view video coding, which comprises the following steps: S1, taking all view images of the first ten frames of a video sequence, performing four times down-sampling, using a SIFT feature extraction algorithm, and calculating average offset vectors V between a reference view and a referenced view offset (x,y); S2, performing prediction coding on the first I frame of a main view and the first I frame of an auxiliary view to obtain a prediction frame after the prediction coding; S3, preparing non-I frame coding of the auxiliary view; and S4, sequentially performing the above optimization on all CTUs of a current coding frame until all CTUs are coded.

Description

Technical Field

[0001] The present invention belongs to the technical field of video coding, and in particular relates to a fast motion and disparity estimation method for multi-viewpoint video coding. Background Art

[0002] With the rapid development of multimedia technology in recent years, people are no longer satisfied with simple high-definition video applications. Instead, they desire to receive and experience video content from multiple perspectives through their senses. This demand has driven the evolution of standard single-viewpoint video content applications towards multi-viewpoint video applications. Multi-viewpoint video refers to video captured simultaneously by multiple cameras from different angles. In this type of video application, multi-viewpoint video compression is a key research and component. International standards groups have expanded upon single-viewpoint compression standards and released standards for multi-viewpoint video. Currently, the most advanced is the MV-HEVC standard, an extension of the HEVC standard. Compared to simulcast encoding, the technology used in the MV-HEVC standard can reduce the bitrate by 20% to 30% while maintaining equivalent image quality.

[0003] MV-HEVC uses similar prediction techniques to HEVC during predictive coding, including 35 angle predictions for intra-frame prediction and AMVP, Merge, and Skip modes for inter-frame prediction. Similarly, during rate-distortion optimization, the CTU is recursively partitioned into 64×64, 32×32, 16×16, and 8×8 configurations.

[0004] However, the increase in the number of viewpoints to be compressed also brings a more complex encoding process. In addition to the intra-frame prediction and inter-frame prediction of the newly added viewpoint itself, different viewpoints at the same POC (Picture Order Count) will use a new prediction encoding method - inter-view prediction to pursue smaller residuals. The principle of inter-view prediction is basically the same as that of inter-frame prediction. The main difference lies in the different reference frames. The reference frame of inter-frame prediction is the adjacent frame in the time domain, and the motion estimation performed on this frame is called ME (Motion Estimation). The reference frame of inter-view prediction is the adjacent viewpoint frame at the same time, and the motion estimation performed on this frame is called disparity estimation DE (Disparity Estimation).

[0005] According to statistics, during multi-view video encoding, predictive coding accounts for approximately 80% of the total encoding time. Within the predictive coding process, motion estimation for inter-frame prediction and disparity estimation for inter-view prediction account for approximately 90% of the time. This process is the primary factor contributing to the increased complexity of multi-view video encoding. In multi-view video coding standards, primarily MV-HEVC, inter-view prediction almost completely reuses the inter-frame prediction module, modifying only the reference frame settings without leveraging the correlation between different viewpoints. This results in the disparity estimation complexity for inter-view prediction being almost identical to the motion estimation complexity for inter-frame prediction. Furthermore, the encoding process for the auxiliary viewpoint is essentially the same as that for the primary viewpoint, without leveraging the similarity between the primary and auxiliary viewpoints, leaving significant room for acceleration. Summary of the Invention

[0006] In view of the above problems, the present invention provides a fast motion and disparity estimation method for multi-viewpoint video coding, which utilizes the relevant information between different viewpoints to optimize the motion estimation and disparity estimation process of auxiliary viewpoints, and reduces the estimation area while ensuring that the estimation results are as accurate as possible, so as to reduce the coding complexity of the auxiliary viewpoints, thereby reducing the complexity of the entire coding process.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A fast motion and disparity estimation method for multi-view video coding comprises the following steps:

[0009] S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y);

[0010] S2, performing predictive coding on the first I frame of the primary viewpoint and the first I frame of the secondary viewpoint to obtain a predicted frame after predictive coding;

[0011] S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame,

[0012] If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the finally selected inter-frame search area SR and inter-view search area SR'; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR';

[0013] If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the finally selected inter-frame search area SR and inter-view search area SR' are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, first perform a fast reference frame decision to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, complete fast motion estimation on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, complete fast disparity estimation on the reference frame of the main viewpoint based on the inter-view search area SR';

[0014] S4, performing the above optimization on all CTUs of the current coding frame in sequence until all CTUs are coded.

[0015] In one possible implementation, the SIFT feature extraction algorithm is used as described in S1 to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes:

[0016] Extract feature points, and assume that the feature point set of the two viewpoint images of the kth image is {P1, P2, ... P n} and {Q1,Q2,…Q n}, use the matching algorithm to match the feature points, and get several matching pairs represented as (P i ,Q j ), for each matched feature point, calculate their coordinate offsets Δx and Δy:

[0017] Δx i =Q jx -P ix

[0018] Δy i =Q jy -P iy

[0019] Among them, P ix and P iy is the coordinate of the i-th feature point of the first viewpoint of the image, Q jx and Q jy is the coordinate of the jth feature point of the second viewpoint of the image;

[0020] Calculate the average offset of all feature points:

[0021]

[0022] Where N is the number of matching pairs; after calculating the average offset of the viewpoints corresponding to all images, the total average offset can be obtained, that is:

[0023] When the vector component value is negative, it means it is shifting to the left or downward; when it is positive, it means it is shifting to the right or upward. offset The vector value of (x,y) is an integer multiple of the minimum CU size, that is:

[0024] V offset (x,y)=(V offsetx +MiniSize CU -V offsetx %MiniSize CU ,V offsety +MiniSize CU -V offsety %MiniSize CU )

[0025] Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

[0026] In one possible implementation, the step of preparing the non-I frame encoding of the auxiliary viewpoint in S3 and determining the current frame type specifically includes:

[0027] Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

[0028] In one possible implementation, if the current frame is a P frame, then the predicted frame of the P frame obtained after encoding, and the inter-frame search area SR and inter-view search area SR' finally selected specifically include:

[0029] Predictive coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block is the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block.

[0030]

[0031] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is:

[0032]

[0033] To further improve the accuracy of the search area, when performing inter-frame AMVP prediction on the sub-CU of the current block, a refined Accurate SR search area for the current sub-CU is obtained based on the initial search area, the MV of the sub-CU on the offset block corresponding to the block of the sub-CU to be encoded in the current block, and the MV of the adjacent encoded MV of the current block. The error percentage of the width and height of InitialSR and Accurate SR is calculated. If either error is greater than 0.5 or both errors are greater than 0.2, Accurate SR is applied; otherwise, Initial SR is applied.

[0034] When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV is

[0035]

[0036] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for inter-view AMVP prediction of the current block. Similarly, the width and height of this area are the average values ​​of all DV components, that is:

[0037]

[0038] Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is:

[0039] (x 偏 ,y 偏 )=(x 当前 ,y 当前 )

[0040] Then, based on the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in the AMVP mode.

[0041] In one possible implementation, if the current frame is a B frame, then after encoding, the predicted frame of the B frame of the auxiliary viewpoint, and the finally selected inter-frame search area SR and inter-view search area SR' specifically include:

[0042] If the current frame is a B frame, the fast reference frame decision method is first enabled to determine the reference frame selected by the main viewpoint of the current viewpoint and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, discarding the other two possible options; after completing the reference frame decision, the final selected inter-frame search area SR and inter-viewpoint search area SR' are obtained to complete the fast coding optimization.

[0043] The present invention has the following beneficial effects:

[0044] (1) The disparity vector (DV) information of the encoded reference viewpoint is used to optimize the disparity estimation process of the current reference viewpoint to be encoded, and the search area is adaptively narrowed.

[0045] (2) Using the motion vector (MV) of the main viewpoint that has been encoded with the same POC, the motion search area of ​​the auxiliary viewpoint to be encoded is reasonably limited to simplify the motion estimation process.

[0046] (3) For B frames using inter-frame bidirectional prediction, quickly select reference frames to speed up encoding.

[0047] (4) Significantly reduce the complexity of the AMVP mode for inter-frame prediction of auxiliary viewpoint non-I frames and inter-view prediction, thereby reducing the overall complexity of multi-view coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for fast motion and disparity estimation for multi-view video coding according to an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of the internal relationship between the current frame and the reference frame in AMVP mode optimization. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] See also Figure 1 , which is a flowchart of a method for fast motion and disparity estimation for multi-view video coding according to an embodiment of the present invention, includes the following steps:

[0052] S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y);

[0053] S2, performing predictive coding on the first I frame of the primary viewpoint and the first I frame of the secondary viewpoint to obtain a predicted frame after predictive coding;

[0054] S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame,

[0055] If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the inter-frame search area SR and inter-view search area SR' that are finally selected; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR'. Figure 2 The internal relationship between the current frame and the reference frame in AMVP mode optimization is given;

[0056] If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the finally selected inter-frame search area SR and inter-view search area SR' are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, first perform a fast reference frame decision to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, complete fast motion estimation on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, complete fast disparity estimation on the reference frame of the main viewpoint based on the inter-view search area SR';

[0057] S4, performing the above optimization on all CTUs of the current coding frame in sequence until all CTUs are coded.

[0058] Specifically, the SIFT feature extraction algorithm is used as described in S1 to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes:

[0059] Extract feature points, and assume that the feature point set of the two viewpoint images of the kth image is {P1, P2, ... P n} and {Q1,Q2,…Q n}, use matching algorithm (such as BFMatcher algorithm) to match the feature points, and get several matching pairs represented as (P i ,Q j), for each matched feature point, calculate their coordinate offsets Δx and Δy:

[0060] Δx i =Q jx -P ix

[0061] Δy i =Q jy -P iy

[0062] Among them, P ix and P iy is the coordinate of the i-th feature point of the first viewpoint of the image, Q jx and Q jy is the coordinate of the jth feature point of the second viewpoint of the image;

[0063] Calculate the average offset of all feature points:

[0064]

[0065] Where N is the number of matching pairs; after calculating the average offset of the viewpoints corresponding to all images, the total average offset can be obtained, that is:

[0066] When the vector component value is negative, it means it is shifting to the left or downward; when it is positive, it means it is shifting to the right or upward. offset The vector value of (x,y) is an integer multiple of the minimum CU size, that is:

[0067] V offset (x,y)=(V offsetx +MiniSize CU -V offsetx %MiniSize CU ,V offsety +MiniSize CU -V offsety %MiniSize CU )

[0068] Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

[0069] Specifically, the preparation of the non-I frame encoding of the auxiliary viewpoint in S3 and the determination of the current frame type specifically include:

[0070] Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

[0071] Specifically, if the current frame is a P frame, the predicted frame of the P frame obtained after encoding, and the inter-frame search area SR and inter-view search area SR' finally selected specifically include:

[0072] Predictive coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block is the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block.

[0073]

[0074] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is:

[0075]

[0076] To further improve the accuracy of the search area, when performing inter-frame AMVP prediction on the sub-CU of the current block, a refined Accurate SR search area for the current sub-CU is obtained based on the initial search area, the MV of the sub-CU on the offset block corresponding to the block of the sub-CU to be encoded in the current block, and the MV of the adjacent encoded MV of the current block. The error percentage of the width and height of InitialSR and Accurate SR is calculated. If either error is greater than 0.5 or both errors are greater than 0.2, Accurate SR is applied; otherwise, Initial SR is applied.

[0077] That is, the method of obtaining the sub-CU on the offset block of the corresponding block of the sub-CU to be encoded in the current block is the same as that of obtaining the offset block of the corresponding block, and the offset vector V is also used. offset (x,y) calculated coordinates:

[0078] (x 偏 ,y 偏 )=(x 当前 +V offsetx ,y当前 +V offsety )

[0079] Get the MV stored in the offset block 偏x and MV 偏y Used for the following calculations.

[0080] Then, take the MVs of the adjacent coded blocks of the current block and calculate their average:

[0081]

[0082] Where M is the number of adjacent coded blocks, and k represents the kth adjacent coded block. Based on the above calculation information, the fine search area Accurate SR is calculated for the current coded block:

[0083]

[0084] Among them, H ASR and W ASR Represents the height and width of Accurate SR, α and β represent the calculation weights. Based on actual test experiments, when α = 0.65 and β = 0.35, the designed algorithm can achieve the optimal balance between coding complexity and performance loss.

[0085] Calculate the width and height error percentage of the Initial SR and Accurate SR, that is:

[0086]

[0087] If P 误x or P 误y Greater than 0.5, or P 误x and P 误y If both are greater than 0.2, Accurate SR is applied; otherwise, Initial SR is applied.

[0088] When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV is

[0089]

[0090] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for inter-view AMVP prediction of the current block. Similarly, the width and height of this area are the average values ​​of all DV components, that is:

[0091]

[0092] Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is:

[0093] (x 偏 ,y 偏 )=(x 当前 ,y 当前 )

[0094] Then, based on the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in the AMVP mode.

[0095] Specifically, if the current frame is a B frame, then after encoding, the predicted frame of the B frame of the auxiliary viewpoint is obtained, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include:

[0096] If the current frame is a B frame, the fast reference frame decision method is first enabled to determine the reference frame selected by the main viewpoint of the current viewpoint and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, discarding the other two possible options; after completing the reference frame decision, the final selected inter-frame search area SR and inter-viewpoint search area SR' are obtained to complete the fast coding optimization.

[0097] Table 1 shows the objective quality changes and encoding speed changes achieved by the proposed method compared to the original encoder for three 1080p and three 720p 3D sequences. As can be seen, the proposed method reduces encoding complexity by approximately 32% on average, with a performance loss of approximately 0.4%. Furthermore, the data in the table shows that the acceleration effect of the proposed method is even more pronounced for sequences with lower resolutions. While the performance loss does not show a clear trend across sequences with different resolutions, a performance loss of less than 1% is negligible and will not significantly impact subjective experience.

[0098] Table 1 Objective quality change and encoding speed change of the method of the present invention compared with the original encoder

[0099]

[0100] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A fast motion and disparity estimation method for multi-view video coding, characterized in that: The following steps are involved: S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y); S2, performing predictive coding on the first I frame of the primary viewpoint and the first I frame of the secondary viewpoint to obtain a predicted frame after predictive coding; S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame, If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the finally selected inter-frame search area SR and inter-view search area SR'; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR'; If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the finally selected inter-frame search area SR and inter-view search area SR' are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, first perform a fast reference frame decision to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, complete fast motion estimation on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, complete fast disparity estimation on the reference frame of the main viewpoint based on the inter-view search area SR'; S4, performing the optimization of steps S1 to S3 on all CTUs of the current coding frame in sequence until all CTUs are coded.

2. The fast motion and disparity estimation method for multi-view video coding according to claim 1, wherein: In S1, the SIFT feature extraction algorithm is used to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes: Extract feature points, and assume that the feature point set of the two viewpoint images of the kth image is {P1, P2, ... P n } and {Q1,Q2,…Q n }, use the matching algorithm to match the feature points, and get several matching pairs represented as (P i ,Q j ), for each matched feature point, calculate their coordinate offsets Δx and Δy: Δx i =Q jx -P ix Δy i =Q jy -P iy Among them, P ix and P iy is the coordinate of the i-th feature point of the first viewpoint of the image, Q jx and Q jy is the coordinate of the jth feature point of the second viewpoint of the image; Calculate the average offset of all feature points: Where N is the number of matching pairs; after calculating the average offset of the viewpoints corresponding to all images, the total average offset can be obtained, that is: When the vector component value is negative, it means it is shifting to the left or downward; when it is positive, it means it is shifting to the right or upward. offset The vector value of (x,y) is an integer multiple of the minimum CU size, that is: V offset (x,y)=(V offsetx +MiniSize CU -V offsetx MiniSize CU ,V offsety +MiniSize CU -V offsety MiniSize CU ) Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

3. The fast motion and disparity estimation method for multi-view video coding according to claim 1, wherein: In S3, the preparation of the non-I frame encoding of the auxiliary viewpoint and the determination of the current frame type specifically include: Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

4. The fast motion and disparity estimation method for multi-view video coding according to claim 1, wherein: As described in S3, if the current frame is a P frame, the predicted frame of the P frame obtained after encoding, and the inter-frame search area SR and inter-view search area SR' finally selected specifically include: Predictive coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block is the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block. Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is: To further improve the accuracy of the search area, when performing inter-frame AMVP prediction on the sub-CU of the current block, a refined Accurate SR for the search area of ​​the current sub-CU is obtained based on the initial search area, the MV of the sub-CU on the offset block corresponding to the block of the sub-CU to be encoded in the current block, and the MV of the adjacent encoded MV of the current block. The error percentage of the width and height of the Initial SR and Accurate SR is calculated. If either error is greater than 0.5 or both errors are greater than 0.2, Accurate SR is applied; otherwise, Initial SR is applied. When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV is Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for inter-view AMVP prediction of the current block. Similarly, the width and height of this area are the average values ​​of all DV components, that is: Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is: (x 偏 ,and 偏 )=(x 当前 ,and 当前 ) Then, based on the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in the AMVP mode.

5. The fast motion and disparity estimation method for multi-view video coding according to claim 1, wherein: As described in S3, if the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, and the finally selected inter-frame search area SR and inter-view search area SR' specifically include: If the current frame is a B frame, the fast reference frame decision method is first enabled to determine the reference frame selected by the main viewpoint of the current viewpoint and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, and the other two possible options are discarded. After completing the reference frame decision, the final selected inter-frame search area SR and inter-viewpoint search area SR' are obtained to complete the fast coding optimization.

Images