Fast motion and parallax estimation method for multi-view video coding

Abstract

The invention discloses a fast motion and parallax estimation method for multi-view video coding, which comprises the following steps of: S1, taking all view images of the first ten frames of a video sequence, performing quadruple downsampling, and calculating an average offset vector Voffset (x, y) between a reference view and a referenced view by using an SIFT (Scale Invariant Feature Transform) feature extraction algorithm; s2, executing predictive coding of the first I frame of the main viewpoint and the first I frame of the auxiliary viewpoint to obtain a predictive frame after the predictive coding; s3, preparing non-I frame coding of the auxiliary viewpoint; and S4, sequentially executing the optimization on all CTUs of the current coding frame until all CTUs are coded.

Description

Technical Field

[0001] The invention belongs to the technical field of video encoding, and in particular relates to a fast motion and disparity estimation method for multi-viewpoint video encoding. Background Art

[0002] In recent years, with the rapid development of multimedia technology, people are no longer satisfied with simple high-definition video applications, but hope to receive and experience video content in a multi-faceted manner through their own senses. This demand has promoted the development of ordinary single-view video content applications to multi-view video applications. Multi-view video refers to video that is recorded by multiple cameras at different angles at the same time. In this type of video application, multi-view video compression is an important research and component. The international standards group has expanded on the basis of the single-view compression standard and released standards for multi-view video. The most advanced one is the MV-HEVC standard based on the HEVC standard extension. Compared with multi-channel simulcast encoding, the technology used in the MV-HEVC standard can reduce the bit rate by 20% to 30% under the same picture quality.

[0003] The prediction technology used by MV-HEVC in the process of predictive coding is basically the same as that of HEVC, including 35 angle predictions for intra-frame prediction and AMVP, Merge, and Skip modes for inter-frame prediction. Similarly, in the process of rate-distortion optimization, CTU needs to be recursively divided into 64×64, 32×32, 16×16, and 8×8.

[0004] However, the increase in the number of viewpoints to be compressed also brings a more complicated encoding process. In addition to the intra-frame prediction and inter-frame prediction of the newly added viewpoint itself, different viewpoints in the same POC (Picture Order Count) will use a new prediction encoding method - inter-view prediction to pursue smaller residuals. The principle of inter-view prediction is basically the same as that of inter-frame prediction, and the main difference lies in the different reference frames. The reference frame of inter-frame prediction is the adjacent frame in the time domain, and the motion estimation performed on this frame is called ME (Motion Estimation). The reference frame of inter-view prediction is the adjacent viewpoint frame at the same time, and the motion estimation performed on this frame is called disparity estimation DE (Disparity Estimation).

[0005] According to statistics, in the process of multi-view video encoding, the predictive encoding process accounts for about 80% of the total encoding time. In the predictive encoding process, the motion estimation of inter-frame prediction and the disparity estimation of inter-view prediction account for about 90% of the time. It can be seen that this process is the main factor that makes multi-view video encoding more complicated. In the multi-view video coding standard based on MV-HEVC, inter-view prediction almost completely reuses the inter-frame prediction module, only modifies the setting of the reference frame, and does not utilize the correlation between different viewpoints, resulting in the complexity of the disparity estimation of inter-view prediction being almost the same as the complexity of the motion estimation of inter-frame prediction. In addition, the encoding process of the auxiliary viewpoint is basically the same as that of the main viewpoint, and the similarity between the main and auxiliary viewpoints is not utilized, so there is a lot of room for acceleration here. Summary of the invention

[0006] In view of the above problems, the present invention provides a fast motion and disparity estimation method for multi-viewpoint video encoding, which utilizes the relevant information between different viewpoints to optimize the motion estimation and disparity estimation processes of auxiliary viewpoints, and reduces the estimation area while ensuring that the estimation results are as accurate as possible, so as to reduce the encoding complexity of the auxiliary viewpoints, thereby reducing the complexity of the entire encoding process.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A fast motion and disparity estimation method for multi-view video coding comprises the following steps:

[0009] S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y);

[0010] S2, performing predictive coding on the first I frame of the main viewpoint and the first I frame of the auxiliary viewpoint to obtain a predicted frame after predictive coding;

[0011] S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame,

[0012] If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the inter-frame search area SR and the inter-view search area SR' finally selected; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR';

[0013] If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the inter-frame search area SR and the inter-view search area SR' finally selected are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, firstly, a fast reference frame decision is performed to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, a fast motion estimation is completed on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, a fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR';

[0014] S4, performing the above optimization on all CTUs of the current coding frame in sequence until all CTUs are coded.

[0015] In one possible implementation, the SIFT feature extraction algorithm described in S1 is used to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes:

[0016] Extract feature points, and set the feature point set of the two viewpoint images of the kth image as {P 1 ,P 2 ,…P n} and {Q 1 ,Q 2 ,…Q n}, use the matching algorithm to match the feature points, and get several matching pairs represented as (P i ,Q j ), for each matched feature point, calculate their coordinate offsets Δx and Δy:

[0017] Δx i =Q "x -P ix

[0018] Δy i =Q "$ -P i$

[0019] Among them, P ix and P i$ is the coordinate of the i-th feature point of the first viewpoint of the image, Q "x and Q "$ is the coordinate of the jth feature point of the second viewpoint of the image;

[0020] Calculate the average offset of all feature points:

[0021]

[0022] Where N is the number of matching pairs; after the average offset of the viewpoints corresponding to all images is calculated, the total average offset can be obtained, that is:

[0023] When the vector component value is negative, it means it is shifted to the left or downward; when it is positive, it means it is shifted to the right or upward. offset The vector value of (x, y) is an integer multiple of the minimum CU size, that is:

[0024] V offset (x,y)=(V offsetx +MiniSize CU -V offsetx %MiniSize CU ,V offsety +MiniSize CU -V offsety %MiniSize CU )

[0025] Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

[0026] In a possible implementation manner, the preparing the non-I frame encoding of the auxiliary viewpoint in S3 and determining the current frame type specifically include:

[0027] Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

[0028] In a possible implementation manner, if the current frame is a P frame, the predicted frame of the P frame obtained after encoding, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include:

[0029] Prediction coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block has the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block.

[0030]

[0031] Where N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is:

[0032]

[0033] In order to further improve the accuracy of the search area, when the sub-CU of the current block performs inter-frame AMVP prediction, the MV of the sub-CU on the offset block of the corresponding block of the sub-CU to be encoded in the current block and the adjacent encoded MV of the current block are comprehensively considered to obtain a refined search area Accurate SR for the current sub-CU; the error percentage of the width and height of InitialSR and Accurate SR is calculated; if one of the errors is greater than 0.5, or the errors are both greater than 0.2, it is considered that Accurate SR is applied; otherwise, Initial SR is applied;

[0034] When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV

[0035]

[0036] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, and x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for the current block to perform inter-view AMVP prediction. Similarly, the width and height of this area are the average values ​​of all DV components, that is:

[0037]

[0038] Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is:

[0039] (x 偏 ,y 偏 )=(x 当前 ,y 当前 )

[0040] Then, according to the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in AMVP mode.

[0041] In a possible implementation manner, if the current frame is a B frame, then after encoding, the predicted frame of the B frame of the auxiliary viewpoint, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include:

[0042] If the current frame is a B frame, first enable the fast reference frame decision method to determine the reference frame selected by the main viewpoint of the current viewpoint, and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, and the other two possible choices are discarded; after completing the reference frame decision, obtain the final selected inter-frame search area SR and inter-viewpoint search area SR' to complete the fast encoding optimization.

[0043] The present invention has the following beneficial effects:

[0044] (1) The disparity vector (DV) information of the encoded reference viewpoint is used to optimize the disparity estimation process of the current reference viewpoint to be encoded, and the search area is adaptively reduced.

[0045] (2) Using the motion vector (MV) of the main viewpoint that has been encoded with the same POC, the motion search area of ​​the auxiliary viewpoint to be encoded is reasonably limited to simplify the motion estimation process.

[0046] (3) For B frames using inter-frame bidirectional prediction, reference frames are quickly selected to speed up encoding.

[0047] (4) Greatly reduce the complexity of the AMVP mode for inter-frame prediction of auxiliary viewpoint non-I frames and inter-view prediction, so as to reduce the overall complexity of multi-view coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for fast motion and disparity estimation for multi-view video coding according to an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of the internal relationship between the current frame and the reference frame in AMVP mode optimization. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1, which is a flowchart of a method for fast motion and disparity estimation of multi-view video coding according to an embodiment of the present invention, comprises the following steps:

[0052] S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y);

[0053] S2, performing predictive coding on the first I frame of the main viewpoint and the first I frame of the auxiliary viewpoint to obtain a predicted frame after predictive coding;

[0054] S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame,

[0055] If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the inter-frame search area SR and inter-view search area SR' finally selected; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR'. Figure 2 The internal relationship between the current frame and the reference frame in AMVP mode optimization is given;

[0056] If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the inter-frame search area SR and the inter-view search area SR' finally selected are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, firstly, a fast reference frame decision is performed to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, a fast motion estimation is completed on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, a fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR';

[0057] S4, performing the above optimization on all CTUs of the current coding frame in sequence until all CTUs are coded.

[0058] Specifically, the SIFT feature extraction algorithm described in S1 is used to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes:

[0059] Extract feature points, and set the feature point set of the two viewpoint images of the kth image as {P 1 ,P 2 ,…P n} and {Q 1,Q 2 ,…Q n}, use the matching algorithm (such as BFMatcher algorithm) to match the feature points, and obtain several matching pairs represented as (P i ,Q j ), for each matched feature point, calculate their coordinate offsets Δx and Δy:

[0060] Δx i =Q "x -P ix

[0061] Δy i =Q "$ -P i$

[0062] Among them, P ix and P i$ is the coordinate of the i-th feature point of the first viewpoint of the image, Q "x and Q "$ is the coordinate of the jth feature point of the second viewpoint of the image;

[0063] Calculate the average offset of all feature points:

[0064]

[0065] Where N is the number of matching pairs; after the average offset of the viewpoints corresponding to all images is calculated, the total average offset can be obtained, that is:

[0066] When the vector component value is negative, it means it is shifted to the left or downward; when it is positive, it means it is shifted to the right or upward. offset The vector value of (x, y) is an integer multiple of the minimum CU size, that is:

[0067] V offset (x,y)=(V offsetx +MiniSize CU -V offsetx %MiniSize CU ,V offsety +MiniSize CU -V offsety %MiniSize CU )

[0068] Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

[0069] Specifically, the preparation of non-I frame encoding of the auxiliary viewpoint in S3 and the determination of the current frame type specifically include:

[0070] Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

[0071] Specifically, if the current frame is a P frame, the predicted frame of the P frame obtained after encoding, and the inter-frame search area SR and inter-view search area SR' finally selected specifically include:

[0072] Prediction coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block has the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block.

[0073]

[0074] Where N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is:

[0075]

[0076] In order to further improve the accuracy of the search area, when the sub-CU of the current block performs inter-frame AMVP prediction, the MV of the sub-CU on the offset block of the corresponding block of the sub-CU to be encoded in the current block and the adjacent encoded MV of the current block are comprehensively considered to obtain a refined search area Accurate SR for the current sub-CU; the error percentage of the width and height of InitialSR and Accurate SR is calculated; if one of the errors is greater than 0.5, or the errors are both greater than 0.2, it is considered that Accurate SR is applied; otherwise, Initial SR is applied;

[0077] That is, the method of obtaining the sub-CU on the offset block of the corresponding block of the sub-CU to be encoded in the current block is the same as the method of obtaining the offset block of the corresponding block, and the offset vector V is also used. offset (x,y) Calculate coordinates:

[0078] (x 偏 ,y 偏 )=(x 当前 +V offsetx ,y 当前 +V offsety )

[0079] Get the MV stored in the offset block 偏x and MV 偏y Used for the following calculations.

[0080] Then, take the MV of the adjacent coded blocks of the current block and calculate their average:

[0081]

[0082] Where M is the number of adjacent coded blocks, and k represents the kth adjacent coded block. Based on the above calculation information, the refined search area Accurate SR is calculated for the current coded block:

[0083]

[0084] Among them, H ASR and W ASR Represents the height and width of Accurate SR, α and β represent the calculation weights. Based on actual test experiments, when α=0.65 and β=0.35, the designed algorithm can achieve the optimal balance between coding complexity and performance loss.

[0085] Calculate the width and height error percentage of the Initial SR and Accurate SR, that is:

[0086]

[0087] If P 误x or P 误$ Greater than 0.5, or P 误x and P 误$ If both are greater than 0.2, Accurate SR is applied; otherwise, Initial SR is applied.

[0088] When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV

[0089]

[0090] Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, and x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for the current block to perform inter-view AMVP prediction. Similarly, the width and height of this area are the average values ​​of all DV components, that is:

[0091]

[0092] Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is:

[0093] (x 偏 ,y 偏 )=(x 当前 ,y 当前 )

[0094] Then, according to the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in AMVP mode.

[0095] Specifically, if the current frame is a B frame, then after encoding, the predicted frame of the B frame of the auxiliary viewpoint is obtained, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include:

[0096] If the current frame is a B frame, first enable the fast reference frame decision method to determine the reference frame selected by the main viewpoint of the current viewpoint, and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, and the other two possible choices are discarded; after completing the reference frame decision, obtain the final selected inter-frame search area SR and inter-viewpoint search area SR' to complete the fast encoding optimization.

[0097] Table 1 shows the objective quality changes and encoding speed changes of the method of the present invention compared with the original encoder under three 1080p and three 720p 3D sequence tests. It can be seen that the method of the present invention can reduce the encoding complexity by about 32% on average, with a performance loss of about 0.4%. And through the data shown in the table, for sequences with lower resolutions, the acceleration effect of the method of the present invention is more obvious. The changing characteristics of the encoding performance loss do not have an obvious trend in sequences with different resolutions, but the performance loss of less than 1% is almost negligible and will not have much impact on subjective feelings.

[0098] Table 1 Objective quality change and encoding speed change of the method of the present invention compared with the original encoder

[0099]

[0100] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A fast motion and disparity estimation method for multi-view video coding, characterized in that: The following steps are involved: S1, take all viewpoint images of the first ten frames of the video sequence, perform four-fold downsampling, use the SIFT feature extraction algorithm to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint offset (x,y); S2, performing predictive coding on the first I frame of the main viewpoint and the first I frame of the auxiliary viewpoint to obtain a predicted frame after predictive coding; S3, prepare the non-I frame encoding of the auxiliary viewpoint, first determine the type of the current frame, If the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, as well as the inter-frame search area SR and the inter-view search area SR' finally selected; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, fast motion estimation is completed on the reference frame of the auxiliary viewpoint based on the inter-frame search area SR; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR'; If the current frame is a B frame, after encoding, the predicted frame of the B frame of the auxiliary viewpoint, as well as the inter-frame search area SR and the inter-view search area SR' finally selected are obtained; when executing the AMVP mode of inter-frame prediction of each sub-CU of the current frame, firstly, a fast reference frame decision is performed to determine the inter-frame reference combination of the frame corresponding to the main viewpoint; then, based on the reference frame combination and the inter-frame search area SR, a fast motion estimation is completed on the reference frame of the main viewpoint; when executing the AMVP mode of inter-view prediction of each sub-CU of the current frame, a fast disparity estimation is completed on the reference frame of the main viewpoint based on the inter-view search area SR'; S4, performing the above optimization on all CTUs of the current coding frame in sequence until all CTUs are coded.

2. The fast motion and disparity estimation method for multi-view video coding according to claim 1, characterized in that: In S1, the SIFT feature extraction algorithm is used to calculate the average offset vector V between the reference viewpoint and the referenced viewpoint. offset (x,y) includes: Extract feature points. Let the feature point set of the two viewpoint images of the kth image be {P1, P2, ... P n } and {Q1,Q2,…Q n }, use the matching algorithm to match the feature points, and get several matching pairs represented as (P i ,Q j ), for each matched feature point, calculate their coordinate offsets Δx and Δy: Δx i =Q "x -P ix Δy i =Q "$ -P i$ Among them, P ix and P i$ is the coordinate of the i-th feature point of the first viewpoint of the image, Q "x and Q "$ is the coordinate of the jth feature point of the second viewpoint of the image;. Calculate the average offset of all feature points: Where N is the number of matching pairs; after the average offset of the viewpoints corresponding to all images is calculated, the total average offset can be obtained, that is: When the vector component value is negative, it means it is shifted to the left or downward; when it is positive, it means it is shifted to the right or upward. offset The vector value of (x, y) is an integer multiple of the minimum CU size, that is: V offset (x,y)=(V offsetx +MiniSize CU -V offsetx %MiniSize CU ,V offsety +MiniSize CU -V offsety %MiniSize CU ) Among them, MiniSize CU Represents the minimum value of the CU height set by the encoder, usually 8, V offsetx and V offsety V offset (x,y) Component values ​​in the x and y directions.

3. The fast motion and disparity estimation method for multi-view video coding according to claim 1, characterized in that: In S3, the preparation of non-I frame encoding of the auxiliary viewpoint and the determination of the current frame type specifically include: Determine whether the current frame is a P frame or a B frame by observing whether it has a backward reference frame. If so, it is a B frame, otherwise it is a P frame.

4. The fast motion and disparity estimation method for multi-view video coding according to claim 1, characterized in that: In S3, if the current frame is a P frame, the predicted frame of the P frame is obtained after encoding, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include: Prediction coding starts from the first CTU of the current frame. When the inter-frame AMVP mode is executed, if it is a P frame, based on the offset vector V offset (x, y), find the offset block of the corresponding block of the current block. The offset block has the same size as the current block and contains N encoded CUs. Each CU stores the optimal MV selected during inter-frame prediction. Count the average area of ​​the matrix formed by all MVs in the offset block. Where N represents the number of CUs in the encoded offset block, that is, the number of stored MVs, x and y represent the component size of the MV in the horizontal direction and the component size in the vertical direction respectively; the area That is, the initial search area Initial SR when the current block performs inter-frame AMVP prediction. The width and height of this area are the average values ​​of all MV components, that is: In order to further improve the accuracy of the search area, when the sub-CU of the current block performs inter-frame AMVP prediction, the MV of the sub-CU on the offset block of the corresponding block of the sub-CU to be encoded in the current block and the adjacent encoded MV of the current block are comprehensively considered to obtain a refined search area Accurate SR for the current sub-CU; the error percentage of the width and height of Initial SR and Accurate SR is calculated; if one of the errors is greater than 0.5, or the errors are both greater than 0.2, it is considered that Accurate SR is applied; otherwise, Initial SR is applied; When the inter-view AMVP mode is executed, the average sum of the areas of the matrix formed by all DVs in the offset block calculated based on the DV Among them, N represents the number of CUs in the encoded offset block, that is, the number of stored DVs, and x and y represent the component size of the DV in the horizontal direction and the component size in the vertical direction respectively; so the calculated area is the initial search area Initial SR' for the current block to perform inter-view AMVP prediction. Similarly, the width and height of this area are the average values ​​of all DV components, that is: Since the offset vector of inter-view prediction is set to (0,0), the coordinates of the offset block are consistent with the current block, that is: (x 偏 ,and 偏 )=(x 当前 ,and 当前 ) Then, according to the DV and Initial SR' obtained in this step, the area size and position of Accurate SR' are calculated, and one of them is selected for optimization in AMVP mode.

5. The fast motion and disparity estimation method for multi-view video coding according to claim 1, characterized in that: If the current frame is a B frame, then after encoding, the predicted frame of the B frame of the auxiliary viewpoint is obtained, and the inter-frame search area SR and the inter-view search area SR' finally selected specifically include: If the current frame is a B frame, first enable the fast reference frame decision method to determine the reference frame selected by the main viewpoint of the current viewpoint, and directly use it as the only reference frame of the current viewpoint. That is, assuming that the reference frame combination selected by the main viewpoint for inter-frame prediction only uses backward reference frames, the current viewpoint also only uses backward reference frames, and the other two possible choices are discarded; after completing the reference frame decision, obtain the final selected inter-frame search area SR and inter-viewpoint search area SR' to complete the fast encoding optimization.

Images