Video encoding method and device

Abstract

The present application discloses a video coding method, system, apparatus, and device. The method utilizes the inter-coding mode information of neighboring CUs of a target coding unit and employs a weighted scoring mechanism for voting by neighboring CUs to make decisions on skipping the inter-coding mode of the target coding CU. This early termination of inter-coding mode decisions allows for rapid selection of merge mode or skip mode as the optimized inter-coding mode, and premature termination of the time-consuming search process for inter-coding and other modes. This effectively reduces the complexity of inter-coding for large-sized CUs, thereby improving overall coding efficiency.

Description

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 3, 2022, with application number 202210926788.3 and invention name “Video Coding Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of video processing technology, and in particular to a video encoding method and apparatus, a video encoding system, and an electronic device. Background Art

[0003] Video encoding is the process of converting an original video file into another format through compression technology. The compression rate and encoding complexity of a video encoder are key indicators of the competitiveness of video products (such as live streaming, on-demand video, and video calls).

[0004] The primary time overhead of video coding occurs in the inter-frame coding phase. Early termination of inter-frame mode decision can reduce coding complexity in inter-frame mode. A typical early termination method for inter-frame mode decision follows this process: the mean rate-distortion (RD) cost of spatially neighboring blocks is calculated and used as a threshold, T. If the current coding unit (CU) is a small CU (4×N or N×4 CU) and the optimal rate-distortion after other non-inter-frame prediction modes is less than the threshold, inter-frame mode prediction is skipped. This approach allows the inter-frame mode decision to be skipped early.

[0005] However, in the process of implementing the present invention, the inventors found that the above scheme has at least the following problems: it is only applicable to small-sized coding units (4×N or N×4 CUs) and is not applicable to large-sized coding units. Therefore, the inter-frame coding complexity of large-sized CUs is relatively high. Summary of the Invention

[0006] The present application provides a video coding method to solve the problem of high inter-frame coding complexity of large-sized CUs in the prior art. The present application also provides a video coding device, a video coding system, and an electronic device.

[0007] The present application provides a video encoding method, comprising:

[0008] Obtain a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from the neighbor coding units of the target coding unit;

[0009] Obtaining an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0010] Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0011] Perform inter-frame coding in merge mode or skip mode on the target coding unit;

[0012] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score;

[0013] If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

[0014] Optionally, also include:

[0015] If the inter-frame coding result of the merge mode or skip mode performed on the target coding unit is an all-zero block, the inter-frame coding of other inter-frame coding modes on the target coding unit is terminated.

[0016] Optionally, also include:

[0017] If the above judgment result is no and the inter-frame coding result of the merge mode or skip mode on the target coding unit is not an all-zero block, the inter-frame coding mode of the target coding unit is selected from the other inter-frame coding modes.

[0018] Optionally, the selecting the inter-frame coding mode of the target coding unit from the other inter-frame coding modes includes:

[0019] Get the forward prediction rate-distortion cost and the backward prediction rate-distortion cost;

[0020] Obtain a predicted value of a bidirectional prediction rate-distortion cost according to the forward prediction rate-distortion cost and the backward prediction rate-distortion cost;

[0021] adjusting the predicted value according to a scaling factor;

[0022] If the adjusted prediction value is greater than or equal to the forward prediction rate-distortion cost, or the adjusted prediction value is greater than or equal to the backward prediction rate-distortion cost, bidirectional prediction processing on the target coding unit is skipped.

[0023] Optionally, also include:

[0024] If the adjusted prediction value is less than the forward prediction rate-distortion cost and less than the backward prediction rate-distortion cost, bidirectional prediction processing is performed on the target coding unit.

[0025] Optionally, obtaining the inter-coding mode correlation between the target neighbor coding unit and the target coding unit includes:

[0026] An inter-frame coding mode correlation between the neighbor coding unit and the target coding unit is determined according to the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit.

[0027] Optionally, the determining whether to terminate the inter-frame coding mode early according to the inter-frame merging mode score includes:

[0028] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score and the score threshold;

[0029] The method further comprises:

[0030] The score threshold is determined according to the encoding quantization parameter.

[0031] Optionally, also include:

[0032] Get the number of neighboring coding units of the target coding unit;

[0033] If the number is greater than the number threshold, it is determined whether to terminate the decision of the inter-frame coding mode in advance according to the score.

[0034] Optionally, also include:

[0035] If the target coding unit is not a coding unit at an edge of the image and the number is greater than the number threshold, whether to terminate the decision of the inter-frame coding mode in advance is determined according to the score.

[0036] The present application also provides a video encoding device, comprising:

[0037] A neighbor CU acquisition unit, configured to acquire a target neighbor coding unit whose inter-frame coding mode is a merge skip mode from neighbor coding units of the target coding unit;

[0038] A correlation obtaining unit, configured to obtain an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0039] a score determining unit, configured to determine a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0040] A first coding unit, configured to perform inter-frame coding in a merge mode or a skip mode on a target coding unit;

[0041] a judgment unit, configured to judge whether to terminate the decision of the inter-frame coding mode in advance according to the score;

[0042] The first ending unit is configured to terminate the inter-frame coding of the target coding unit in other inter-frame coding modes if the above judgment result is yes.

[0043] The present application also provides an electronic device, comprising:

[0044] processor and memory;

[0045] The memory is used to store a program for implementing the above-mentioned video encoding method. The device is powered on and runs the program of the method through the processor.

[0046] The present application also provides a video processing system, comprising:

[0047] According to the above video encoding device; and, video decoding device.

[0048] This application also provides a video live broadcast method, including:

[0049] For a target coding unit of a video frame included in the live video, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit;

[0050] Obtaining an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0051] Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0052] Perform inter-frame coding in merge mode or skip mode on the target coding unit;

[0053] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score;

[0054] If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

[0055] The present application also provides a video encoding method for video on demand, comprising:

[0056] For a target coding unit of a video frame included in the on-demand video, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit;

[0057] Obtaining an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0058] Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0059] Perform inter-frame coding in merge mode or skip mode on the target coding unit;

[0060] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score;

[0061] If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

[0062] The present application also provides a video encoding method for video calling, comprising:

[0063] For a target coding unit of a video frame included in the video call, obtaining a target neighboring coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighboring coding units of the target coding unit;

[0064] Obtaining an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0065] Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0066] Perform inter-frame coding in merge mode or skip mode on the target coding unit;

[0067] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score;

[0068] If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

[0069] The present application also provides a video encoding method for video conferencing, comprising:

[0070] For a target coding unit of a video frame included in the video conference, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit;

[0071] Obtaining an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0072] Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0073] Perform inter-frame coding in merge mode or skip mode on the target coding unit;

[0074] Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score;

[0075] If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

[0076] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned various methods.

[0077] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to perform the above methods.

[0078] Compared with the prior art, this application has the following advantages:

[0079] The video coding method provided in the embodiment of the present application utilizes the inter-coding mode information of the neighboring CUs of the target coding unit and adopts a "voting weighted scoring mechanism" of the neighboring CUs to make a decision on skipping the inter-coding mode (Inter_ME mode) of the target coding CU. By adopting this processing method of early termination of the inter-coding mode decision, the merge mode or skip mode can be quickly selected as the optimized inter-coding mode, and the time-consuming search process of inter-coding (Inter_ME) and other modes can be terminated early, which can effectively reduce the inter-coding complexity of large-size CUs, thereby improving the overall coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of a flow chart of an embodiment of a video encoding method provided by the present application;

[0081] Figure 2 A position relationship diagram of neighboring CUs of a current CU in an embodiment of the video encoding method provided by the present application;

[0082] Figure 3 A flow chart of early termination of inter-mode decision in an embodiment of a video coding method provided by the present application;

[0083] Figure 4 Different QP and different thresholds TH of the embodiment of the video encoding method provided by this application Select_MS The test results of precision and recall are as follows;

[0084] Figure 5 A three-dimensional distribution diagram of test samples at different QPs in an embodiment of the video encoding method provided in this application;

[0085] Figure 6 A diagram of a single-layer perceptron structure of an embodiment of the video encoding method provided by this application;

[0086] Figure 7 A three-dimensional distribution diagram of the model prediction results and actual test samples of an embodiment of the video encoding method provided by this application;

[0087] Figure 8The impact of different scaling factors α on the algorithm accuracy and recall rate of the video encoding method provided in this application;

[0088] Figure 9 The present application provides an overall flow chart of inter-mode decision in an embodiment of a video encoding method. DETAILED DESCRIPTION

[0089] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0090] This application provides a video encoding method and apparatus, a video encoding system, and an electronic device. In the following embodiments, various solutions are described in detail one by one.

[0091] Objects in adjacent images often have temporal correlation, and inter-frame coding uses the association between previous and next frames of video information for compression. Inter-frame coding modes can be divided into merge mode (Merge), skip mode (SKIP), affine merge mode (Affine Merge), ordinary inter-frame mode (Inter_ME) and affine mode (Affine). The inventors of this application discovered through experiments that the temporal complexity distribution of each inter-frame coding mode in the entire inter-frame mode decision process during the coding process has certain characteristics. Among them, merge mode or skip mode (Merge_SKIP mode) is the most commonly used coding mode in the inter-frame mode process of the coding unit (CU), and its temporal complexity accounts for 31.9% of the entire inter-frame mode decision process. Inter_ME and Affine modes together account for 48.4% of the temporal complexity of inter-frame mode decision. Based on the above implementation data, the inventors determined the optimization goal: by quickly selecting Merge_SKIP as the optimized inter-frame coding mode and terminating the time-consuming search process of Inter_ME and other modes in advance, the overall coding efficiency can be improved.

[0092] In order to quickly predict that the Merge_SKIP mode is the optimized mode for the target CU, the inventors conducted two analysis experiments, including: 1) spatial correlation analysis of inter-frame modes; 2) analysis of the correlation between all-zero blocks and mode decisions.

[0093] 1) Spatial correlation analysis of inter-frame patterns

[0094] To test the spatial correlation of the Merge_SKIP mode, the inventors conducted a test experiment using the random access RA (random_access.cfg) configuration file of the VVC (Versatile Video Coding, a multi-purpose video coding standard such as H.266) encoder to perform encoding experiments on six standard test sequences under four different encoding quantization parameters (QP).

[0095] Figure 2 The positional relationship between the target CU's five neighbor CUs and the parent CU and the target CU is shown. The five neighbor CUs include the left CU (L), the lower left CU (LB), the upper left CU (AL), the upper CU (A), and the upper right CU (AR) (hereinafter collectively referred to as neighbor CUs). Figure 2 The Parent CU in ( ) represents the CU corresponding to the previous depth d-1 of the target CU depth d.

[0096] When encoding the target CU, the neighboring CUs have basically been encoded, so their preferred encoding modes are already known. Therefore, the experiment makes probability statistics on the six neighboring CUs of the target CU, and counts the probability of the neighboring CUs being encoded during the encoding process. i When Merge_SKIP is selected as the preferred mode, the target CU's optimal mode is also the probability Pi of Merge_SKIP, and the probability Pi is used as the neighbor CU i Merge_SKIP mode correlation metric R with the target CU i , calculated as shown in formula (1):

[0097]

[0098] Formula (1) is the conditional probability, i represents the subscript of the neighbor CU, R i ∈{R p , R L , R LB , R A , R AL , R AR}, corresponding to the Merge_SKIP mode correlation of the parent CU, left CU, lower left CU, upper CU, upper left CU, upper right CU and target CU respectively. Event A indicates that the target CU optimization mode is Merge_SKIP mode, and event B i Indicates that the i-th neighbor CU selects Merge_SKIP mode as the optimization mode, event AB i Represents event A and event B i Occurrence at the same time, N(*) represents the number of times event* occurs during the encoding process.

[0099] It can be seen that R i The larger the value, the more neighboring CUsi The greater the correlation with the target CU selecting the Merge_SKIP mode as the preferred mode. The experimental results show that the average Ri values ​​of different neighbor CUs are all over 70%, indicating that the neighbor CUs and the target CU have a high correlation in selecting the Merge_SKIP mode as the preferred inter-frame mode.

[0100] 2) Analyze the correlation between all-zero blocks and mode decisions

[0101] Because all-zero blocks are crucial for mode decisions, the inventors conducted a test experiment to explore their impact. The experiment employed the configuration shown in Table 1, using the VVC encoder's RA (random_access.cfg) configuration file, and encoded six standard test sequences at four different QPs.

[0102] An all-zero block is a transform unit (TU) where all coefficients are quantized to 0 after transformation and quantization. This is called an all-zero block (AZB). If a CU is encoded using a certain mode and an all-zero block is obtained after transformation and quantization of its transform unit (TU), the CU is said to have generated an AZB using this mode.

[0103] In order to quickly select the Merge_SKIP mode and skip the Inter_ME mode and other modes that are time-consuming and have a very low probability of being selected by the CU in advance, the inventors calculated the probability P of the encoder finally selecting Merge_SKIP as the optimization mode after the frame rate distortion mode decision when the CU is encoded in the Merge_SKIP mode and AZB is generated during the statistical coding process. AZB_MS , the calculation formula is shown in (2):

[0104]

[0105] Formula (2) is the conditional probability. Event A indicates that the optimal mode of the target CU is Merge_SKIP. Event B indicates that AZB can be generated after Merge_SKIP encoding of the CU. Event AB indicates that event A and event B occur at the same time. N(*) indicates the number of times event * occurs during the encoding process.

[0106] It can be seen that P AZB_MS The higher the value, the higher the probability that the target CU will choose Merge_SKIP as the optimization mode if Merge_SKIP is used to encode the target CU and AZB is generated. The experimental results show that, whether from different QPs or from the average results of different sequences, P AZB_MSThe values ​​of are all above 90%, indicating that for the CU that adopts Merge_SKIP encoding and generates AZB, it has a very high probability of selecting Merge_SKIP mode as the preferred encoding mode.

[0107] Based on the above two experimental analyses, the inventors proposed a technical concept for quickly predicting that the Merge_SKIP mode is the optimization mode for the target CU: utilizing the inter-frame mode information of the neighboring CUs of the target coding unit, adopting a "voting weighted scoring mechanism" to make decisions on skipping the inter-frame mode (Inter_ME mode) of the target coding CU, and using the feature of whether there are all-zero blocks in the result of encoding in the merge mode or skip mode (Merge_SKIP) to further optimize the decision effect of skipping the inter-frame mode (Inter_ME mode).

[0108] First embodiment

[0109] Please refer to Figure 1 , which is a flow chart of an embodiment of the video encoding method of the present application. In this embodiment, the method may include the following steps:

[0110] Step S101: obtaining a target neighboring coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighboring coding units of a target coding unit.

[0111] Video coding is performed on a block basis. A frame of video is divided into different blocks, and each block is then encoded separately. Each block is called a coding unit (CU), and coding units are divided into layers.

[0112] The method provided in the embodiment of the present application adopts a "voting weighted scoring mechanism". In order to predict whether the optimized coding mode of the target CU is merge mode or skip mode (Merge_SKIP mode), it is necessary to vote through its neighboring CUs. If the optimized mode of a neighboring CU is merge mode or skip mode (Merge_SKIP mode), its voting value is 1, otherwise it is 0. Therefore, it is necessary to obtain the target neighboring coding units whose inter-frame coding mode is merge mode or skip mode from the neighboring coding units of the target coding unit.

[0113] The neighbor coding units of the target coding unit may include a parent CU, a left CU (L), a lower left CU (LB), an upper left CU (AL), an upper CU (A), an upper right CU (AR), etc. Figure 2 As shown, the target coding unit (Current CU, also called current CU) with a depth of d is a child coding unit of a parent CU with a depth of d-1, and the parent CU is divided into four sub-units.

[0114] When encoding the target CU, the neighboring CUs have already been encoded, so their preferred coding modes can be obtained. In specific implementations, based on the preferred coding mode, the neighboring CUs with inter-frame coding modes of merge mode or skip mode can be selected as the target neighboring CUs.

[0115] Step S103: Obtaining the inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit.

[0116] The correlation refers to the influence of neighboring coding units on the target coding unit's selection of either merge mode or skip mode. The first part of the experiment above verifies that different neighboring CUs have different correlations (influences) on the target CU's selection of Merge_SKIP mode.

[0117] In one example, the correlation is related to the quantization parameter QP and the neighbor CU, and the method may further include the following steps: determining the inter-frame coding mode correlation between the neighbor coding unit and the target coding unit based on the encoding quantization parameter and the position of the neighbor coding unit relative to the target coding unit.

[0118] In a specific implementation, the correlation corresponding to a certain quantization parameter QP and a neighboring CU at a certain position can be determined in the following manner: i When Merge_SKIP is selected as the preferred mode, the probability P that the optimal mode of the target CU is also Merge_SKIP is i , the probability P i As a neighbor CU i Merge_SKIP mode correlation metric R with the target CU i In specific implementation, the above formula (1) can be used to calculate R i , I will not go into details here.

[0119] The following table shows the correlation R between the neighbor CU and the current CU under different QP i List:

[0120]

[0121] The table shows the R values ​​of the six neighbor CUs under different QPs. i Overall, the average R i The values ​​are all over 70%, indicating that the neighbor CU and the current CU have a high correlation in selecting the Merge_SKIP mode as the optimal inter-frame mode.

[0122] Step S105: Determine the inter-frame merge mode score of the target coding unit according to the correlation.

[0123] This step determines the score of the early termination inter-frame coding mode decision of the target coding unit according to the inter-frame coding mode correlation between each neighbor coding unit and the target coding unit.

[0124] In specific implementation, the voting values ​​of different neighbor CUs can be weighted by correlation. The weighted voting score of the neighbor CU can be calculated using formula (3).

[0125]

[0126] In formula (3), N represents the number of neighboring CUs of the target CU. i Represents the Merge_SKIP mode correlation (ie, the correlation) between the i-th neighbor CU and the target CU. i Indicates the voting value of the i-th neighbor CU. If the i-th neighbor CU optimization mode is Merge_SKIP mode, then M i =1, otherwise M i =0.

[0127] Step S107: Determine whether to terminate the decision of the inter-frame coding mode early according to the score.

[0128] This step determines whether to terminate the inter-frame coding mode decision early based on the weighted scores of all votes. In specific implementations, the decision to terminate the inter-frame coding mode decision early can be based on the score and score range, or based on the score and score threshold.

[0129] In one example, the weighted score of all votes exceeds a preset adjustable score threshold TH. Select_MS To determine whether the target CU selects Merge_SKIP as the preferred mode. Figure 3 As shown, if the score is greater than the score threshold, the decision to terminate the inter-frame coding mode early is determined.

[0130] In one example, the method may further include the following steps: obtaining the number of neighboring coding units of the target coding unit; if the number is greater than a threshold, determining whether to prematurely terminate the inter-coding mode decision based on the score and the threshold. For example, if the threshold is set to 5, the "voting weighted scoring mechanism" will only be used to predict the Merge_SKIP mode of the target CU if at least 5 of the target CU's 6 neighboring CUs exist. This method will not be used for CUs with fewer than 5 neighbors. This approach can effectively improve prediction accuracy and ensure the scientific nature of the experiment.

[0131] In one example, the method may further include the following step: if the target coding unit is not a coding unit at an image edge and the number is greater than a number threshold, determining whether to prematurely terminate the inter-frame coding mode decision based on the score and the score threshold. This processing method does not apply to image edge CUs or other CUs with less than five neighbors.

[0132] In an example, the method may further include the following step: determining the score threshold according to an encoding quantization parameter.

[0133] Adjustable score threshold TH Select_MS As an important parameter for Merge_SKIP mode prediction, it needs to be set reasonably and scientifically. A too large threshold will cause too many CUs to not quickly select Merge_SKIP mode as the optimized coding mode, so that the optimal coding speed improvement brought by early termination of Inter_ME and other modes cannot be achieved; while a too small threshold will cause too many CUs to select Merge_SKIP mode as the optimized mode in advance, so that the rate distortion cost (RD_Cost) is skipped. m ) other encoding modes that are better, resulting in a greater loss of compression performance for the encoder.

[0134] To address this situation, the inventors set up an experiment to find the optimized score threshold for different QPs. The experiment used the configuration in Table 1, set the test threshold group {0.5, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95} at different QPs, and statistically analyzed the accuracy and recall of the algorithm under different threshold settings, as shown in Table 1. Figure 4 shown.

[0135] In order to achieve a higher recall rate while ensuring the accuracy of the algorithm, different TH can be selected for different Qp {22, 27, 32, 37} Select_MS , TH Select_MS ={0.9, 0.85, 0.85, 0.85}, which basically guarantees acceptable encoder compression performance loss and maximizes the acceleration of the encoder's inter-frame mode decision process.

[0136] Step S109: performing inter-frame coding in a merge mode or a skip mode on the target coding unit.

[0137] In specific implementation, a relatively mature existing technology can be used to determine whether to perform inter-frame coding in merge mode or skip mode on the target coding unit. For example, a method of comparing coding costs can be used to select the mode with the lower coding cost between merge mode and skip mode.

[0138] Step S111: If the above determination result is yes, then terminate the inter-frame coding of the target coding unit in other inter-frame coding modes.

[0139] In an example, the method may further include the following step: if the inter-frame encoding result of the merge mode or skip mode performed on the target coding unit is an all-zero block, terminating the inter-frame encoding of the target coding unit in other inter-frame encoding modes.

[0140] The second part of the experiment above verified that a CU using Merge_SKIP encoding and generating AZB (All Zero Block) has a high probability of selecting Merge_SKIP as the preferred encoding mode. Therefore, AZB detection can be performed after Merge_SKIP encoding. AZBs are blocks with all zero coefficients after quantization. Early AZB identification allows for quick block classification, reducing D and R calculations.

[0141] In one example, the method may further include the following steps: if the judgment result of step S109 is no and the inter-frame coding result of the merge mode or skip mode for the target coding unit is not an all-zero block, then the inter-frame coding mode of the target coding unit is selected from the other inter-frame coding modes.

[0142] like Figure 3 As shown, first initialize the early termination identifier to 0 and search to establish a neighbor CU list. When the number of neighbor CUs is greater than or equal to 5, calculate the average score according to formula (3) and add the average score to the score threshold TH. Select_MS Compare, if it is greater than the threshold TH Select_MS Then set the termination identifier to 1; then use the Merge_SKIP mode for inter-frame coding, and determine whether the result after transform quantization is an all-zero block. If so, set the termination identifier to 1; finally, determine whether the early termination identifier is 1 after the above process. If it is 1, skip the encoding process of other inter-frame modes and select Merge_SKIP as the preferred inter-frame coding mode.

[0143] In addition, in the process of implementing the present invention, the inventors also focused on improving the overall coding efficiency by fast skipping the inter-frame motion estimation bidirectional prediction process. The inventors studied the existing technology of fast skipping the inter-frame motion estimation bidirectional prediction process and found that a typical fast skip bidirectional prediction method adopts the following process: based on the all-zero block (AZB) and motion estimation (ME) information of the inter-frame 2N×2N mode, early merge mode decision is implemented for the root CU (i.e., 64×64 CU); considering the mode selection correlation between the root CU and the sub-CU, early merge mode decision is implemented for the sub-CU (i.e., 32×32, 16×16 and 8×8 CU).

[0144] However, the inventors found that this solution is only applicable to the inter-frame mode fast scene of HEVC, and it is not fully applicable to complex encoders such as VVC. Taking the VVC encoder as an example, this type of video encoder has a complex QTMTT division mode, a fine intra-frame angle prediction mode, and an affine motion prediction mode that adapts to complex motions. When doing the Inter_ME mode, the following directional prediction mode is adopted: the encoder first performs a motion estimation search (forward prediction) on the reference frame in the forward list (LIST_0), and then performs a motion estimation search on the reference frame in the backward list (LIST_1), and then performs a bidirectional motion estimation search process (bidirectional prediction) on the two lists (LIST_0 and LIST_1), and finally selects the prediction direction with the smallest rate-distortion cost (SAD_Cost is used here to represent the rate-distortion cost) as the optimized prediction direction of the Inter_ME mode.

[0145] Through experiments, the inventors found that only 16% of CUs used bidirectional prediction, but all CUs did. Therefore, the inventors identified another optimization goal: to improve overall coding efficiency by quickly skipping the inter-frame motion estimation bidirectional prediction process. The "Fast Skip Motion Estimation Bidirectional Prediction Algorithm" aims to leverage the correlation between unidirectional prediction (forward and backward prediction are collectively referred to as unidirectional prediction) and bidirectional prediction to predict in advance whether the target CU needs to undergo bidirectional prediction motion estimation.

[0146] In order to quickly skip the inter-frame motion estimation and bidirectional prediction process, the inventors analyzed the correlation between unidirectional prediction and bidirectional prediction. In this embodiment, to explore the correlation between unidirectional prediction and bidirectional prediction, the inventors conducted a test experiment to analyze the relationship between the rate-distortion cost RD_Cost of unidirectional prediction and bidirectional prediction.

[0147] The experiment used 18 standard video sequences for testing, including five categories of videos with different resolutions, A to E. To generate test samples, the experiment used the VVC video encoder with the RA (random_access.cfg) fast gear configuration. Each of the 18 sequences was encoded at four different QPs {22, 27, 32, 37}. The forward prediction rate-distortion cost list0_cost, backward prediction rate-distortion cost list1_cost, and bidirectional prediction rate-distortion cost Bi_cost of the corresponding CU during the encoding process were extracted, and [list0_cost, list1_cost, Bi_cost] was used as a test sample.

[0148] For different QP {22, 27, 32, 37}, a total of {2010132, 1235560, 687445, 365477} test samples are extracted, and the list0_cost, list1_cost, and Bi_cost values ​​in the samples correspond to the X, Y, and Z axes of the three-dimensional coordinates and are plotted in three-dimensional space. Figure 5 As shown in the figure, from the three-dimensional distribution diagram, the sample points in the figure can almost be fitted by a plane, that is, list0_cost, list1_cost and Bi_cost should have a strong linear relationship.

[0149] The inventors found through experiments that the predicted value of the bidirectional prediction rate-distortion cost costBi pred There is a strong linear relationship between the unidirectional prediction rate-distortion cost (costList0, costList1). Based on this conclusion, the inventors proposed to use the correlation between unidirectional prediction (forward and backward prediction are collectively referred to as unidirectional prediction) and bidirectional prediction to predict in advance whether the target CU needs to perform bidirectional prediction motion estimation, thus achieving "fast skip motion estimation bidirectional prediction."

[0150] In an example, selecting the inter-frame coding mode of the target coding unit from the other inter-frame coding modes may include the following steps:

[0151] Step S201: Obtain forward prediction rate-distortion cost and backward prediction rate-distortion cost.

[0152] Step S203: Obtain a predicted value of a bidirectional prediction rate-distortion cost according to the forward prediction rate-distortion cost and the backward prediction rate-distortion cost.

[0153] In a specific implementation, a predicted value of the bidirectional prediction rate-distortion cost can be obtained based on the forward prediction rate-distortion cost and the backward prediction rate-distortion cost through a machine learning model. The machine learning model can be a linear model, and the structure of the linear model includes but is not limited to a single-layer perceptron.

[0154] The core of the Inter_ME mode optimization prediction direction decision process is to try different prediction directions and select the direction with the minimum rate-distortion cost as the optimized prediction direction. Through the above analysis, the rate-distortion cost costBi of the bidirectional prediction is the same as the rate-distortion cost of the two unidirectional predictions (the forward prediction rate-distortion cost costList o It has a strong linear relationship with the backward prediction rate-distortion cost costList1), so a linear function can be used to predict the rate-distortion cost costBi of the bidirectional prediction. pred , the linear function can be expressed by formula (4):

[0155] costBi pred=w0*costList0+w1*costList1+bias (4)

[0156] In order to find the optimized linear function, that is, to determine the most appropriate w0, w1 and bias, this embodiment trains a single-layer perceptron for fitting a linear model. The perceptron structure is as follows: Figure 6 shown.

[0157] To train the perceptron, the inventors experimentally used 111 YUV test sequences as a database for establishing a dataset. The experiments divided the 111 sequences into three non-overlapping databases: a training set database, a validation set database, and a test set database. The training set database contained 83 sequences for constructing the training set, the validation set database contained 10 sequences for constructing the validation set, and the test set contained 18 standard test sequences for constructing the test set (this test set had already been constructed in the aforementioned test experiment). The aforementioned method for generating test samples was then used on the VVC encoder to generate training and validation samples. Finally, for different QPs {22, 27, 32, 37}, the resulting training set contained {5,615,302, 3,483,382, 2,050,603, 1,136,089} training samples, and the validation set contained {878,953, 528,828, 289,946, 158,824} validation samples.

[0158] Since the data gradient distribution is very scattered and the value is very large, in order to make the model converge quickly and effectively, the data normalization method commonly used in data processing can be used, that is, the standard score (z-score, z score) method, so that the processed data conforms to the standard normal distribution (distribution with a mean of 0 and a variance of 1). This standardization method can be expressed by formula (5):

[0159]

[0160] In formula (5), x * Represents the data after standardization, x represents the data before standardization, μ represents the mean of multiple data samples, and σ represents the standard deviation of multiple data samples. The multiple data samples can be multiple forward prediction rate-distortion costs, and the mean and standard deviation of the multiple forward prediction rate-distortion costs can be obtained through formula (5); the multiple data samples can also be multiple backward prediction rate-distortion costs, and the mean and standard deviation of the multiple backward prediction rate-distortion costs can be obtained through formula (5). The multiple data samples can also include multiple forward prediction rate-distortion costs and multiple backward prediction rate-distortion costs, and the mean and standard deviation of the multiple forward prediction rate-distortion costs and multiple backward prediction rate-distortion costs can be obtained through formula (5).

[0161] Substituting formula (5) into formula (4) yields formula (6):

[0162]

[0163] That is: predict the standardized bidirectional prediction rate distortion cost (stdCostBi by the two standardized unidirectional prediction data pred ), in this embodiment, the mean and standard of all training, validation and test samples are calculated, and finally μ=8185,σ=23011 are determined.

[0164] To adapt to different QPs {22, 27, 32, 37}, this embodiment uses the training and validation sets corresponding to different QPs to train the parameters (w0, w1, and bias) of the single-layer perceptron, and uses the mean square error (MSE) (calculated as shown in Formula 7) as the loss function of this model.

[0165] costBi pred =w0*costList0+w1*costList1+B (7)

[0166] In formula (7), B = bias*σ+(1-w0-w1)*μ, where bias, w0, and w1 are all trained model parameters, μ represents the mean of multiple data samples, which is set to 8185 in the experiment, and σ represents the standard deviation of multiple data samples, which is set to 23011 in the experiment.

[0167] The model predicts the result stdCostBi pred It is the result of standardization, but in the actual coding process, it needs to be de-standardized so that the relevant methods can be studied in the original numerical space. After de-standardization, the predicted value costBi can be derived pred The linear relationship with the unidirectional prediction rate-distortion cost (costList0, costList1) is shown in formula (7), where w0, w1 and bias are parameters obtained by model training, and the parameter B = bias*σ+(1-w0-w1)*μ. Finally, the parameters of the linear model under different QPs are determined accordingly.

[0168] Figure 7 The results predicted by the linear model parameters at different QPs on the test set and the three-dimensional space distribution of the actual test sample points are shown. The results show that the plane formed by the linear model in three-dimensional space can basically fit the majority of the sample points, and performs better at larger QPs such as QP32 and QP37.

[0169] Step S205: adjusting the predicted value according to the scaling factor.

[0170] The predicted value of the bidirectional prediction rate-distortion cost costBi predIt has a strong linear relationship with the unidirectional prediction rate distortion cost (costList0, costList1). Figure 7 As shown in Figure 2, the linear model trained with a single-layer perceptron for different QPs can fit the distribution of test points relatively well in three-dimensional space. However, Figure 7 The test sample points in are not completely on the model plane, that is, the actual bidirectional prediction rate-distortion cost and the unidirectional prediction rate-distortion cost are not completely linear. Therefore, the method proposed in the embodiment of the present application uses a scaling factor α to adjust the bidirectional prediction rate-distortion cost costBi obtained by model prediction. pred The scaling factor α plays an important role in ensuring the accuracy of the method. Reasonable setting of α can ensure the accuracy of the method and skip unnecessary bidirectional prediction processes as much as possible in advance.

[0171] In an example, the method may further include the following step: determining a scaling factor according to the encoding quantization parameter.

[0172] In order to find a suitable α, this embodiment conducts experiments on the six standard sequences provided by VVC under four different QPs. The experiments test the accuracy and recall of the method under different α. The test results are shown in Figure 2. Figure 8 To balance the compression performance of the encoder and the effectiveness of the method, the method adopts α values ​​of {0.80, 0.85, 0.90, 0.95} for different QPs {22, 27, 32, 37}, ensuring that the accuracy (precision) of the calculation is between 93% and 95% while the recall (recall) reaches 50% to 80%.

[0173] Step S207: If the adjusted prediction value is greater than or equal to the forward prediction rate-distortion cost, or the adjusted prediction value is greater than or equal to the backward prediction rate-distortion cost, skipping bidirectional prediction processing on the target coding unit.

[0174] In specific implementation, the inter-frame coding mode of the target coding unit selected from the other inter-frame coding modes may also include the following steps: if the adjusted prediction value is less than the forward prediction rate-distortion cost and less than the backward prediction rate-distortion cost, the target coding unit is subjected to bidirectional prediction processing.

[0175] The method provided in the embodiment of the present application, through the above steps S201 to 207, uses the perceptron in deep learning to fit the linear relationship between bidirectional prediction and unidirectional prediction in inter-frame prediction, and quickly skips the bidirectional prediction coding process of the inter-frame mode through this linear relationship. This processing method is not only applicable to relatively simple video coding scenarios (such as the HETV coding standard), but also to complex video coding scenarios with complex QTMTT partitioning modes, fine intra-frame angle prediction modes, and affine motion prediction modes that adapt to complex motion (such as the VVC coding standard).

[0176] like Figure 9 As shown, in a specific embodiment, the processing process of the method is as follows. First, establish the available neighbor CU list (NeighborCULists) of the target CU, and set the early termination decision parameter of the target CU to no, such as initializing isEarlyTerminate=faIse; then, determine whether the number of available neighbor CUs is greater than or equal to the number threshold (such as the number threshold = 5), and whether the score is greater than the score threshold (TH select_MS ), if the judgment result is yes, the parameter for whether to terminate the decision in advance is set to yes; next, inter-frame coding of the target CU in merge mode or skip mode is performed; next, it is detected whether the inter-frame coding result of the target CU in merge mode or skip mode is an all-zero block, if the judgment result is yes, the parameter for whether to terminate the decision in advance is set to yes; thereafter, it is judged whether the parameter value is yes, if the judgment result is yes, the inter-frame coding of other inter-frame coding modes for the target coding unit is terminated; if the parameter value is no, the processing flow of selecting the inter-frame coding mode of the target coding unit from other inter-frame coding modes is entered.

[0177] The process of selecting the inter-frame coding mode of the target coding unit from other inter-frame coding modes is as follows. First, forward prediction and backward prediction are performed to obtain the forward prediction rate-distortion cost costList0 and the backward prediction rate-distortion cost costList1; then, through the linear model (costBi pred =w0*costList0+w1*costList1+B), according to the forward prediction rate distortion cost and the backward prediction rate distortion cost, obtain the predicted value costBi of the bidirectional prediction rate distortion cost pred ; According to the scaling factor α, adjust the predicted value costBi pred; If the adjusted prediction value is greater than or equal to the forward prediction rate-distortion cost costList0, or the adjusted prediction value is greater than or equal to the backward prediction rate-distortion cost costList1, then skip the bidirectional prediction processing of the target coding unit; if the adjusted prediction value is less than costList0 and less than costList1, then perform bidirectional prediction processing on the target coding unit; finally, the target coding unit can be encoded in affine transformation mode (Affine mode) and intra-frame mode (Intra mode).

[0178] As can be seen from the above embodiments, the video coding method provided by the embodiments of the present application utilizes the inter-frame coding mode information of the neighboring CUs of the target coding unit and adopts the neighboring CU "voting weighted scoring mechanism" to make the decision on skipping the inter-frame coding mode (Inter_ME mode) of the target coding CU. With this processing method, the merge mode or skip mode can be quickly selected as the optimized inter-frame coding mode, and the time-consuming search process of inter-frame coding (Inter_ME) and other modes can be terminated in advance, which can effectively reduce the inter-frame coding complexity of large-size CUs, thereby improving the overall coding efficiency.

[0179] Second embodiment

[0180] In the above embodiment, a video encoding method is provided. Accordingly, this application also provides a video encoding device. This device corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0181] A video encoding device provided in the present application includes: a neighbor CU acquisition unit, a correlation acquisition unit, a score determination unit, a first encoding unit, a first judgment unit, and a first ending unit.

[0182] A neighbor CU acquisition unit, configured to acquire a target neighbor coding unit whose inter-frame coding mode is a merge skip mode from neighbor coding units of the target coding unit;

[0183] A correlation obtaining unit, configured to obtain an inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit;

[0184] a score determining unit, configured to determine a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation;

[0185] A first coding unit, configured to perform inter-frame coding in a merge mode or a skip mode on a target coding unit;

[0186] A first judgment unit, configured to judge whether to terminate the decision of the inter-frame coding mode in advance according to the score;

[0187] The first ending unit is configured to terminate the inter-frame coding of the target coding unit in other inter-frame coding modes if the above judgment result is yes.

[0188] In one example, the apparatus may further include: a second ending unit for terminating inter-frame encoding of the target coding unit in other inter-frame encoding modes if the inter-frame encoding result of the merge mode or skip mode on the target coding unit is an all-zero block.

[0189] In one example, the device may also include: a second encoding unit, used to select the inter-frame coding mode of the target coding unit from the other inter-frame coding modes if the above judgment result is no and the inter-frame coding result of the merge mode or skip mode of the target coding unit is not an all-zero block.

[0190] In one example, the second coding unit is specifically used to obtain a forward prediction rate-distortion cost and a backward prediction rate-distortion cost; through a linear model, based on the forward prediction rate-distortion cost and the backward prediction rate-distortion cost, a predicted value of the bidirectional prediction rate-distortion cost is obtained; according to a scaling factor, the predicted value is adjusted; if the adjusted predicted value is greater than or equal to the forward prediction rate-distortion cost, or the adjusted predicted value is greater than or equal to the backward prediction rate-distortion cost, then bidirectional prediction processing of the target coding unit is skipped.

[0191] In an example, the second coding unit is further configured to perform bidirectional prediction processing on the target coding unit if the adjusted prediction value is less than the forward prediction rate-distortion cost and less than the backward prediction rate-distortion cost.

[0192] In an example, the correlation obtaining unit is specifically configured to determine the inter-frame coding mode correlation between the neighbor coding unit and the target coding unit according to the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit.

[0193] In one example, the first judgment unit is specifically used to judge whether to terminate the decision of the inter-frame coding mode early based on the inter-frame merge mode score and the score threshold; the device may also include: a threshold determination unit, used to determine the score threshold based on the coding quantization parameter.

[0194] In one example, the device may also include: a neighbor CU number acquisition unit, used to obtain the number of neighbor coding units of the target coding unit; and a second judgment unit, used to judge whether to terminate the decision of the inter-frame coding mode in advance based on the score if the number is greater than the number threshold.

[0195] In one example, the first judgment unit is specifically used to determine whether to terminate the decision of the inter-frame coding mode early based on the score if the target coding unit is not a coding unit at the edge of the image and the number is greater than the number threshold.

[0196] Third embodiment

[0197] In the above-mentioned embodiment, a video encoding method is provided. Accordingly, the present application also provides an electronic device. Since the device embodiment is substantially similar to the method embodiment, the description is relatively brief. For relevant details, please refer to the description of the method embodiment. The device embodiment described below is merely illustrative.

[0198] An electronic device according to this embodiment includes: a processor and a memory; the memory is used to store a program for implementing the above-mentioned video encoding method; the device is powered on and runs the program of the method through the processor.

[0199] Fourth embodiment

[0200] In the above embodiment, a video encoding method is provided. Accordingly, this application also provides a video encoding system. This system corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0201] This application provides a video encoding system comprising any of the aforementioned video encoding devices and a video decoding device. The video decoding device is state-of-the-art and can utilize relatively mature video decoding technology, which will not be described in detail here. The video encoding system can be used in live broadcasting, on-demand video, video conferencing, and other scenarios.

[0202] Fifth embodiment

[0203] In the above embodiment, a video encoding method is provided. Accordingly, this application also provides a video live broadcast method for a live broadcast scenario. This method corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0204] A video live broadcast method provided in this application includes the following steps:

[0205] Step 301: For a target coding unit of a video frame included in a live video, obtain a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit.

[0206] The live video also includes multiple video frames. The processing method of the encoding unit of the video frame is as described in Example 1 and will not be repeated here.

[0207] Step 303: Obtain the inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit.

[0208] Step 305: Determine a score for early termination of inter-frame coding mode decision of the target coding unit according to the correlation.

[0209] Step 307: Perform inter-frame coding in merge mode or skip mode on the target coding unit.

[0210] Step 309: Determine whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score.

[0211] Step 311: If the above determination result is yes, then terminate the inter-frame coding of the target coding unit in other inter-frame coding modes.

[0212] This processing method can quickly select the merge mode or skip mode as the optimized inter-frame coding mode, and terminate the time-consuming search process of inter-frame coding and other modes in advance, which can effectively reduce the inter-frame coding complexity of large-size CUs, thereby improving the smoothness of live video at higher picture quality.

[0213] Sixth embodiment

[0214] In the above embodiment, a video encoding method is provided. Correspondingly, this application also provides a video encoding method for video on demand. This method corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0215] The present application provides a video encoding method for video on demand, comprising the following steps:

[0216] Step 401: For a target coding unit of a video frame included in a video on demand, obtain a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit.

[0217] The on-demand video also includes multiple video frames. The processing method of the encoding unit of the video frame is as described in the first embodiment and will not be repeated here.

[0218] Step 403: Obtain the inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit.

[0219] Step 405: Determine a score for early termination of inter-frame coding mode decision of the target coding unit according to the correlation.

[0220] Step 407: Perform inter-frame coding in merge mode or skip mode on the target coding unit.

[0221] Step 409: Determine whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score.

[0222] Step 411: If the above determination result is yes, then terminate the inter-frame coding of the target coding unit in other inter-frame coding modes.

[0223] This processing method can quickly select the merge mode or skip mode as the optimized inter-frame coding mode, and terminate the time-consuming search process of inter-frame coding and other modes in advance, which can effectively reduce the inter-frame coding complexity of large-size CUs, thereby improving the smoothness of on-demand video at higher picture quality.

[0224] Seventh embodiment

[0225] In the above embodiment, a video encoding method is provided. Correspondingly, this application also provides a video encoding method for video calling. This method corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0226] The present application provides a video encoding method for video calling, comprising the following steps:

[0227] Step 501: For a target coding unit of a video frame included in a video call, obtain a target neighboring coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighboring coding units of the target coding unit.

[0228] The call video also includes multiple video frames. The processing method of the encoding unit of the video frame is as described in Example 1 and will not be repeated here.

[0229] Step 503: Obtain the inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit.

[0230] Step 505: Determine a score for early termination of inter-frame coding mode decision of the target coding unit according to the correlation.

[0231] Step 507: Perform inter-frame coding in merge mode or skip mode on the target coding unit.

[0232] Step 509: Determine whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score.

[0233] Step 511: If the above determination result is yes, then terminate the inter-frame coding of the target coding unit in other inter-frame coding modes.

[0234] This processing method can quickly select merge mode or skip mode as the optimized inter-frame coding mode, and terminate the time-consuming search process of inter-frame coding and other modes in advance, which can effectively reduce the inter-frame coding complexity of large-size CUs, thereby improving the smoothness of video call pictures at higher picture quality.

[0235] Eighth embodiment

[0236] In the above embodiment, a video encoding method is provided. Accordingly, this application also provides a video encoding method for video conferencing. This method corresponds to the embodiment of the above method. The parts of this embodiment that are identical to the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.

[0237] The present application provides a video encoding method for video conferencing, comprising the following steps:

[0238] Step 601: For a target coding unit of a video frame included in a video conference, obtain a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit.

[0239] The conference video also includes multiple video frames. The processing method of the encoding unit of the video frame is as described in the first embodiment and will not be repeated here.

[0240] Step 603: Obtain the inter-frame coding mode correlation between the target neighbor coding unit and the target coding unit.

[0241] Step 605: Determine a score for early termination of inter-frame coding mode decision of the target coding unit according to the correlation.

[0242] Step 607: Perform inter-frame coding in merge mode or skip mode on the target coding unit.

[0243] Step 609: Determine whether to terminate the inter-frame coding mode decision early based on the inter-frame merging mode score.

[0244] Step 611: If the above determination result is yes, then terminate the inter-frame coding of the target coding unit in other inter-frame coding modes.

[0245] This processing method can quickly select merge mode or skip mode as the optimized inter-frame coding mode, and terminate the time-consuming search process of inter-frame coding and other modes in advance, which can effectively reduce the inter-frame coding complexity of large-size CUs, thereby improving the smoothness of video call pictures at higher picture quality.

[0246] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0247] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0248] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0249] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0250] 2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A video encoding method, characterized in that: include: Obtain a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from the neighbor coding units of the target coding unit; Obtaining, based on the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit, an inter-coding mode correlation between the target neighbor coding unit and the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either a merge mode or a skip mode; Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; Perform inter-frame coding in merge mode or skip mode on the target coding unit; Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score; If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

2. The method according to claim 1, characterized in that Also includes: If the inter-frame coding result of the merge mode or skip mode performed on the target coding unit is an all-zero block, the inter-frame coding of other inter-frame coding modes on the target coding unit is terminated.

3. The method according to claim 1, characterized in that Also includes: If the above judgment result is no and the inter-frame coding result of the merge mode or skip mode on the target coding unit is not an all-zero block, the inter-frame coding mode of the target coding unit is selected from the other inter-frame coding modes.

4. The method according to claim 3, characterized in that The selecting the inter-frame coding mode of the target coding unit from the other inter-frame coding modes includes: Get the forward prediction rate-distortion cost and the backward prediction rate-distortion cost; Obtain a predicted value of a bidirectional prediction rate-distortion cost according to the forward prediction rate-distortion cost and the backward prediction rate-distortion cost; adjusting the predicted value according to a scaling factor; If the adjusted prediction value is greater than or equal to the forward prediction rate-distortion cost, or the adjusted prediction value is greater than or equal to the backward prediction rate-distortion cost, bidirectional prediction processing on the target coding unit is skipped.

5. The method according to claim 4, characterized in that Also includes: If the adjusted prediction value is less than the forward prediction rate-distortion cost and less than the backward prediction rate-distortion cost, bidirectional prediction processing is performed on the target coding unit.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: Get the number of neighboring coding units of the target coding unit; If the number is greater than the number threshold, it is determined whether to terminate the decision of the inter-frame coding mode in advance according to the score.

7. The method according to claim 6, characterized in that Also includes: If the target coding unit is not a coding unit at an edge of the image and the number is greater than the number threshold, whether to terminate the decision of the inter-frame coding mode in advance is determined according to the score.

8. A video encoding device, characterized in that: include: A neighbor CU acquisition unit, configured to acquire a target neighbor coding unit whose inter-frame coding mode is a merge skip mode from neighbor coding units of the target coding unit; a correlation obtaining unit, configured to obtain an inter-coding mode correlation between the target neighbor coding unit and the target coding unit based on a coding quantization parameter and a position of the target neighbor coding unit relative to the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either the merge mode or the skip mode; a score determining unit, configured to determine a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; A first coding unit, configured to perform inter-frame coding in a merge mode or a skip mode on a target coding unit; A first judgment unit, configured to judge whether to terminate the decision of the inter-frame coding mode in advance according to the score; The first ending unit is configured to terminate the inter-frame coding of the target coding unit in other inter-frame coding modes if the above judgment result is yes.

9. An electronic device, characterized in that: include: processor and memory; A memory for storing a program for implementing the video encoding method according to any one of claims 1 to 7, wherein the device is powered on and runs the program of the method through the processor.

10. A video processing system, characterized in that: include: The video encoding device according to claim 8; and the video decoding device.

11. A video encoding method for live video broadcasting, characterized in that: include: For a target coding unit of a video frame included in the live video, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit; Obtaining, based on the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit, an inter-coding mode correlation between the target neighbor coding unit and the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either a merge mode or a skip mode; Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; Perform inter-frame coding in merge mode or skip mode on the target coding unit; Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score; If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

12. A video encoding method for video on demand, characterized in that: include: For a target coding unit of a video frame included in the on-demand video, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit; Obtaining, based on the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit, an inter-coding mode correlation between the target neighbor coding unit and the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either a merge mode or a skip mode; Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; Perform inter-frame coding in merge mode or skip mode on the target coding unit; Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score; If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

13. A video encoding method for video calling, characterized in that: include: For a target coding unit of a video frame included in the video call, obtaining a target neighboring coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighboring coding units of the target coding unit; Obtaining, based on the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit, an inter-coding mode correlation between the target neighbor coding unit and the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either a merge mode or a skip mode; Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; Perform inter-frame coding in merge mode or skip mode on the target coding unit; Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score; If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

14. A video encoding method for video conferencing, characterized in that: include: For a target coding unit of a video frame included in the video conference, obtaining a target neighbor coding unit whose inter-frame coding mode is a merge mode or a skip mode from neighbor coding units of the target coding unit; Obtaining, based on the coding quantization parameter and the position of the target neighbor coding unit relative to the target coding unit, an inter-coding mode correlation between the target neighbor coding unit and the target coding unit, wherein the correlation refers to an influence of the neighbor coding unit on the target coding unit's selection of either a merge mode or a skip mode; Determining a score for early termination of inter-frame coding mode decision of a target coding unit according to the correlation; Perform inter-frame coding in merge mode or skip mode on the target coding unit; Determining whether to terminate the inter-frame coding mode decision in advance according to the inter-frame merging mode score; If the above determination result is yes, then the inter-frame coding of the target coding unit in other inter-frame coding modes is terminated.

Images