Coding mode prediction method and device

Abstract

The invention relates to a coding mode prediction method and device, and the method comprises the steps: carrying out the down-sampling operation of a to-be-coded video frame, and obtaining a first down-sampling video frame; performing motion search on a down-sampling coding unit in the first down-sampling video frame to obtain the complexity of the down-sampling coding unit; obtaining a Lagrange adjustment factor of the to-be-coded unit according to the complexity of the down-sampling coding unit; according to the Lagrangian adjustment factor, adjusting the Lagrangian multiplier of the to-be-coded unit to obtain an adjusted Lagrangian multiplier; and performing coding mode prediction on the to-be-coded unit by using the adjusted Lagrange multiplier to obtain an optimal coding mode of the to-be-coded unit. According to the method and the device, the Lagrange multiplier is adaptively adjusted along with different complexity degrees of the coding units, the pertinence of a coding mode prediction result is improved, and the compression performance of video coding is integrally improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video encoding, and in particular to a method and device for predicting an encoding mode in a video encoding process. Background Art

[0002] Video coding is a technology that reduces the size or bit rate of video data while minimizing the distortion caused by compression compared to the original image, and tries to maintain good video quality.

[0003] In existing popular coding technologies, such as H.264 and H.265 (HEVC), a video frame is divided into coding units (CU) of different sizes, and the coding mode is predicted for each coding unit, and then the optimal coding mode is selected for each coding unit. Finally, each coding unit is encoded using its optimal coding mode to complete the encoding of the video frame.

[0004] In the process of coding mode prediction, Lagrange multipliers (λ) are used to calculate the rate-distortion cost. Lagrange multipliers are related to the quantization parameters of the quantization units in the video coding process, and have nothing to do with the specific coding units in the video frame. That is to say, if a video frame or a region is determined to use the same QP, the values ​​of the Lagrange multipliers used in the prediction of all coding units in the same video frame are the same. It can be seen that the setting of the Lagrange multiplier does not take into account the influence of the different complexity of different coding units in the video frame. Using a unified Lagrange multiplier for coding mode prediction may result in the predicted coding mode not being the optimal coding mode, especially in the prediction of P frames and B frames. There is a correlation between the previous and next frames or even multiple consecutive frames in the inter-frame prediction. The use of a unified Lagrange multiplier for the same video frame cannot further improve the compression performance of the video in the coding compression performance of continuous videos composed of multiple frames. Summary of the invention

[0005] In view of this, the present disclosure provides a coding mode prediction method and device, which utilizes motion search of coding units in downsampled video frames to obtain Lagrangian adjustment factors for adjusting Lagrangian multipliers used for coding prediction of coding units in to-be-encoded video frames, and then utilizes the Lagrangian adjustment factors to adjust the Lagrangian multipliers, thereby achieving adaptive adjustment of the Lagrangian multipliers with different complexities of the coding units, thereby improving the pertinence of coding mode prediction results, thereby improving the compression performance of video coding as a whole.

[0006] The technical solution of the present disclosure is achieved as follows:

[0007] A coding mode prediction method, comprising:

[0008] Acquire a video frame to be encoded, and perform a downsampling operation on the video frame to be encoded to obtain a first downsampled video frame associated with the video frame to be encoded;

[0009] Obtaining the complexity of the down-sampled coding unit by performing motion search on the down-sampled coding unit in the first down-sampled video frame, wherein the down-sampled coding unit is obtained by performing the down-sampling operation on the original image of the unit to be encoded in the video frame to be encoded;

[0010] Obtaining, according to the complexity of the down-sampling coding unit, a Lagrangian adjustment factor of the unit to be coded corresponding to the down-sampling coding unit;

[0011] According to the Lagrangian adjustment factor, the Lagrangian multiplier of the unit to be encoded is adjusted to obtain an adjusted Lagrangian multiplier;

[0012] The adjusted Lagrange multiplier is used to predict the coding mode of the unit to be coded, so as to obtain the optimal coding mode of the unit to be coded.

[0013] Further, obtaining the complexity of the down-sampled coding unit by motion search of the down-sampled coding unit in the first down-sampled video frame includes:

[0014] Performing motion search on the down-sampled coding unit using a second down-sampled video frame to obtain an optimal motion vector of the down-sampled coding unit, wherein the second down-sampled video frame is obtained by performing the down-sampling operation on a reference image, and the reference image is associated with the video frame to be encoded;

[0015] According to the optimal motion vector, obtaining a cost value corresponding to the optimal motion vector;

[0016] A cost value corresponding to the optimal motion vector is determined as the complexity of the down-sampling coding unit.

[0017] Further, the reference image is an original image of the reference frame of the video frame to be encoded before encoding;

[0018] Alternatively, the reference image is a reconstructed image obtained by decoding the reference frame after encoding.

[0019] Further, obtaining a cost value corresponding to the optimal motion vector according to the optimal motion vector includes:

[0020] The cost value corresponding to the optimal motion vector is obtained according to the following formula:

[0021] cost=sad+λ*mvdbits

[0022] Among them, cost is the cost value corresponding to the optimal motion vector, sad is the absolute error between the downsampled coding unit and the predicted coding unit, the predicted coding unit is obtained in the second downsampled video frame according to the optimal motion vector, mvdbits is the bit value of the optimal motion vector, and λ is the Lagrange multiplier.

[0023] Further, obtaining the Lagrangian adjustment factor of the unit to be encoded corresponding to the down-sampling coding unit according to the complexity of the down-sampling coding unit includes:

[0024] Obtaining the complexity of all encoded downsampled coding units in the first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the already encoded coding units in the video frame to be encoded in the first downsampled video frame;

[0025] Obtaining a second average value according to the complexity of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit, where the second average value is an average value of all complexities including the complexity of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit;

[0026] The Lagrangian adjustment factor is obtained according to the second average value and the complexity of the downsampling coding unit.

[0027] Further, obtaining the Lagrangian adjustment factor of the unit to be encoded corresponding to the down-sampling coding unit according to the complexity of the down-sampling coding unit includes:

[0028] Obtaining a first average value, where the first average value is an average value of the complexity of all encoded downsampled coding units in the first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first downsampled video frame;

[0029] Obtaining a second average value according to the first average value and the complexity of the down-sampling coding unit, where the second average value is an average value of the first average value and the complexity of the down-sampling coding unit;

[0030] The Lagrangian adjustment factor is obtained according to the second average value and the complexity of the downsampling coding unit.

[0031] Further, obtaining a second average value according to the first average value and the complexity of the downsampling coding unit includes:

[0032] A binary result of adding the first average value and the complexity of the down-sampling coding unit is right-shifted by one bit to obtain the second average value.

[0033] Further, obtaining the Lagrangian adjustment factor according to the second average value and the complexity of the downsampling coding unit includes:

[0034] The quotient of the complexity of the down-sampling coding unit and the second average value is determined as the Lagrangian adjustment factor.

[0035] Further, the Lagrangian multiplier of the unit to be encoded is adjusted according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier, including:

[0036] The Lagrangian adjustment factor is multiplied by the Lagrangian multiplier to obtain the adjusted Lagrangian multiplier.

[0037] Further, obtaining the Lagrangian adjustment factor according to the second average value and the complexity of the downsampling coding unit includes:

[0038] Shifting the binary value of the complexity of the down-sampling coding unit left by a preset number of bits to obtain a left-shifted complexity binary value;

[0039] According to a plurality of preset segmentation intervals and a fixed-point integer value corresponding to each segmentation interval, the fixed-point integer value corresponding to the segmentation interval where the left-shift complexity binary value is located is determined as the Lagrangian adjustment factor.

[0040] Further, the Lagrangian multiplier of the unit to be encoded is adjusted according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier, including:

[0041] The binary value of the product of the Lagrangian adjustment factor and the Lagrangian multiplier is right-shifted by the preset number of bits to obtain the adjusted Lagrangian multiplier.

[0042] A coding mode prediction device, comprising:

[0043] A downsampling module is configured to obtain a video frame to be encoded, perform a downsampling operation on the video frame to be encoded, and obtain a first downsampled video frame associated with the video frame to be encoded;

[0044] a complexity acquisition module, configured to perform motion search on a downsampled coding unit in the first downsampled video frame to obtain the complexity of the downsampled coding unit, wherein the downsampled coding unit is obtained by the to-be-encoded unit in the to-be-encoded video frame through the downsampling operation;

[0045] A Lagrangian adjustment factor acquisition module is configured to obtain a Lagrangian adjustment factor of the to-be-encoded unit corresponding to the down-sampling coding unit according to the complexity of the down-sampling coding unit;

[0046] A Lagrangian multiplier adjustment module is configured to adjust the Lagrangian multiplier of the to-be-encoded unit according to the Lagrangian adjustment factor to obtain an adjusted Lagrangian multiplier;

[0047] The coding mode prediction module is configured to perform coding mode prediction on the unit to be coded by using the adjusted Lagrange multiplier to obtain an optimal coding mode of the unit to be coded.

[0048] It can be seen from the above scheme that the coding mode prediction method and device disclosed in the present invention obtain a first downsampled video frame with a resolution lower than the video frame to be encoded by downsampling the video frame to be encoded and perform motion search in the first downsampled video frame, thereby reducing the calculation amount of the motion search, accelerating the speed of the motion search, and shortening the time consumption of the motion search. The cost value corresponding to the best motion vector obtained by the motion search is used as the complexity of the downsampled coding unit, and then the complexity of the encoded coding unit in the video frame to be encoded is combined to obtain the Lagrangian adjustment factor with the coding unit, and the Lagrangian multiplier of the unit to be encoded is adjusted according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier, and then the adjusted Lagrangian multiplier is used to predict the coding mode of the unit to be encoded, and the optimal coding mode of the unit to be encoded is obtained, thereby realizing targeted adaptive adjustment of the Lagrangian multipliers of different coding units in the video frame, improving the pertinence of the coding mode prediction result, and improving the compression performance of video coding as a whole. Moreover, in the process of coding mode prediction, the shifting of binary values, the fixed-point integer valuing of the Lagrangian adjustment factor by segmented intervals and other related operations are used to replace division and floating-point calculations, thereby reducing the complexity and cost of the chip hardware that performs video coding and improving the cost-effectiveness of video coding in hardware implementation.

[0049] By adopting the coding mode prediction method and device disclosed in the present invention, under the condition of fixed QP, on the one hand, it is possible to select a coding mode that is more suitable for static or slow-motion features by lowering the λ value, so that the areas where the static or slow-motion features are located obtain higher image quality, which is beneficial to improving the video coding quality of related areas that refer to these areas in subsequent video frames; on the other hand, it is possible to select a coding mode that is more suitable for violent motion or rapid texture change features by raising the λ value, so that the areas where the violent motion or rapid texture change features are located can reduce the number of coding bits required for coding, so that within one frame, the number of bits can be used in areas that are more likely to improve the image quality of subsequent frames, thereby improving the video coding quality as a whole and improving the compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a coding mode prediction method according to an exemplary embodiment;

[0051] Figure 2 is a schematic diagram showing the relationship between a to-be-encoded video frame, an encoding unit, a first down-sampled video frame and a down-sampled encoding unit according to an exemplary embodiment;

[0052] Figure 3 is a flow chart showing a method for obtaining the complexity of a down-sampling coding unit according to an exemplary embodiment;

[0053] Figure 4 is a schematic diagram showing the relationship between a to-be-encoded video frame, a reference image, a first down-sampled video frame, and a second down-sampled video frame according to an exemplary embodiment;

[0054] Figure 5 is a flow chart of a first embodiment of obtaining a Lagrangian adjustment factor according to an exemplary embodiment;

[0055] Figure 6 is a flow chart of a second embodiment of obtaining a Lagrangian adjustment factor according to an exemplary embodiment;

[0056] Figure 7 is a flow chart showing a method of obtaining a Lagrangian adjustment factor by shifting according to an exemplary embodiment;

[0057] Figure 8 is a flow chart of a first application scenario of a coding mode prediction method according to an exemplary embodiment;

[0058] Fig. 9 is a flow chart of a second application scenario of a coding mode prediction method according to an exemplary embodiment;

[0059] Fig.10is a schematic diagram of the logical structure of a coding mode prediction device according to an exemplary embodiment;

[0060] Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.

[0062] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0063] In current video encoders, the rate-distortion cost is commonly used to select the encoding mode. The cost function is as follows:

[0064] J mode =D mode +λ*R mode

[0065] Among them, D mode is the distortion of a certain coding mode used by the coding unit, R mode is the number of bits after the coding unit is coded using a certain coding mode, J mode is the rate-distortion cost of the coding unit adopting the coding mode, λ is the Lagrange multiplier, and mode represents the coding mode.

[0066] D modeIt can be SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSD (Sum of Squared Difference), MAD (Mean Absolute Difference), MSD (Mean Squared Difference). Among them, SAD is SAE (Sum of Absolute Error), SATD is the sum of absolute values ​​after Hadamard transformation, SSD is SSE (Sum of Squared Error), MAD is MAE (Mean Absolute Error), and MSD is MSE (Mean Squared Error).

[0067] Among them, the Lagrange multiplier can be obtained using the following formula:

[0068] λ=a*2 (QP-12) / 3

[0069] Where a is a constant related to the frame type, QP is a frame-level quantization parameter, and under a fixed QP configuration, all coding units in the entire frame use the same QP.

[0070] It can be seen from the above Lagrange multiplier formula that once the QP is determined, λ is also determined, and all coding units of the entire frame are encoded using the same λ.

[0071] Generally speaking, the complexity (determined by texture and motion) of different coding units in a frame is different, and the same λ is used for prediction of the coding mode for the entire frame. For coding units with different complexities in the same frame, the predicted coding mode will be suboptimal. It can be seen that from the overall perspective of video coding, if different coding units in the same frame can adjust λ according to their complexity, the predicted coding mode can be optimized, thereby improving the quality of video coding. For example, if the display objects in some areas of a frame are still or moving slowly, they will be referenced by subsequent frames at a high frequency. In this case, if the λ value is lowered to select a coding mode that is more suitable for the still or slow-moving characteristics, the image quality of these areas where the display objects are still or moving slowly can be higher, which can be beneficial to improving the video coding quality of the related areas that refer to these areas in subsequent video frames; while for those areas where the display objects move violently or the texture changes rapidly, the correlation between the subsequent video frames and them will become weaker. In this case, if the λ value is increased to select a coding mode that is more suitable for the characteristics of violent movement or rapid texture change, the number of coding bits required for these areas can be reduced, so that within a frame, the number of bits can be used in areas that are easier to improve the image quality of subsequent frames. Therefore, using the same λ for the entire frame does not necessarily enable all areas (coding units) in a frame to obtain the best compression performance. If the complexity characteristics of the coding unit are used to adaptively adjust λ, the compression performance can be effectively improved.

[0072] In view of this, the embodiment of the present disclosure provides a coding mode prediction method, which defines the complexity of different coding units in the same frame, and adaptively adjusts λ to select a better coding mode for different complexities, thereby optimizing the balance between the number of bits and quality of different coding units, so as to improve the coding compression performance of the entire video frame or sequence. The complexity of the coding unit in a frame is defined from the perspective of the coding unit level, and the λ factor of the coding unit in the frame is calculated by the complexity of the coding unit level to adjust the λ of each coding unit in the frame, so that in the entire video encoding, λ is adaptive at the coding unit level to improve the compression performance of the video.

[0073] Figure 1 is a flow chart of a coding mode prediction method according to an exemplary embodiment. Figure 1 As shown, the coding mode prediction method includes the following steps 101 to 105.

[0074] Step 101: Obtain a video frame to be encoded, perform a downsampling operation on the video frame to be encoded, and obtain a first downsampled video frame associated with the video frame to be encoded.

[0075] In the exemplary embodiment, the downsampling operation refers to adjusting the to-be-encoded video frame from a higher resolution to a lower resolution. Since the video frame with a lower resolution contains relatively less information and the operation calculation amount is lower, in the embodiment of the present disclosure, the Lagrange multiplier is adjusted using the first downsampled video frame with a lower resolution, which can reduce the calculation amount and improve the video encoding speed compared with using the original to-be-encoded video frame with a higher resolution.

[0076] In an exemplary embodiment, the length and width of the first downsampled video frame obtained after the downsampling operation are respectively half of the original video frame to be encoded. For example, the resolution of the video frame to be encoded is 1280*720, and the resolution of the first downsampled video frame is 640*360.

[0077] Step 102: Obtain the complexity of the down-sampled coding unit by performing motion search on the down-sampled coding unit in the first down-sampled video frame, wherein the down-sampled coding unit is obtained by down-sampling the original image of the unit to be encoded in the video frame to be encoded.

[0078] In an exemplary embodiment, the resolution of the coding unit is 64*64, and the resolution of the downsampled coding unit obtained by the downsampling operation is 32*32. In an exemplary embodiment, downsampling can also be performed according to 1 / 4, the resolution of the coding unit is 64*64, and the resolution of the downsampled coding unit obtained by the 1 / 4 downsampling operation is 16*16. In practice, the downsampling ratio can be selected as needed.

[0079] Figure 2 is a schematic diagram showing the relationship between a video frame to be encoded, a coding unit, a first down-sampled video frame and a down-sampled coding unit according to an exemplary embodiment, such as Figure 2 As shown, the first down-sampled video frame obtained after the to-be-encoded video frame is down-sampled (resolution) to 1 / 4 of the original, that is, the length and width of the first down-sampled video frame are 1 / 2 of the to-be-encoded video frame, and the to-be-encoded unit in the to-be-encoded video frame is also changed into a 1 / 4 down-sampled coding unit with the down-sampling operation, that is, the length and width of the down-sampled coding unit are 1 / 2 of the to-be-encoded video frame. Figure 2 This is only an illustrative example and does not represent the actual division method of the coding unit.

[0080] Figure 3 is a flow chart showing a method for obtaining the complexity of a downsampling coding unit according to an exemplary embodiment, such as Figure 3 As shown, in the exemplary embodiment, the process of obtaining the complexity of the down-sampling coding unit in step 102 includes the following steps 301 to 303.

[0081] Step 301: Perform motion search on a down-sampled coding unit using a second down-sampled video frame to obtain an optimal motion vector for the down-sampled coding unit, wherein the second down-sampled video frame is obtained by down-sampling a reference image, and the reference image is associated with the video frame to be encoded.

[0082] In an exemplary embodiment, the reference image is an original image of a reference frame of a video frame to be encoded before encoding; or, the reference image is a reconstructed image obtained by decoding the reference frame after encoding. The reference frame is a video frame used as a comparison benchmark in the motion search process. In an exemplary embodiment, the reference frame is a video frame before the video frame to be encoded (current video frame), and the reference frame can be an I frame or a P frame.

[0083] In the exemplary embodiment, the original image of the reference frame before encoding refers to the original image obtained by the image sensor in the video acquisition device, that is, the reference image can be the original image obtained by the image sensor in the video acquisition device.

[0084] In the exemplary embodiment, the reconstructed image obtained by decoding the reference frame after encoding refers to the image obtained by decoding the original image of the reference frame after the encoding is completed to obtain the bit stream. In the video encoding process such as H.264, H.265 (HEVC), for the encoding of subsequent frames, the encoded video frame will also be decoded to obtain a reconstructed image as a reference frame used in the encoding of subsequent frames, and the subsequent frames are encoded using the reconstructed image (reference frame).

[0085] Among them, the motion vector is MV (Motion Vector). In video coding, the motion vector is a parameter used to describe the motion information between video frames. In the process of video coding, in order to compress video data, the motion similarity between frames can be used, that is, the motion relationship between the previous frame or other reference frame and the current frame can be used to represent the pixel changes in the current frame. Motion vectors are usually used to represent the pixel position offset between two frames. The motion vector indicates which area in the previous frame can find the same or similar pixels as those in the current frame, thereby reducing the amount of data that needs to be encoded and transmitted. By predicting and reconstructing the motion vector, a higher compression ratio can be achieved while maintaining video quality. Motion vectors can be represented in different ways, such as pixel-based motion vectors, block-based motion vectors, etc. The calculation and representation of these motion vectors will vary depending on the specific video coding algorithm.

[0086] Taking P frame as an example, in the exemplary embodiment, the video frame to be encoded is a P frame after encoding, and the reference image can be the original image of the reference frame of the video frame to be encoded (for example, the previous video frame of the video frame to be encoded) before encoding, and the reference image can also be the reconstructed image obtained by decoding the reference frame of the video frame to be encoded after encoding (it can be a P frame or an I frame). Based on the coding mode prediction method of the embodiment of the present disclosure, in the video encoding process of each frame (for example, a P frame), a downsampling operation is required to obtain the corresponding first downsampled video frame. Based on this, in step 301, the reference image of the video frame to be encoded is also downsampled in the video encoding process of the video frame to be encoded to obtain a reference video frame corresponding to the reference image. In step 301, the reference video frame corresponding to the reference image is referred to as the second downsampled video frame, see Figure 4 The diagram shows the relationship between the to-be-encoded video frame, the reference image, the first down-sampled video frame, and the second down-sampled video frame.

[0087] Regarding using the second down-sampled video frame to perform motion search on the down-sampled coding unit to obtain the optimal motion vector of the down-sampled coding unit, this can be achieved through an existing motion search solution, which will not be described in detail here.

[0088] Step 302: According to the best motion vector, obtain a cost value corresponding to the best motion vector.

[0089] In an exemplary embodiment, step 302 specifically includes obtaining a cost value corresponding to the optimal motion vector according to the following formula:

[0090] cost=sad+λ*mvdbits

[0091] Among them, cost is the cost value corresponding to the best motion vector, sad is the absolute error between the downsampled coding unit and the predicted coding unit, the predicted coding unit is obtained in the second downsampled video frame according to the best motion vector, mvdbits is the bit value of the best motion vector, and λ is the Lagrange multiplier.

[0092] The absolute error between the down-sampled coding unit and the predicted coding unit can be obtained through related technologies of existing video coding, which will not be described in detail here.

[0093] Step 303: Determine the cost value corresponding to the optimal motion vector as the complexity of the down-sampling coding unit.

[0094] In an exemplary embodiment, the cost is directly determined as the complexity of the down-sampled coding unit in step 303. A smaller cost indicates a lower complexity of the down-sampled coding unit, and a larger cost indicates a higher complexity of the down-sampled coding unit.

[0095] Step 103: Obtain, according to the complexity of the down-sampling coding unit, a Lagrangian adjustment factor of the unit to be coded corresponding to the down-sampling coding unit.

[0096] Figure 5 is a flow chart of a first embodiment of obtaining a Lagrangian adjustment factor according to an exemplary embodiment, such as Figure 5 As shown, in the exemplary embodiment, step 103 may further include the following steps 501 to 503 .

[0097] Step 501: Obtain the complexity of all encoded downsampled coding units in a first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first downsampled video frame.

[0098] In an exemplary embodiment, the coding units in the video frame are encoded one by one in sequence. For example, when there is one coding unit that has been encoded in the video frame to be encoded, the complexity of one encoded downsampled coding unit corresponding to the one encoded coding unit in the first downsampled video frame is obtained; when there are two coding units that have been encoded in the video frame to be encoded, the complexity of two encoded downsampled coding units corresponding to the two encoded coding units in the first downsampled video frame is obtained, and so on.

[0099] Step 502: Obtain a second average value according to the complexity of all encoded downsampled coding units and the complexity of the downsampled coding unit in the first downsampled video frame, where the second average value is the average value of all complexities including the complexity of all encoded downsampled coding units and the complexity of the downsampled coding unit in the first downsampled video frame.

[0100] In an exemplary embodiment, the second average value is obtained using the following formula:

[0101]

[0102] Among them, when 1≤i≤n-1, cost i is the complexity of the ith coded downsampled coding unit among the n-1 coded downsampled coding units in the first downsampled video frame, cost n is the complexity of the current down-sampled coding unit, and mean2 is the second average value.

[0103] In the video encoding process, each coding unit in the currently encoded video frame is encoded one by one according to a set order. In an exemplary embodiment, in the encoding mode prediction method of the embodiment of the present disclosure, the down-sampled coding units in the first down-sampled video frame are also encoded one by one according to the set order. Based on this situation, when the first down-sampled coding unit in the first down-sampled video frame is encoded, there is no encoded down-sampled coding unit, or in other words, when the first down-sampled coding unit in the first down-sampled video frame is encoded, the number of encoded down-sampled coding units is 0. In this case, in the exemplary embodiment, the second average value corresponding to the first down-sampled coding unit in the first down-sampled video frame is the complexity of the first down-sampled coding unit, that is, cost1.

[0104] Step 503: Obtain a Lagrangian adjustment factor according to the second average value and the complexity of the down-sampling coding unit.

[0105] In the above process, in the process of calculating the average value in step 502, a division operation is required and floating-point operations are involved. However, the division operation and floating-point operations consume a lot of resources in the hardware implementation of the chip. In order to meet the requirements of friendly and simple hardware implementation, the following can be used: Figure 6 The illustrated related scheme implements this division operation at the hardware level.

[0106] Figure 6 is a flow chart of a second embodiment of obtaining a Lagrangian adjustment factor according to an exemplary embodiment, such as Figure 6 As shown, in the exemplary embodiment, step 103 may further include the following steps 601 to 603 .

[0107] Step 601: Obtain a first average value, where the first average value is an average value of the complexity of all encoded downsampled coding units in a first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first downsampled video frame.

[0108] In an exemplary embodiment, the first average value may be stored in a hardware buffer and updated as each down-sampled coding unit in the first down-sampled video frame is encoded.

[0109] In the video encoding process, each coding unit in the currently encoded video frame is encoded one by one according to a set order. In the exemplary embodiment, in the coding mode prediction method of the embodiment of the present disclosure, the down-sampled coding units in the first down-sampled video frame are also encoded one by one according to the set order. Based on this situation, when the first down-sampled coding unit in the first down-sampled video frame is encoded, there is no encoded down-sampled coding unit, or in other words, when the first down-sampled coding unit in the first down-sampled video frame is encoded, the number of encoded down-sampled coding units is 0. In this case, in the exemplary embodiment, in step 601, because there is no encoded down-sampled coding unit, the first average value obtained when the first down-sampled coding unit in the first down-sampled video frame is encoded is 0 or a null value.

[0110] Step 602: Obtain a second average value according to the first average value and the complexity of the down-sampled coding unit, where the second average value is an average value of the first average value and the complexity of the down-sampled coding unit.

[0111] In an exemplary embodiment, step 602 includes: right-shifting a binary result of adding the first average value and the complexity of the down-sampled coding unit by one bit to obtain a second average value.

[0112] In an exemplary embodiment, the second average value corresponding to the first down-sampled coding unit in the first down-sampled video frame is the complexity of the first down-sampled coding unit. In this case, no shift operation is performed on the complexity of the first down-sampled coding unit, and the complexity of the first down-sampled coding unit is directly determined as the second average value.

[0113] Step 603: Obtain a Lagrangian adjustment factor according to the second average value and the complexity of the down-sampling coding unit.

[0114] For example, when the number of coding units that have been encoded in the video frame to be encoded is 0, and the downsampled coding unit being processed is the first coding unit in the first downsampled video frame, in step 601, the corresponding downsampled coding unit of the coding unit that has been encoded in the video frame to be encoded in the first downsampled video frame is also 0. At this time, the first average value is 0. In step 602, the downsampled coding unit is the first coding unit processed in the first downsampled video frame. At this time, because the first average value is 0, the binary result of adding the first average value (which is 0) and the complexity of the first coding unit processed in the first downsampled video frame is right shifted by one bit to obtain a second average value corresponding to the first coding unit processed in the first downsampled video frame. When the second average value corresponding to the first coding unit processed in the first downsampled video frame is obtained, the second average value can be used as the first average value used to obtain the Lagrangian adjustment factor of the next coding unit to be encoded in the video frame to be encoded; it can be expressed as:

[0115] mean2=(0+cost1)>>1

[0116] Among them, mean2 is the second average value corresponding to the first coding unit processed in the first downsampled video frame, cost1 is the complexity of the first coding unit in the first downsampled video frame (the downsampled coding unit currently being processed), and (0+cost1)>>1 means shifting the binary result of 0+cost1 right by one place.

[0117] When there is one coding unit that has been encoded in the video frame to be encoded and the downsampled coding unit being processed is the second coding unit in the first downsampled video frame, in step 601, the corresponding downsampled coding unit in the first downsampled video frame for the coding unit that has been encoded in the video frame to be encoded is also one. At this time, the first average value is the second average value obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (corresponding to the first downsampled coding unit in the first downsampled video frame). In step 602, the downsampled coding unit is the second coding unit processed in the first downsampled video frame. At this time, the binary result of adding the first average value and the complexity of the second coding unit processed in the first downsampled video frame is right shifted by one bit to obtain the second average value corresponding to the second coding unit processed in the first downsampled video frame. When the second average value corresponding to the second coding unit processed in the first downsampled video frame is obtained, the second average value can be used as the first average value used to obtain the Lagrangian adjustment factor of the next coding unit to be encoded in the video frame to be encoded; it can be expressed as:

[0118] mean2=(mean1+cost2)>>1

[0119] Among them, mean2 is the second average value corresponding to the second coding unit processed in the first down-sampled video frame, mean1 is the second average value (i.e., mean2 in the previous formula) obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (here corresponding to the first down-sampled coding unit in the first down-sampled video frame), cost2 is the complexity of the second coding unit in the first down-sampled video frame (the down-sampled coding unit currently being processed), and (mean1+cost2)>>1 means that the binary result of mean1+cost2 is shifted right by one place.

[0120] When there are two coding units that have been encoded in the video frame to be encoded, and the downsampled coding unit being processed is the third coding unit in the first downsampled video frame, in step 601, the number of downsampled coding units corresponding to the coding units that have been encoded in the video frame to be encoded in the first downsampled video frame is also two. At this time, the first average value is the second average value obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (corresponding to the second downsampled coding unit in the first downsampled video frame). In step 602, the downsampled coding unit is the third coding unit processed in the first downsampled video frame. At this time, the binary result of adding the first average value and the complexity of the third coding unit processed in the first downsampled video frame is right shifted by one bit to obtain the second average value corresponding to the third coding unit processed in the first downsampled video frame. When the second average value corresponding to the third coding unit processed in the first downsampled video frame is obtained, the second average value can be used as the first average value used to obtain the Lagrangian adjustment factor of the next coding unit to be encoded in the video frame to be encoded; it can be expressed as follows:

[0121] mean2=(mean1+cost3)>>1

[0122] Among them, mean2 is the second average value corresponding to the third coding unit processed in the first down-sampled video frame, mean1 is the second average value (i.e., mean2 in the previous formula) obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (here corresponding to the second down-sampled coding unit in the first down-sampled video frame), and cost3 is the complexity of the third coding unit in the first down-sampled video frame (the down-sampled coding unit currently being processed).

[0123] By analogy, when the number of coding units that have been encoded in the video frame to be encoded is n-1, and the downsampled coding unit being processed is the nth coding unit in the first downsampled video frame, in step 601, the number of downsampled coding units corresponding to the coding units that have been encoded in the video frame to be encoded in the first downsampled video frame is also n-1. At this time, the first average value is the second average value obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (corresponding to the n-1th downsampled coding unit in the first downsampled video frame). In step 602, The downsampled coding unit is the nth coding unit processed in the first downsampled video frame. At this time, the binary result of adding the first average value and the complexity of the nth coding unit processed in the first downsampled video frame is right-shifted by one bit to obtain a second average value corresponding to the nth coding unit processed in the first downsampled video frame. When the second average value corresponding to the nth coding unit processed in the first downsampled video frame is obtained, the second average value can be used as the first average value used to obtain the Lagrangian adjustment factor of the next coding unit to be encoded in the video frame to be encoded; it is expressed by the formula:

[0124] mean2=(mean1+cost n )>>1

[0125] Wherein, mean2 is the second average value corresponding to the nth coding unit processed in the first down-sampled video frame, mean1 is the second average value (i.e., mean2 in the previous formula) obtained in the process of obtaining the Lagrangian adjustment factor of the previous coding unit to be encoded (here corresponding to the n-1th down-sampled coding unit in the first down-sampled video frame), and cost n is the complexity of the nth coding unit in the first down-sampled video frame (the down-sampled coding unit currently being processed).

[0126] In the hardware implementation of the chip, the operation of adding two numbers and then shifting them right by one position is equivalent to summing and averaging the two numbers. Although there are differences between the scheme of using the above steps 601 to 603 and the scheme of direct division from steps 501 to 503, in terms of execution efficiency and results, the calculation speed of the shift operation from steps 601 to 603 is much higher than the division operation from steps 501 to 503, and the deviation between the two is very small, and the effects are similar. Therefore, compared with the scheme of using the division operation, using the shift operation in the process of obtaining the second average value can significantly reduce the complexity of the hardware implementation.

[0127] In an exemplary embodiment, step 503 and / or step 603 includes: determining a quotient of the complexity of the downsampled coding unit and the second average value as a Lagrangian adjustment factor.

[0128] Specifically, step 503 and / or step 603 may use the following formula to obtain the Lagrangian adjustment factor:

[0129] factor n =cost n / mean2

[0130] Among them, factor n is the Lagrangian adjustment factor, cost n is the complexity of the downsampled coding unit, and mean2 is the second mean value.

[0131] It can be seen from the above formula that the process of obtaining the Lagrangian adjustment factor in step 503 and / or step 603 still includes a division operation. In order to meet the requirements of reducing the area and power consumption of hardware implementation, in the exemplary embodiment, the following can be used to obtain the Lagrangian adjustment factor. Figure 7 The illustrated related scheme implements this division operation at the hardware level.

[0132] Figure 7 is a flow chart showing a method of obtaining a Lagrangian adjustment factor by shifting according to an exemplary embodiment, such as Figure 7 As shown, in the exemplary embodiment, step 503 and / or step 603 includes the following steps 701 to 702 .

[0133] Step 701: Left-shift the binary value of the complexity of the down-sampled coding unit by a preset number of bits to obtain a left-shifted complexity binary value.

[0134] Step 702: According to a plurality of preset segmentation intervals and the fixed-point integer value corresponding to each segmentation interval, the fixed-point integer value corresponding to the segmentation interval where the left-shift complexity binary value is located is determined as a Lagrangian adjustment factor.

[0135] In the exemplary embodiment, the preset number of bits can be set to 5 bits, because 2 5 =32, so the left shift of 5 bits is equivalent to expanding the binary value of the complexity of the downsampled coding unit by 2 5 Times, which is equivalent to a multiplication operation here, can convert decimals (floating points) into integers, which is beneficial to improving the hardware processing speed. In addition, the fixed-point integer determined by the left shift complexity binary value obtained in the integer form after the shift is combined with subsequent multiplication operations and shift operations (see subsequent description for details). Compared with division and floating-point operations, the processing speed of the chip hardware can be improved. The encoding mode prediction method of the embodiment of the present disclosure can be used to reduce the hardware implementation complexity of the chip.

[0136] Wherein, step 701 is expressed by the formula:

[0137] factor nl =costn <<5

[0138] Among them, factor nl Indicates the binary value of left shift complexity, cost n is the complexity of the downsampling coding unit (in binary form), cost n <<5 means cost n The binary value of is shifted left by 5 bits, where 5 bits are the preset number of bits.

[0139] In a specific example, the fixed-point integer values ​​of the Lagrangian adjustment factors of the segmented intervals in step 702 can refer to the following Table 1.

[0140] Table 1 Fixed-point integer values ​​of segment intervals and Lagrangian adjustment factors

[0141]

[0142]

[0143]

[0144] In Table 1, the cost in hardware calculation form n <<5 is the binary value factor of the left shift complexity nl , factor nl’ is the Lagrangian adjustment factor.

[0145] The explanation for row 1 in Table 1 is as follows:

[0146] The cost in the first row of segment intervals n / mean2>52 / 32 means cost n / mean2 is greater than 52 / 32, in cost n / mean2>52 / 32, no matter cost n / mean2 or 52 / 32 both involve division operations. In order to avoid division operations as much as possible in the calculation form of chip hardware, cost n / mean2>52 / 32 equivalent transformation to cost n *32>mean2*52, so the cost in the segmented interval n / When mean2 is greater than 52 / 32, it is equivalent to cost n *32 is greater than mean2*52, thus eliminating the division operation here. In the hardware calculation form, 32=2 5 , so cost n *32 is equivalent to cost n Shift left five bits to get costn <<5, and thus the cost in the hardware calculation form is obtained. n <<5>mean2 * 52. In this way, division operations can be avoided in the hardware calculation form, thereby improving the computing performance of the chip hardware. In the example shown in Table 1, when cost n / mean2 > 52 / 32, that is, when cost n <<5>mean2 * 52, the Lagrange adjustment factor factor nl’ is set to 53. From the actual operation of video encoding, when cost n / mean2 > 52 / 32, setting factor nl’ higher than 53 will weaken the contribution to further improving the video encoding quality. For example, it will not further increase the compression ratio while ensuring clarity. Therefore, in the illustrative embodiment, the maximum value of factor nl’ can be set to 53.

[0147] Among them, the explanation of the second row in Table 1 above is as follows:

[0148] The cost in the first row's segmented interval n / mean2 > 51 / 32 and cost n / mean2 <= 52 / 32 means the situation where cost n / mean2 is greater than 51 / 32 and less than or equal to 52 / 32. Referring to the explanation in the first row above, the equivalent hardware calculation form in the second row for the segmented interval is expressed as cost n <<5>mean2 * 51 and cost n <<5 <= mean2 * 52. When cost n / mean2 > 51 / 32 and cost n / mean2 <= 52 / 32, that is, when cost n <<5>mean2 * 51 and cost n <<5 <= mean2 * 52, the Lagrange adjustment factor factor nl’ is set to 52.

[0149] The representations of the remaining rows are the same as the explanations in the first and second rows above, and will not be elaborated here.

[0150] The situation of the last row in Table 1 is that cost n / mean2 < 27 / 32, that is, the situation where cost n <<5 < mean2 * 27. In this case, the Lagrange adjustment factor factornl’ The value is 26. According to the actual operation of video encoding, the cost n / mean2<27 / 32, factor nl’ Although a value lower than 26 can further improve the video encoding quality, the corresponding bit rate will also increase, thereby reducing the compression rate. Therefore, in the exemplary embodiment, the factor nl’ The minimum value is set to 26.

[0151] Step 104: adjust the Lagrangian multiplier of the coding unit according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier.

[0152] In an exemplary embodiment, step 104 includes: multiplying the Lagrangian adjustment factor by the Lagrangian multiplier to obtain an adjusted Lagrangian multiplier. This exemplary embodiment matches the above step of "determining the quotient of the complexity of the downsampled coding unit and the second average value as the Lagrangian adjustment factor". In an exemplary embodiment, the adjusted Lagrangian multiplier is obtained using the following formula:

[0153] λ'=factor n *λ

[0154] Among them, λ' is the adjusted Lagrange multiplier.

[0155] In an exemplary embodiment, step 104 includes: shifting the binary value of the product of the Lagrangian adjustment factor and the Lagrangian multiplier right by a preset number of bits to obtain an adjusted Lagrangian multiplier. This exemplary embodiment matches the above steps 701 and 702. In an exemplary embodiment, the adjusted Lagrangian multiplier is obtained using the following formula:

[0156] λ'=(factor nl’ *λ)>>5

[0157] Among them, (factor nl’ *λ)>>5 means factor nl’ *The binary result of λ is shifted right by 5 bits, where 5 bits are the preset number of bits.

[0158] Since the Lagrange adjustment factor obtained after step 701 and step 702 is an integer, it is actually obtained by shifting the binary value of the complexity of the down-sampled coding unit to the left by a preset number of bits and then using the segment interval and taking the fixed-point integer value. The Lagrange adjustment factor here is amplified by being shifted to the left by the preset number of bits. Therefore, in step 104, the Lagrange adjustment factor is multiplied by the Lagrange multiplier by an integer multiplication of the Lagrange multiplier, which can avoid the problem of large amount of calculation caused by decimal floating-point operations, which is conducive to improving the hardware processing speed. degree, and then right-shift the result of multiplying the Lagrange adjustment factor and the Lagrange multiplier by the preset number of bits (reduced), which is equivalent to multiplying the Lagrange adjustment factor by the Lagrange multiplier. Since the hardware module complexity of division and floating-point operations in chip hardware is higher than that of shift and multiplication operations, and the speed at which chip hardware processes division and floating-point operations is lower than that of shift and multiplication operations, using shift and integer multiplication operations instead of division and floating-point operations can simplify the chip hardware circuit design and improve the processing speed of chip hardware.

[0159] Step 105: Use the adjusted Lagrange multiplier to predict the coding mode of the unit to be coded, and obtain the optimal coding mode of the unit to be coded.

[0160] In an exemplary embodiment, the optimal encoding mode of the unit to be encoded is obtained using the following formula:

[0161] min{J mode =D mode +λ'*R mode}

[0162] Among them, min{} means taking the J of all coding modes in {} mode In step 105, the minimum value of the rate-distortion costs of all coding modes is obtained by the above formula, and the coding mode corresponding to the minimum value of the rate-distortion costs of all coding modes is determined as the optimal coding mode of the unit to be encoded.

[0163] After obtaining the optimal coding mode of the unit to be encoded using the coding mode prediction method of the embodiment of the present invention, the optimal coding mode can be used to perform video encoding on the unit to be encoded. Video encoding can be implemented using existing technologies, such as related technologies in H.264 and H.265 (HEVC) video encoding, which will not be repeated here.

[0164] The coding mode prediction method of the disclosed embodiment obtains a first downsampled video frame with a resolution lower than that of the video frame to be encoded by downsampling the video frame to be encoded and performs motion search in the first downsampled video frame, thereby reducing the computational complexity of the motion search, accelerating the speed of the motion search, and shortening the time consumption of the motion search. The cost value corresponding to the best motion vector obtained by the motion search is used as the complexity of the downsampled coding unit, and then the Lagrangian adjustment factor with the coding unit is obtained in combination with the complexity of the encoded coding unit in the video frame to be encoded. The Lagrangian multiplier of the unit to be encoded is adjusted according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier, and the adjusted Lagrangian multiplier is used to predict the coding mode of the unit to be encoded to obtain the optimal coding mode of the unit to be encoded, thereby realizing targeted adaptive adjustment of the Lagrangian multipliers of different coding units in the video frame, improving the targetedness of the coding mode prediction result, and improving the compression performance of video encoding as a whole. Moreover, in the process of coding mode prediction, the shifting of binary values, the fixed-point integer valuing of the Lagrangian adjustment factor by segmented intervals and other related operations are used to replace division and floating-point calculations, thereby reducing the complexity and cost of the chip hardware that performs video coding and improving the cost-effectiveness of video coding in hardware implementation.

[0165] By adopting the coding mode prediction method of the embodiment of the present disclosure, under the condition of fixed QP, on the one hand, it is possible to select a coding mode that is more suitable for still or slow-motion features by lowering the λ value, so that the areas where the still or slow-motion features are located obtain higher image quality, which is beneficial to improving the video coding quality of related areas that refer to these areas in subsequent video frames; on the other hand, it is possible to select a coding mode that is more suitable for features of violent motion or rapid texture changes by raising the λ value, so that the areas where the violent motion or rapid texture changes are located can reduce the number of coding bits required for coding, so that within one frame, the number of bits can be used in areas that are more likely to improve the image quality of subsequent frames, thereby improving the coding quality of the video as a whole and improving the compression performance.

[0166] Figure 8 is a flowchart of a first application scenario of a coding mode prediction method according to an exemplary embodiment, such as Figure 8 As shown, the process mainly includes the following steps 801 to 814.

[0167] Step 801, obtain the video frame to be encoded, and then execute step 802.

[0168] Step 802 , determining whether the to-be-encoded video frame is an I frame, if so, executing the intra-frame prediction process for the I frame, otherwise executing step 803 .

[0169] Step 803 : perform a down-sampling operation on the video frame to be encoded to obtain a first down-sampled video frame associated with the video frame to be encoded, and then execute step 804 .

[0170] Step 804 , obtaining the unit to be encoded from the video frame to be encoded, and then executing step 805 .

[0171] Step 805 , obtaining a downsampled coding unit associated with the unit to be encoded from the first downsampled video frame, and then executing step 806 .

[0172] The down-sampled coding unit is obtained by down-sampling the to-be-coded unit in the to-be-coded video frame.

[0173] Step 806 : Use the second down-sampled video frame to perform motion search on the down-sampled coding unit to obtain the best motion vector of the down-sampled coding unit, and then execute step 807 .

[0174] Among them, the second down-sampled video frame is obtained by downsampling the reference image associated with the video frame to be encoded. The reference image can be the original image of the reference frame of the video frame to be encoded before encoding, or the reference image can also be the reconstructed image obtained by decoding the reference frame of the video frame to be encoded after encoding.

[0175] The encoded video frame is associated with the video frame to be encoded, for example, the encoded video frame is a previous video frame of the video frame to be encoded.

[0176] Step 807 : According to the best motion vector, a cost value corresponding to the best motion vector is obtained, and the cost value corresponding to the best motion vector is determined as the complexity of the down-sampling coding unit, and then step 808 is executed.

[0177] Step 808 , obtaining the complexity of all encoded down-sampled coding units in the first down-sampled video frame, and then executing step 809 .

[0178] The encoded down-sampled coding unit is a down-sampled coding unit corresponding to the coding unit that has been encoded in the video frame to be encoded in the first down-sampled video frame.

[0179] Step 809 : Obtain a second average value according to the complexity of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit, and then execute step 810 .

[0180] The second average value is an average value of all complexities including the complexities of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit.

[0181] In an exemplary embodiment, when performing coding mode prediction for a first down-sampled coding unit in a first down-sampled video frame, the complexity of the first down-sampled coding unit is determined as a second average value corresponding to the first down-sampled coding unit.

[0182] Step 810: Determine the quotient of the complexity of the down-sampled coding unit and the second average value as the Lagrangian adjustment factor, and then execute step 811.

[0183] Step 811: multiply the Lagrangian adjustment factor and the Lagrangian multiplier to obtain the adjusted Lagrangian multiplier, and then execute step 812.

[0184] Step 812: Use the adjusted Lagrange multiplier to predict the coding mode of the unit to be coded to obtain the optimal coding mode of the unit to be coded, and then execute step 813.

[0185] Step 813: Encode the unit to be encoded using the optimal encoding mode, and then execute step 814.

[0186] Step 814 , determining whether all units to be encoded in the video frame to be encoded have been encoded; if so, ending the encoding process of the current video frame to be encoded; otherwise, executing step 804 .

[0187] Fig. 9 is a flow chart of a second application scenario of a coding mode prediction method according to an exemplary embodiment, such as Fig. 9 As shown, the process mainly includes the following steps 901 to 916.

[0188] Step 901, obtain the video frame to be encoded, and then execute step 902.

[0189] Step 902 , determining whether the to-be-encoded video frame is an I frame, if so, executing the intra-frame prediction process for the I frame, otherwise executing step 903 .

[0190] Step 903 : perform a down-sampling operation on the video frame to be encoded to obtain a first down-sampled video frame associated with the video frame to be encoded, and then execute step 904 .

[0191] Step 904 , obtaining the unit to be encoded from the video frame to be encoded, and then executing step 905 .

[0192] Step 905 : Obtain a downsampled coding unit associated with the unit to be encoded from the first downsampled video frame, and then execute step 906 .

[0193] The down-sampled coding unit is obtained by down-sampling the to-be-coded unit in the to-be-coded video frame.

[0194] Step 906 : Use the second down-sampled video frame to perform motion search on the down-sampled coding unit to obtain the best motion vector of the down-sampled coding unit, and then execute step 907 .

[0195] Among them, the second down-sampled video frame is obtained by downsampling the reference image associated with the video frame to be encoded. The reference image can be the original image of the reference frame of the video frame to be encoded before encoding, or the reference image can also be the reconstructed image obtained by decoding the reference frame of the video frame to be encoded after encoding.

[0196] Step 907 : According to the best motion vector, a cost value corresponding to the best motion vector is obtained, and the cost value corresponding to the best motion vector is determined as the complexity of the down-sampling coding unit, and then step 908 is executed.

[0197] Step 908 : Obtain a first average value, where the first average value is an average value of the complexity of all encoded down-sampled coding units in the first down-sampled video frame, and then execute step 909 .

[0198] Among them, all the encoded down-sampled coding units in the first down-sampled video frame are down-sampled coding units corresponding to the already encoded coding units in the to-be-encoded video frame in the first down-sampled video frame.

[0199] In an exemplary embodiment, when the first down-sampled coding unit in the first down-sampled video frame is encoded, the obtained first average value is 0 or a null value.

[0200] Step 909: Obtain a second average value according to the first average value and the complexity of the down-sampling coding unit, and then execute step 910.

[0201] The second average value is an average value of the first average value and the complexity of the down-sampling coding unit.

[0202] Step 910: Left-shift the binary value of the complexity of the down-sampled coding unit by a preset number of bits to obtain a left-shifted complexity binary value, and then execute step 911.

[0203] In an exemplary embodiment, the second average value corresponding to the first down-sampled coding unit in the first down-sampled video frame is the complexity of the first down-sampled coding unit. In this case, no shift operation is performed on the complexity of the first down-sampled coding unit, and the complexity of the first down-sampled coding unit is directly determined as the second average value.

[0204] Step 911: According to the preset multiple segmentation intervals and the fixed-point integer value corresponding to each segmentation interval, the fixed-point integer value corresponding to the segmentation interval where the left-shift complexity binary value is located is determined as the Lagrangian adjustment factor, and then step 912 is executed.

[0205] Step 912: right-shift the binary value of the product of the Lagrange adjustment factor and the Lagrange multiplier by a preset number of bits to obtain an adjusted Lagrange multiplier, and then execute step 913.

[0206] Step 913: multiply the Lagrangian adjustment factor and the Lagrangian multiplier to obtain the adjusted Lagrangian multiplier, and then execute step 914.

[0207] Step 914: Use the adjusted Lagrange multiplier to predict the coding mode of the unit to be coded to obtain the optimal coding mode of the unit to be coded, and then execute step 915.

[0208] Step 915: Encode the unit to be encoded using the optimal encoding mode, and then execute step 916.

[0209] Step 916 , determining whether all units to be encoded in the video frame to be encoded have been encoded; if so, ending the encoding process of the current video frame to be encoded; otherwise, executing step 904 .

[0210] Fig.10 is a schematic diagram of the logical structure of a coding mode prediction device according to an exemplary embodiment, such as Fig.10 As shown, the coding mode prediction device includes a downsampling module 1001, a complexity acquisition module 1002, a Lagrangian adjustment factor acquisition module 1003, a Lagrangian multiplier adjustment module 1004, and a coding mode prediction module 1005.

[0211] The down-sampling module 1001 is configured to obtain a video frame to be encoded, perform a down-sampling operation on the video frame to be encoded, and obtain a first down-sampled video frame associated with the video frame to be encoded;

[0212] The complexity acquisition module 1002 is configured to perform a motion search on a downsampled coding unit in a first downsampled video frame to obtain a complexity of the downsampled coding unit, wherein the downsampled coding unit is obtained by downsampling the coding unit in the to-be-encoded video frame;

[0213] The Lagrangian adjustment factor acquisition module 1003 is configured to obtain the Lagrangian adjustment factor of the to-be-encoded unit corresponding to the down-sampled coding unit according to the complexity of the down-sampled coding unit;

[0214] The Lagrangian multiplier adjustment module 1004 is configured to adjust the Lagrangian multiplier of the coding unit according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier;

[0215] The coding mode prediction module 1005 is configured to perform coding mode prediction on the unit to be coded using the adjusted Lagrange multiplier to obtain the optimal coding mode of the unit to be coded.

[0216] In an exemplary embodiment, the complexity acquisition module 1002 includes:

[0217] The best motion vector obtaining submodule is configured to perform motion search on the down-sampled coding unit using the second down-sampled video frame to obtain the best motion vector of the down-sampled coding unit, wherein the second down-sampled video frame is obtained by down-sampling the reference image, and the reference image is associated with the video frame to be encoded;

[0218] The cost value obtaining submodule is configured to obtain a cost value corresponding to the best motion vector according to the best motion vector;

[0219] The complexity determination submodule is configured to determine the cost value corresponding to the best motion vector as the complexity of the down-sampled coding unit.

[0220] In an exemplary embodiment, the reference image is an original image of a reference frame of the to-be-encoded video frame before encoding; or, the reference image is a reconstructed image obtained by decoding the reference frame after encoding.

[0221] In an exemplary embodiment, the cost value obtaining submodule is further configured to perform:

[0222] The cost value corresponding to the optimal motion vector is obtained according to the following formula:

[0223] cost=sad+λ*mvdbits

[0224] Among them, cost is the cost value corresponding to the best motion vector, sad is the absolute error between the downsampled coding unit and the predicted coding unit, the predicted coding unit is obtained in the second downsampled video frame according to the best motion vector, mvdbits is the bit value of the best motion vector, and λ is the Lagrange multiplier.

[0225] In an exemplary embodiment, the Lagrangian adjustment factor acquisition module 1003 includes:

[0226] The complexity acquisition submodule is configured to execute the acquisition of the complexity of all the encoded down-sampled coding units in the first down-sampled video frame, wherein all the encoded down-sampled coding units in the first down-sampled video frame are the down-sampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first down-sampled video frame;

[0227] A second average value obtaining submodule is configured to obtain a second average value according to the complexity of all encoded downsampled coding units and the complexity of the downsampled coding unit in the first downsampled video frame, where the second average value is an average value of all complexities including the complexity of all encoded downsampled coding units and the complexity of the downsampled coding unit in the first downsampled video frame;

[0228] The Lagrangian adjustment factor obtaining submodule is configured to obtain the Lagrangian adjustment factor according to the second average value and the complexity of the down-sampling coding unit.

[0229] In an exemplary embodiment, the Lagrangian adjustment factor acquisition module 1003 includes:

[0230] A first average value acquisition submodule is configured to execute acquisition of a first average value, where the first average value is an average value of the complexity of all encoded downsampled coding units in the first downsampled video frame, wherein all the encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first downsampled video frame;

[0231] A second average value obtaining submodule is configured to obtain a second average value according to the first average value and the complexity of the down-sampling coding unit, where the second average value is an average value of the first average value and the complexity of the down-sampling coding unit;

[0232] The Lagrangian adjustment factor obtaining submodule is configured to obtain the Lagrangian adjustment factor according to the second average value and the complexity of the down-sampling coding unit.

[0233] In an exemplary embodiment, the second average value obtaining submodule is further configured to execute:

[0234] A binary result of adding the first average value and the complexity of the down-sampled coding unit is right-shifted by one bit to obtain a second average value.

[0235] In an exemplary embodiment, the Lagrangian adjustment factor obtaining submodule is further configured to execute:

[0236] The quotient of the complexity of the down-sampled coding unit and the second average value is determined as a Lagrangian adjustment factor.

[0237] In the exemplary embodiment, the Lagrange multiplier adjustment module 1004 is further configured to perform:

[0238] The Lagrange adjustment factor is multiplied by the Lagrange multiplier to obtain the adjusted Lagrange multiplier.

[0239] In an exemplary embodiment, the Lagrangian adjustment factor obtaining submodule is further configured to execute:

[0240] Shifting the binary value of the complexity of the down-sampled coding unit left by a preset number of bits to obtain a left-shifted complexity binary value;

[0241] According to a plurality of preset segmentation intervals and a fixed-point integer value corresponding to each segmentation interval, the fixed-point integer value corresponding to the segmentation interval where the left-shift complexity binary value is located is determined as a Lagrangian adjustment factor.

[0242] In the exemplary embodiment, the Lagrange multiplier adjustment module 1004 is further configured to perform:

[0243] The binary value of the product of the Lagrange adjustment factor and the Lagrange multiplier is right-shifted by a preset number of bits to obtain an adjusted Lagrange multiplier.

[0244] The coding mode prediction device of the embodiment of the present disclosure obtains a first downsampled video frame with a resolution lower than the video frame to be encoded by downsampling the video frame to be encoded and performs motion search in the first downsampled video frame, thereby reducing the calculation amount of the motion search, accelerating the speed of the motion search, and shortening the time consumption of the motion search. The cost value corresponding to the best motion vector obtained by the motion search is used as the complexity of the downsampled coding unit, and then the Lagrangian adjustment factor with the coding unit is obtained in combination with the complexity of the encoded coding unit in the video frame to be encoded. The Lagrangian multiplier of the unit to be encoded is adjusted according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier, and then the adjusted Lagrangian multiplier is used to predict the coding mode of the unit to be encoded to obtain the optimal coding mode of the unit to be encoded, thereby realizing targeted adaptive adjustment of the Lagrangian multipliers of different coding units in the video frame, improving the targetedness of the coding mode prediction result, and improving the compression performance of video encoding as a whole. Moreover, in the process of coding mode prediction, the shifting of binary values, the fixed-point integer valuing of the Lagrangian adjustment factor by segmented intervals and other related operations are used to replace division and floating-point calculations, thereby reducing the complexity and cost of the chip hardware that performs video coding and improving the cost-effectiveness of video coding in hardware implementation.

[0245] By using the coding mode prediction device of the embodiment of the present disclosure, under the condition of fixed QP, on the one hand, it is possible to select a coding mode that is more suitable for still or slow-motion features by lowering the λ value, so that the areas where the still or slow-motion features are located obtain higher image quality, which is beneficial to improving the video coding quality of related areas that refer to these areas in subsequent video frames; on the other hand, it is possible to select a coding mode that is more suitable for violent motion or rapid texture change features by raising the λ value, so that the areas where the violent motion or rapid texture change features are located can reduce the number of coding bits required for coding, so that within one frame, the number of bits can be used in areas that are more likely to improve the image quality of subsequent frames, thereby improving the coding quality of the video as a whole and improving the compression performance.

[0246] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0247] Regarding the coding mode prediction device in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the coding mode prediction method, and will not be elaborated here.

[0248] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0249] Fig.11 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. In some embodiments, the electronic device is a server. The electronic device 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1101 and one or more memories 1102, wherein at least one program code is stored in the memory 1102, and the at least one program code is loaded and executed by the processor 1101 to implement the coding mode prediction method provided by the above-mentioned various embodiments. Of course, the electronic device 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 1100 may also include other components for implementing device functions, which will not be repeated here.

[0250] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the coding mode prediction method in the above embodiment.

[0251] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.

[0252] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A coding mode prediction method, comprising: Acquire a video frame to be encoded, and perform a downsampling operation on the video frame to be encoded to obtain a first downsampled video frame associated with the video frame to be encoded; Obtaining the complexity of the down-sampled coding unit by performing motion search on the down-sampled coding unit in the first down-sampled video frame, wherein the down-sampled coding unit is obtained by performing the down-sampling operation on the original image of the unit to be encoded in the video frame to be encoded; Obtaining, according to the complexity of the down-sampling coding unit, a Lagrangian adjustment factor of the unit to be coded corresponding to the down-sampling coding unit; According to the Lagrangian adjustment factor, the Lagrangian multiplier of the unit to be encoded is adjusted to obtain an adjusted Lagrangian multiplier; The adjusted Lagrange multiplier is used to predict the coding mode of the unit to be coded, so as to obtain the optimal coding mode of the unit to be coded.

2. The coding mode prediction method according to claim 1, characterized in that: The obtaining the complexity of the down-sampled coding unit by motion searching the down-sampled coding unit in the first down-sampled video frame comprises: Performing motion search on the down-sampled coding unit using a second down-sampled video frame to obtain an optimal motion vector of the down-sampled coding unit, wherein the second down-sampled video frame is obtained by performing the down-sampling operation on a reference image, and the reference image is associated with the video frame to be encoded; According to the optimal motion vector, obtaining a cost value corresponding to the optimal motion vector; A cost value corresponding to the optimal motion vector is determined as the complexity of the down-sampling coding unit.

3. The coding mode prediction method according to claim 2, characterized in that: The reference image is an original image of the reference frame of the to-be-encoded video frame before encoding; Alternatively, the reference image is a reconstructed image obtained by decoding the reference frame after encoding.

4. The coding mode prediction method according to claim 2, characterized in that: The step of obtaining a cost value corresponding to the optimal motion vector according to the optimal motion vector includes: The cost value corresponding to the optimal motion vector is obtained according to the following formula: cost=sad+λ*mvdbits Among them, cost is the cost value corresponding to the optimal motion vector, sad is the absolute error between the downsampled coding unit and the predicted coding unit, the predicted coding unit is obtained in the second downsampled video frame according to the optimal motion vector, mvdbits is the bit value of the optimal motion vector, and λ is the Lagrange multiplier.

5. The coding mode prediction method according to claim 1, characterized in that: The obtaining, according to the complexity of the down-sampling coding unit, a Lagrangian adjustment factor of the to-be-coded unit corresponding to the down-sampling coding unit comprises: Obtaining the complexity of all encoded downsampled coding units in the first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the already encoded coding units in the video frame to be encoded in the first downsampled video frame; Obtaining a second average value according to the complexity of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit, where the second average value is an average value of all complexities including the complexity of all encoded downsampled coding units in the first downsampled video frame and the complexity of the downsampled coding unit; The Lagrangian adjustment factor is obtained according to the second average value and the complexity of the downsampling coding unit.

6. The coding mode prediction method according to claim 1, characterized in that: The obtaining, according to the complexity of the down-sampling coding unit, a Lagrangian adjustment factor of the to-be-coded unit corresponding to the down-sampling coding unit comprises: Obtaining a first average value, where the first average value is an average value of the complexity of all encoded downsampled coding units in the first downsampled video frame, wherein all encoded downsampled coding units in the first downsampled video frame are downsampled coding units corresponding to the encoded coding units in the video frame to be encoded in the first downsampled video frame; Obtaining a second average value according to the first average value and the complexity of the down-sampling coding unit, where the second average value is an average value of the first average value and the complexity of the down-sampling coding unit; The Lagrangian adjustment factor is obtained according to the second average value and the complexity of the downsampling coding unit.

7. The coding mode prediction method according to claim 6, characterized in that: The obtaining a second average value according to the first average value and the complexity of the downsampling coding unit includes: A binary result of adding the first average value and the complexity of the down-sampling coding unit is right-shifted by one bit to obtain the second average value.

8. The coding mode prediction method according to claim 5 or 6, characterized in that: The obtaining the Lagrangian adjustment factor according to the second average value and the complexity of the downsampling coding unit includes: The quotient of the complexity of the down-sampling coding unit and the second average value is determined as the Lagrangian adjustment factor.

9. The coding mode prediction method according to claim 8, characterized in that: The step of adjusting the Lagrangian multiplier of the unit to be encoded according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier comprises: The Lagrangian adjustment factor is multiplied by the Lagrangian multiplier to obtain the adjusted Lagrangian multiplier.

10. The coding mode prediction method according to claim 5 or 6, characterized in that: The obtaining the Lagrangian adjustment factor according to the second average value and the complexity of the downsampling coding unit includes: Shifting the binary value of the complexity of the down-sampling coding unit left by a preset number of bits to obtain a left-shifted complexity binary value; According to a plurality of preset segmentation intervals and a fixed-point integer value corresponding to each segmentation interval, the fixed-point integer value corresponding to the segmentation interval where the left-shift complexity binary value is located is determined as the Lagrangian adjustment factor.

11. The coding mode prediction method according to claim 10, characterized in that: The step of adjusting the Lagrangian multiplier of the unit to be encoded according to the Lagrangian adjustment factor to obtain the adjusted Lagrangian multiplier comprises: The binary value of the product of the Lagrangian adjustment factor and the Lagrangian multiplier is right-shifted by the preset number of bits to obtain the adjusted Lagrangian multiplier.

12. A coding mode prediction device, characterized in that: include: A downsampling module is configured to obtain a video frame to be encoded, perform a downsampling operation on the video frame to be encoded, and obtain a first downsampled video frame associated with the video frame to be encoded; a complexity acquisition module, configured to perform motion search on a downsampled coding unit in the first downsampled video frame to obtain the complexity of the downsampled coding unit, wherein the downsampled coding unit is obtained by the to-be-encoded unit in the to-be-encoded video frame through the downsampling operation; A Lagrangian adjustment factor acquisition module is configured to obtain a Lagrangian adjustment factor of the to-be-encoded unit corresponding to the down-sampling coding unit according to the complexity of the down-sampling coding unit; A Lagrangian multiplier adjustment module is configured to adjust the Lagrangian multiplier of the to-be-encoded unit according to the Lagrangian adjustment factor to obtain an adjusted Lagrangian multiplier; The coding mode prediction module is configured to perform coding mode prediction on the unit to be coded by using the adjusted Lagrange multiplier to obtain an optimal coding mode of the unit to be coded.

Images