Video coding method and device, electronic equipment, storage medium, program product and method for generating bit stream

Abstract

The invention provides a video coding method and device, electronic equipment, a storage medium, a program product and a bit stream generation method, and the video coding method comprises the steps: traversing a plurality of preset coding modes for a current block, and obtaining the rate distortion cost in each preset coding mode; selecting a coding mode of the current block from a plurality of preset coding modes based on the rate distortion cost in each preset coding mode; encoding the current block based on the encoding mode of the current block; wherein the step of acquiring the rate distortion cost in each preset coding mode comprises the following steps: acquiring a subjective quality index corresponding to the current coding mode; a penalty coefficient associated with the subjective quality index is calculated, and the penalty coefficient is in positive correlation with the possibility of the subjective quality problem indicated by the subjective quality index; and based on the penalty coefficient, amplifying the original rate-distortion cost of the current coding mode to obtain the rate-distortion cost of the current coding mode. The method can improve the subjective quality of the coded video.

Description

Technical Field

[0001] The present disclosure relates to video coding and compression. More specifically, the present disclosure relates to a video coding method, apparatus, electronic device, storage medium, program product and method for generating a bit stream. Background Art

[0002] In recent years, with the popularity of live streaming, outdoor live streaming, video conferencing and other businesses, the application scenarios of client-side video encoding have become increasingly extensive. How to maximize video quality within limited bandwidth has become one of the research hotspots in academia and industry.

[0003] In the existing coding framework, there are often multiple candidate coding modes and parameters to cope with different video contents. One of the main tasks of a video encoder is to select the optimal coding mode and parameters with a "certain strategy" to achieve relatively good coding performance. To this end, most video encoders will calculate the rate-distortion cost for different coding modes and parameters to reflect the trade-off between the number of bits (data volume) consumed by coding and the degree of distortion (video quality loss) under a given bit rate, and then determine the final coding mode and parameters based on the rate-distortion cost.

[0004] However, when using existing coding strategies, problems such as blurring, artifacts, and block effects often occur, making it difficult to ensure coding quality. Summary of the invention

[0005] Embodiments of the present disclosure provide a video encoding method, apparatus, electronic device, storage medium, program product, and method for generating a bitstream, for solving at least one of the above problems.

[0006] According to one aspect of the present disclosure, a video encoding method is provided, the video encoding method comprising: for a current block, traversing multiple preset encoding modes to obtain a rate-distortion cost in each preset encoding mode; based on the rate-distortion cost in each preset encoding mode, selecting a coding mode for the current block from the multiple preset coding modes; based on the coding mode of the current block, encoding the current block; wherein obtaining the rate-distortion cost in each preset encoding mode comprises: obtaining a subjective quality indicator corresponding to the current encoding mode, wherein the subjective quality indicator is related to a subjective quality problem existing in the current encoding mode; calculating a penalty coefficient associated with the subjective quality indicator, wherein the penalty coefficient is positively correlated with the possibility of the existence of the subjective quality problem indicated by the subjective quality indicator; based on the penalty coefficient, amplifying the original rate-distortion cost of the current encoding mode to obtain the rate-distortion cost of the current encoding mode.

[0007] Optionally, the calculation of the penalty coefficient associated with the subjective quality indicator includes: when the size relationship between the subjective quality indicator and a preset indicator threshold satisfies a preset condition, calculating the penalty coefficient based on the subjective quality indicator and the preset indicator threshold, wherein different subjective quality indicators are set with different preset indicator thresholds, and the preset condition indicates that the possibility of the existence of the subjective quality problem exceeds the possibility indicated by the preset indicator threshold.

[0008] Optionally, calculating the penalty coefficient according to the subjective quality indicator and the preset indicator threshold includes: calculating a candidate penalty coefficient according to the subjective quality indicator and the preset indicator threshold; and determining the smaller one of the candidate penalty coefficient and the preset coefficient threshold as the penalty coefficient.

[0009] Optionally, obtaining the subjective quality indicator corresponding to the current coding mode includes: when the current coding mode is an intra-frame angle coding mode, obtaining the statistical value of the horizontal gradient and the statistical value of the vertical gradient of the current block; and determining the subjective quality indicator corresponding to the intra-frame angle coding mode based on the statistical value of the horizontal gradient and the statistical value of the vertical gradient.

[0010] Optionally, obtaining the subjective quality indicator corresponding to the current coding mode includes: when the current coding mode is an intra-frame non-angular coding mode, obtaining a quantization parameter and energy of the current block as a subjective quality indicator corresponding to the intra-frame non-angular coding mode.

[0011] Optionally, obtaining the subjective quality indicator corresponding to the current coding mode includes: when the current coding mode is an inter-frame skip coding mode, determining the subjective quality indicator corresponding to the inter-frame skip coding mode based on the pixel difference between two adjacent columns of pixels between the current block and the adjacent block on the left and the pixel difference between two adjacent rows of pixels between the current block and the adjacent block above.

[0012] Optionally, when the current coding mode is an inter-frame adaptive motion vector coding mode, the penalty coefficient is set to a value that maintains the original rate-distortion cost.

[0013] Optionally, amplifying the original rate-distortion cost of the current coding mode based on the penalty coefficient includes: amplifying the distortion corresponding to the current coding mode in the original rate-distortion cost based on the penalty coefficient; or amplifying the original rate-distortion cost as a whole based on the penalty coefficient.

[0014] According to another aspect of the present disclosure, a video encoding device is provided, the video encoding device comprising: an acquisition unit, configured to traverse multiple preset encoding modes for a current block, and acquire the rate-distortion cost under each preset encoding mode; a selection unit, configured to select the encoding mode of the current block from the multiple preset encoding modes based on the rate-distortion cost under each preset encoding mode; an encoding unit, configured to encode the current block based on the encoding mode of the current block; the acquisition unit is further configured to: acquire a subjective quality indicator corresponding to the current encoding mode, wherein the subjective quality indicator is related to a subjective quality problem existing in the current encoding mode; calculate a penalty coefficient associated with the subjective quality indicator, wherein the penalty coefficient is positively correlated with the possibility of the existence of the subjective quality problem indicated by the subjective quality indicator; based on the penalty coefficient, amplify the original rate-distortion cost of the current encoding mode to obtain the rate-distortion cost of the current encoding mode.

[0015] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the video encoding method as described above.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method as described above.

[0017] According to another aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the video encoding method as described above when the computer instructions are executed by at least one processor.

[0018] According to another aspect of the present disclosure, a computer program product is provided, having instructions for storing a bit stream, wherein the bit stream includes: video data generated according to the video encoding method as described above.

[0019] According to another aspect of the present disclosure, a method for generating a bitstream is provided, comprising: generating a bitstream according to the video encoding method as described above.

[0020] According to the video encoding method, apparatus, electronic device, storage medium, program product and method for generating a bitstream of the exemplary embodiments of the present disclosure, a penalty coefficient that is positively correlated with the possibility of the existence of a subjective quality problem is introduced, and the original rate-distortion cost is amplified and corrected to a corresponding degree accordingly, so that the rate-distortion cost obtained thereby increases with the increase in the possibility of the existence of the subjective quality problem, thereby reducing the possibility of the corresponding encoding mode being selected when other conditions remain unchanged, thereby reducing the risk of subjective quality problems occurring after encoding, and being able to effectively consider the subjective quality problem when calculating the rate-distortion cost, thereby improving the subjective quality of the video obtained by encoding.

[0021] It is to be understood that both the foregoing general description and the following detailed description are examples only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate examples according to the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0023] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0024] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0025] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0026] Figure 4 is a flowchart illustrating a video encoding method according to some embodiments of the present disclosure.

[0027] Figure 5 is a flow chart illustrating acquisition rate-distortion cost according to some embodiments of the present disclosure.

[0028] Figure 6 is a schematic diagram showing the calculation principle of the left pixel difference and value according to some embodiments of the present disclosure.

[0029] Figure 7 is a flow chart showing a method of obtaining rate-distortion cost according to a specific embodiment of the present disclosure.

[0030] Figure 8 is a block diagram illustrating a video encoding apparatus according to some embodiments of the present disclosure.

[0031] Fig. 9is a diagram illustrating a computing environment coupled with a user interface according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0032] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, various alternatives may be used without departing from the scope of the claims, and the subject matter may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0034] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a target device 14. Source device 12 and target device 14 may include any of a wide variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0035] In some implementations, target device 14 may receive the encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving the encoded video data from source device 12 to target device 14.

[0036] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28.

[0037] like Figure 1, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as, or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video.

[0038] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback.

[0039] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0040] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present disclosure is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0041] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0042] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in the present disclosure. The video encoder 20 can perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that in the field of video coding and decoding, the term "frame" can be used as a synonym for the term "image" or "picture".

[0043] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter may be omitted, and the decoded video blocks may be provided directly by adder 62 to DPB 64. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0044] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 Video source 18 is shown obtaining video data in video data memory 40. DPB 64 is a buffer that stores reference video data (eg, reference frames or pictures) for use by video encoder 20 in encoding the video data (eg, in intra or inter prediction coding modes).

[0045] like Figure 2 As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into strips, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree (QT) structure. It should be noted that the terms "block" or "video block" used herein may be a portion of a frame or picture, in particular a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0046] The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on the error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

[0047] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0048] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0049] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0050] Intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicating the intra-prediction mode selected for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0051] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0052] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0053] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0054] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0055] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0056] Figure 3 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present disclosure. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process described with respect to the video encoder 20 is essentially the inverse of the decoding process. For example, the motion compensation unit 82 may generate prediction data based on the motion vector received from the entropy decoding unit 80, and the intra-frame prediction unit 84 may generate prediction data based on the intra-frame prediction mode indicator received from the entropy decoding unit 80. A loop filter 91 (e.g., a deblocking filter, an SAO filter, a CCSAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted.

[0057] As mentioned above, Figure 2 The prediction processing unit 41 in the video processing unit 41 can select one of multiple feasible prediction coding modes for the current video block based on the error result (for example, the coding rate and the distortion level), and the commonly used method is to select based on the rate-distortion cost of each coding mode.

[0058] At present, most video encoders use Lagrangian rate-distortion optimization (RDO) for mode selection, which satisfies the following formula:

[0059] J=D+λ·R

[0060] Among them, J represents the rate-distortion cost, D represents the distortion in the current coding mode, which is usually an objective distortion index such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSD (Sum of Squared Difference), λ is the Lagrangian factor, and R represents the number of bits required for all information in the current coding mode. When comparing the coding performance of different coding modes and parameters, the encoder will select the coding mode and parameters corresponding to the minimum rate-distortion cost as the optimal value recognized under this strategy. However, the above Lagrangian rate-distortion optimization formula mainly considers the objective quality in coding, and does not consider the relationship between the selection of different coding modes and parameters and subjective quality. In other words, when the rate-distortion cost calculated in this way is the smallest, it does not mean that the subjective quality at this time is very good. Therefore, if a certain coding mode and parameter is prone to subjective quality problems in certain scenarios, continuing to use the above formula may fail to identify subjective quality risks, which in turn leads to subjective image quality deterioration, such as blurring, artifacts, and block effects.

[0061] In this regard, there is a solution in the related technology, which is to directly disable a certain encoding mode or parameter based on development and testing experience when it is known that a certain encoding mode or parameter is prone to cause subjective problems. However, this one-size-fits-all approach of disabling a certain encoding mode or parameter will lead to a significant decrease in compression performance, resulting in an increase in bit rate and bandwidth.

[0062] Refer to the following Figures 4 to 7 A video encoding method according to some embodiments of the present disclosure is introduced.

[0063] Figure 4 FIG. 2 is a flowchart showing an exemplary video encoding method according to some embodiments of the present disclosure. The video encoding method can be implemented in the video encoder 20 described above.

[0064] Reference Figure 4 In step S410, for the current block, multiple preset coding modes are traversed to obtain the rate-distortion cost in each preset coding mode. The present disclosure will improve the calculation method of the rate-distortion cost for this step, which will be described in detail later and will not be expanded here.

[0065] In step S420, based on the rate-distortion cost of each preset coding mode, a coding mode for the current block is selected from multiple preset coding modes. For example, the coding mode with the smallest rate-distortion cost may be selected from multiple preset coding modes as the coding mode for the current block.

[0066] In step S430, the current block is encoded based on the encoding mode of the current block. The specific encoding process can be referred to the detailed description above, which will not be repeated here.

[0067] Next, step S410 is further introduced. Figure 5 The specific processing flow of obtaining the rate-distortion cost of each preset coding mode in step S410 is shown, including steps S411 to S413. It should be noted that step S410 of the present disclosure includes these steps, which means that when traversing multiple preset coding modes for the current block, at least one preset coding mode can be used as follows Figure 5 The process shown in FIG. 1 is used to obtain its rate-distortion cost, rather than forcing each preset coding mode to adopt the following method. Figure 5 The process shown can achieve a more flexible video encoding scheme.

[0068] Reference Figure 5 , in step S411, a subjective quality indicator corresponding to the current encoding mode is obtained.

[0069] The subjective quality index is related to the subjective quality problems existing in the current coding mode, so that the subjective quality problems that may exist in the current coding mode can be quantitatively expressed in a targeted manner. As an example, the subjective quality index can be positively or negatively correlated with the possibility of subjective quality problems existing in the current coding mode. It should be noted that the subjective quality index can be calculated in advance for standby before traversing different preset coding modes, or it can be calculated accordingly when traversing each preset coding mode, and the present disclosure does not limit this.

[0070] In step S412, a penalty coefficient associated with the subjective quality indicator is calculated.

[0071] The penalty coefficient is positively correlated with the possibility of the existence of subjective quality problems indicated by the subjective quality index, that is, if the subjective quality index indicates that the possibility of the existence of subjective quality problems is greater, the calculated penalty coefficient is greater.

[0072] In step S413, the original rate-distortion cost of the current coding mode is amplified based on the penalty coefficient to obtain the rate-distortion cost of the current coding mode.

[0073] The original rate-distortion cost is the rate-distortion cost obtained by using the existing video encoding method, which mainly considers the objective quality problem. By introducing a penalty coefficient that is positively correlated with the possibility of subjective quality problems, and accordingly amplifying and correcting the original rate-distortion cost to a corresponding degree, the rate-distortion cost obtained thereby will increase with the increase in the possibility of subjective quality problems, thereby reducing the possibility of the corresponding encoding mode being selected when other conditions remain unchanged, and thus reducing the risk of subjective quality problems after encoding. The subjective quality problem can be effectively considered when calculating the rate-distortion cost, thereby improving the subjective quality of the video obtained by encoding.

[0074] Regarding the calculation of the penalty coefficient in step S412, a penalty function may be constructed, and the penalty coefficient may be calculated based on the subjective quality indicator using the subjective quality indicator as an independent variable and the penalty coefficient as a dependent variable.

[0075] As an example, step S412 includes: when the magnitude relationship between the subjective quality indicator and the preset indicator threshold satisfies the preset condition, the penalty coefficient is calculated according to the subjective quality indicator and the preset indicator threshold, wherein different subjective quality indicators are set with different preset indicator thresholds, and the preset condition is to indicate that the possibility of the existence of a subjective quality problem exceeds the possibility indicated by the preset indicator threshold. By further introducing the preset indicator threshold to indicate the acceptable possibility of subjective quality problems, and using it as a reference, the corresponding penalty coefficient is calculated for the subjective quality indicator exceeding the preset indicator threshold, and the calculation of the penalty coefficient can be triggered on demand. In addition, by considering both the subjective quality indicator and the preset indicator threshold when calculating the penalty coefficient, instead of considering the subjective quality indicator alone, the relative magnitude of the possibility of the subjective quality problem can be reflected by the deviation between the subjective quality indicator and the preset indicator threshold, which helps to calculate a more reasonable penalty coefficient. It should be understood that, as mentioned above, the subjective quality indicator can be positively correlated or negatively correlated with the possibility of the existence of a subjective quality problem in the current coding mode. Based on this, when positively correlated, the preset condition can be that the subjective quality indicator is greater than the preset indicator threshold, and when negatively correlated, the preset condition can be that the subjective quality indicator is less than the preset indicator threshold. For the case where the relationship between the subjective quality index and the preset index threshold does not meet the preset conditions, that is, the possibility of the subjective quality problem can be introduced, the penalty coefficient may not be calculated, that is, the original rate distortion cost is not amplified, or the penalty coefficient may be calculated in a different way from that when the preset conditions are met to reduce the degree of amplification of the original rate distortion cost, and the present disclosure does not limit this. It should also be understood that the specific value of the preset index threshold can be set to a fixed value associated with coding parameters such as block size and quantization parameter (QP) according to engineering experience or calculated by a formula, and the present disclosure does not limit this.

[0076] Regarding the specific calculation method of the penalty coefficient when the preset conditions are met, the deviation between the subjective quality index and the preset index threshold can be expressed in different ways (for example, including but not limited to absolute difference, growth rate, reduction rate, etc.), and the deviation can be further converted to a numerical value so that the final penalty coefficient is greater than 1 to represent the amplification of the original rate distortion cost. It should be understood that if the penalty coefficient is equal to 1 at this time, it means that the original rate distortion cost is maintained without amplification. Of course, other values ​​other than 1 can also be used to represent the maintenance of the original rate distortion cost. At this time, the calculation method of the penalty coefficient needs to adapt to it, which is also an implementation method of the present disclosure and falls within the protection scope of the present disclosure.

[0077] For this deviation, optionally, the operation of calculating the penalty coefficient according to the subjective quality index and the preset index threshold in the above step S412 includes: calculating the penalty coefficient according to the ratio of the subjective quality index and the preset index threshold. This embodiment uses a ratio to represent the deviation between the subjective quality index and the preset index threshold. Since the subjective quality index and the preset index threshold are of the same order of magnitude, the ratio of the two will be of the same order of magnitude as 1, and a penalty coefficient greater than 1 can be obtained more conveniently, thereby improving the calculation efficiency. It should be understood that for embodiments that use values ​​other than 1 to represent the maintenance rate distortion cost, the ratio of the two can also be calculated, and the ratio needs to be further processed.

[0078] Further optionally, the above-mentioned operation of calculating the penalty coefficient according to the ratio of the subjective quality index to the preset index threshold includes: determining the ratio of the larger to the smaller of the subjective quality index and the preset index threshold as the penalty coefficient. As mentioned above, since the correlation between the subjective quality index and the possibility of the existence of a subjective quality problem in the current coding mode can be positively correlated or negatively correlated, the size relationship between the subjective quality index represented by the preset condition and the preset index threshold can also be greater than or less than. By specifically setting the calculated ratio to be the ratio of the larger to the smaller of the two, it can be ensured that the obtained ratio is greater than 1, and can be directly used as the penalty coefficient, further improving the calculation efficiency.

[0079] In addition to the embodiment of directly using the calculation result of the subjective quality index and the preset index threshold as the penalty coefficient, optionally, the operation of calculating the penalty coefficient according to the subjective quality index and the preset index threshold in the above step S412 may also include: calculating the candidate penalty coefficient according to the subjective quality index and the preset index threshold; determining the smaller of the candidate penalty coefficient and the preset coefficient threshold as the penalty coefficient. Considering that if the result directly calculated according to the subjective quality index and the preset index threshold is too large, the subjective quality problem will have too much impact on the rate distortion cost, which may cause the compression performance loss to be correspondingly too large. By further introducing the preset coefficient threshold as the upper limit of the penalty coefficient, when the direct calculation result (i.e., the candidate penalty coefficient) is too large (i.e., greater than the preset coefficient threshold), the preset coefficient threshold is used as the final penalty coefficient, which helps to reasonably control the compression performance loss and improve the overall encoding quality. It should be understood that the calculation of the subjective quality index and the preset index threshold here can adopt any calculation method described above. It should also be understood that the specific value of the preset coefficient threshold can be set as a fixed value associated with coding parameters such as block size and quantization parameter according to engineering experience or calculated by a formula, and the present disclosure does not limit this.

[0080] Next, how to process the specific current encoding mode in step S411 is introduced.

[0081] Optionally, step S411 may include: when the current coding mode is the intra-frame angle coding mode, obtaining the statistical value of the horizontal gradient and the statistical value of the vertical gradient of the current block; and determining the subjective quality index corresponding to the intra-frame angle coding mode according to the statistical value of the horizontal gradient and the statistical value of the vertical gradient. For the intra-frame angle coding mode, after analysis, when the pixel gradient of the current block does not change much, that is, it is relatively flat, the subjective quality problem of artifacts is prone to occur. Based on this, by respectively counting the horizontal gradient and the vertical gradient of the current block and determining the subjective quality index accordingly, it is possible to quantitatively reflect the pixel gradient change of the current block, and realize the quantitative representation of the possibility of the occurrence of the subjective quality problem of artifacts.

[0082] As an example, the method for calculating the gradient statistics is to first apply the Sobel operator (of course, other operators can also be used, such as but not limited to the Canny operator) to calculate the horizontal gradient and vertical gradient of each pixel in the current block, and then perform statistics on the horizontal gradient and vertical gradient of all pixels in the current block to obtain the statistical value of the horizontal gradient and the statistical value of the vertical gradient of the current block. Specifically, when calculating the horizontal gradient of each pixel, the pixel values ​​of the current pixel and the 9 pixels around it are compared with the horizontal Sobel operator G x Convolution, taking the absolute value of the convolution result as the horizontal gradient of the current pixel, and combining the pixel values ​​of the current pixel and its surrounding 9 pixels with the vertical Sobel operator G yConvolution, the absolute value of the convolution result is used as the vertical gradient of the current pixel. Among them, the horizontal Sobel operator G x , the horizontal gradient G of pixel (i, j) x (i,j), vertical Sobel operator G y , the vertical gradient G of pixel (i, j) y (i,j) can be expressed as:

[0083]

[0084]

[0085] Among them, I represents the pixel value of the corresponding pixel in the original image.

[0086] As an example, when counting the horizontal gradients and vertical gradients of all pixels in the current block, the average value may be counted, or other statistical values ​​such as the mode and the median may be counted, and the present disclosure does not limit this. Taking the statistical average value as an example, for a current block of size n×n (referring to the number of rows and columns of pixels included in the current block), the horizontal gradient average value and the vertical gradient average value may be respectively expressed as:

[0087]

[0088] Among them, (i, j) is the position coordinate of the first pixel in the upper left corner of the current block, p and q represent the distances that the pixels move along the rows and columns, so that (i+p, j+q) can be used to represent each pixel in the current block.

[0089] Specifically, before encoding the current frame, it can be divided into multiple non-overlapping n×n small blocks from left to right and from top to bottom. If the size of the edge block of the current frame is less than n×n, it can be processed in a variety of different ways, such as including but not limited to pixel filling (such as filling with the mean or a constant) and abandoning calculation, which is not limited in the present disclosure.

[0090] As an example, after the statistics are completed, the ratio of the larger and smaller of the statistical values ​​of the horizontal gradient and the statistical values ​​of the vertical gradient can be determined as the gradient ratio, and as a subjective quality indicator corresponding to the intra-frame angle coding mode, the gradient ratio dir_rate can be expressed as:

[0091]

[0092] For the intra-frame angle coding mode, further optionally, step S412 includes: when the subjective quality index is less than the preset index threshold corresponding to the intra-frame angle coding mode, the ratio of the preset index threshold to the subjective quality index is determined as a candidate penalty coefficient; and the smaller of the candidate penalty coefficient and the preset coefficient threshold is determined as the penalty coefficient. According to the above analysis, the smaller the pixel gradient change of the current block, that is, the flatter the image, the greater the possibility of the subjective quality problem of artifacts. Based on this, by configuring the preset index threshold as a safety lower limit, when the subjective quality index is less than the preset index threshold, it can be considered that there is a greater possibility of subjective quality problems, and the preset conditions are met. Then, the penalty coefficient is determined based on the ratio (greater than 1) of the larger preset index threshold to the smaller subjective quality index, and efficient calculation of the penalty coefficient can be achieved. Specifically, the ratio is first determined as a candidate penalty coefficient, and then the preset coefficient threshold is used to achieve the upper limit of the penalty coefficient. Please refer to the previous introduction, and no further details will be repeated here.

[0093] Optionally, step S411 may further include: when the current coding mode is an intra non-angle coding mode, obtaining a quantization parameter (QP) and energy (Energy) of the current block as a subjective quality indicator corresponding to the intra non-angle coding mode. For the intra non-angle coding mode, it is found through analysis that when the quantization parameter and energy of the current block are large, a subjective quality problem of blur is prone to occur. Based on this, by obtaining the quantization parameter and energy of the current block, a quantitative representation of the possibility of occurrence of the subjective quality problem of blur can be achieved together.

[0094] Specifically, the quantization parameter can be determined in advance before encoding, and can be directly obtained here. Energy has many different but similar calculation methods in this field, and often the sum of the pixel values ​​(Summation, abbreviated as SUM) and the sum of squares (Sum of Squares, abbreviated as SS) of each pixel in the current block are combined to achieve a similar effect to variance. For example, the following formulas can be used to calculate the pixel square sum S of the current block: SS , pixels and S SUM 、Energy E:

[0095]

[0096] Among them, I, i, j, p, and q have the same meanings as above, and the current frame can also be divided into multiple non-overlapping n×n small blocks, which will not be repeated here.

[0097] As an example, in addition to directly calculating the absolute energy E, a normalization step may be further added, for example, using the following formula for normalization:

[0098] Energy = (2 × E) 0.1

[0099] Wherein, Energy represents the normalized energy, and 2 and 0.1 are parameters that can be adjusted as needed (for example, according to the preset energy threshold to be introduced later). Of course, other methods can also be used for normalization, such as using functions such as sigmoid, and the present disclosure does not limit this.

[0100] For the intra-frame non-angle coding mode, further optionally, step S412 includes: when the quantization parameter is greater than the preset quantization parameter threshold and the energy is greater than the preset energy threshold, calculating a first ratio of the quantization parameter to the quantization parameter threshold, and calculating a second ratio of the energy to the energy threshold, determining the product of the first ratio and the second ratio as a candidate penalty coefficient; and determining the smaller of the candidate penalty coefficient and the preset coefficient threshold as the penalty coefficient. According to the above analysis, the larger the quantization parameter and the gradient of the current block, the greater the possibility of the subjective quality problem of blur. By configuring the preset quantization parameter threshold and the preset energy threshold as the safety upper limit, it can be considered that there is a greater possibility of subjective quality problems when the quantization parameter and the energy are both greater than the corresponding preset index thresholds, and the preset conditions are met. Then, for the quantization parameter and the energy, respectively, the ratio of the larger subjective quality index to the smaller preset index threshold (i.e., the first ratio and the second ratio, both of which are greater than 1) is calculated, and the penalty coefficient is determined according to the product of the two ratios, so that efficient calculation of the penalty coefficient can be achieved. Specifically, the product of the two ratios is first determined as a candidate penalty coefficient, and then a preset coefficient threshold is used to implement an upper limit on the penalty coefficient. Please refer to the previous introduction and will not be repeated here.

[0101] Optionally, step S411 may also include: when the current coding mode is the inter-frame skip coding mode, according to the pixel difference between the two adjacent columns of pixels of the current block and the adjacent block on the left, and the pixel difference between the two adjacent rows of pixels of the current block and the adjacent block above, determining the subjective quality index corresponding to the inter-frame skip coding mode. For the inter-frame skip coding mode, it is found through analysis that when the boundary pixel difference between two adjacent blocks is large, the subjective quality problem of block effect is prone to occur. Based on this, by respectively calculating the pixel difference between the current block and the adjacent block on the left, and the pixel difference between the current block and the adjacent block above, and then determining the subjective quality index accordingly, it is possible to quantitatively reflect the pixel difference between the current block and its adjacent blocks, and realize the quantitative representation of the possibility of the occurrence of the subjective quality problem of block effect.

[0102] As an example, it is possible to first determine whether there are reconstructed pixels of the left adjacent block and the upper adjacent block of the current block. If so, the pixel difference is calculated using the reconstructed pixels of the two adjacent blocks. If not, the calculation is no longer continued and the corresponding amplification of the original rate-distortion cost is abandoned.

[0103] As an example, when determining the subjective quality index, the pixel difference sum value between the two adjacent columns of pixels of the current block and the adjacent block on the left and the pixel difference sum value between the current block and the adjacent block above can be calculated, and the larger of the two calculated pixel difference sum values ​​is determined as the subjective quality index corresponding to the inter-frame skip coding mode. The method for calculating the pixel difference sum value using the reconstructed pixels of the adjacent block on the left side of the current block (size is m×m) is as follows: Figure 6 First, obtain the two adjacent columns of pixels of the left adjacent block and the current block, and use L1, L2, ..., LS and C1, C2, ..., CS to represent the pixel values ​​of these two columns of pixels, where S = min (m, n). Then, calculate the pixel difference of the two adjacent pixels on the left and right one by one and finally sum them up to get the pixel difference and value, which can be expressed as:

[0104]

[0105] The calculation method of the pixel difference and value of the two adjacent rows of pixels of the current block and the upper adjacent block (size is m×m) is similar to the above. First, the two adjacent rows of pixels of the upper adjacent block and the current block are obtained, and the pixel values ​​of the two rows of pixels are represented by T1, T2, ..., TS and C1, C2, ..., CS respectively. Then, the pixel difference of the two adjacent pixels above and below is calculated one by one and finally summed to obtain the pixel difference and value, which can be expressed as:

[0106]

[0107] For the inter-frame skip coding mode, further optionally, step S412 includes: when the subjective quality index is less than the preset index threshold corresponding to the inter-frame skip coding mode, the ratio of the subjective quality index to the preset index threshold is determined as a candidate penalty coefficient; the smaller of the candidate penalty coefficient and the preset coefficient threshold is determined as the penalty coefficient. According to the above analysis, the greater the boundary pixel difference and value between the current block and the adjacent block, the greater the possibility of the subjective quality problem of block effect. Based on this, by configuring the preset index threshold as a safety upper limit, when the subjective quality index is greater than the preset index threshold, it can be considered that there is a greater possibility of subjective quality problems, and the preset conditions are met. Then, the penalty coefficient is determined based on the ratio (greater than 1) of the larger subjective quality index to the smaller preset index threshold, and efficient calculation of the penalty coefficient can be achieved. Specifically, the ratio is first determined as a candidate penalty coefficient, and then the upper limit of the penalty coefficient is realized using the preset coefficient threshold. Please refer to the previous introduction, and no further details will be repeated here.

[0108] Optionally, when the current encoding mode is an inter-frame adaptive motion vector (Adaptive Motion Vector Prediction, referred to as AMVP) encoding mode, the penalty coefficient is set to a value that maintains the original rate distortion cost, such as but not limited to 1. Since the inter-frame adaptive motion vector encoding mode is less likely to have subjective quality problems, by directly setting the penalty coefficient to a value that maintains the original rate distortion cost, the mode can be used as a bottom-line option, ensuring that video encoding can be performed closely.

[0109] Regarding the amplification of the original rate-distortion cost in step S413, optionally, the operation of amplifying the original rate-distortion cost of the current coding mode based on the penalty coefficient in step S413 includes: amplifying the distortion corresponding to the current coding mode in the original rate-distortion cost based on the penalty coefficient; or amplifying the original rate-distortion cost as a whole based on the penalty coefficient. As an example, the former may multiply the penalty coefficient with the distortion corresponding to the current coding mode in the original rate-distortion cost, and the latter may multiply the penalty coefficient with the original rate-distortion cost. Both of these amplification methods can effectively amplify the original rate-distortion cost through simple calculations, which helps to improve processing efficiency.

[0110] Next, a video encoding method according to a specific embodiment of the present disclosure is introduced. This specific embodiment is mainly implemented by a preprocessing module and an amplification processing module. The preprocessing module is used to calculate the gradient (including the statistical value of the horizontal gradient) of each n×n block in the current frame in units of n×n size (for example, including but not limited to 16×16, which can be set according to actual business needs) before the amplification processing module. And the vertical gradient statistics ) and energy information (normalized Energy). The specific calculation method is as mentioned above and will not be repeated here. The amplification processing module is used to perform the amplification processing for the current block when encoding each block. Figure 4 To simplify the description, only the step S410 is combined here. Figure 7 Describes the processing flow performed by the amplification processing module.

[0111] Step S1: Initialize rate=1.0, wherein rate is a penalty coefficient for the distortion part newly added in the present disclosure for the Lagrangian rate-distortion optimization formula J=D+λ·R.

[0112] Step S2: Predict multiple modes such as intra, inter AMVP, and inter skip. Different standards and encoders have different implementation logics for this part, which is not limited here. It only needs to comply with the semantic syntax specified by the corresponding standard. After completing the intra mode prediction, go to step S3; after completing the inter AMVP mode prediction, go to step S13; after completing the inter skip mode prediction, go to step S9.

[0113] Step S3: Enter this step after completing the intra mode prediction in step S2. Determine whether the mode selected by the intra prediction is the intra angle mode. If yes, enter Step 4; otherwise, enter Step S7.

[0114] Step S4: Calculate the gradient ratio dir_rate using the gradient information calculated by the preprocessing module, and proceed to step S5. The calculation formula is as mentioned above.

[0115] Step S5: Determine whether the gradient ratio dir_rate is less than the threshold thr0. If so, proceed to step S6; otherwise, proceed to step S13. Wherein, thr0 is an empirical value greater than 1, and the present disclosure does not limit its specific value. The purpose of this step is to reduce the subjective quality problem of artifacts. The reason is that artifacts are prone to appear in the intra angle mode when the pixel gradient of the current block does not change much, that is, it is relatively flat. Therefore, this step uses the gradient ratio dir_rate and thr0 to determine whether the possibility of the subjective quality problem of artifacts is high.

[0116] Step S6: Update the penalty coefficient rate = min (thr0 / dir_rate, thr1), and go to step S13. Among them, thr1 is also an empirical value greater than 1, which is used to limit the penalty coefficient so that it does not become too large and cause excessive compression performance loss. The present disclosure does not limit its specific value. In addition, it can be seen from the introduction of step S5 that the intra angle mode is prone to artifacts when the pixel gradient of the current block does not change much, that is, it is relatively flat. Therefore, the smaller the dir_rate, that is, the smaller the pixel gradient change, the larger the penalty coefficient will be, which can effectively reduce the risk of artifacts and improve subjective quality.

[0117] Step S7: Determine whether the quantization parameter (QP) of the current block is greater than the threshold thr2 and whether the energy Energy is greater than the threshold thr3. If so, proceed to step S8; otherwise, proceed to step S13. Among them, thr2 and thr3 are empirical values ​​greater than 0, and the present disclosure does not limit their specific values. The purpose of this step is to reduce the subjective quality problem of blur. The reason is that blur is prone to occur in the intra non-angular mode when the QP and energy are relatively large, that is, the bit rate is relatively insufficient (QP is large) to encode non-flat areas (energy is large). Therefore, this step uses the quantization parameter QP, energy Energy and thr2, thr3 to determine whether the possibility of the subjective quality problem of blur is large.

[0118] Step S8: Update the penalty coefficient rate = min (QP / thr2 * Energy / thr3, th1), and enter step S13. In the introduction of step S7, it can be seen that the intra non-angle mode is prone to blurring when the QP and energy are relatively large, that is, the bit rate is relatively insufficient (QP is large) to encode non-flat areas (energy is large). Therefore, the larger the QP and Energy, that is, the less sufficient the bit rate and the more uneven the coding area, the larger the penalty coefficient will be, which can effectively reduce the risk of blurring and improve subjective quality.

[0119] Step S9: Enter this step after completing the inter skip mode prediction in step S2. Determine whether the reconstructed pixels of the left adjacent block and the upper adjacent block are obtained. If yes, enter step S10; otherwise, enter step S13.

[0120] Step S10: Calculate the pixel difference sum value diff_left on the left and the pixel difference sum value diff_top on the top, and then proceed to step S11. The calculation method is described above and will not be repeated here.

[0121] Step S11: Determine whether the maximum value of diff_left and diff_top is greater than thr4. If so, proceed to step S12; otherwise, proceed to step S13. Wherein, thr4 is an empirical value greater than 0, and the present disclosure does not limit its specific value. The purpose of this step is to reduce the subjective quality problem of block effect, because the inter skip mode is prone to block effect when the boundary pixel difference between two adjacent blocks is large. Therefore, this step can use diff_left, diff_top and thr4 to determine whether the possibility of the subjective quality problem of block effect is high.

[0122] Step S12: Update the penalty coefficient rate = min (max (diff_left, diff_top) / thr4, th1), and enter step S13. In the introduction of step S11, it can be seen that the inter skip mode is prone to block effect problems when the difference between the boundary pixels of two adjacent blocks is large. Therefore, the larger the diff_left and diff_top are, that is, the larger the difference between the boundary pixels of two adjacent blocks is, the larger the penalty coefficient will be, which can effectively reduce the risk of block effect and improve subjective quality.

[0123] Step S13: Update the Lagrangian rate-distortion optimization formula J=rate·D+λ·R. After comparison, it can be found that compared with the existing formula, this specific embodiment adds a penalty coefficient rate to the distortion D part to reduce the risk of subjective quality problems such as artifacts, blur, and block effects, which can improve subjective quality.

[0124] After the amplification processing module is finished, the rate-distortion costs J corresponding to various modes such as intra, inter AMVP, and inter skip can be obtained respectively, and the coding mode and parameters corresponding to the minimum J are selected as the optimal choice.

[0125] This specific embodiment proposes a subjective quality-friendly video coding mode selection rate-distortion optimization algorithm. By calculating the gradient, energy, pixel difference and other features of the current block, the rate-distortion optimization logic of mode selection in the video coding process is optimized, which can significantly reduce image quality problems such as blur, artifacts, and block effects, and improve subjective quality. On the one hand, since the logic of the Lagrangian rate-distortion optimization formula is optimized during mode selection, a rate penalty coefficient is added to the distortion D part, and factors such as coding mode and quantization parameters are taken into consideration, different indicators and methods are used to measure different subjective image quality risks in the face of intra-angle prediction mode, intra-non-angle prediction mode, and interskip mode, and the penalty coefficient rate is designed in a targeted manner, so in addition to ensuring objective compression rate and other indicators, the subjective quality can also be effectively improved. On the other hand, since the Lagrangian rate-distortion optimization formula is modified by using characteristic information such as the gradient, energy, and pixel difference of the current block, compared with the solution of disabling a certain coding mode or parameter in a one-size-fits-all manner, it can effectively guarantee compression performance and suppress unnecessary bit rate and bandwidth increases.

[0126] Figure 8 is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment of the present disclosure.

[0127] Reference Figure 8 , the video encoding device 800 includes an acquisition unit 801, a selection unit 802, and an encoding unit 803.

[0128] The acquisition unit 801 may traverse multiple preset coding modes for the current block, and acquire the rate-distortion cost in each preset coding mode.

[0129] The selection unit 802 may select a coding mode for the current block from a plurality of preset coding modes based on the rate-distortion cost of each preset coding mode.

[0130] The encoding unit 803 may encode the current block based on the encoding mode of the current block.

[0131] The acquisition unit 801 can also: acquire a subjective quality indicator corresponding to the current coding mode, wherein the subjective quality indicator is related to the subjective quality problem existing in the current coding mode; calculate a penalty coefficient associated with the subjective quality indicator, wherein the penalty coefficient is positively correlated with the possibility of the existence of a subjective quality problem indicated by the subjective quality indicator; based on the penalty coefficient, amplify the original rate-distortion cost of the current coding mode to obtain the rate-distortion cost of the current coding mode.

[0132] Optionally, the acquisition unit 801 may also: calculate a penalty coefficient based on the subjective quality indicator and the preset indicator threshold when the size relationship between the subjective quality indicator and the preset indicator threshold meets a preset condition, wherein different subjective quality indicators are set with different preset indicator thresholds, and the preset condition indicates that the possibility of the existence of a subjective quality problem exceeds the possibility indicated by the preset indicator threshold.

[0133] Optionally, the acquisition unit 801 may further: calculate a candidate penalty coefficient according to the subjective quality index and a preset index threshold; and determine the smaller one between the candidate penalty coefficient and the preset coefficient threshold as the penalty coefficient.

[0134] Optionally, the acquisition unit 801 may also: when the current coding mode is an intra-frame angle coding mode, obtain the statistical value of the horizontal gradient and the statistical value of the vertical gradient of the current block; determine the subjective quality indicator corresponding to the intra-frame angle coding mode based on the statistical value of the horizontal gradient and the statistical value of the vertical gradient.

[0135] Optionally, the acquisition unit 801 may further: when the current coding mode is the intra non-angle coding mode, acquire the quantization parameter and energy of the current block as a subjective quality indicator corresponding to the intra non-angle coding mode.

[0136] Optionally, the acquisition unit 801 may also: when the current coding mode is the inter-frame skip coding mode, determine the subjective quality indicator corresponding to the inter-frame skip coding mode based on the pixel difference between the two adjacent columns of pixels between the current block and the adjacent block on the left and the pixel difference between the two adjacent rows of pixels between the current block and the adjacent block above.

[0137] Optionally, when the current coding mode is an inter-frame adaptive motion vector coding mode, the penalty coefficient is set to a value that maintains the original rate-distortion cost.

[0138] Optionally, the acquisition unit 801 may further: amplify the distortion corresponding to the current coding mode in the original rate-distortion cost based on the penalty coefficient; or amplify the entire original rate-distortion cost based on the penalty coefficient.

[0139] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] Fig. 9 A computing environment 910 is shown coupled to a user interface 950. The computing environment 910 may be part of a data processing server. The computing environment 910 includes a processor 920, a memory 930, and an input / output (I / O) interface 940.

[0141] The processor 920 generally controls the overall operation of the computing environment 910, such as operations associated with display, data acquisition, data communication, and image processing. The processor 920 may include one or more processors for executing instructions to perform all or some steps in the above method. In addition, the processor 920 may include one or more modules that facilitate the interaction between the processor 920 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0142] The memory 930 is configured to store various types of data to support the operation of the computing environment 910. The memory 930 may include predetermined software 932. Examples of such data include instructions for any application or method operating on the computing environment 910, video data sets, image data, etc. The memory 930 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0143] The I / O interface 940 provides an interface between the processor 920 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 940 may be coupled to an encoder and a decoder.

[0144] In an embodiment, a non-transitory computer-readable storage medium including, for example, a plurality of programs in a memory 930 and / or storing a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is also provided, wherein the plurality of programs can be executed by a processor 920 in a computing environment 910 to perform the above method. In one example, the plurality of programs can be executed by a processor 920 in a computing environment 910 to (for example, from Figure 2 The video encoder 20 in the computing environment 910 receives a bit stream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.), and can also be executed by the processor 920 in the computing environment 910 to perform the above-mentioned decoding method according to the received bit stream or data stream. In another example, the multiple programs can be executed by the processor 920 in the computing environment 910 to perform the above-mentioned encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more associated syntax elements, etc.) into a bit stream or data stream, and can also be executed by the processor 920 in the computing environment 910 to (e.g., to Figure 3 Alternatively, a non-transitory computer-readable storage medium may store the bit stream or data stream generated by the encoder (e.g., Figure 2 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Figure 3 A bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or associated one or more syntax elements, etc.) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0145] In an embodiment, a bit stream generated by the above encoding method or a bit stream to be decoded by the above decoding method is provided. In an embodiment, a bit stream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method is provided.

[0146] In an embodiment, a computing device is also provided, which includes: one or more processors (e.g., processor 920); and a non-transitory computer-readable storage medium or memory 930 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0147] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is also provided, wherein the bitstream includes encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method. In an embodiment, a computer program product including, for example, a plurality of programs in a memory 930 is also provided, wherein the plurality of programs can be executed by a processor 920 in a computing environment 910 for performing the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0148] In an embodiment, the computing environment 910 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0149] In an embodiment, a method for storing a bit stream is also provided, comprising: storing the bit stream on a digital storage medium, wherein the bit stream comprises encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0150] In an embodiment, a method for transmitting a bit stream generated by the above encoder is also provided. In an embodiment, a method for receiving a bit stream to be decoded by the above decoder is also provided.

[0151] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0152] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0153] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video encoding method, characterized in that: The video encoding method comprises: For the current block, traverse multiple preset coding modes to obtain the rate-distortion cost under each preset coding mode; Selecting a coding mode for a current block from the plurality of preset coding modes based on a rate-distortion cost in each preset coding mode; Encoding the current block based on the encoding mode of the current block; The rate-distortion cost of each preset coding mode is obtained, including: Acquire a subjective quality indicator corresponding to a current coding mode, wherein the subjective quality indicator is related to a subjective quality problem existing in the current coding mode; Calculating a penalty coefficient associated with the subjective quality indicator, wherein the penalty coefficient is positively correlated with the possibility of the presence of the subjective quality problem indicated by the subjective quality indicator; Based on the penalty coefficient, the original rate-distortion cost of the current coding mode is amplified to obtain the rate-distortion cost of the current coding mode.

2. The video encoding method according to claim 1, characterized in that: The calculating a penalty coefficient associated with the subjective quality indicator comprises: In the case where the magnitude relationship between the subjective quality indicator and the preset indicator threshold satisfies a preset condition, the penalty coefficient is calculated based on the subjective quality indicator and the preset indicator threshold, wherein different subjective quality indicators are set with different preset indicator thresholds, and the preset condition indicates that the possibility of the existence of the subjective quality problem exceeds the possibility indicated by the preset indicator threshold.

3. The video encoding method according to claim 2, characterized in that: The calculating the penalty coefficient according to the subjective quality indicator and the preset indicator threshold comprises: Calculating a candidate penalty coefficient according to the subjective quality indicator and the preset indicator threshold; The smaller one of the candidate penalty coefficient and a preset coefficient threshold is determined as the penalty coefficient.

4. The video encoding method according to any one of claims 1 to 3, characterized in that: The obtaining of a subjective quality indicator corresponding to the current coding mode includes: When the current coding mode is an intra-frame angle coding mode, obtaining a statistical value of a horizontal gradient and a statistical value of a vertical gradient of the current block; A subjective quality indicator corresponding to the intra-frame angle coding mode is determined according to the statistical value of the horizontal gradient and the statistical value of the vertical gradient.

5. The video encoding method according to any one of claims 1 to 3, characterized in that: The obtaining of a subjective quality indicator corresponding to the current coding mode includes: In a case where the current coding mode is an intra non-angular coding mode, a quantization parameter and energy of the current block are obtained as a subjective quality indicator corresponding to the intra non-angular coding mode.

6. The video encoding method according to any one of claims 1 to 3, characterized in that: The obtaining of a subjective quality indicator corresponding to the current coding mode includes: When the current coding mode is the inter-frame skip coding mode, the subjective quality indicator corresponding to the inter-frame skip coding mode is determined according to the pixel difference between two adjacent columns of pixels between the current block and the adjacent block on the left and the pixel difference between two adjacent rows of pixels between the current block and the adjacent block above.

7. The video encoding method according to any one of claims 1 to 3, characterized in that: In the case where the current coding mode is the inter-frame adaptive motion vector coding mode, the penalty coefficient is set to a value that maintains the original rate-distortion cost.

8. The video encoding method according to any one of claims 1 to 3, characterized in that: The amplifying the original rate-distortion cost of the current coding mode based on the penalty coefficient includes: Based on the penalty coefficient, amplifying the distortion corresponding to the current coding mode in the original rate-distortion cost; or Based on the penalty coefficient, the original rate-distortion cost is amplified as a whole.

9. A video encoding device, characterized in that: The video encoding device comprises: An acquisition unit is configured to traverse multiple preset coding modes for a current block and acquire a rate-distortion cost in each preset coding mode; A selection unit, configured to select a coding mode of a current block from the plurality of preset coding modes based on a rate-distortion cost in each preset coding mode; an encoding unit, configured to encode the current block based on the encoding mode of the current block; The acquisition unit is further configured to: Acquire a subjective quality indicator corresponding to a current coding mode, wherein the subjective quality indicator is related to a subjective quality problem existing in the current coding mode; Calculating a penalty coefficient associated with the subjective quality indicator, wherein the penalty coefficient is positively correlated with the possibility of the presence of the subjective quality problem indicated by the subjective quality indicator; Based on the penalty coefficient, the original rate-distortion cost of the current coding mode is amplified to obtain the rate-distortion cost of the current coding mode.

10. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by at least one processor, the at least one processor is prompted to execute the video encoding method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the video encoding method according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the video encoding method according to any one of claims 1 to 8 is implemented.

13. A method for generating a bit stream, characterized in that: include: A bit stream is generated according to the video encoding method according to any one of claims 1 to 8.

Images