Image encoding method, image encoding device, electronic device, and storage medium
The image encoding method enhances coding efficiency by analyzing gradient histograms to select optimal coding modes based on rate-distortion cost, addressing the complexity of intraframe prediction in video encoders.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-03-04
Smart Images
Figure 2026035844000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to the field of image processing technology, particularly to the field of video compression and image encoding / decoding technology. More specifically, the present disclosure provides an image encoding method, an image encoding device, an electronic device, a storage medium, and a computer program product. [Background technology]
[0002] Intraframe prediction is an important part of image processing in video encoders. By predicting pixel values within an image, the image can be compressed more efficiently. However, since there are many coding modes to predict, determining the optimal coding mode often requires relatively complex calculations, resulting in poor coding efficiency. Summary of the Invention [Means for solving the problem]
[0003] The present disclosure provides an image encoding method, an image encoding device, an electronic device, a storage medium, and a computer program product.
[0004] According to a first aspect, an image encoding method is provided, the method including: dividing an image to be processed into a plurality of coding units; for one of the coding units, determining a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit based on gradients of pixels in the coding unit; determining a consistency between a main gradient direction of each of the plurality of sub-blocks and a main gradient direction of the coding unit based on the gradient histograms; determining a set of candidate coding modes from a plurality of angular coding modes based on the consistency; and determining a target angular coding mode based on a rate-distortion cost of each angular coding mode in the set of candidate coding modes, thereby encoding the coding unit.
[0005] According to a second aspect, an image encoding apparatus is provided, the apparatus including: a division module for dividing an image to be processed into a plurality of coding units; a gradient calculation module for, for each of the coding units, determining a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit based on gradients of pixels in the coding unit; a consistency determination module for determining a consistency between a main gradient direction of each of the plurality of sub-blocks and a main gradient direction of the coding unit based on the gradient histograms; a mode candidate module for determining a set of candidate coding modes from a plurality of angular coding modes based on the consistency; and an encoding module for determining a target angular coding mode based on a rate-distortion cost of each angular coding mode in the set of candidate coding modes and encoding the coding unit.
[0006] According to a third aspect, there is provided an electronic device, the electronic device including at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor performs a method provided by the present disclosure.
[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions that cause a computer to perform the methods provided by the present disclosure.
[0008] In a fifth aspect, a computer program product is provided, the computer program product including a computer program stored on at least one of a readable storage medium and an electronic device, the computer program performing a method provided by the present disclosure when executed by a processor.
[0009] It should be understood that the content of this section is not intended to identify key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily apparent from the following description.
[0010] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present disclosure. [Brief explanation of the drawings]
[0011] [Figure 1] FIG. 1 is a flowchart of an image encoding method according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of an angle encoding mode according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a flowchart of an image encoding method according to another embodiment of the present disclosure. [Figure 4] FIG. 4 is a block diagram of an image encoding device according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a block diagram of an electronic device relating to an image encoding method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0012] Exemplary embodiments of the present disclosure will now be described with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are shown by way of example to facilitate understanding. These are merely examples. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0013] The collection, storage, use, processing, transmission, provision and disclosure of users' personal information related to the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and are not contrary to public order and morals.
[0014] In the technical solution of the present disclosure, the permission or consent of the user is obtained before obtaining or collecting the user's personal information.
[0015] First, technical terms related to the embodiments of the present disclosure will be explained.
[0016] Mode prediction refers to improving compression efficiency by predicting the coding mode of the current coding unit. In coding mode prediction, an encoder predicts the most likely coding mode for the current block by taking into account the surrounding environment of the current coding unit, such as the coding modes of neighboring blocks, motion information, texture information, etc. Then, based on the prediction result, an optimal coding mode is selected for compression.
[0017] Intra-frame prediction refers to predicting a current pixel using adjacent coded pixels within the same frame image, exploiting the correlation in the video spatial domain to effectively remove redundancy in the video spatial domain.
[0018] Distortion refers to the degradation of image quality caused by data compression. In video coding, to reduce the amount of data, the images to be processed are usually compressed to some extent, and this compression can cause some details in the image to be lost, resulting in video distortion.
[0019] RDO (Rate Distortion Optimized) is a policy that takes into account constraints on both bitrate and distortion when calculating the cost function, ensuring low distortion and bitrate.
[0020] RDOCost (Rate Distortion Optimized Cost) is the cost required to balance bit rate and distortion in the RDO policy.
[0021] The quantization parameter (QP) is used to control the bitrate and coding distortion; the larger the QP, the greater the distortion and the lower the bitrate.
[0022] The Sum of Absolute Transformed Difference (SATD) is an index for measuring the magnitude of a video residual signal, and is calculated by summing the absolute values of coefficients after Hadamard transforming the residual.
[0023] STADCost (Sum of Absolute Transformed Difference_Cost, or residual cost) is a cost calculated using the SATD metric method. STADCost can quickly evaluate the cost of a prediction mode and has low computational complexity.
[0024] In some examples, the process of intraframe prediction may include obtaining an image to be processed, dividing each frame of the image to be processed into several coding units (CUs), and predicting pixel values of the coding units based on correlation in the image space domain. Here, texture and configuration information of the image affect the prediction accuracy of different coding modes. The optimal coding mode for the coding unit can be determined through mode prediction, and compression can be performed. Therefore, determining a coding mode that matches the coding unit through mode prediction can improve coding quality.
[0025] The encoder can determine an optimal coding mode by calculating the residual cost and the rate-distortion cost of different modes. Here, the calculation complexity of the residual cost is relatively low, but the calculation complexity of the rate-distortion cost is relatively high. The encoder can first calculate the residual cost of different coding modes for a coding unit to perform pre-screening, and then calculate the rate-distortion costs of the selected coding modes to obtain a coding mode that matches the coding unit, thereby reducing the calculation complexity of intra-frame prediction.
[0026] Image intra-frame prediction includes numerous coding modes. For example, the H.266 encoder increases the number of angle-based directional intra-frame prediction modes from 33 in the H.265 encoder to 65. Including the traditional DC (direct current average) mode and Planar (planar weighted average) mode, there are now 67 coding modes. In addition to the 67 coding modes, the H.266 encoder's intra-frame prediction also includes additional coding modes, such as a multi-reference row coding mode, a wide-angle coding mode, and a matrix-weighted average coding mode. When performing intra-frame prediction, the cost of each coding mode is calculated to determine the optimal coding mode, which increases the computational complexity of intra-frame prediction and impacts coding efficiency. Based on this, this disclosure proposes an image coding method that performs feature analysis on information such as the texture and structure of the image to be processed and filters out unnecessary coding modes, thereby reducing the number of intra-frame prediction calculations and improving coding efficiency.
[0027] Next, the image encoding method of the present disclosure will be described with reference to FIGS.
[0028] FIG. 1 is a flowchart of an image encoding method according to an embodiment of the present disclosure.
[0029] As shown in FIG. 1, the image coding method 100 of this embodiment includes operations S110 to S150.
[0030] In operation S110, the image to be processed is divided into a number of coding units.
[0031] The image to be processed may be an image frame of a video file that is input to the encoder and processed. According to encoding needs, the image to be processed is divided into multiple CUs (Coding Units), and intra-frame prediction is performed for each CU of the image to be processed.
[0032] In operation S120, a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit are determined based on the gradients of the pixels of the coding unit.
[0033] In the embodiment of the present disclosure, the gradient of a pixel refers to the degree of change in pixel value between the pixel immediately before and the pixel immediately after the current pixel. A sub-block is a coding region obtained by dividing a coding unit. The magnitude of the gradient of each pixel is calculated using the following formula:
[0034]
number
[0035] where G x (x,y) indicates the magnitude of the gradient in the x-axis direction of the pixel at coordinate (x,y). G y (x,y) refers to the magnitude of the gradient in the y-axis direction of the pixel with coordinates (x,y). G(x,y) refers to the magnitude of the gradient of the pixel with coordinates (x,y). I(x,y) refers to the pixel value of the pixel with coordinates (x,y). θ(x,y) refers to the gradient direction of the pixel with coordinates (x,y).
[0036] The range of pixel gradient directions is divided into multiple angle intervals, and the gradient magnitudes of pixels with gradient directions belonging to the same angle interval are summed to obtain the sum of gradient magnitudes corresponding to each angle interval. Here, the gradient histogram of a coding unit may refer to a distribution map of gradient magnitudes across different angle intervals based on all pixel values of the coding unit, and the gradient histogram of a coding unit may represent the gradient distribution of each pixel of the coding unit. The gradient histogram of a subblock may refer to a distribution map of gradient magnitudes across different angle intervals obtained based on all pixel values within a subblock of the coding unit, and the gradient histogram of a subblock may represent the gradient distribution of each pixel within the subblock.
[0037] For example, the range of gradient directions of a pixel can be divided into 32 angle intervals. By calculating the gradient of each pixel in the coding unit, the gradient magnitude and direction of each pixel can be obtained. The angle interval to which the pixel belongs is determined based on the gradient direction of the pixel. For example, the sum of the gradient magnitudes of angle interval 1 is 10, the sum of the gradient magnitudes of angle interval 2 is 15, ..., the sum of the gradient magnitudes of angle interval 32 is 5. Based on different angle intervals and their corresponding sums of gradient magnitudes, a gradient histogram of the coding unit can be obtained, which shows the distribution of the gradient directions of the pixels in the coding unit in different angle intervals.
[0038] In operation S130, the consistency of the main gradient direction of each of the plurality of sub-blocks with the main gradient direction of the coding unit is determined based on the gradient histogram.
[0039] In the embodiments of the present disclosure, the dominant gradient direction may refer to the angle interval in which the sum of the gradient magnitudes in the gradient histogram is the largest, and the consistency may refer to the number or percentage of sub-blocks that have the same dominant gradient direction in the coding unit.
[0040] For example, gradient histograms of a coding unit and four sub-blocks are calculated, and the main gradient direction of the coding unit is determined to be B based on the gradient histograms, and the main gradient directions of the four sub-blocks are determined to be B01 to B04, respectively. The number of sub-blocks that match the main gradient direction B is counted, and the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit is determined. The more sub-blocks that match the main gradient direction B, the higher the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit.
[0041] In operation S140, a set of candidate encoding modes is determined from the plurality of angle encoding modes based on consistency.
[0042] In the embodiments of the present disclosure, the angular coding mode refers to a coding mode based on different angular directions. An angular interval includes multiple angular coding modes whose angular directions are located within the angular interval. The consistency of the main gradient direction between a subblock and a coding unit indicates the consistency of the texture direction of the subblock with that of the coding unit. High consistency indicates that the coding unit has prominent texture features and texture directions, and the feature distribution of pixels in each subblock is close to that of the coding unit. Low consistency indicates that the texture features of the coding unit are unclear, and the feature distribution of pixels in each subblock is different from that of the coding unit. The candidate coding mode set refers to a set of angular coding modes similar to the feature distribution of the coding unit.
[0043] For example, angle interval 1 indicates that the angle direction is between 0 degrees and 5 degrees. The angle coding modes include coding modes with angle directions of 1 degree, 3 degrees, and 5 degrees. Therefore, angle interval 1 corresponds to three different angle coding modes of 1 degree, 3 degrees, and 5 degrees.
[0044] In one example, the main gradient direction of the coding unit is angle interval 1, and the main gradient direction of the sub-block is highly consistent with the main gradient direction of the coding unit. Three angle coding modes corresponding to angle interval 1 can be determined as a set of candidate coding modes.
[0045] In another example, the main gradient direction of the coding unit is in angle interval 1, but the main gradient direction of the sub-block is not consistent with the main gradient direction of the coding unit. This indicates that the feature distribution within the coding unit is relatively complex. If the three angle coding modes corresponding to angle interval 1 are directly determined as the candidate coding mode set, the coding accuracy of the coding unit will be reduced. Therefore, other angle coding modes need to be added to the candidate coding mode set and calculated to obtain a coding mode with higher accuracy.
[0046] In operation S150, a target angular coding mode is determined to code the coding unit based on the rate-distortion cost of each angular coding mode in the set of candidate coding modes.
[0047] In the embodiments of the present disclosure, the rate distortion optimized cost (RdoCost) refers to the cost required to control and balance the bit rate and distortion. The rate distortion cost truly reflects the coding quality of a coding unit using different angle coding modes. Based on the rate distortion cost, a coding mode with the best cost performance in coding quality can be determined as a target coding mode from a set of candidate coding modes.
[0048] According to an embodiment of the present disclosure, the gradient values and gradient directions of pixels in the coding unit are statistically analyzed, and a set of candidate coding modes that are compatible with the texture direction of the coding unit is further selected based on the consistency between the main gradient direction of the coding unit and the main gradient direction of each sub-block of the coding unit. By calculating the rate-distortion cost of each angle coding mode in the set of candidate coding modes, a target angle coding mode with the lowest rate-distortion cost is selected to encode the coding unit. This reduces the number of calculations for intra-frame coding modes and improves image coding efficiency.
[0049] In an embodiment of the present disclosure, determining a target angular coding mode based on the rate-distortion cost of each angular coding mode in the set of candidate coding modes includes determining an angular coding mode from the set of candidate coding modes with the lowest rate-distortion cost as the target angular coding mode and encoding the coding unit.
[0050] In embodiments of the present disclosure, the lower the rate-distortion cost corresponding to an angle coding mode, the better the cost-performance in coding quality of the angle coding mode. By calculating the rate-distortion cost of each angle coding mode in a set of candidate coding modes, a target angle coding mode with the lowest rate-distortion cost can be determined as the optimal coding mode for the coding unit.
[0051] FIG. 2 is a schematic diagram of an angle encoding mode according to an embodiment of the present disclosure.
[0052] 2, arrows 201 pointing from the center of the coding unit to the edge represent the angular directions corresponding to the angle coding modes. An angle interval 210 can include angle coding modes oriented at angles between 30 degrees and 60 degrees relative to the horizontal.
[0053] In an embodiment of the present disclosure, the multiple angular coding modes include M. Determining a gradient histogram of the coding unit and a gradient histogram of each of the multiple sub-blocks of the coding unit based on gradients of pixels of the coding unit includes: determining N gradient histograms for the coding unit, each distributed among N angle intervals, based on a distribution of gradient directions of each pixel of the coding unit in N angle intervals, where the N angle intervals are obtained by dividing the M angle directions corresponding to the M angular coding modes, respectively, where N is an integer smaller than M; and determining, for each sub-block, N gradient histograms for the sub-block, each distributed among N angle intervals, based on a distribution of gradient directions of each pixel of the sub-block in the N angle intervals.
[0054] According to an embodiment of the present disclosure, M angular encoding modes correspond to M angular directions, respectively. N angular intervals can be obtained by dividing the M angular directions, and each angular interval can include angular encoding modes corresponding to one or more angular directions. A sum of the gradient magnitudes of pixels located within each angular interval can be obtained by calculating the gradient directions of pixels. Based on the pixel values of the encoding unit, a sum of the gradient magnitudes corresponding to each of the N angular intervals can be obtained, thereby determining a gradient histogram corresponding to the N angular intervals. Based on the pixel values of each sub-block, a sum of the gradient magnitudes corresponding to each of the N angular intervals can be obtained, thereby determining a gradient histogram corresponding to each sub-block's N angular intervals.
[0055] For example, as shown in FIG. 2, angle interval 210 corresponds to angular orientations from 30 degrees to 60 degrees from horizontal and includes angle encoding modes corresponding to three angular orientations.
[0056] In an embodiment of the present disclosure, there may be 65 types of angle coding modes and 32 angle intervals, each angle interval including three angle coding modes, and the angular direction of the third angle coding mode included in the previous angle interval overlaps with the angular direction of the first angle coding mode included in the subsequent angle interval.
[0057] For example, angle interval 1 may include angle encoding modes corresponding to 1 degree, 3 degrees, and 5 degrees, and angle interval 2 may include angle encoding modes corresponding to 5 degrees, 7 degrees, and 9 degrees.
[0058] In an embodiment of the present disclosure, determining N gradient histograms respectively distributed in N angle intervals by the encoding unit based on the distribution of gradient directions of each pixel of the encoding unit in the N angle intervals includes: for each angle interval, determining a gradient histogram corresponding to the angle interval based on the gradient magnitude of the pixels whose gradient directions are distributed in the angle interval; and determining N gradient histograms respectively corresponding to the N angle intervals as the N gradient histograms respectively distributed by the encoding unit in the N angle intervals.
[0059] According to an embodiment of the present disclosure, the gradient magnitudes of pixels whose gradient directions fit within the range of an angle interval are summed to obtain a sum of the gradient magnitudes corresponding to the angle interval, and a gradient histogram corresponding to the angle interval is determined. Based on the gradient magnitudes and gradient directions of all pixels in the coding unit, the gradient magnitudes corresponding to N angle intervals are summed, and a gradient histogram for the N angle intervals is determined.
[0060] For example, as shown in Figure 2, angle interval 210 corresponds to an angle direction of 30 degrees to 60 degrees with respect to the horizontal direction. When calculating the gradient direction of a pixel in a coding unit, the gradient magnitude of pixels whose gradient direction is within the range of 30 degrees to 60 degrees can be integrated to obtain the sum of the gradient magnitude of angle interval 210. Similarly, a gradient histogram consisting of the sum of the gradient magnitudes of N angle intervals can be obtained.
[0061] In the embodiment of the present disclosure, the dominant gradient direction can be determined as follows: among the N gradient histograms of the coding unit, the angle interval corresponding to the gradient histogram with the largest magnitude is determined as the dominant gradient direction of the coding unit, and for each sub-block, among the N gradient histograms of the sub-block, the angle interval corresponding to the gradient histogram with the largest magnitude is determined as the dominant gradient direction of the sub-block.
[0062] According to an embodiment of the present disclosure, after determining the sums of gradient magnitudes corresponding to N angle intervals, the main gradient direction of the coding unit and the main gradient direction of the sub-block can be determined based on the size of the sums of gradient magnitudes.
[0063] For example, a gradient histogram corresponding to N angle intervals is obtained based on the gradient direction and gradient magnitude of each pixel in the coding unit. Based on the gradient histogram, angle interval 1 in which the sum of the gradient magnitudes is maximum is determined, and the angle direction range of 0 to 5 degrees corresponding to this angle interval is set as the main gradient direction of the coding unit.
[0064] For example, a gradient histogram corresponding to N angle intervals is obtained based on the gradient direction and gradient magnitude of each pixel in the sub-block. Based on the gradient histogram, angle interval 2 in which the sum of the gradient magnitudes is maximum is determined, and the angle direction range of 5 to 10 degrees corresponding to this angle interval is set as the main gradient direction of the sub-block.
[0065] In an embodiment of the present disclosure, in response to the consistency satisfying the preset condition, a candidate coding mode set is determined from the M angle coding modes based on the angle interval of the main gradient direction of the coding unit; and in response to the consistency not satisfying the preset condition, a candidate coding mode set is determined from the M angle coding modes according to the reliability of the main gradient direction of the coding unit.
[0066] In an embodiment of the present disclosure, the consistency satisfying a predetermined condition indicates that the feature distribution of pixels in each subblock is close to that of the coding unit. The consistency satisfying a predetermined condition may refer to the number of subblocks that match the main gradient direction of the coding unit being equal to or greater than a quantity threshold, or the proportion of the number of subblocks that match the main gradient direction of the coding unit being equal to or greater than a percentage threshold. For example, if the number of subblocks is four, the quantity threshold may be 2, or the percentage threshold may be 50%. In this case, an angle coding mode that matches the main gradient direction of the coding unit may be added to the candidate coding mode set.
[0067] For example, the main gradient direction of the coding unit corresponds to angle interval 1, and angle interval 1 includes three angle coding modes. The three angle coding modes located within angle interval 1 can be added to the candidate coding mode set.
[0068] According to an embodiment of the present disclosure, the consistency satisfies a predetermined condition indicating that the pixel feature distribution of each sub-block is different from that of the coding unit. The consistency satisfies the predetermined condition may refer to the number of sub-blocks that match the main gradient direction of the coding unit being equal to or less than a quantity threshold, or the proportion of the number of sub-blocks that match the main gradient direction of the coding unit being equal to or less than a proportion threshold. In this case, the reliability of the main gradient direction needs to be taken into consideration.
[0069] In an embodiment of the present disclosure, determining a candidate coding mode set from the M angle coding modes based on a confidence level of the main gradient direction of the coding unit includes determining the confidence level of the main gradient direction of the coding unit based on a ratio of a magnitude of a gradient histogram of the main gradient direction to a magnitude of a gradient histogram of the minor gradient direction of the coding unit. In response to the confidence level being greater than a first threshold, a candidate coding mode set is determined from the M angle coding modes based on an angle interval of the main gradient direction of the coding unit, and in response to the confidence level being equal to or less than the first threshold, a candidate coding mode is determined from the M angle coding modes based on an angle interval of the main gradient direction and an angle interval of the minor gradient direction of the coding unit.
[0070] According to an embodiment of the present disclosure, the main gradient direction refers to the angle interval with the largest sum of gradient magnitudes, and the minor gradient direction refers to the angle interval with the second largest sum of gradient magnitudes. The magnitude ratio may refer to the ratio between the sum of gradient magnitudes in the main gradient direction and the sum of magnitudes in the minor gradient direction. The confidence may indicate the reliability that the main gradient direction is the main texture direction of the coding unit.
[0071] In one example, a confidence level higher than a first threshold indicates a high confidence that the dominant gradient direction of the coding unit is the dominant texture direction of the coding unit, and the angle coding mode corresponding to the dominant gradient direction of the coding unit can be included in the set of candidate coding modes.
[0072] In another example, a confidence level equal to or less than a first threshold indicates that the dominant gradient direction of the coding unit is not highly reliable as the dominant texture direction of the coding unit. The set of candidate coding modes cannot be determined based only on the dominant gradient direction of the coding unit, but can be determined based on the angular intervals of the minor gradient directions and the angular coding modes corresponding to the dominant gradient direction and the minor gradient direction.
[0073] In an embodiment of the present disclosure, determining a candidate coding mode set from the M angular coding modes based on an angle interval of a main gradient direction and an angle interval of a minor gradient direction of the coding unit in response to the confidence being less than or equal to a first threshold includes: determining an initial coding mode set from the M angular coding modes based on an angle interval of a main gradient direction and an angle interval of a minor gradient direction in response to the confidence being less than or equal to the first threshold and greater than a second threshold; and determining the M angular coding modes as the initial coding mode set in response to the confidence being less than or equal to the second threshold.
[0074] According to an embodiment of the present disclosure, if the confidence is less than or equal to the first threshold, it indicates that the texture of the coding unit is complex, the dominant texture direction cannot be determined, and the candidate coding mode set cannot be directly determined based on the dominant gradient direction of the coding unit.
[0075] The present disclosure classifies the case where the confidence level is equal to or less than the first threshold into two cases. In one example, the confidence level is equal to or less than the first threshold and greater than the second threshold. This case indicates that the similarity between the main gradient direction of the coding unit and the image texture direction of the coding unit is not high, but the image texture direction is mainly distributed in the main gradient direction and the sub-gradient direction of the coding unit. Therefore, based on the angle interval of the main gradient direction and the angle interval of the sub-gradient direction of the coding unit, the corresponding angle coding mode can be determined as a set of candidate coding modes.
[0076] According to an embodiment of the present disclosure, the angular coding modes are selected according to the reliability of the main gradient direction of the coding unit, and the candidate coding mode set and the target coding mode are determined based on the angular coding modes corresponding to the main gradient direction and the secondary gradient direction, thereby reducing the computational complexity of intra-frame prediction and improving coding efficiency.
[0077] In another example, the confidence level is less than or equal to a second threshold. This indicates that the texture features of the coding unit are relatively complex, making it impossible to determine the main texture direction of the image based on the main gradient direction and the subgradient direction. Therefore, there may be no optimal coding mode for either the angle interval of the main gradient direction or the angle interval of the subgradient direction. In this case, to maintain the accuracy of intra-frame prediction, all angle coding modes need to be added to the initial coding mode set. The initial coding mode set can be first selected based on the residual cost of each angle coding mode in the initial coding mode set. By comparing the residual costs of each angle coding mode, angle coding modes with smaller residual costs are determined as the candidate coding mode set.
[0078] According to an embodiment of the present disclosure, after determining an initial coding mode set based on reliability, a low-complexity residual cost is calculated to filter the initial coding mode set and obtain a candidate coding mode set. Next, a rate-distortion cost of the candidate coding mode set is calculated to obtain a target angle coding mode. Compared with calculating the rate-distortion costs of all angle coding modes, the present disclosure reduces the computational complexity and improves the coding efficiency of the encoder.
[0079] In the embodiment of the present disclosure, the values of the first and second thresholds are related to the size of the coding unit and the QP. For example, for each QP in each coding unit, the impact of different thresholds on coding quality and speed can be measured to determine the most cost-effective threshold.
[0080] FIG. 3 is a flowchart of an image encoding method according to another embodiment of the present disclosure.
[0081] As shown in FIG. 3, the image coding method of this embodiment includes operations S301 to S307 and operations S311 to S316.
[0082] In operation S301, a coding unit is divided into a plurality of sub-blocks.
[0083] In operation S302, the gradient histogram and the dominant gradient direction of the coding unit are determined.
[0084] In operation S303, a gradient histogram and a dominant gradient direction for each of the plurality of sub-blocks are determined.
[0085] In operation S304, the consistency of the main gradient direction of each of the plurality of sub-blocks with the main gradient direction of the coding unit is determined.
[0086] In operation S305, it is determined whether the consistency satisfies a preset condition. If the consistency between the main gradient direction of each of the plurality of sub-blocks and the main gradient direction of the coding unit satisfies the preset condition, operation S306 is performed; if the consistency between the main gradient direction of each of the plurality of sub-blocks and the main gradient direction of the coding unit does not satisfy the preset condition, operation S311 is performed.
[0087] The predetermined condition may refer to the number of sub-blocks that match the main gradient direction of the coding unit being equal to or greater than a quantity threshold, or the proportion of the number of sub-blocks that match the main gradient direction of the coding unit being equal to or greater than a proportion threshold.
[0088] In operation S306, a set of candidate coding modes is determined based on the direction interval of the main gradient of the coding unit.
[0089] In operation S307, the rate-distortion cost of each angle coding mode in the set of candidate coding modes is calculated to determine a target angle coding mode.
[0090] In the embodiment of the present disclosure, if the consistency does not satisfy the preset condition, the confidence of the main gradient direction needs to be further calculated. In operation S311 to operation S316, the confidence is compared with different thresholds to determine a set of candidate coding modes.
[0091] In operation S311, the reliability of the main gradient direction of the coding unit is calculated.
[0092] In operation S312, it is determined whether the reliability is greater than a first threshold. If the reliability of the main gradient direction is greater than the first threshold, the process returns to and continues executing operation S306. If the reliability of the main gradient direction is less than the first threshold, the process executes operation S313.
[0093] In operation S313, it is determined whether the reliability is greater than a second threshold. If the reliability of the main gradient direction is smaller than the first threshold and greater than the second threshold, operation S314 is executed. If the reliability of the main gradient direction is less than the second threshold, operation S315 is executed.
[0094] In operation S314, an initial encoding mode set is determined based on the angle interval of the main gradient direction and the angle interval of the secondary gradient direction of the encoding unit.
[0095] In operation S315, M angle encoding modes are determined as an initial encoding mode set.
[0096] In operation S316, a residual cost of each angle coding mode in the initial coding mode set is calculated to determine a candidate coding mode set.
[0097] In operation S316, after the candidate encoding mode set is determined, the process returns to operation S307.
[0098] FIG. 4 is a block diagram of an image encoding device according to an embodiment of the present disclosure.
[0099] As shown in FIG. 4, the image encoding device 400 includes a segmentation module 410, a gradient calculation module 420, a consistency determination module 430, a mode candidate module 440, and an encoding module 450.
[0100] The division module 410 is used to divide the image to be processed into multiple coding units.
[0101] The gradient calculation module 420 is used to determine, for any one coding unit, a gradient histogram of the coding unit and a gradient histogram of each of multiple sub-blocks of the coding unit based on the gradients of the pixels of the coding unit.
[0102] The consistency determination module 430 is used to determine the consistency between the main gradient direction of each of the plurality of sub-blocks and the main gradient direction of the coding unit based on the gradient histogram.
[0103] The mode candidate module 440 determines a set of candidate coding modes from the plurality of angle coding modes based on consistency.
[0104] The encoding module 450 is used to determine a target angular coding mode to encode the coding unit based on the rate-distortion cost of each angular coding mode in the set of candidate coding modes.
[0105] In the embodiment of the present disclosure, the plurality of angle encoding modes includes M, and the gradient calculation module 420 includes a first determination sub-module and a second determination sub-module.
[0106] The first determination sub-module is used to determine N gradient histograms in which the coding units are distributed in N angle intervals, respectively, based on the distribution of gradient directions of each pixel of the coding units in the N angle intervals, where the N angle intervals are obtained by dividing M angle directions corresponding to the M angle coding modes, respectively, and N is an integer less than M. The second determination sub-module is used to determine N gradient histograms in which the subblocks are distributed in N angle intervals, respectively, based on the distribution of gradient directions of each pixel of the subblocks in the N angle intervals for each subblock.
[0107] In an embodiment of the present disclosure, the image encoding device 400 further includes a first direction determining module and a second direction determining module.
[0108] The first direction determination module is used to determine, from among the N gradient histograms of the coding unit, an angle interval corresponding to the gradient histogram with the largest magnitude as the main gradient direction of the coding unit. The second direction determination module is used for each sub-block to determine, from among the N gradient histograms of the sub-block, an angle interval corresponding to the gradient histogram with the largest magnitude as the main gradient direction of the sub-block.
[0109] In an embodiment of the present disclosure, the mode candidate module 440 includes a first mode candidate sub-module and a second mode candidate sub-module.
[0110] The first mode candidate sub-module is used to determine a candidate coding mode set from the M angle coding modes based on an angle interval of a principal gradient direction of the coding unit in response to the consistency satisfying a preset condition, and the second mode candidate sub-module is used to determine a candidate coding mode set from the M angle coding modes based on a reliability of the principal gradient direction of the coding unit in response to the consistency not satisfying the preset condition.
[0111] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0112] 5 shows an exemplary block diagram for implementing an example electronic device 500 according to an embodiment of the present disclosure. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are exemplary only and do not limit the implementation of the present disclosure as described and / or claimed herein.
[0113] 5, the device 500 includes a computing unit 501, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 can further store various programs and data necessary for the operation of the device 500. The computing unit 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0114] The components of the device 500 are connected to an I / O interface 505, which includes an input unit 506, such as a keyboard, a mouse, etc., an output unit 507, such as various types of displays, speakers, etc., a storage unit 508, such as a magnetic disk, an optical disk, etc., and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 enables the device 500 to exchange information / data with other devices via a computer network, such as the Internet, and / or various telecommunication networks.
[0115] The computing unit 501 may be any of a variety of general-purpose and / or specialized processing modules having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the methods and processes described above, such as the image encoding method. For example, in some embodiments, the image encoding method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, it may perform one or more steps of the image encoding method described above. Alternatively, in another embodiment, the computing unit 501 may be configured to perform the image encoding method in any other suitable manner (eg, via firmware).
[0116] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be embodied in one or more computer programs that can be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and that can receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
[0117] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on a device, partially on a device, partially on a device as a separate software package and partially on a remote device, or entirely on a remote device or server.
[0118] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may contain or store a program for use in or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection of one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0119] To provide interaction with a user, a computer may implement the systems and techniques described herein and include a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices may also provide interaction with a user; for example, the feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and may receive input from the user in any form (including voice input, speech input, or tactile input).
[0120] The systems and techniques described herein can be implemented in a computing system including background components (e.g., a data server), or middleware components (e.g., an application server), or front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such background, middleware, or front-end components. The components of the system can be connected to each other by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include, by way of example, a local area network (LAN), a wide area network (WAN), and the Internet.
[0121] The computer system may include clients and servers. Clients and servers are generally remote and typically interact through a communication network. The relationship of client and server is created by computer programs running on the corresponding computers and having the client-server relationship.
[0122] It should be understood that various types of flows shown above may be used, and steps may be rearranged, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present disclosure can be achieved, and the present specification is not limited thereto.
[0123] The specific embodiments described above do not limit the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.
Claims
1. Dividing an image to be processed into a plurality of coding units; For any one coding unit, determining a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit based on gradients of pixels of the coding unit; determining a consistency between a main gradient direction of each of the plurality of sub-blocks and a main gradient direction of the coding unit based on the gradient histogram; determining a set of candidate coding modes from a plurality of angle coding modes based on the consistency; determining a target angular coding mode based on a rate-distortion cost of each angular coding mode in the set of candidate coding modes to encode the coding unit; Image encoding method.
2. the plurality of angle encoding modes includes M; Determining a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit based on gradients of pixels of the coding unit includes: According to the distribution of gradient directions of each pixel of the encoding unit in N angle intervals, the encoding unit determines N gradient histograms, each distributed in N angle intervals, where the N angle intervals are obtained by dividing M angle directions corresponding to M angle encoding modes, respectively, and N is an integer smaller than M; and determining, for each sub-block, N gradient histograms in which the sub-block is distributed across the N angular intervals based on a distribution of gradient directions of each pixel of the sub-block across the N angular intervals. The method of claim 1.
3. determining an angle interval corresponding to the gradient histogram with the largest magnitude from the N gradient histograms of the coding unit as a main gradient direction of the coding unit; and determining, for each sub-block, an angle interval corresponding to the gradient histogram with the largest magnitude from the N gradient histograms of the sub-block as a main gradient direction of the sub-block. The method of claim 2.
4. determining a set of candidate coding modes from the M angle coding modes based on the consistency, determining a set of candidate coding modes from the M angular coding modes based on an angle interval of a main gradient direction of the coding unit in response to the consistency satisfying a predetermined condition; and determining a set of candidate coding modes from the M angle coding modes based on a reliability of a principal gradient direction of the coding unit in response to the consistency not satisfying the predetermined condition. The method of claim 2.
5. determining a set of candidate coding modes from the M angle coding modes based on the reliability of the principal gradient direction of the coding unit, determining a reliability of the main gradient direction of the coding unit based on a ratio of a magnitude of a gradient histogram of the main gradient direction to a magnitude of a gradient histogram of the minor gradient direction of the coding unit; determining a set of candidate coding modes from the M angular coding modes based on an angular interval of a main gradient direction of the coding unit in response to the confidence being greater than a first threshold; and determining a set of candidate coding modes from the M angular coding modes based on an angle interval of a primary gradient direction and an angle interval of a secondary gradient direction of the coding unit in response to the confidence being less than or equal to a first threshold. The method of claim 4.
6. determining a set of candidate coding modes from the M angular coding modes based on an angle interval of a primary gradient direction and an angle interval of a secondary gradient direction of the coding unit in response to the confidence being equal to or less than the first threshold, determining the initial encoding mode set from the M angular encoding modes based on an angle interval of the primary gradient direction and an angle interval of the secondary gradient direction in response to the confidence being less than or equal to the first threshold and greater than a second threshold; determining the M angle coding modes as an initial coding mode set in response to the confidence being less than or equal to the second threshold; determining the candidate coding mode set from the initial coding mode set based on a residual cost of each angle coding mode in the initial coding mode set. The method of claim 5.
7. Determining N gradient histograms in which the encoding units are respectively distributed in the N angle intervals based on the distribution of gradient directions of each pixel of the encoding units in the N angle intervals includes: determining a gradient histogram corresponding to each angle interval based on the gradient magnitude of pixels whose gradient directions are distributed in the angle interval; The encoding unit determines N gradient histograms corresponding to the N angle intervals, respectively, as N gradient histograms distributed in the N angle intervals, The method of claim 2.
8. determining a target angular coding mode based on a rate-distortion cost of each angular coding mode in the set of candidate coding modes, determining an angular coding mode from the candidate coding mode set that minimizes a rate-distortion cost as the target angular coding mode, and encoding the coding unit using the target angular coding mode. The method of claim 1.
9. a segmentation module for segmenting an image to be processed into a plurality of coding units; a gradient calculation module that determines, for any one coding unit, a gradient histogram of the coding unit and a gradient histogram of each of a plurality of sub-blocks of the coding unit based on gradients of pixels of the coding unit; a consistency determination module that determines a consistency between a main gradient direction of each of the plurality of sub-blocks and a main gradient direction of the coding unit based on the gradient histogram; a mode candidate module that determines a set of candidate coding modes from a plurality of angle coding modes based on the consistency; a coding module for determining a target angular coding mode based on a rate-distortion cost of each angular coding mode in the set of candidate coding modes to code the coding unit. Image encoding device.
10. at least one processor; a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor performs the method of any one of claims 1 to 8; electronic equipment.
11. A non-transitory computer readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 8.
12. A computer program stored on at least one of a readable storage medium and an electronic device, comprising: The computer program, when executed by a processor, performs the method according to any one of claims 1 to 8. Computer program.