Image coding method, image coding device, electronic equipment and storage medium
By dividing the coding unit and analyzing the gradient histogram of the image, the consistency of the main gradient direction is determined and the candidate encoding mode is screened, which solves the problem of high intra prediction calculation complexity and improves the video encoding efficiency.
Patent Information
- Application Number
- CN202510669351.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-22
- Publication Date
- 2025-07-04
AI Technical Summary
The existing video encoding technology has high computational complexity during intra prediction, resulting in low encoding efficiency and making it difficult to quickly determine the optimal encoding mode.
By dividing the coding unit of the image to be processed, the gradient histogram of the coding unit and the sub-block is calculated, the consistency of the main gradient direction is determined, the candidate coding mode set is filtered, and the target coding mode is determined according to the cost of rate distortion.
The calculation complexity of intra prediction is reduced, the image encoding efficiency is improved, unnecessary encoding mode calculation is reduced, and the encoding quality is improved.
Smart Images
Figure CN120263988A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technologies, and in particular to the fields of video compression and image encoding and decoding technologies. More specifically, the present disclosure provides an image encoding method, an image encoding apparatus, an electronic device, a storage medium, and a computer program product. Background Art
[0002] Intra prediction technology is an important part of video encoders for image processing. By predicting the pixel values in an image, the compression efficiency of the image is improved. Since there are many prediction coding modes, in order to determine the optimal coding mode, relatively complex calculations are often required, resulting in low coding efficiency. Summary of the Invention
[0003] The present disclosure provides an image encoding method, an image encoding apparatus, an electronic device, a storage medium, and a computer program product.
[0004] According to a first aspect, an image encoding method is provided. The method includes: dividing an image to be processed into a plurality of coding units; for any one of the coding units, determining a gradient histogram of the coding unit and gradient histograms of a plurality of sub-blocks in the coding unit according to the gradients of the pixels in the coding unit; determining the consistency between the main gradient directions of the plurality of sub-blocks and the main gradient direction of the coding unit according to the gradient histogram; determining a candidate coding mode set from a plurality of angular coding modes according to the consistency; and determining a target angular coding mode to encode the coding unit according to the rate-distortion cost of each angular coding mode in the candidate coding mode set.
[0005] According to a second aspect, an image encoding apparatus is provided. The apparatus includes: a dividing module configured to divide an image to be processed into a plurality of coding units; a gradient calculation module configured to, for any one of the coding units, determine a gradient histogram of the coding unit and gradient histograms of a plurality of sub-blocks in the coding unit according to the gradients of the pixels in the coding unit; a consistency determination module configured to determine the consistency between the main gradient directions of the plurality of sub-blocks and the main gradient direction of the coding unit according to the gradient histogram; a mode candidate module configured to determine a candidate coding mode set from a plurality of angular coding modes according to the consistency; and an encoding module configured to determine a target angular coding mode to encode the coding unit according to the rate-distortion cost of each angular coding mode in the candidate coding mode set.
[0006] According to a third aspect, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by the present disclosure.
[0007] According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method provided according to the present disclosure.
[0008] According to a fifth aspect, there is provided a computer program product including a computer program, the computer program being stored on at least one of a readable storage medium and an electronic device, and the computer program, when executed by a processor, implementing the method provided according to the present disclosure.
[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:
[0011] Figure 1 is a flowchart of an image encoding method according to an embodiment of the present disclosure;
[0012] Figure 2 is a schematic diagram of an angular encoding mode according to an embodiment of the present disclosure;
[0013] Figure 3 is a flowchart of an image encoding method according to another embodiment of the present disclosure;
[0014] Figure 4 is a block diagram of an image encoding apparatus according to an embodiment of the present disclosure; and
[0015] Figure 5 is a block diagram of an electronic device for an image encoding method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0016] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist in understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0017] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.
[0018] In the technical solutions of the present disclosure, the authorization or consent of the user is obtained before acquiring or collecting the user's personal information.
[0019] First, the technical terms involved in the embodiments of the present disclosure are explained.
[0020] Mode prediction refers to improving the compression efficiency by predicting the coding mode of the current coding unit. In the mode prediction for the coding mode, the encoder will consider the surrounding environment of the current coding unit, such as the coding modes of adjacent blocks, the motion information and texture information of the current block, etc., to predict the most likely coding mode of the current block. Then, according to this prediction result, the most suitable coding mode is selected for compression.
[0021] Intra prediction refers to using the correlation in the video spatial domain to predict the current pixel using the neighboring encoded pixels within the same frame image, so as to effectively remove the redundancy in the video spatial domain.
[0022] Distortion refers to the degradation of image quality caused by data compression. In order to reduce the amount of data, video coding usually compresses the image to be processed to a certain extent, and this compression may cause some details of the image to be lost, thus causing video distortion.
[0023] RDO (Rate Distortion Optimized) refers to considering the constraints of both bitrate and distortion factors when calculating the cost function, and ensuring both low distortion and bitrate.
[0024] RDOCost (Rate Distortion Optimized Cost) is the cost required to balance bitrate and distortion under the RDO strategy.
[0025] QP (quantization parameter) is used to control the bitrate and coding distortion. The larger the QP, the greater the distortion and the lower the bitrate.
[0026] SATD (Sum of Absolute Transformed Difference) is a measure of the size of the video residual signal, which is the sum of the absolute values of the coefficients after performing Hadamard transform on the residual.
[0027] STADCost (Sum of Absolute Transformed Difference_Cost) is the cost calculated in the SATD metric. STADCost can quickly evaluate the cost of the prediction mode and has a relatively low computational complexity.
[0028] In some examples, the process of intra prediction may include obtaining an image to be processed, dividing each frame of the image to be processed into a plurality of CUs (Coding Units), and predicting the pixel values in the coding units based on the correlation in the spatial domain of the image. Among them, the texture and structure information of the image will affect the prediction accuracy of different coding modes, and the most suitable coding mode for the coding unit can be determined through mode prediction for compression. Therefore, determining the coding mode adapted to the coding unit through mode prediction can improve the coding quality.
[0029] The encoder can determine the optimal coding mode by calculating the residual cost and rate-distortion cost of different modes. Among them, the calculation complexity of the residual cost is relatively low, and the calculation complexity of the rate-distortion cost is relatively high. The encoder can first calculate the residual cost of different coding modes for the coding unit for pre-screening, and then calculate the rate-distortion cost for multiple screened coding modes to obtain the coding mode adapted to the coding unit, reducing the calculation complexity of intra prediction.
[0030] The intra prediction of an image includes a relatively large number of coding modes. For example, the H.266 encoder increases the angular-directional intra prediction mode from 33 in the H265 encoder to 65. Together with the original DC (Direct Current average) and Planar (Planar weighted average) modes, there are 67 coding modes. In addition, in addition to the above 67 coding modes, the intra prediction of the H266 encoder also extends the following several: multi-reference row coding mode, wide-angle coding mode, matrix weighted average coding mode, etc. When performing intra prediction, the costs of all coding modes are calculated separately to determine the best coding mode, resulting in too high a calculation complexity of intra prediction and affecting the coding efficiency. Based on this, the present disclosure proposes an image coding method, which screens out unnecessary coding modes by performing feature analysis on information such as the texture and structure of the image to be processed, thereby reducing the number of calculations of intra prediction and improving the coding efficiency.
[0031] Next, in combination with Figures 1 - 3 the image coding method of the present disclosure will be described.
[0032] Figure 1 is a flowchart of an image coding method according to an embodiment of the present disclosure.
[0033] As Figure 1 shown, the image coding method 100 of this embodiment includes operations S110 - S150.
[0034] In operation S110, the image to be processed is divided into a plurality of coding units.
[0035] The image to be processed can be an image frame of a video file input to an encoder for processing. According to the encoding requirements, the image to be processed can be divided into multiple CUs (Coding Units), and intra-frame prediction is performed on each CU of the image to be processed respectively.
[0036] In operation S120, according to the gradients of the pixels in the coding unit, the gradient histogram of the coding unit and the gradient histograms of multiple sub-blocks in the coding unit are determined respectively.
[0037] In an embodiment of the present disclosure, the gradient of a pixel refers to the degree of change in the pixel values of the previous pixel and the next pixel of the current pixel. A sub-block is a coding area obtained by dividing the coding unit. The gradient amplitude of each pixel can be obtained by the following formula:
[0038] (1)
[0039] (2)
[0040] (3)
[0041] (4)
[0042] Among them, G x (x, y) refers to the gradient amplitude of the pixel with coordinates (x, y) in the x-axis direction. G y (x, y) refers to the gradient amplitude of the pixel with coordinates (x, y) in the y-axis direction. G(x, y) refers to the gradient amplitude of the pixel with coordinates (x, y). I(x, y) refers to the pixel value of the pixel with coordinates (x, y). θ(x, y) refers to the gradient direction of the pixel with coordinates (x, y).
[0043] The range of the gradient directions of the pixels is divided into multiple angular intervals, and the gradient amplitudes of the pixels whose gradient directions belong to the same angular interval are accumulated, and the sum of the gradient amplitudes corresponding to each angular interval can be obtained. Among them, the gradient histogram of the coding unit can refer to the distribution diagram of the gradient amplitudes in different angular intervals obtained based on all the pixel values in the coding unit, and the gradient histogram of the coding unit can characterize the gradient distribution of each pixel in the coding unit. The gradient histogram of the sub-block can refer to the distribution diagram of the gradient amplitudes in different angular intervals obtained based on all the pixel values in the sub-block of the coding unit, and the gradient histogram of the sub-block can characterize the gradient distribution of each pixel in the sub-block.
[0044] For example, the range of the gradient direction of pixels can be divided into 32 angular intervals. By calculating the gradient of each pixel in the coding unit, the gradient amplitude and direction of each pixel can be obtained. According to the gradient direction of the pixel, the belonging angular interval can be determined. For example, the total gradient amplitude of angular interval 1 is 10, the total gradient amplitude of angular interval 2 is 15, …… the total gradient amplitude of angular interval 32 is 5. According to different angular intervals and the corresponding total gradient amplitudes, the gradient histogram of the coding unit can be obtained, which characterizes the distribution of the gradient directions of the pixels in the coding unit in different angular intervals.
[0045] In operation S130, according to the gradient histogram, determine the consistency between the main gradient directions of the multiple sub-blocks and the main gradient direction of the coding unit.
[0046] In the embodiments of the present disclosure, the main gradient direction may refer to the angular interval with the largest total gradient amplitude in the gradient histogram. The consistency may refer to the number or proportion of sub-blocks having the same main gradient direction as the coding unit.
[0047] For example, by calculating the gradient histograms of the coding unit and 4 sub-blocks, the main gradient direction of the coding unit is determined to be B according to the gradient histogram, and the main gradient directions of the 4 sub-blocks are B01 - B04 respectively. By counting the number of sub-blocks consistent with the main gradient direction B, the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit can be determined. The more sub-blocks are consistent with the main gradient direction B, the higher the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit.
[0048] In operation S140, according to the consistency, determine a candidate coding mode set from multiple angular coding modes.
[0049] In the embodiments of the present disclosure, the angular coding mode refers to a coding mode based on different angular directions. In an angular interval, multiple angular coding modes with angular directions within the angular interval may be included. The consistency between the sub-block and the main gradient direction of the coding unit characterizes the consistency between the texture direction of the sub-block and the texture direction of the coding unit. A higher consistency indicates that the coding unit has relatively prominent texture features and texture directions, and the feature distribution of the pixels in each sub-block is close to the feature distribution of the coding unit; a lower consistency indicates that the texture features of the coding unit are not obvious, and there are differences in the feature distribution of the pixels in each sub-block and the feature distribution of the coding unit. The candidate coding mode set refers to a set of angular coding modes with feature distributions similar to that of the coding unit.
[0050] For example, the angle interval 1 indicates that the angle direction is between 0 degrees and 5 degrees. The angle coding mode may include coding modes with angle directions of 1 degree, 3 degrees, and 5 degrees. Therefore, the angle interval 1 corresponds to three different angle coding modes of 1 degree, 3 degrees, and 5 degrees.
[0051] In one example, if the main gradient direction of the coding unit is the angle interval 1 and the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit is relatively high, the three angle coding modes corresponding to the angle interval 1 can be determined as the candidate coding mode set.
[0052] In another example, if the main gradient direction of the coding unit is the angle interval 1, but the consistency between the main gradient direction of the sub-block and the main gradient direction of the coding unit is relatively low, it indicates that the feature distribution within the coding unit is relatively complex. If the three angle coding modes corresponding to the angle interval 1 are directly determined as the candidate coding mode set, the coding accuracy of the coding unit will be relatively low. Other angle coding modes need to be added to the candidate coding mode set for calculation in order to obtain a coding mode with higher accuracy.
[0053] In operation S150, according to the rate-distortion cost of each angle coding mode in the candidate coding mode set, a target angle coding mode is determined to encode the coding unit.
[0054] In the embodiments of the present disclosure, the rate-distortion cost (Rate Distortion Optimized Cost, RdoCost) refers to the cost required to balance the control of the bit rate and the distortion. The rate-distortion cost can truly reflect the coding quality of the coding unit using different angle coding modes. According to the rate-distortion cost, the coding mode with the highest cost performance in terms of coding quality can be determined from the candidate coding mode set as the target coding mode.
[0055] According to the embodiments of the present disclosure, by statistically analyzing the gradient values and gradient directions of the pixels within the coding unit, and based on the consistency between the main gradient direction of the coding unit and the main gradient directions of each sub-block of the coding unit, a candidate coding mode set that conforms to the texture direction of the coding unit is further screened out. By calculating the rate-distortion cost of each angle coding mode in the candidate coding mode set and selecting the target angle coding mode with the minimum rate-distortion cost to encode the coding unit, the number of calculations for the intra-frame coding mode can be reduced and the image coding efficiency can be improved.
[0056] In the embodiments of the present disclosure, determining the target angle coding mode according to the rate-distortion cost of each angle coding mode in the candidate coding mode set includes: determining the angle coding mode with the minimum rate-distortion cost from the candidate coding mode set as the target angle coding mode to encode the coding unit.
[0057] In an embodiment of the present disclosure, the smaller the rate - distortion cost corresponding to the angular coding mode, the higher the cost performance of the coding quality of the angular coding mode. By calculating the rate - distortion cost of each angular coding mode in the candidate coding mode set, the target angular coding mode with the minimum rate - distortion cost can be determined as the optimal coding mode of the coding unit.
[0058] Figure 2 It is a schematic diagram of an angular coding mode according to an embodiment of the present disclosure.
[0059] As Figure 2 shown, the arrow 201 pointing from the center of the coding unit to the edge represents the angular direction corresponding to an angular coding mode. The angular interval 210 may include angular coding modes in the direction with an included angle of 30 degrees to 60 degrees with the horizontal.
[0060] In an embodiment of the present disclosure, there are M angular coding modes; determining the gradient histogram of the coding unit and the gradient histograms of multiple sub - blocks in the coding unit according to the gradients of the pixels in the coding unit includes: determining N gradient histograms respectively distributed in N angular intervals of the coding unit according to the distribution of the gradient directions of each pixel in the coding unit in N angular intervals, where the N angular intervals are obtained by dividing the M angular directions respectively corresponding to the M angular coding modes, and N is an integer less than M; and for each sub - block, determining N gradient histograms respectively distributed in N angular intervals of the sub - block according to the distribution of the gradient directions of each pixel in the sub - block in N angular intervals.
[0061] According to an embodiment of the present disclosure, the M angular coding modes respectively correspond to M angular directions. The N angular intervals are obtained by dividing the M angular directions, and each angular interval may include angular coding modes corresponding to one or more angular directions. By statistically analyzing the gradient directions of the pixels, the sum of the gradient amplitudes of the pixels located in each angular interval can be obtained. According to the pixel values of the coding unit, the sum of the gradient amplitudes respectively corresponding to the N angular intervals can be obtained, so as to determine the gradient histograms corresponding to the N angular intervals. According to the pixel values of each sub - block, the sum of the gradient amplitudes respectively corresponding to the N angular intervals can be obtained, so as to determine the gradient histograms corresponding to the N angular intervals of each sub - block.
[0062] For example, as Figure 2 shown, the angular interval 210 corresponds to the angular direction with an included angle of 30 degrees to 60 degrees with the horizontal and includes angular coding modes corresponding to three angular directions.
[0063] In an embodiment of the present disclosure, there can be 65 angle encoding patterns and 32 angle intervals. Each angle interval includes 3 angle encoding patterns, and the angle direction of the third angle encoding pattern included in the previous angle interval coincides with the angle direction of the first angle encoding pattern included in the subsequent angle interval.
[0064] For example, angle interval 1 can include the angle encoding patterns corresponding to 1 degree, 3 degrees, and 5 degrees, and angle interval 2 can include the angle encoding patterns corresponding to 5 degrees, 7 degrees, and 9 degrees.
[0065] In an embodiment of the present disclosure, according to the distribution of the gradient directions of each pixel in the encoding unit among N angle intervals, determining the N gradient histograms respectively distributed by the encoding unit in the N angle intervals includes: for each angle interval, determining the gradient histogram corresponding to the angle interval according to the gradient magnitudes of the pixels whose gradient directions are distributed in the angle interval; and determining the N gradient histograms respectively corresponding to the N angle intervals as the N gradient histograms respectively distributed by the encoding unit in the N angle intervals.
[0066] According to an embodiment of the present disclosure, by performing an accumulation calculation on the gradient magnitudes of the pixels whose gradient directions conform to the angle interval range, the total gradient magnitude corresponding to the angle interval can be obtained, and the gradient histogram corresponding to the angle interval can be determined. Based on the gradient magnitudes and gradient directions of all pixels in the encoding unit, the total gradient magnitudes corresponding to N angle intervals can be obtained, and the gradient histograms of the N angle intervals can be determined.
[0067] For example, as Figure 2 shown, angle interval 210 corresponds to an angle direction with a horizontal included angle of 30 degrees to 60 degrees. When calculating the gradient directions of the pixels in the encoding unit, by performing an accumulation calculation on the gradient magnitudes of the pixels whose gradient directions are in the range of 30 degrees to 60 degrees, the total gradient magnitude of angle interval 210 can be obtained, and so on, and the gradient histogram composed of the total gradient magnitudes of N angle intervals can be obtained.
[0068] In an embodiment of the present disclosure, the main gradient direction can be determined in the following manner: from the N gradient histograms of the encoding unit, determining the angle interval corresponding to the gradient histogram with the largest magnitude as the main gradient direction of the encoding unit; for each sub-block, from the N gradient histograms of the sub-block, determining the angle interval corresponding to the gradient histogram with the largest magnitude as the main gradient direction of the sub-block.
[0069] According to an embodiment of the present disclosure, after determining the total gradient magnitudes corresponding to N angle intervals respectively, the main gradient direction of the encoding unit and the main gradient direction of the sub-block can be determined based on the magnitudes of the total gradient magnitudes.
[0070] For example, gradient histograms corresponding to N angular intervals are obtained based on the gradient directions and gradient magnitudes of the pixels in the coding unit. According to the gradient histogram, the angular interval 1 with the largest sum of gradient magnitudes is determined, and the angular direction range of 0 - 5 degrees corresponding to this angular interval is used as the main gradient direction of the coding unit.
[0071] For example, gradient histograms corresponding to N angular intervals are obtained based on the gradient directions and gradient magnitudes of the pixels in the sub - block. According to the gradient histogram, the angular interval 2 with the largest sum of gradient magnitudes is determined, and the angular direction range of 5 - 10 degrees corresponding to this angular interval is used as the main gradient direction of the sub - block.
[0072] In an embodiment of the present disclosure, in response to the consistency meeting the preset condition, a candidate coding mode set is determined from M angular coding methods according to the angular interval of the main gradient direction of the coding unit; and in response to the consistency not meeting the preset condition, a candidate coding mode set is determined from M angular coding methods according to the confidence of the main gradient direction of the coding unit.
[0073] According to an embodiment of the present disclosure, the consistency meeting the preset condition indicates that the characteristic distribution of the pixels within each sub - block is close to the characteristic distribution of the coding unit. The consistency meeting the preset condition may mean that the number of sub - blocks consistent with the main gradient direction of the coding unit is greater than or equal to the quantity threshold, or it may mean that the proportion of the number of sub - blocks consistent with the main gradient direction of the coding unit is greater than or equal to the proportion threshold. For example, when the number of sub - blocks is 4, the quantity threshold may be 2, or the proportion threshold is 50%. In this case, the angular coding mode that conforms to the main gradient direction of the coding unit can be added to the candidate coding mode set.
[0074] For example, the main gradient direction of the coding unit corresponds to the angular interval 1, and the angular interval 1 includes three angular coding methods. The three angular coding methods within the range of the angular interval 1 can be added to the candidate coding mode set.
[0075] According to an embodiment of the present disclosure, the consistency not meeting the preset condition indicates that the characteristic distribution of the pixels within each sub - block is different from the characteristic distribution of the coding unit. The consistency not meeting the preset condition may mean that the number of sub - blocks consistent with the main gradient direction of the coding unit is less than or equal to the quantity threshold, or it may mean that the proportion of the number of sub - blocks consistent with the main gradient direction of the coding unit is less than or equal to the proportion threshold. In this case, the confidence of the main gradient direction needs to be considered.
[0076] In an embodiment of the present disclosure, determining a candidate coding pattern set from M angular coding manners according to the confidence of the main gradient direction of a coding unit includes: determining the confidence of the main gradient direction of the coding unit according to the amplitude ratio of the gradient histogram of the main gradient direction of the coding unit to the gradient histogram of the secondary gradient direction. In response to the confidence being greater than a first threshold, determining a candidate coding pattern set from M angular coding manners according to the angular interval of the main gradient direction of the coding unit; and in response to the confidence being less than or equal to the first threshold, determining a candidate coding pattern from M angular coding manners according to the angular interval of the main gradient direction of the coding unit and the angular interval of the secondary gradient direction.
[0077] According to an embodiment of the present disclosure, the main gradient direction refers to the angular interval with the largest total gradient amplitude, and the secondary gradient direction refers to the angular interval with the second largest total gradient amplitude. The amplitude ratio may refer to the ratio of the total gradient amplitude of the main gradient direction to the total gradient amplitude of the secondary gradient direction. The confidence can characterize the credibility that the main gradient direction is the main texture direction of the coding unit.
[0078] In one example, that the confidence is greater than the first threshold indicates that the credibility that the main gradient direction of the coding unit is the main texture direction of the coding unit is relatively high, and the angular coding pattern corresponding to the main gradient direction of the coding unit can be used as the candidate coding pattern set.
[0079] In another example, that the confidence is less than or equal to the first threshold indicates that the credibility that the main gradient direction of the coding unit is the main texture direction of the coding unit is not high, and the candidate coding pattern set cannot be determined only based on the main gradient direction of the coding unit. The angular interval of the secondary gradient direction can be considered, and the candidate coding pattern set is determined according to the angular coding patterns corresponding to the main gradient direction and the secondary gradient direction.
[0080] In an embodiment of the present disclosure, in response to the confidence being less than or equal to the first threshold, determining a candidate coding pattern set from M angular coding manners according to the angular interval of the main gradient direction of the coding unit and the angular interval of the secondary gradient direction includes: in response to the confidence being less than or equal to the first threshold and greater than a second threshold, determining an initial coding pattern set from M angular coding manners according to the angular interval of the main gradient direction and the angular interval of the secondary gradient direction; in response to the confidence being less than or equal to the second threshold, determining the M angular coding manners as the initial coding pattern set.
[0081] According to an embodiment of the present disclosure, when the confidence is less than or equal to the first threshold, it indicates that the texture of the coding unit is complex, the main texture direction cannot be determined, and the candidate coding pattern set cannot be directly determined based on the main gradient direction of the coding unit.
[0082] The present disclosure divides the confidence level less than or equal to the first threshold into two cases. In one example, the confidence level is less than or equal to the first threshold and greater than the second threshold. This case indicates that the similarity between the main gradient direction of the coding unit and the image texture direction of the coding unit is not high, but the image texture direction is mainly distributed in the angular intervals of the main gradient direction and the secondary gradient direction of the coding unit. Thus, the corresponding angular coding mode can be determined based on the angular intervals of the main gradient direction and the secondary gradient direction of the coding unit as the candidate coding mode set.
[0083] According to an embodiment of the present disclosure, by the confidence level of the main gradient direction of the coding unit, the angular coding modes are screened, and the candidate coding mode set is determined according to the angular coding modes corresponding to the main gradient direction and the secondary gradient direction, and the target coding mode is determined, reducing the computational complexity of intra prediction and improving the coding efficiency.
[0084] In another example, the confidence level is less than or equal to the second threshold. This case indicates that the texture features of the coding unit are relatively complex, and the main texture direction of the image cannot be determined based on the main gradient direction and the secondary gradient direction. Therefore, there may be no optimal coding mode in the angular intervals of the main gradient direction and the secondary gradient direction. In this case, in order to maintain the accuracy of intra prediction, all angular coding modes need to be added to the initial coding mode set. The initial coding mode set can be preliminarily screened according to the residual costs of the angular coding modes in the initial coding mode set. By comparing the residual costs of the angular coding modes, the angular coding mode with a smaller residual cost is determined as the candidate coding mode set.
[0085] According to an embodiment of the present disclosure, after determining the initial coding mode set based on the confidence level, the initial coding mode set can be screened by the residual cost with a lower computational complexity to obtain the candidate coding mode set, and then the rate-distortion cost of the candidate coding mode set is calculated to obtain the target angular coding mode. Compared with calculating the rate-distortion costs of all angular coding modes, the computational complexity of the present disclosure is reduced, and the coding efficiency of the encoder is improved.
[0086] In the embodiment of the present disclosure, the values of the first threshold and the second threshold are related to the size of the coding unit and the QP. For example, by testing the influence of different thresholds on the coding quality and speed under each coding unit and each QP, the threshold with the highest cost performance can be determined.
[0087] Figure 3 It is a flowchart of an image coding method according to another embodiment of the present disclosure.
[0088] As Figure 3 shown, the image coding method of this embodiment includes operations S301 - S307 and operations S311 - S316.
[0089] In operation S301, the coding unit is divided into multiple sub-blocks.
[0090] In operation S302, the gradient histogram and the main gradient direction of the coding unit are determined.
[0091] In operation S303, the gradient histogram and the main gradient direction of each of the multiple sub-blocks are determined.
[0092] In operation S304, the consistency between the main gradient direction of each of the multiple sub-blocks and the main gradient direction of the coding unit is determined.
[0093] In operation S305, it is determined whether the consistency meets a preset condition. If the consistency between the main gradient direction of each of the multiple sub-blocks and the main gradient direction of the coding unit meets the preset condition, operation S306 is executed; if the consistency between the main gradient direction of each of the multiple sub-blocks and the main gradient direction of the coding unit does not meet the preset condition, operation S311 is executed.
[0094] The preset condition may refer to that the number of sub-blocks consistent with the main gradient direction of the coding unit is greater than or equal to a quantity threshold, or that the proportion of the number of sub-blocks consistent with the main gradient direction of the coding unit is greater than or equal to a proportion threshold.
[0095] In operation S306, according to the direction interval of the main gradient of the coding unit, a candidate coding mode set is determined.
[0096] In operation S307, the rate-distortion cost of each angular coding mode in the candidate coding mode set is calculated to determine the target angular coding mode.
[0097] In the embodiment of the present disclosure, if the consistency does not meet the preset condition, it is necessary to further calculate the confidence of the main gradient direction. Through operations S311 - S316, the confidence is compared with different thresholds to determine the candidate coding mode set.
[0098] In operation S311, the confidence of the main gradient direction of the coding unit is calculated.
[0099] In operation S312, it is determined whether the confidence is greater than a first threshold. If the confidence of the main gradient direction is greater than the first threshold, return to continue executing operation S306; if the confidence of the main gradient direction is less than the first threshold, operation S313 is executed.
[0100] In operation S313, it is determined whether the confidence is greater than a second threshold. If the confidence of the main gradient direction is less than the first threshold and greater than the second threshold, operation S314 is executed; if the confidence of the main gradient direction is less than the second threshold, operation S315 is executed.
[0101] In operation S314, an initial coding mode set is determined according to the angular intervals of the main gradient direction and the secondary gradient direction of the coding unit.
[0102] In operation S315, M angular coding modes are determined as the initial coding mode set.
[0103] In operation S316, the residual cost of each angular coding mode in the initial coding mode set is calculated to determine a candidate coding mode set.
[0104] After the candidate coding mode set is determined in operation S316, operation S307 is returned.
[0105] Figure 4 It is a block diagram of an image coding device according to an embodiment of the present disclosure.
[0106] As Figure 4 shown, the image coding device 400 includes a partitioning module 410, a gradient calculation module 420, a consistency determination module 430, a mode candidate module 440, and an encoding module 450.
[0107] The partitioning module 410 is configured to partition the image to be processed into a plurality of coding units.
[0108] The gradient calculation module 420 is configured to, for any coding unit, determine the gradient histogram of the coding unit and the gradient histograms of multiple sub-blocks in the coding unit according to the gradients of the pixels in the coding unit.
[0109] The consistency determination module 430 is configured to determine the consistency between the main gradient direction of each of the multiple sub-blocks and the main gradient direction of the coding unit according to the gradient histogram.
[0110] The mode candidate module 440 determines a candidate coding mode set from multiple angular coding modes according to the consistency.
[0111] The encoding module 450 is configured to determine a target angular coding mode to encode the coding unit according to the rate-distortion cost of each angular coding mode in the candidate coding mode set.
[0112] In the embodiment of the present disclosure, the multiple angular coding modes include M; the gradient calculation module 420 includes a first determination sub-module and a second determination sub-module.
[0113] The first determination sub-module is configured to determine N gradient histograms respectively distributed in N angular intervals for a coding unit according to the distribution of the gradient directions of each pixel in the coding unit in the N angular intervals, where the N angular intervals are obtained by dividing M angular directions respectively corresponding to M angular coding modes, and N is an integer less than M. The second determination sub-module is configured to determine, for each sub-block, N gradient histograms respectively distributed in the N angular intervals according to the distribution of the gradient directions of each pixel in the sub-block in the N angular intervals.
[0114] In an embodiment of the present disclosure, the image coding apparatus 400 further includes a first direction determination module and a second direction determination module.
[0115] The first direction determination module is configured to determine, from the N gradient histograms of the coding unit, the angular interval corresponding to the gradient histogram with the largest amplitude as the main gradient direction of the coding unit. The second direction determination module is configured to determine, for each sub-block, the angular interval corresponding to the gradient histogram with the largest amplitude as the main gradient direction of the sub-block from the N gradient histograms of the sub-block.
[0116] In an embodiment of the present disclosure, the mode candidate module 440 includes a first mode candidate sub-module and a second mode candidate sub-module.
[0117] The first mode candidate sub-module is configured to, in response to the consistency meeting a preset condition, determine a candidate coding mode set from the M angular coding modes according to the angular interval of the main gradient direction of the coding unit. The second mode candidate sub-module is configured to, in response to the consistency not meeting the preset condition, determine a candidate coding mode set from the M angular coding modes according to the confidence of the main gradient direction of the coding unit.
[0118] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0119] Figure 5 FIG. shows a schematic block diagram of an exemplary electronic device 500 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0120] As Figure 5As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0121] A plurality of components in device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disc, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0122] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 executes the various methods and processes described above, such as an image encoding method. For example, in some embodiments, the image encoding method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the image encoding method described above can be executed. Alternatively, in other embodiments, the computing unit 501 can be configured to execute the image encoding method in any other appropriate manner (e.g., by means of firmware).
[0123] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.
[0124] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.
[0125] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).
[0127] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
[0128] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of the client and the server is generated by computer programs that run on the respective computers and have a client-server relationship with each other.
[0129] It should be understood that the various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.
[0130] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of this disclosure shall be included within the protection scope of this disclosure.
Claims
1. An image encoding method, comprising: Dividing an image to be processed into a plurality of coding units; For any one of the coding units, determining a gradient histogram of the coding unit and gradient histograms of multiple sub-blocks in the coding unit respectively according to gradients of pixels in the coding unit; Determining a consistency between a main gradient direction of each of the multiple sub-blocks and a main gradient direction of the coding unit according to the gradient histogram; Determining a candidate coding mode set from a plurality of angular coding modes according to the consistency; And Determining a target angular coding mode to encode the coding unit according to rate-distortion costs of angular coding modes in the candidate coding mode set.
2. The method according to claim 1, wherein The plurality of angular coding modes include M; the determining the gradient histogram of the coding unit and gradient histograms of multiple sub-blocks in the coding unit respectively according to gradients of pixels in the coding unit includes: Determining N gradient histograms respectively distributed in N angular intervals of the coding unit according to a distribution of gradient directions of each pixel in the coding unit in the N angular intervals, where the N angular intervals are obtained by dividing M angular directions respectively corresponding to the M angular coding modes, and N is an integer less than M; and For each sub-block, determining N gradient histograms respectively distributed in the N angular intervals of the sub-block according to a distribution of gradient directions of each pixel in the sub-block in the N angular intervals.
3. The method according to claim 2, further comprising: Determining, from the N gradient histograms of the coding unit, an angular interval corresponding to the gradient histogram with the largest amplitude as the main gradient direction of the coding unit; For each sub-block, determining, from the N gradient histograms of the sub-block, an angular interval corresponding to the gradient histogram with the largest amplitude as the main gradient direction of the sub-block.
4. The method according to claim 2, wherein, The determining the candidate coding mode set from the M angular coding modes according to the consistency includes: In response to the consistency meeting a preset condition, determining the candidate coding mode set from the M angular coding modes according to the angular interval of the main gradient direction of the coding unit; and In response to the consistency not meeting the preset condition, determining the candidate coding mode set from the M angular coding modes according to a confidence level of the main gradient direction of the coding unit.
5. The method according to claim 4, wherein The determining the candidate coding mode set from the M angular coding modes according to the confidence level of the main gradient direction of the coding unit includes: Determining the confidence level of the main gradient direction of the coding unit according to an amplitude ratio between the gradient histogram of the main gradient direction of the coding unit and the gradient histogram of a secondary gradient direction; In response to the confidence level being greater than a first threshold, determining the candidate coding mode set from the M angular coding modes according to the angular interval of the main gradient direction of the coding unit; and In response to the confidence level being less than or equal to the first threshold, determining the candidate coding mode set from the M angular coding modes according to the angular interval of the main gradient direction and the angular interval of the secondary gradient direction of the coding unit.
6. The method according to claim 5, wherein In response to the confidence level being less than or equal to the first threshold, determining a candidate coding mode set from the M angular coding modes according to the angular interval of the main gradient direction and the angular interval of the secondary gradient direction of the coding unit includes: In response to the confidence level being less than or equal to the first threshold and greater than the second threshold, determining an initial coding mode set from the M angular coding modes according to the angular interval of the main gradient direction and the angular interval of the secondary gradient direction; In response to the confidence level being less than or equal to the second threshold, determining the M angular coding modes as the initial coding mode set; and Determining the candidate coding mode set from the initial coding mode set according to the residual cost of each angular coding mode in the initial coding mode set.
7. The method according to claim 2, wherein, Determining N gradient histograms respectively distributed in N angular intervals of the coding unit according to the distribution of the gradient directions of each pixel in the coding unit in N angular intervals includes: For each angular interval, determining a gradient histogram corresponding to the angular interval according to the gradient magnitude of the pixels whose gradient directions are distributed in the angular interval; and Determining the N gradient histograms respectively corresponding to the N angular intervals as the N gradient histograms respectively distributed in the N angular intervals of the coding unit.
8. The method according to claim 1, wherein Determining a target angular coding mode according to the rate-distortion cost of each angular coding mode in the candidate coding mode set includes: Determining the angular coding mode with the minimum rate-distortion cost from the candidate coding mode set as the target angular coding mode to code the coding unit.
9. An image coding device, comprising: A partitioning module, configured to partition a to-be-processed image into a plurality of coding units; A gradient calculation module, configured to, for any coding unit, determine the gradient histogram of the coding unit and the gradient histograms of respective multiple sub-blocks in the coding unit according to the gradients of the pixels in the coding unit; A consistency determination module, configured to determine the consistency between the main gradient direction of each of the multiple sub-blocks and the main gradient direction of the coding unit according to the gradient histogram; A mode candidate module, configured to determine a candidate coding mode set from multiple angular coding modes according to the consistency; And An encoding module, configured to determine a target angular coding mode to code the coding unit according to the rate-distortion cost of each angular coding mode in the candidate coding mode set.
10. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of claims 1 to 8.
11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, where the computer program is stored on at least one of a readable storage medium and an electronic device, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.