Sample point adaptive compensation method, device, equipment, storage medium and product

Abstract

The embodiments of the present application provide a method, device, equipment, storage medium and product for adaptive sample compensation, the method comprising: obtaining coding reference information of a coding tree unit and a corresponding video scene; determining whether to skip sample compensation of the coding tree unit according to the coding frame type and compensation mode information; skipping sample compensation of the coding tree unit in response to the judgment result of skipping sample compensation; determining the compensation mode to be traversed according to the coding sub-block type and the video scene in response to the judgment result of not skipping sample compensation; and performing sample compensation processing on the coding tree unit based on the compensation mode to be traversed. This scheme makes a preliminary judgment on whether compensation can be skipped based on the coding frame type and compensation mode information to achieve a preliminary acceleration effect; simplifies the compensation mode from the perspective of the internal coding information and content features of the coding tree unit, saves computational overhead and efficiently improves the compensation processing speed under the premise of slightly sacrificing compensation accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video coding technology, and in particular to a sample adaptive compensation method, device, equipment, storage medium and product. Background Art

[0002] The new generation of high-efficiency video coding standards, by introducing a series of efficient coding tools, has nearly doubled the compression rate compared to the most widely used advanced video coding standards. However, the considerable compression benefits inevitably bring high computational overhead, especially the coding tools represented by sample adaptive compensation, which has caused obvious computational pressure on mobile video coding. Sample adaptive compensation is mainly born to solve the ringing effect. By traversing all pixels, classifying and compensating according to the relationship between the pixel and the surrounding pixels, it can correct similar problems such as convex corners and concave corners in a specific direction. It can be seen that sample adaptive compensation is a pixel-level distortion compensation technology, which also means high coding overhead. On mobile devices with poor computing power, post-processing processes such as sample adaptive compensation and encoding processes are serial, which will seriously affect the encoding progress and become the main bottleneck of mobile video encoding speed, which is easy to cause a series of problems such as video frame loss and freeze; even on mobile terminals with better computing power, a separate thread can be opened for post-processing processes such as sample adaptive compensation, but the absolute amount of calculation has not been reduced, which will still cause unnecessary power consumption, heat generation and other problems.

[0003] In order to reduce the absolute amount of calculation of sample adaptive compensation, the related technologies mainly adopt two solutions: overall skipping of sample adaptive compensation and mode simplification. For overall skipping, it is mainly judged based on the texture information of the current block or the existing CU (Coding Unit) mode and other coding reference information. If the current block is a large flat block, the benefit of performing sample adaptive compensation is not great, and the related calculations of sample adaptive compensation can be skipped directly. In the related technologies, the judgment conditions for overall skipping are relatively rough, and a good acceleration effect can be achieved on large blocks of size 64×64, but it is easy to miss or misselect small blocks such as 32×32 and 16×16, and the acceleration effect is not good. As for the mode simplification, the main purpose is to reduce the amount of calculation in terms of accuracy. In the pixel traversal process of sample adaptive compensation, the traversal categories, rate distortion optimization and other processes are simplified. In the related art, the mainstream encoder has realized the simplification of the rate distortion optimization process within the sample adaptive compensation, using the formula fitting method to estimate the distortion, and using empirical values ​​instead of the actual coding bits to save the calculation overhead within the mode. Therefore, by simplifying the RDO process of all modes, the acceleration effect is not obvious. In addition, in the related art, the number of boundary compensation modes is directly halved, and the content characteristics of the encoded frame are not considered, which is easy to cause compensation accuracy damage. Summary of the invention

[0004] The embodiments of the present application provide a sample point adaptive compensation method, device, equipment, storage medium and product, which solve the problem of poor acceleration effect due to rough judgment conditions for overall skipping, and serious damage to compensation accuracy due to the mode simplification strategy not combining content features and video scenes, and effectively combine the internal coding information of the coding tree unit and the content features of the coding frame. When the compensation accuracy is slightly damaged, the compensation mode is simplified, the computing overhead is saved, and the compensation processing speed is efficiently improved.

[0005] In a first aspect, an embodiment of the present application provides a sample point adaptive compensation method, the method comprising:

[0006] Acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units;

[0007] Determine whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information;

[0008] In response to a determination result of skipping sample compensation, skipping sample compensation of the coding tree unit;

[0009] In response to a determination result that sample compensation is not skipped, determining a compensation mode to be traversed according to the coding sub-block type and the video scene;

[0010] The coding tree unit is subjected to sample point compensation processing based on the compensation modes to be traversed, wherein the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0011] In a second aspect, an embodiment of the present application further provides a sample point adaptive compensation device, including:

[0012] An acquisition module configured to acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units;

[0013] A determination module, configured to determine whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information;

[0014] an overall skipping module, configured to skip sample compensation of the coding tree unit in response to a judgment result of skipping sample compensation;

[0015] A mode simplification module, configured to determine a compensation mode to be traversed according to the coding sub-block type and the video scene in response to a judgment result that sample compensation is not skipped;

[0016] The sample compensation module is configured to perform sample compensation processing on the coding tree unit based on the compensation modes to be traversed, and the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0017] In a third aspect, an embodiment of the present application further provides a sample point adaptive compensation device, the device comprising:

[0018] one or more processors;

[0019] a storage device configured to store one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the sample adaptive compensation method described in the embodiment of the present application.

[0021] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the sample adaptive compensation method described in the embodiment of the present application when executed by a computer processor.

[0022] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the sample adaptive compensation method described in the embodiment of the present application.

[0023] In the embodiment of the present application, by obtaining the coding reference information of the coding tree unit and the corresponding video scene, and then determining whether to skip the sample compensation of the coding tree unit according to the coding frame type and compensation mode information in the coding reference information; or in response to the judgment result of skipping the sample compensation, skip the sample compensation of the coding tree unit; or in response to the judgment result of not skipping the sample compensation, determine the compensation mode to be traversed according to the coding sub-block type and the video scene in the coding reference information; finally, based on the compensation mode to be traversed, the coding tree unit is subjected to sample compensation processing, wherein the number of compensation modes to be traversed is less than the number of preset compensation modes. By adopting the above technical means, the importance of the current coding tree unit can be known based on the coding frame type, and the necessity of the current coding tree unit compensation can be obtained through the compensation situation of the adjacent coding tree units based on the compensation mode information, and then a preliminary judgment is made on whether the compensation can be skipped to achieve a preliminary acceleration effect; in the case where the compensation cannot be skipped, the compensation mode is further simplified based on the coding sub-block type and the video scene, from the perspective of the internal coding information of the coding tree unit and the content characteristics of the coding frame, and the calculation overhead is saved and the compensation processing speed is efficiently improved under the premise of sacrificing the compensation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of a sample point adaptive compensation method provided in an embodiment of the present application;

[0025] Figure 2 A flowchart of a method for determining whether to skip sample compensation of a coding tree unit provided in an embodiment of the present application;

[0026] Figure 3 A flowchart of a method for determining a compensation mode to be traversed provided in an embodiment of the present application;

[0027] Figure 4 A flowchart of another method for determining a compensation mode to be traversed provided in an embodiment of the present application;

[0028] Figure 5 A flowchart of a method for performing sample compensation processing on a coding tree unit provided in an embodiment of the present application;

[0029] Figure 6 A flowchart of another sample point adaptive compensation method provided in an embodiment of the present application;

[0030] Figure 7 A structural block diagram of a sample point adaptive compensation device provided in an embodiment of the present application;

[0031] Figure 8 A schematic diagram of the structure of a sample point adaptive compensation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the embodiments of the present application, rather than to limit the embodiments of the present application. It should also be noted that, for ease of description, only parts related to the embodiments of the present application are shown in the accompanying drawings, rather than all structures.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] The sample adaptive compensation method provided in the embodiment of the present application belongs to the intermediate link of the video encoding link, wherein the video encoding link includes the acquisition frame, PTQ loop, sample adaptive compensation, and the code stream to be output. When the user initiates the video encoding demand through the video transmitter, the aforementioned video encoding link will be triggered. The specific application scenarios may include: video conferencing, video calls, indoor live broadcast and outdoor live broadcast. Taking indoor live broadcast as an example, the video transmitter calls the camera of the terminal device to collect the picture, and inputs the collected video frames frame by frame into the PTQ loop for encoding to obtain the encoded frame. Next, the sample adaptive compensation method provided in the embodiment of the present application is applied to perform sample compensation processing. Finally, the compensated encoded frame is sent back to the PTQ loop as the prediction process of the next frame on the one hand, and entropy coding is continued on the other hand to obtain the code stream to be transmitted. The video receiving terminals distributed in different locations perform video decoding and playback after receiving the code stream through network transmission. The several application scenarios listed above are only exemplary and explanatory. In actual applications, the sample adaptive compensation method can also be used in video encoding in other scenarios, and the embodiment of the present application does not limit this. The present application aims to provide a sample point adaptive compensation method, which can effectively combine the content characteristics of the coded frame and the video scene, simplify the compensation mode, save computing overhead, and efficiently improve the compensation processing speed while slightly compromising the compensation accuracy.

[0035] In the sample adaptive compensation method provided in the embodiment of the present application, the execution subject of each step may be a computer device, which refers to any electronic device with data calculation, processing and storage capabilities, such as a mobile phone, a PC (Personal Computer), a tablet computer and other terminal devices, and may also be a server and other devices, which is not limited in the embodiment of the present application.

[0036] Figure 1 A flow chart of a sample point adaptive compensation method provided in an embodiment of the present application specifically includes the following steps:

[0037] Step S101: Obtain coding reference information of a coding tree unit and a corresponding video scene, where the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units.

[0038] Among them, the video scene can be a scene divided based on the picture content of the video, the region where the video is located, and the unidirectional and bidirectional transmission of the video. For example, the video scene can include different scenes such as video conferencing, video calls, indoor live broadcasts, and outdoor live broadcasts. Among them, the coding reference information can be the coding information of the video frame after the PTQ loop processing and the compensation results of other coding tree units in the sample adaptive compensation process. Among them, the coding frame type can be divided into I frames, P frames, and B frames. The current coding frame can be further divided into multiple coding tree unit rows, each coding tree unit row includes multiple coding tree units, and each coding tree unit can be divided into coding sub-blocks of different sizes according to different division depths. Taking the 64×64 coding tree unit as an example, it can be further divided into 32×32, 16×16, 8×8, and 4×4 coding sub-blocks, and the coding sub-block type of the coding sub-block can include skip type and non-skip type. The sample adaptive compensation method provided in the embodiment of the present application performs sample compensation processing based on the coding tree unit as the basic processing unit. In one embodiment, the sample adaptive compensation method can traverse each coding tree unit of the current coding frame in sequence according to the Z-shaped scanning order, so that the adjacent coding tree unit that has completed the sample compensation and is referenced for obtaining the compensation mode information can be the coding tree unit at the left end or the coding tree unit at the top end.

[0039] Step S102: Determine whether to skip sample compensation of a coding tree unit according to the coding frame type and compensation mode information.

[0040] Among them, the coding frame types can be divided into I frames, P frames and B frames. In the video coding strategy, I frames represent key frames. The probability of I frames appearing is not high, but since I frames are reference frames for P frames and B frames, their quality directly affects the quality of subsequent frames in the same group, so I frames are very important. In order to avoid distortion of I frames due to skipping sample compensation, thereby affecting the entire video sequence, the embodiment of the present application does not skip the sample compensation processing of I frames. For P frames, it can be a reference frame of other P frames behind it, or it can be a reference frame of B frames before and after it. If more sample compensation is skipped, the introduced error may be continuously propagated in the subsequent reference chain. For B frames, it is necessary to refer to the previous I frame and the following P frame, or the P frames before and after it for prediction. Therefore, the importance of B frames is relatively the lowest, and the possibility of error propagation is small. Therefore, for P frames, relatively strict judgment conditions can be set to determine whether to skip the sample compensation of the current coding tree unit, and for B frames, relatively loose judgment conditions can be set to determine whether to skip the sample compensation of the current coding tree unit.

[0041] In one embodiment, Figure 2 A flowchart of a method for determining whether to skip sample compensation of a coding tree unit provided in an embodiment of the present application is shown in FIG. Figure 2 The specific implementation process is as follows:

[0042] Step S1021: When the coding frame type is an I frame, determine the sample compensation for skipping the coding tree unit.

[0043] Step S1022: When the coding frame type is a B frame and the compensation mode information records skip compensation, determine to skip the sample compensation of the coding tree unit.

[0044] Specifically, when it is confirmed that the coding frame type is a B frame, combined with the traversal method of the Z-shaped scanning order, the upper coding tree unit and the left coding tree unit adjacent to the current coding tree unit can be selected as reference objects, and the compensation mode information of the upper coding tree unit and the left coding tree unit is read. When the sample compensation of the upper coding tree unit and the left coding tree unit are skipped, the sample compensation of the current coding tree unit is skipped.

[0045] Step S1023: When the coding frame type is a P frame, the compensation mode information records skip compensation, and the division depth of the coding tree unit is not greater than a preset value, determine to skip sample compensation of the coding tree unit.

[0046] Specifically, when the coding frame type is confirmed to be a P frame, combined with the traversal method of the Z-shaped scanning order, the upper coding tree unit and the left coding tree unit adjacent to the current coding tree unit can be selected as reference objects, and the compensation mode information of the upper coding tree unit and the left coding tree unit can be read. When the sample compensation of the upper coding tree unit and the left coding tree unit is skipped, the division depth of the current coding tree unit is obtained, wherein the division depth can be the size specification of the coding sub-blocks further divided into the coding tree unit. For example, if the size of the coding tree unit is 64×64, when the division depth is 1, a 32×32 coding sub-block can be obtained; when the division depth is 2, 16×16 can be obtained; when the division depth is 3, 8×8 coding sub-blocks can be obtained. Therefore, based on the higher importance of P frames than B frames, relatively strict judgment conditions can be set to ensure the detailed expression of important things in the video screen. For example, for the hair in the video screen, the division depth of the coding tree unit belonging to the hair is large. In order to retain the detailed expression of the hair, it is chosen not to skip the sample compensation. For the wall in the video screen, the division depth of the coding tree unit belonging to the wall does not exceed 1, and the details of the wall are relatively insignificant, so sample compensation can be skipped. Therefore, optionally, the preset value corresponding to the division depth is set to 1, and when the division depth is not greater than 1, it is confirmed to skip the sample compensation of the current coding tree unit.

[0047] Optionally, of course, the aforementioned division depth setting is only an illustrative example, and the embodiment of the present application may also set other division depths according to other specific scene requirements, and the embodiment of the present application is not limited to this.

[0048] As described above, by combining the coding frame type and the compensation mode information of adjacent coding tree units, the reference role of different types of coding frames in video coding is fully considered, and different judgment conditions are reasonably set to achieve overall skipping of the sample compensation mode of the coding tree unit, thereby effectively speeding up the sample compensation processing speed of the coding frame.

[0049] Step S103: In response to the judgment result of skipping sample compensation, skipping sample compensation of the coding tree unit.

[0050] Step S104: In response to the judgment result that sample compensation is not skipped, a compensation mode to be traversed is determined according to the coding sub-block type and the video scene.

[0051] Among them, the preset compensation mode in step S105 can be a possible compensation mode provided by the sample adaptive compensation technology, and the compensation mode to be traversed can be a compensation mode obtained by deleting and simplifying the aforementioned possible compensation mode in the embodiment of the present application. For example, if the sample compensation of the current coding tree unit is not skipped under the judgment condition of step S102, the current coding tree unit may correspond to multiple preset compensation modes, and the optimal compensation mode is selected for compensation by traversing multiple preset compensation modes. In this way, the preset compensation mode can be simplified by combining the coding sub-block type and the video scene to achieve the effect of accelerating the compensation processing. Specifically, the sample adaptive compensation technology in the embodiment of the present application includes boundary compensation technology and sideband compensation technology. Among them, the boundary compensation technology corresponds to 4 compensation modes, namely horizontal mode, vertical mode, 135° direction mode and 45° direction mode. In each compensation mode, according to the size relationship between the current pixel and the two adjacent pixels, the one-dimensional three-pixel template is used to further subdivide the pixels into five categories, namely valley, concave angle, convex angle, peak and other scenes. Among them, the valley is a scene with obvious concave, and the concave angle is a scene with unilateral concave. Both of these scenes require positive compensation values ​​to achieve a smooth transition of three pixels, while the convex angle is a scene with unilateral convexity and the peak is a scene with obvious convexity. Both of these scenes require negative compensation values ​​to reduce the protrusion of the current pixel; the representative categories of other scenes are not obvious and are not processed specifically. Therefore, the boundary compensation technology is divided into 21 categories in different compensation modes, and different categories use different compensation values. It takes a high computational cost to complete the calculation of all categories. In addition, the sideband compensation technology is classified according to the intensity value of the current pixel. Taking the encoded 8-bit format video as an example, the sideband compensation technology will divide the 256 pixel values ​​of 0 to 255 into 32 sidebands, and each sideband contains 8 consecutive pixel values. The sideband compensation technology requires the statistical and mean calculation of the pixel values ​​in a specific sideband at the encoder end, and transmits the difference between the original pixel mean and the reconstructed pixel mean to the decoder end, where the difference is compensated to the corresponding sideband, thereby achieving the purpose of reducing pixel distortion. If both boundary compensation technology and sideband compensation technology are considered at the same time, a pixel may have 53 types of preset compensation modes, so sample adaptive compensation will become a speed bottleneck at the encoder end.

[0052] In one embodiment, the coding sub-block type includes a skip type and a non-skip type. The more the coding sub-blocks of the skip type are included in the coding tree unit, the smaller the distortion effect caused by simplifying the number of preset compensation modes to be traversed is. In addition, the preset compensation mode that can be simplified is selected in combination with the video scene, which is more in line with the content characteristics of the video frame where the coding tree unit is located. Optionally, Figure 3 A flowchart of a method for determining a compensation mode to be traversed provided in an embodiment of the present application is shown in FIG. Figure 3 The specific implementation process is as follows:

[0053] Step S1041: Count the percentage of skipped coding sub-blocks in the coding tree unit.

[0054] In one embodiment, the current coding tree unit can be divided into 256 coding sub-blocks with a size of 4×4 as a unit, and a cache matrix cuType can be generated corresponding to the 256 coding sub-blocks, wherein the element with a value of 1 in the cache matrix cuType represents that the corresponding coding sub-block type is a skip type, and the element with a value of 0 represents that the corresponding coding sub-block type is a non-skipped type. Then, by counting the element values ​​of the cache matrix, the ratio of the skipped coding sub-blocks skipRatio can be calculated, wherein the calculation formula is as follows:

[0055]

[0056] Where i is the index of the coding sub-block of size 4×4.

[0057] Step S1042: determining the compensation mode to be traversed based on the proportion value, the set skip threshold and the video scene.

[0058] Among them, the skip threshold can be the middle value of the sub-interval division of the value interval [0,1] of the proportion value, and can include one or more. The size of the skip threshold determines the interval length of the corresponding sub-interval. It can be understood that the value range of the proportion value calculated in step S1041 is [0,1], and the interval [0,1] can be divided into different sub-intervals by setting the skip threshold. The number of simplifications of the preset compensation mode corresponding to each sub-interval is different. After determining the number of simplifications of the preset compensation mode, the preset compensation mode to be simplified is selected by distinguishing different video scenes, and then the compensation mode to be traversed is determined from all the preset compensation modes.

[0059] In one embodiment, the skip threshold includes a first skip threshold and a second skip threshold, wherein the first skip threshold is smaller than the second skip threshold. Figure 4 A flowchart of another method for determining a compensation mode to be traversed provided in an embodiment of the present application, such as Figure 4 The specific implementation process is as follows:

[0060] Step S10421: when the proportion value is less than or equal to the first skip threshold, delete the first preset mode from the preset compensation modes to obtain the compensation mode to be traversed;

[0061] Step S10422: when the proportion value is greater than the first skip threshold value and less than the second skip threshold value, determine the second preset mode to be skipped based on the video scene, delete the second preset mode from the preset compensation modes to obtain the compensation mode to be traversed, and the number of the second preset modes is greater than the number of the first preset modes;

[0062] Step S10423: When the proportion value is greater than or equal to the second skip threshold, skip the sample compensation of the coding tree unit.

[0063] Among them, both the first preset mode and the second preset mode can be compensation modes that can be simplified through calculation and judgment in the preset compensation mode. The number of compensation modes of the first preset mode and the second preset mode is different, and is set according to the specific degree of simplification. It can be understood that the sample compensation of the current coding tree unit needs to be performed in both brightness and chrominance dimensions, that is, the boundary compensation mode and the sideband compensation mode need to be considered in the brightness dimension, and the boundary compensation mode and the sideband compensation mode need to be considered in the chrominance dimension. Therefore, the simplification of the preset compensation modes to be traversed also needs to consider these two dimensions at the same time. For example, if the proportion value is less than or equal to the first skip threshold Th_min, that is, it is located between the sub-interval [0, Th_min], the area occupied by the skip-type coding sub-block in the current coding tree unit is small, and a conservative level 1 mode simplification can be adopted, that is, the first preset mode is deleted to obtain the compensation mode to be traversed, where the first preset mode is all sideband compensation modes of brightness and all sideband compensation modes of chrominance. If the proportion value is greater than the first skip threshold Th_min and less than the second skip threshold Th_max, that is, it is located between the sub-intervals (Th_min, Th_max), the area occupied by the skip-type coding sub-block in the current coding tree unit is moderate, and the two-level mode simplification can be adopted, that is, the second preset mode is deleted to obtain the compensation mode to be traversed, wherein the second preset mode is all sideband compensation modes of luminance and all sideband compensation modes of chrominance, and the boundary compensation mode of luminance is reduced by half, and the boundary compensation mode of chrominance is reduced by one. Taking the scenes of video conferencing and video calls as examples, the picture texture is simple, and the 135° direction mode and 45° direction mode of the boundary compensation mode can be selected in the luminance dimension, and the 45° direction mode of the boundary compensation mode can be selected in the chrominance dimension. If the proportion value is greater than or equal to the second skip threshold Th_max, that is, it is located between the sub-intervals [Th_max, 1], the area occupied by the skip-type coding sub-block in the current coding tree unit is large, and the sample compensation processing can be skipped.

[0064] Optionally, of course, the reduction of the aforementioned specific compensation mode is only an illustrative example, and the embodiments of the present application may also perform reductions of different quantities and specific modes according to the characteristics of other specific scenarios, and the embodiments of the present application are not limited to this.

[0065] In one embodiment, the first skip threshold Th_min and the second skip threshold Th_max may be set based on empirical thresholds of experimental data, such as the first skip threshold Th min =0.5, the second skip threshold Th max =0.75;

[0066] In another embodiment, adjustments can also be made based on the empirical thresholds of the first skip threshold and the second skip threshold in combination with the details of the specific video scene. For example, when the video scene is a preset detailed video scene, the first skip threshold and the second skip threshold are increased, wherein the increase value of the first skip threshold is greater than the increase value of the second skip threshold. Taking the outdoor live broadcast scene as an example, since the outdoor live broadcast involves many scene details, such as flowers, grass, etc., you can choose to increase the first skip threshold Th_min and the second skip threshold Th_max, and reduce the interval length of the interval (Th_min, Th_max) to increase the probability of using the level 1 mode simplification, while reducing the probability of skipping the sample compensation processing, so as to achieve better compensation accuracy while improving the compensation processing speed. Specifically, the first skip threshold Th min =0.5, the second skip threshold Th max =0.75, then the interval [0,1] is divided into subintervals [0,0.5], (0.5,0.75) and [0.75,1], and the first skip threshold Th min Adjust to 0.6, the second skip threshold Th max Adjust to 0.8, and get new sub-intervals [0,0.6], (0.6,0.8) and [0.8,1], which is equivalent to increasing the first skip threshold Th at the same time. min , and the second skip threshold Th max Moreover, the interval length of (0.6, 0.8) is also smaller than that of (0.5, 0.75), so as to increase the probability of adopting the 1st level mode simplification and reduce the probability of skipping the sample point compensation processing.

[0067] Optionally, of course, the aforementioned skip threshold setting and adjustment of the skip threshold based on the scenario are merely illustrative examples, and the embodiments of the present application may also perform other settings based on data statistics and other scenario characteristics, which are not limited in the embodiments of the present application.

[0068] Step S105 : performing sample compensation processing on the coding tree unit based on the compensation modes to be traversed, wherein the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0069] For example, Figure 5 A flowchart of a method for performing sample point compensation processing on a coding tree unit provided in an embodiment of the present application is shown in FIG. Figure 5The specific implementation process is as follows:

[0070] Step S1051: Calculate the cumulative mean of compensation for each compensation category in each compensation mode to be traversed.

[0071] Specifically, after determining the compensation mode to be traversed in step S104, it is necessary to first determine the cumulative mean of compensation for each compensation category under each compensation mode. Taking the horizontal mode and the vertical mode under the brightness dimension that need to be traversed as an example, first use a horizontal one-dimensional three-pixel template to classify each pixel in the current coding tree unit, determine which compensation category the pixel belongs to, such as valley, concave angle, convex angle, peak, and others, and calculate the compensation value of the pixel. Accumulate the compensation value of each pixel corresponding to the compensation category to which it belongs, and obtain the total compensation value of the five compensation categories in the horizontal mode. Then, average the total compensation value of each compensation category for all pixels belonging to the compensation category to obtain the average compensation value, that is, obtain the cumulative mean of compensation for each compensation category. Then use a vertical one-dimensional three-pixel template to perform the same steps of traversal and accumulation to obtain the cumulative mean of compensation for the five compensation categories in the vertical mode.

[0072] Step S1052: According to a preset cost calculation formula, the total rate distortion cost corresponding to each compensation mode is calculated based on the compensation cumulative mean value, and the compensation mode with the smallest total rate distortion cost is determined as the optimal compensation mode.

[0073] Among them, the rate-distortion cost represents the minimum amount of information required to restore the output of the source with a preset distortion degree, and the average amount of information required for each source symbol. Specifically, in step S1051, the cumulative mean of compensation for each compensation category is calculated, and the distortion caused by the use of each cumulative mean of compensation can be determined according to the preset cost value calculation formula, and then the rate-distortion cost of each compensation category is determined, and finally the total rate-distortion cost corresponding to each compensation mode is determined. For example, the preset cost value calculation formula is as follows:

[0074] J=D+λ mode ·R

[0075] Where D represents the distortion value in the current coding mode, R represents the bit rate value in the current coding mode, and λ mode is the Lagrangian factor. By combining the entropy coding experience value and the estimated value, the total rate distortion cost of the five categories in the horizontal mode and the total rate distortion cost of the five categories in the vertical mode are determined, and the total rate distortion cost of different modes is compared, and the mode with the smallest cost is taken as the optimal compensation mode.

[0076] Step S1053: Add the cumulative mean values ​​of compensation of each compensation category in the optimal compensation mode to the pixel values ​​of the corresponding category of the coding tree unit.

[0077] Optionally, after completing the sample compensation of the current coding tree unit, the optimal supplementary mode corresponding to the brightness and chrominance of the current coding tree unit may be cached for use in acquiring the domain information of the next coding tree unit.

[0078] As can be seen from the above, by obtaining the coding reference information of the coding tree unit and the corresponding video scene, and then determining whether to skip the sample compensation of the coding tree unit according to the coding frame type and compensation mode information in the coding reference information; or in response to the judgment result of skipping the sample compensation, skip the sample compensation of the coding tree unit; or in response to the judgment result of not skipping the sample compensation, determine the compensation mode to be traversed according to the coding sub-block type and the video scene in the coding reference information; finally, based on the compensation mode to be traversed, the coding tree unit is subjected to sample compensation processing, wherein the number of compensation modes to be traversed is less than the number of preset compensation modes. By adopting the above technical means, the importance of the current coding tree unit can be known based on the coding frame type, and the necessity of the current coding tree unit compensation can be obtained through the compensation situation of the adjacent coding tree units based on the compensation mode information, and then a preliminary judgment is made on whether the compensation can be skipped to achieve a preliminary acceleration effect; in the case where the compensation cannot be skipped, the compensation mode is further simplified based on the coding sub-block type and the video scene, from the perspective of the internal coding information of the coding tree unit and the content characteristics of the coding frame, and the calculation overhead is saved and the compensation processing speed is efficiently improved under the premise of sacrificing the compensation accuracy. .

[0079] Figure 6 A flowchart of another sample adaptive compensation method provided in an embodiment of the present application is provided, which provides a process of filling the border of the current coding frame after sample compensation processing, such as Figure 6 As shown, including:

[0080] Step S201: Obtain coding reference information of a coding tree unit and a corresponding video scene, where the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units;

[0081] Step S202: determining whether to skip sample compensation of a coding tree unit according to the coding frame type and compensation mode information;

[0082] Step S203: In response to the judgment result of skipping sample compensation, skipping sample compensation of the coding tree unit;

[0083] Step S204: in response to the judgment result that sample compensation is not skipped, determining the compensation mode to be traversed according to the coding sub-block type and the video scene;

[0084] Step S205: Perform sample compensation processing on the coding tree unit based on the compensation modes to be traversed, where the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0085] Step S206: When the coding tree unit is a boundary unit, determine a boundary filling direction based on the position of the coding tree unit;

[0086] Step S207: Fill the boundaries of pixels in the boundary filling direction.

[0087] In one embodiment, when the current coding tree unit is the first position of the current unit row, after completing the sample compensation, the updated pixel value needs to be filled to the left boundary, and when the current coding tree unit is the last position of the current unit row, the updated pixel value needs to be filled to the right boundary, thereby ensuring that when the motion vector points to an area outside the current coding frame, the correct reference pixel value can still be obtained to avoid causing coding errors. In addition, the above-mentioned boundary filling process is not only applicable to the sample adaptive compensation method of the embodiment of the present application, but also applicable to the sample adaptive compensation method provided by any embodiment of the present application.

[0088] Figure 7 This is a structural block diagram of a sample point adaptive compensation device provided in an embodiment of the present application. The device is configured to execute the sample point adaptive compensation method provided in the above embodiment and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the device specifically includes:

[0089] The acquisition module 101 is configured to acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units;

[0090] A determination module 102, configured to determine whether to skip sample compensation of a coding tree unit according to a coding frame type and compensation mode information;

[0091] The overall skipping module 103 is configured to skip the sample compensation of the coding tree unit in response to the judgment result of skipping the sample compensation;

[0092] The mode simplification module 104 is configured to determine the compensation mode to be traversed according to the coding sub-block type and the video scene in response to the judgment result of not skipping sample compensation;

[0093] The sample compensation module 105 is configured to perform sample compensation processing on the coding tree unit based on the compensation modes to be traversed, and the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0094] It can be seen from the above scheme that by obtaining the coding reference information of the coding tree unit and the corresponding video scene, it is determined whether to skip the sample compensation of the coding tree unit according to the coding frame type and compensation mode information in the coding reference information; or in response to the judgment result of skipping the sample compensation, the sample compensation of the coding tree unit is skipped; or in response to the judgment result of not skipping the sample compensation, the compensation mode to be traversed is determined according to the coding sub-block type in the coding reference information and the video scene; finally, the coding tree unit is subjected to sample compensation processing based on the compensation mode to be traversed, wherein the number of compensation modes to be traversed is less than the number of preset compensation modes. By adopting the above technical means, the importance of the current coding tree unit can be known based on the coding frame type, and the necessity of the current coding tree unit compensation can be obtained through the compensation situation of the adjacent coding tree units based on the compensation mode information, and then a preliminary judgment is made on whether the compensation can be skipped to achieve a preliminary acceleration effect; in the case where the compensation cannot be skipped, the compensation mode is further simplified based on the coding sub-block type and the video scene, from the perspective of the internal coding information of the coding tree unit and the content characteristics of the coding frame, so as to save the calculation overhead and efficiently improve the compensation processing speed under the premise of sacrificing the compensation accuracy. .

[0095] In a possible embodiment, the determination module 102 is specifically configured as follows:

[0096] In the case where the coding frame type is an I frame, determining a sample compensation for skipping a coding tree unit;

[0097] When the coding frame type is a B frame and the compensation mode information records skip compensation, determining the sample compensation of the skip coding tree unit;

[0098] When the coding frame type is a P frame, the compensation mode information records skip compensation, and the division depth of the coding tree unit is not greater than a preset value, it is determined to skip sample compensation of the coding tree unit.

[0099] In a possible embodiment, the coding tree unit is pre-divided into a plurality of coding sub-blocks, each coding sub-block corresponds to a coding sub-block type, and the coding sub-block type includes a skip type and a non-skip type. The mode simplification module 104 is specifically configured as follows:

[0100] Count the percentage of skipped coding sub-blocks in the coding tree unit;

[0101] The compensation mode to be traversed is determined based on the ratio value, the set skip threshold and the video scene.

[0102] In a possible embodiment, the skip threshold includes a first skip threshold and a second skip threshold, the first skip threshold is less than the second skip threshold, and the mode simplification module 104 is specifically configured as follows:

[0103] When the proportion value is less than or equal to the first skip threshold, deleting the first preset mode from the preset compensation modes to obtain the compensation mode to be traversed;

[0104] When the proportion value is greater than the first skip threshold and less than the second skip threshold, determining the second preset mode to be skipped based on the video scene, deleting the second preset mode from the preset compensation modes to obtain the compensation mode to be traversed, and the number of the second preset modes is greater than the number of the first preset modes;

[0105] When the proportion value is greater than or equal to the second skip threshold, sample compensation of the coding tree unit is skipped.

[0106] In a possible embodiment, the mode simplification module 104 is further configured to:

[0107] When the video scene is a preset detail video scene, the first skip threshold and the second skip threshold are increased, wherein the increase value of the first skip threshold is greater than the increase value of the second skip threshold.

[0108] In a possible embodiment, the sample point compensation module 105 is specifically configured as follows:

[0109] Calculate the cumulative mean of compensation for each compensation category in each compensation mode to be traversed;

[0110] According to a preset cost value calculation formula, the total rate distortion cost corresponding to each compensation mode is calculated based on the compensation cumulative mean value, and the compensation mode with the smallest total rate distortion cost is determined as the optimal compensation mode;

[0111] The cumulative mean of compensation of each compensation category under the optimal compensation mode is added to the pixel value of the corresponding category of the coding tree unit.

[0112] In a possible embodiment, the apparatus further includes a boundary filling module 106 configured to:

[0113] In the case where the coding tree unit is a boundary unit, determining a boundary filling direction based on a position of the coding tree unit;

[0114] Fill the border of pixels in the border filling direction.

[0115] Figure 8 A schematic diagram of the structure of a sample point adaptive compensation device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the number of processors 201 in the device can be one or more. Figure 8 A processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device can be connected by a bus or other means. Figure 8 In the example, the connection through the bus is taken. The memory 202, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the sample point adaptive compensation method in the embodiment of the present application. The processor 201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 202, that is, the above-mentioned sample point adaptive compensation method is implemented. The input device 203 can be configured to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 204 may include a display device such as a display screen.

[0116] The embodiment of the present application further provides a non-volatile storage medium containing computer executable instructions, wherein the computer executable instructions are configured to execute a sample adaptive compensation method described in the above embodiment when executed by a computer processor, including:

[0117] Acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units;

[0118] Determine whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information;

[0119] In response to a determination result of skipping sample compensation, skipping sample compensation of the coding tree unit;

[0120] In response to a determination result that sample compensation is not skipped, determining a compensation mode to be traversed according to the coding sub-block type and the video scene;

[0121] The coding tree unit is subjected to sample point compensation processing based on the compensation modes to be traversed, wherein the number of the compensation modes to be traversed is less than the number of preset compensation modes.

[0122] It is worth noting that in the above-mentioned embodiment of the sample point adaptive compensation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not configured to limit the protection scope of the embodiments of the present application.

[0123] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is configured to enable the computer device to execute the steps of the method according to various exemplary implementations of the present application described above in this specification. For example, the computer device may execute the sample adaptive compensation method described in the embodiment of the present application. The program product may be implemented in any combination of one or more readable media.

Claims

1. Sample point adaptive compensation method, It is characterized in that include: Acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units; Determine whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information; In response to a determination result of skipping sample compensation, skipping sample compensation of the coding tree unit; In response to the judgment result of not skipping sample compensation, determining the compensation mode to be traversed according to the coding sub-block type and the video scene, wherein the method includes counting the proportion of the skipped coding sub-blocks in the coding tree unit, and determining the compensation mode to be traversed based on the proportion, the set skip threshold and the video scene; The coding tree unit is subjected to sample point compensation processing based on the compensation modes to be traversed, wherein the number of the compensation modes to be traversed is less than the number of preset compensation modes.

2. The sample point adaptive compensation method according to claim 1, It is characterized in that The determining whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information includes: When the coding frame type is an I frame, determining a sample compensation for skipping the coding tree unit; When the coding frame type is a B frame and the compensation mode information records skip compensation, determining to skip sample compensation of the coding tree unit; When the coding frame type is a P frame, skip compensation is recorded in the compensation mode information, and the division depth of the coding tree unit is not greater than a preset value, it is determined to skip sample compensation of the coding tree unit.

3. The sample point adaptive compensation method according to claim 1, It is characterized in that The skip threshold includes a first skip threshold and a second skip threshold, the first skip threshold is less than the second skip threshold, and the determining of the compensation mode to be traversed based on the proportion value, the set skip threshold and the video scene includes: When the proportion value is less than or equal to the first skip threshold, the first preset mode is deleted from the preset compensation modes to obtain a compensation mode to be traversed; When the proportion value is greater than the first skip threshold value and less than the second skip threshold value, determining a second preset mode to be skipped based on the video scene, deleting the second preset mode from the preset compensation mode to obtain a compensation mode to be traversed, and the number of the second preset modes is greater than the number of the first preset modes; When the proportion value is greater than or equal to the second skip threshold, sample compensation of the coding tree unit is skipped.

4. The sample point adaptive compensation method according to claim 3, It is characterized in that The sample point adaptive compensation method also includes: In a case where the video scene is a preset detail video scene, the first skip threshold and the second skip threshold are increased, wherein the increase value of the first skip threshold is greater than the increase value of the second skip threshold.

5. The sample point adaptive compensation method according to any one of claims 1 to 4, It is characterized in that The performing sample point compensation processing on the coding tree unit based on the compensation mode to be traversed includes: Calculate the cumulative mean of compensation for each compensation category in each compensation mode to be traversed; According to a preset cost value calculation formula, based on the compensation cumulative mean, the total rate distortion cost corresponding to each compensation mode is calculated, and the compensation mode with the smallest total rate distortion cost is determined as the optimal compensation mode; The accumulated mean values ​​of compensation of each compensation category under the optimal compensation mode are added to the pixel values ​​of the corresponding category of the coding tree unit.

6. The sample point adaptive compensation method according to any one of claims 1 to 4, It is characterized in that After performing sample point compensation processing on the coding tree unit based on the compensation mode to be traversed, the method further includes: In a case where the coding tree unit is a boundary unit, determining a boundary filling direction based on a position of the coding tree unit; The pixel boundaries are filled in the boundary filling direction.

7. Sample point adaptive compensation device, It is characterized in that include: An acquisition module configured to acquire coding reference information of a coding tree unit and a corresponding video scene, wherein the coding reference information includes a coding frame type, a coding sub-block type, and compensation mode information of adjacent coding tree units; A determination module, configured to determine whether to skip sample compensation of the coding tree unit according to the coding frame type and the compensation mode information; an overall skipping module, configured to skip sample compensation of the coding tree unit in response to a judgment result of skipping sample compensation; A mode simplification module is configured to determine the compensation mode to be traversed according to the coding sub-block type and the video scene in response to the judgment result of not skipping sample compensation, wherein the module includes counting the proportion of the coding sub-blocks of the skipped type in the coding tree unit, and determining the compensation mode to be traversed based on the proportion, the set skip threshold and the video scene; The sample compensation module is configured to perform sample compensation processing on the coding tree unit based on the compensation modes to be traversed, and the number of the compensation modes to be traversed is less than the number of preset compensation modes.

8. A sampling point adaptive compensation device, the device include: one or more processors; The storage device is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the sample adaptive compensation method according to any one of claims 1 to 6.

9. A non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the sample adaptive compensation method according to any one of claims 1 to 6 when executed by a computer processor.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the sample adaptive compensation method according to any one of claims 1 to 6 is implemented.

Images