A method, apparatus, and device for fast size determination between HEVC frames.

Abstract

This invention discloses a fast inter-frame size determination method for HEVC, comprising: determining a first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block; adjusting the first depth range based on motion and stillness characteristics, combined with the predicted depth value and the first depth range of the current CTU block; and dividing the size of the current CTU block according to the adjusted first depth range to complete the fast inter-frame size determination. This invention improves the prediction accuracy by judging the first depth range based on multiple features, while accelerating the judgment speed of the first depth range. By traversing and dividing the current CTU block according to the first depth range to complete the fast inter-frame size determination, it reduces the encoding time complexity and accelerates the HEVC encoding process.

Description

Technical Field

[0001] This invention relates to the field of video coding technology, and in particular to a fast size determination method, apparatus and device for HEVC inter-frame frames. Background Technology

[0002] HEVC, as a next-generation video coding standard, doubles the compression efficiency compared to its predecessor, H.264 / AVC. However, it also significantly increases coding complexity. This is mainly due to the introduction of new technologies in HEVC, such as large-size quadtree partitioning, higher prediction accuracy, 35 intra-frame prediction modes, and RDO and SAO techniques. The RDO process, in particular, requires calculating the coding cost of each coding size and prediction unit, selecting the one with the lowest cost as the optimal size and prediction mode. Therefore, the RDO process is the most computationally intensive part of the encoder's coding process. The extremely high coding complexity resulting from the new technologies introduced in HEVC makes it impractical for real-time applications such as video conferencing.

[0003] Existing fast inter-frame size decision methods primarily estimate the depth prediction range, mode selection, and early CU termination of the current CTU from the perspective of spatiotemporal correlation, but do not consider the texture features and motion / static characteristics of the current CTU block itself. Therefore, this approach has limitations in CTU depth decision and reducing coding complexity. Summary of the Invention

[0004] This invention provides a fast inter-frame size determination method and apparatus for HEVC, which solves the technical problem that existing fast inter-frame size determination methods do not estimate the CTU depth prediction range from multiple aspects such as the texture features, motion and stillness features and spatiotemporal correlation of the CTU block itself.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a fast size determination method for HEVC inter-frame frames, comprising:

[0006] Based on the texture complexity and spatiotemporal correlation of the current CTU block, and combined with the predicted depth value and preliminary depth range of the current CTU block, the first depth range is determined.

[0007] Based on the motion and stillness characteristics, and combined with the predicted depth value of the current CTU block and the first depth range, the first depth range is adjusted;

[0008] The size of the current CTU block is divided according to the adjusted first depth range to complete the fast inter-frame size determination.

[0009] This invention first determines the first depth range of a CTU block using multiple features, including texture complexity and temporal correlation. Then, it adjusts the initially determined first depth range based on motion characteristics. This multi-feature-based determination of the first depth range improves prediction accuracy and speeds up the determination of the first depth range. The current CTU block is traversed and divided according to the first depth range to complete fast inter-frame size determination, thereby reducing encoding time complexity and accelerating the HEVC encoding process.

[0010] Furthermore, the determination of the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block, specifically involves:

[0011] Determine whether the current CTU block conforms to the texture characteristics. If it does, determine the texture complexity of the current CTU block and determine the first depth range based on the predicted depth value and the initial depth range of the current CTU block.

[0012] If the conditions are not met, a temporal co-located CTU block is obtained, and a first depth range is determined based on the depth correlation of the temporal co-located CTU block, combined with the predicted depth value of the current CTU block and the preliminary depth range.

[0013] This invention improves the speed of determining the first depth range by combining texture characteristics and spatiotemporal correlation. At the same time, it makes the prediction more accurate by determining the first depth range of the CTU block based on multiple angles and multiple constraints, thus accelerating the subsequent HEVC encoding process.

[0014] Further, the step of acquiring the temporal co-location CTU block and, based on the temporal co-location CTU block, predicting the depth value of the current CTU block and the preliminary depth range, obtaining the first depth range specifically involves:

[0015] Obtain the first average depth value of the time-domain co-located CTU, the second average depth value of the adjacent left CTU of the time-domain co-located CTU, and the third average depth value of the spatially adjacent CTU;

[0016] Based on the first average depth, the second average depth, and the third average depth, combined with the predicted depth value of the current CTU block and the preliminary depth range, the first depth range is determined.

[0017] This invention determines the depth values ​​of adjacent CTUs in the spatiotemporal domain through spatiotemporal correlation, and uses this to predict the first depth range of the current CTU block. Furthermore, it refines the constraints on the first depth range of the CTU block to improve prediction accuracy.

[0018] Furthermore, before determining whether the current CTU block conforms to texture characteristics, the method further includes:

[0019] The gradients of the current CTU block in the horizontal and vertical directions are calculated using the Roberts gradient operator.

[0020] The average gradient of the current CTU block is calculated based on the gradient in the horizontal direction and the gradient in the vertical direction, and the texture complexity of the current CTU block is determined based on the average gradient.

[0021] Further, determining the texture complexity of the current CTU block based on the average gradient specifically involves:

[0022] A first texture complexity threshold and a second texture complexity threshold are preset. If the average gradient of the current CTU block is less than the first texture complexity threshold, then the current CTU block is a texture flat block.

[0023] If the average gradient of the current CTU block is greater than the second texture complexity threshold, then the current CTU block is a texture complex block;

[0024] If the average gradient of the current CTU is greater than the first texture complexity threshold and less than the second texture complexity threshold, then the current CTU block is a normal texture block.

[0025] This invention calculates the average gradient of the current CTU block using the Roberts gradient operator and determines the texture complexity of the CTU block, thereby dividing the CTU block and facilitating the subsequent determination of the first depth range based on the texture complexity of the CTU block.

[0026] Furthermore, the adjustment of the first depth range based on the motion-stationary characteristics, combined with the predicted depth value of the current CTU block and the first depth range, specifically involves:

[0027] Determine the motion and static characteristics of the current CTU block, including stationary blocks, moving blocks, and violently moving blocks;

[0028] If the current CTU is a block with intense movement, and the minimum depth value of the first depth range is less than or equal to 1, then the minimum depth value of the first depth range is incremented by 1;

[0029] If the current CTU is a stationary block, and the maximum depth value of the first depth range is greater than 2, while the predicted depth value is less than 2, then the maximum depth value of the first depth range is reduced by 1.

[0030] Furthermore, before determining the motion-stationary characteristics of the current CTU block, which include stationary blocks, moving blocks, and violently moving blocks, the method further includes:

[0031] The difference map of the current CTU downsampled block is obtained according to the downsampling method and the frame difference method, and the first difference threshold of the difference map is calculated according to the OTSU method.

[0032] The difference map is thresholded according to the first difference threshold to obtain the motion image of the current CTU downsampling block;

[0033] The number of moving pixels in the current CTU downsampling block is obtained from the motion image, and the motion and stillness characteristics of the current CTU block are determined based on the number of moving pixels and the first motion threshold.

[0034] This invention obtains the motion image of the current CTU block through the frame difference method and the OTSU method, and classifies the CTU block according to the motion and stillness characteristics to further refine the constraints and further adjust the first depth range of the CTU, thereby improving the prediction accuracy of the first depth range from multiple angles.

[0035] Furthermore, before determining the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block, the method further includes:

[0036] The predicted depth value and preliminary depth range of the current CTU block are predicted based on the spatiotemporal adjacent CTU blocks of the current CTU. The spatiotemporal adjacent CTU blocks include temporal adjacent CTUs and spatial adjacent CTUs.

[0037] In a second aspect, the present invention provides a fast size determination device for HEVC inter-frame frames, comprising: a first depth range determination module, a first depth range adjustment module, and a size determination module;

[0038] The first depth range determination module is used to determine the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block;

[0039] The first depth range adjustment module is used to adjust the first depth range based on the motion and stillness characteristics, combined with the predicted depth value of the current CTU block and the first depth range;

[0040] The size determination module is used to divide the current CTU block size according to the adjusted first depth range, and complete the fast inter-frame size determination.

[0041] Thirdly, embodiments of the present invention provide a computer device, including: a processor, a communication interface, and a memory, wherein the processor, the communication interface, and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to execute the HEVC inter-frame fast size determination method. Attached Figure Description

[0042] Figure 1This is a schematic flowchart of a fast size determination method for HEVC inter-frame frames provided in an embodiment of the present invention;

[0043] Figure 2 A schematic diagram illustrating the location of adjacent CTU blocks in the spatiotemporal domain of the HEVC inter-frame fast size determination method provided in an embodiment of the present invention.

[0044] Figure 3 A schematic diagram of a HEVC inter-frame fast size determination device provided in an embodiment of the present invention;

[0045] Figure 4 This is another flowchart illustrating the HEVC inter-frame fast size determination method provided in an embodiment of the present invention.

[0046] Figure 5 This is another schematic diagram of the HEVC inter-frame fast size decision device provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1

[0049] Please refer to Figure 1 , Figure 1 A flowchart illustrating a fast size determination method for HEVC inter-frame frames provided in an embodiment of the present invention includes steps 101 to 103, as detailed below:

[0050] Step 101: Based on the texture complexity and spatiotemporal correlation of the current CTU block, and combined with the predicted depth value and preliminary depth range of the current CTU block, determine the first depth range;

[0051] In this embodiment, before determining the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block, the method further includes:

[0052] The predicted depth value and preliminary depth range of the current CTU block are predicted based on the spatiotemporal adjacent CTU blocks of the current CTU. The spatiotemporal adjacent CTU blocks include temporal adjacent CTUs and spatial adjacent CTUs.

[0053] Please refer to Figure 2 , Figure 2A schematic diagram illustrating the location of adjacent CTU blocks in the spatiotemporal domain of the HEVC inter-frame fast size determination method provided in an embodiment of the present invention.

[0054] In this embodiment, there is a strong spatiotemporal correlation between pixels in each frame of the video. This correlation can be used to predict the prediction depth of the current CTU by using the coding depths of spatially adjacent encoded CTUs and temporally encoded co-located CTUs. Spatially adjacent encoded CTU blocks are divided according to their positional relationship: the left CTU is CUL, the upper CTU is CUA, the upper-left CTU is CULA, and the upper-right CTU is CURA; the temporally encoded co-located CTU is CUco l.

[0055] In this embodiment, the minimum and maximum depths of all existing spatiotemporally adjacent CTU blocks of the current CTU block are used as the initial depth range of the current CTU block. Here, the temporally adjacent CTU refers to the co-located CTU, i.e., CUcol, in the temporal reference frame, and the spatially adjacent CTU refers to the spatially adjacent left CTU (CUL) and upper CTU (CUA).

[0056] In this embodiment, the initial depth range is divided as follows:

[0057]

[0058] Among them, D min and D max Represents the minimum and maximum depth values ​​among all existing spatiotemporally adjacent CTUs, and DR represents the initial depth range of the current CTU. If none of the current CTU's spatiotemporally adjacent CTUs exist, then DR is [0,3].

[0059] In this embodiment, utilizing the spatiotemporal correlation between CTUs, when using adjacent CTUs to predict the depth of the current CTU, the depth value of the left CTU block adopts the average depth value of the right half of the CTU block of CUL; similarly, the depth value of the upper CTU adopts the average depth value of the lower half of the CTU block of CUA; since the upper left and upper right blocks have low correlation with the current block, the depth values ​​of the upper left CTU and the upper right CTU adopt the depth values ​​of the CUs (i.e., CULA and CURA) that are closest to the current CTU; in the temporal domain, the average depth value of the temporally co-located CTU (i.e., CUcol) is used for prediction.

[0060] In this embodiment, the current CTU's predicted depth range is... pre The specific calculation formula is as follows:

[0061]

[0062] Where N = 5, ω i The weighting factor for adjacent CTUs in the spatiotemporal domain, level i This represents the encoding depth value of adjacent CTUs in the spatiotemporal domain. Weighting factor ω i It is set based on the correlation between the current CTU and neighboring CTUs in the spatiotemporal domain, as well as the motion and static characteristics of the current CTU.

[0063] In this embodiment, if the current CTU is not a rapidly moving block: because the current CTU has a strong temporal correlation with the time-domain co-located CTU, the ω corresponding to the time-domain LCU... i Set it to 0.2; because the left and top LCUs are more correlated with the current LCU, ω i Set them to 0.25 respectively; the correlation between the top-left LCU and the top-right LCU and the current LCU is smaller, ω i Set them to 0.15 respectively.

[0064] In this embodiment, if the current CTU is a rapidly moving block: because the temporal correlation between the current CTU and its temporal counterpart CTU is weak, the depth prediction value of the current CTU does not use the temporal block. Because the left and upper LCUs have a greater correlation with the current LCU, ω i Set them to 0.3 respectively; the correlation between the top-left LCU and the top-right LCU and the current LCU is smaller, ω i Set them to 0.2 respectively.

[0065] In this embodiment, determining the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block, specifically involves:

[0066] Determine whether the current CTU block conforms to the texture characteristics. If it does, determine the texture complexity of the current CTU block and determine the first depth range based on the predicted depth value and the initial depth range of the current CTU block.

[0067] If the conditions are not met, a temporal co-located CTU block is obtained, and a first depth range is determined based on the depth correlation of the temporal co-located CTU block, combined with the predicted depth value of the current CTU block and the preliminary depth range.

[0068] In this embodiment, the present invention uses texture characteristics and spatiotemporal correlation to jointly determine the first depth range, thereby improving the speed of determining the first depth range. At the same time, the first depth range of the CTU block is determined based on multiple angles and multiple constraints, making the prediction more accurate and accelerating the subsequent HEVC encoding process.

[0069] In this embodiment, before determining whether the current CTU block conforms to texture characteristics, the method further includes:

[0070] The gradients of the current CTU block in the horizontal and vertical directions are calculated using the Roberts gradient operator.

[0071] The average gradient of the current CTU block is calculated based on the gradient in the horizontal direction and the gradient in the vertical direction, and the texture complexity of the current CTU block is determined based on the average gradient.

[0072] In this embodiment, the Roberts gradient operator is used to measure the texture complexity of the CTU block. The formula for calculating the average gradient of the current CTU block based on the Roberts gradient operator is as follows:

[0073] g x =f(x+1,y)-f(x,y) (3)

[0074] g y =f(x,y+1)-f(x,y) (4)

[0075] G(x,y)=|g x +g y | (5)

[0076]

[0077] Where f(x,y) represents the pixel value at position (x,y), and g x Let g represent the horizontal gradient of (x, y). y Let G(x,y) represent the gradient in the vertical direction at (x,y), and let G(x,y) represent the gradient at (x,y). avg This represents the average gradient value of the current CTU.

[0078] In this embodiment, determining the texture complexity of the current CTU block based on the average gradient specifically involves:

[0079] A first texture complexity threshold and a second texture complexity threshold are preset. If the average gradient of the current CTU block is less than the first texture complexity threshold, then the current CTU block is a texture flat block.

[0080] If the average gradient of the current CTU block is greater than the second texture complexity threshold, then the current CTU block is a texture complex block;

[0081] If the average gradient of the current CTU is greater than the first texture complexity threshold and less than the second texture complexity threshold, then the current CTU block is a normal texture block.

[0082] In this embodiment, the formula for dividing texture complexity is as follows:

[0083]

[0084] Where TC represents the texture complexity of the current CTU block, Simple represents that the current CTU is a texture flat block, Common represents that the current CTU is a texture normal block, Complex represents that the current CTU is a texture complex block, Thr1 represents the first texture complexity threshold, and Thr2 represents the first texture complexity threshold.

[0085] In this embodiment, Thr1 is 5 and Thr2 is 10.

[0086] In this embodiment, the average gradient of the current CTU block is calculated using the Roberts gradient operator, and the texture complexity of the CTU block is determined. This is used to divide the CTU block, which facilitates the subsequent determination of the first depth range based on the texture complexity of the CTU block.

[0087] In this embodiment, determining whether the current CTU block conforms to texture characteristics, and if so, determining the texture complexity of the current CTU block, and determining the first depth range based on the predicted depth value and preliminary depth range of the current CTU block, specifically involves:

[0088] If the current CTU is a complex texture block, and its predicted depth value is Depth pre If the value is greater than 2, then the first depth range of the current CTU is [2,3].

[0089] If the current CTU is a simple texture block, and its predicted depth value is Depth pre If the value is less than 1, then the first depth range of the current CTU is [0,1].

[0090] If the current CTU is a complex texture block and the minimum depth value in the initial depth traversal range is less than or equal to 1, then depth expansion is performed, and the first depth range of the current CTU is [2,3].

[0091] If the current CTU is a simple texture block and the maximum depth value in the initial depth traversal range is greater than or equal to 2, then depth clipping is performed, and the first depth range of the current CTU is [0,2].

[0092] In this embodiment, if the current CTU block does not meet any of the conditions for determining the first depth range based on texture characteristics, the first depth range is predicted based on depth correlation, predicted depth value, and preliminary depth range DR.

[0093] In this embodiment, the step of obtaining a temporally co-located CTU block and, based on the temporally co-located CTU block, predicting the depth value of the current CTU block and the preliminary depth range, obtaining a first depth range specifically involves:

[0094] Obtain the first average depth value of the time-domain co-located CTU, the second average depth value of the adjacent left CTU of the time-domain co-located CTU, and the third average depth value of the spatially adjacent CTU;

[0095] Based on the first average depth, the second average depth, and the third average depth, combined with the predicted depth value of the current CTU block and the preliminary depth range, the first depth range is determined.

[0096] In this embodiment, the present invention determines the depth values ​​of adjacent CTUs in the spatiotemporal domain through spatiotemporal domain correlation, and uses this to predict the first depth range of the current CTU block. Furthermore, the constraints on the first depth range of the CTU block are subdivided to improve the prediction accuracy.

[0097] In this embodiment, the average depth value of the temporal co-location CTU is obtained. co The average depth value (DepthL) of the adjacent left CTU of the time-domain co-located CTU. co The average depth value (DepthL) of the CTU adjacent to the left in the airspace;

[0098] If DepthL co <Depth co And DepthL≥2 and Depth pre If the minimum depth in the initial depth range is less than 2, then the minimum depth value of the first depth range is 2.

[0099] If DepthL co <Depth co And DepthL≥1 and Depth pre If the minimum depth in the initial depth range is less than 1, then the minimum depth value of the first depth range is 1.

[0100] If DepthL co >Depth co And DepthL<2 and Depth pre If the depth is less than 2 and the maximum depth in the initial depth range is greater than 2, then the maximum depth value of the first depth range is 2.

[0101] If DepthL co >Depth co And DepthL<1 and Depth pre If the depth is less than 1 and the maximum depth in the initial depth range is greater than 1, then the maximum depth value of the first depth range is 1.

[0102] Step 102: Based on the motion and stillness characteristics, and combined with the predicted depth value of the current CTU block and the first depth range, adjust the first depth range;

[0103] In this embodiment, the adjustment of the first depth range based on the motion-stationary characteristics, combined with the predicted depth value of the current CTU block and the first depth range, specifically involves:

[0104] Determine the motion and static characteristics of the current CTU block, including stationary blocks, moving blocks, and violently moving blocks;

[0105] If the current CTU is a block with intense movement, and the minimum depth value of the first depth range is less than or equal to 1, then the minimum depth value of the first depth range is incremented by 1;

[0106] If the current CTU is a stationary block, and the maximum depth value of the first depth range is greater than 2, while the predicted depth value is less than 2, then the maximum depth value of the first depth range is reduced by 1.

[0107] In this embodiment, before determining the motion-stationary characteristics of the current CTU block, which include stationary blocks, moving blocks, and violently moving blocks, the method further includes:

[0108] The difference map of the current CTU downsampled block is obtained according to the downsampling method and the frame difference method, and the first difference threshold of the difference map is calculated according to the OTSU method.

[0109] The difference map is thresholded according to the first difference threshold to obtain the motion image of the current CTU downsampling block;

[0110] The number of moving pixels in the current CTU downsampling block is obtained from the motion image, and the motion and stillness characteristics of the current CTU block are determined based on the number of moving pixels and the first motion threshold.

[0111] In this embodiment, frame difference method and OTSU threshold are used to detect the motion and stillness features of CTU, specifically including:

[0112] Obtain a 1 / 4 downsampled image of the current frame and the previous frame; calculate the difference between each pixel in the current frame 1 / 4 downsampled image and the previous frame 1 / 4 downsampled image, and calculate the absolute value to obtain the difference map;

[0113] The first difference threshold of the difference map is calculated using the OTSU method and denoted as Thr3;

[0114] Thresh is applied to the difference map using a first difference threshold Thr3. Pixels with pixel values ​​greater than or equal to Thr3 are considered moving pixels and their pixel values ​​are set to 255. Pixels with pixel values ​​less than Thr3 are considered non-moving pixels and their pixel values ​​are set to 0. The thresholded difference map is then recorded as a motion image.

[0115] If the number N of moving pixels in the current CTU downsampling block is obtained from the moving image, then the motion-stationary features of the current CTU are:

[0116]

[0117] Where MS represents the motion-stationary characteristics of the current CTU, Static indicates that the CTU is a stationary block, MotionLess indicates that the current CTU is a gently moving block, Motion indicates that the current CTU is a violently moving block, and Thr4 represents the first motion threshold. In this embodiment, the first motion threshold Thr4 is 32.

[0118] In this embodiment, the present invention obtains the motion image of the current CTU block through the frame difference method and the OTSU method, and classifies the CTU block according to the motion and stillness characteristics to further refine the constraint conditions, and further adjusts the first depth range of CTU to improve the prediction accuracy of the first depth range from multiple angles.

[0119] Step 103: Divide the current CTU block size according to the adjusted first depth range to complete the fast inter-frame size decision.

[0120] In this embodiment, if the first depth range of the current CTU is [2,3], the calculation of depth 0 and depth 1 is skipped, that is, the current CTU needs to calculate two dimensions, 16x16 and 8x8.

[0121] If the first depth range of the current CTU is [0,1], skip the calculation of depth 2 and depth 3. That is, the current CTU needs to calculate two dimensions, 64x64 and 32x32.

[0122] If the current CTU's first depth range is [0,2], skip the calculation for depth 3. That is, the current CTU needs to calculate three dimensions: 64x64, 32x32, and 16x16.

[0123] In this embodiment, the first depth range of the CTU block is first determined by multiple features such as texture complexity and temporal correlation. Then, the initially determined first depth range is adjusted according to motion characteristics. The first depth range is determined based on multiple features, which improves the accuracy of prediction and speeds up the determination of the first depth range. The current CTU block is traversed and divided according to the first depth range to complete the fast size decision between frames, thereby reducing the traversal time and the coding time complexity and accelerating the HEVC coding process.

[0124] Please refer to Figure 3 , Figure 3 A schematic diagram of a HEVC inter-frame fast size determination device provided in an embodiment of the present invention includes: a first depth range determination module 301, a first depth range adjustment module 302, and a size determination module 303.

[0125] The first depth range determination module 301 is used to determine the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block;

[0126] The first depth range adjustment module 302 is used to adjust the first depth range based on the motion and stillness characteristics, combined with the predicted depth value of the current CTU block and the first depth range;

[0127] The size decision module 303 is used to divide the size of the current CTU block according to the adjusted first depth range, and complete the fast size decision between frames.

[0128] This invention also provides a computer device, including: a processor, a communication interface, and a memory, wherein the processor, the communication interface, and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to execute the HEVC inter-frame fast size determination method.

[0129] Example 2

[0130] Please refer to Figure 4 , Figure 4 This is another flowchart illustrating the HEVC inter-frame fast size determination method provided in an embodiment of the present invention.

[0131] In this embodiment, the HEVC inter-frame fast size determination method specifically includes:

[0132] Calculate the texture complexity of the current CTU block: The Roberts gradient operator is used to calculate the texture complexity of the current CTU block;

[0133] Motion and stillness detection: Frame difference method and OTSU are used to detect motion and stillness features on 1 / 4 downsampled resolution images.

[0134] Estimated initial depth range: The minimum and maximum depth values ​​of all existing spatiotemporally adjacent CTUs of the current CTU are used as the initial depth traversal range of the current CTU. If none of the spatiotemporally adjacent CTUs of the current CTU exist, the initial depth traversal range is [0,3].

[0135] Estimated predicted depth value: The predicted depth value of the current CTU is obtained by weighting the depth values ​​of adjacent CTUs in the spatiotemporal domain. The predicted depth value of CTUs with violent motion does not use the depth value of the CTU block in the same time domain.

[0136] Multi-feature fusion depth decision: Based on the initial depth range DR, multi-feature fusion depth decision is performed according to the texture complexity, motion and stillness features, predicted depth value and depth correlation of the current CTU to obtain the accurate first depth range, and the size of the current CTU block is divided by traversing according to the first depth range.

[0137] Please refer to Figure 5 , Figure 5 Another schematic diagram of the HEVC inter-frame fast size decision device provided in the embodiment of the present invention includes a texture complexity calculation module, a motion and stillness detection module, a preliminary depth traversal range estimation module, a predicted depth value estimation module, and a multi-feature fusion depth decision module.

[0138] The texture complexity calculation module is used to measure the texture complexity of the current CTU block using the Roberts gradient operator.

[0139] The motion and stillness detection module is used to detect motion and stillness features on a 1 / 4 downsampled resolution image using frame difference method and OTSU.

[0140] The preliminary depth traversal range estimation module uses the minimum and maximum depth values ​​of all existing spatiotemporal adjacent CTUs of the current CTU as the preliminary depth traversal range of the current CTU. If none of the spatiotemporal adjacent CTUs of the current CTU exist, the preliminary depth traversal range is [0,3].

[0141] The predicted depth estimation module is used to obtain the predicted depth value of the current CTU by weighting the depth values ​​of adjacent CTUs in the spatiotemporal domain, wherein the predicted depth value of the CTU with violent motion does not use the depth value of the CTU block in the same time domain.

[0142] The multi-feature fusion depth decision module is used to perform multi-feature fusion depth decision based on the texture complexity, motion and stillness features, predicted depth value and depth correlation of the current CTU, based on the initial depth range DR, to obtain an accurate first depth range, and to traverse and divide the size of the current CTU block according to the first depth range.

[0143] In this embodiment, the texture complexity of the current CTU is first calculated using the Roberts gradient operator, and the motion and stillness features of the current CTU are calculated using the frame difference method and OTSU. Then, the initial depth traversal range and predicted depth value of the current CTU are estimated using spatiotemporal correlation. Finally, based on texture complexity, motion and stillness features, predicted depth value, and depth correlation, a multi-feature fusion depth decision is made for the CTU to determine the accurate depth traversal range, thus accelerating the HEVC encoding process. Compared with the open-source encoder x265, under low-latency P-test conditions, the average rate-distortion performance is reduced without loss, while the encoding time complexity is reduced on average.

[0144] In this embodiment, the method combines the spatiotemporal correlation, texture features, motion / static features, and depth correlation of the current CTU to determine the precise depth traversal range through multiple constraints. First, the texture complexity of the current CTU is calculated using the Roberts gradient operator, and the motion / static features of the current CTU are calculated using the frame difference method and OTSU. Then, the initial depth traversal range and predicted depth value of the current CTU are estimated using the spatiotemporal correlation. Finally, based on the texture complexity, motion / static features, predicted depth value, and depth correlation, a multi-feature fusion depth decision is made for the CTU to determine the precise depth traversal range, thereby effectively reducing the computational complexity of quadtree traversal.

[0145] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A fast size determination method for HEVC inter-frame frames, characterized in that, include: Based on the texture complexity and spatiotemporal correlation of the current CTU block, and combined with the predicted depth value and preliminary depth range of the current CTU block, the first depth range is determined. Specifically: Calculate the texture complexity of the current CTU block and determine whether the current CTU block conforms to texture characteristics; if it does, obtain the texture complexity of the current CTU block, and determine the first depth range based on the predicted depth value and preliminary depth range of the current CTU block; if it does not conform, obtain the temporal co-located CTU block, and determine the first depth range based on the depth correlation of the temporal co-located CTU block combined with the predicted depth value and preliminary depth range of the current CTU block; the current CTU block is the current largest coding unit block; wherein, if the current CTU block is a texture complex block or a texture simple block, it is determined that the current CTU block conforms to texture characteristics; the CTU block is a coding tree unit; Based on the motion and stillness characteristics, and combined with the predicted depth value of the current CTU block and the first depth range, the first depth range is adjusted; The size of the current CTU block is divided according to the adjusted first depth range to complete the fast size decision between frames; the texture complexity of the current CTU block is calculated by using the gradient of the current CTU block in the horizontal direction and the gradient in the vertical direction of the current frame, and the texture complexity of the current CTU block is determined according to the average gradient.

2. The HEVC inter-frame fast size determination method as described in claim 1, characterized in that, The process of acquiring the temporally co-located CTU block and predicting the depth value and preliminary depth range of the current CTU block based on the temporally co-located CTU block to obtain the first depth range specifically involves: Obtain the first average depth value of the time-domain co-located CTU, the second average depth value of the adjacent left CTU of the time-domain co-located CTU, and the third average depth value of the spatially adjacent CTU; Based on the first average depth, the second average depth, and the third average depth, combined with the predicted depth value of the current CTU block and the preliminary depth range, the first depth range is determined.

3. The HEVC inter-frame fast size determination method as described in claim 1, characterized in that, Before determining whether the current CTU block conforms to texture characteristics, the method further includes: The gradients of the current CTU block in the horizontal and vertical directions are calculated using the Roberts gradient operator. The average gradient of the current CTU block is calculated based on the gradient in the horizontal direction and the gradient in the vertical direction, and the texture complexity of the current CTU block is determined based on the average gradient.

4. The HEVC inter-frame fast size determination method as described in claim 3, characterized in that, The determination of the texture complexity of the current CTU block based on the average gradient is specifically as follows: A first texture complexity threshold and a second texture complexity threshold are preset. If the average gradient of the current CTU block is less than the first texture complexity threshold, then the current CTU block is a texture flat block. If the average gradient of the current CTU block is greater than the second texture complexity threshold, then the current CTU block is a texture complex block; If the average gradient of the current CTU is greater than the first texture complexity threshold and less than the second texture complexity threshold, then the current CTU block is a normal texture block.

5. The HEVC inter-frame fast size determination method as described in claim 1, characterized in that, The adjustment of the first depth range based on the motion and stillness characteristics, combined with the predicted depth value of the current CTU block and the first depth range, is specifically as follows: Determine the motion and static characteristics of the current CTU block, including stationary blocks, moving blocks, and violently moving blocks; If the current CTU is a block with intense movement, and the minimum depth value of the first depth range is less than or equal to 1, then the minimum depth value of the first depth range is incremented by 1; If the current CTU is a stationary block, and the maximum depth value of the first depth range is greater than 2, while the predicted depth value is less than 2, then the maximum depth value of the first depth range is reduced by 1.

6. The HEVC inter-frame fast size determination method as described in claim 5, characterized in that, Before determining the motion-stationary characteristics of the current CTU block, which include stationary blocks, moving blocks, and violently moving blocks, the method further includes: The difference map of the current CTU downsampled block is obtained according to the downsampling method and the frame difference method, and the first difference threshold of the difference map is calculated according to the OTSU method; the OTSU method is the maximum inter-class difference method. The difference map is thresholded according to the first difference threshold to obtain the motion image of the current CTU downsampling block; The number of moving pixels in the current CTU downsampling block is obtained from the motion image, and the motion and stillness characteristics of the current CTU block are determined based on the number of moving pixels and the first motion threshold.

7. The HEVC inter-frame fast size determination method as described in claim 1, characterized in that, Before determining the first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block, the method further includes: The predicted depth value and preliminary depth range of the current CTU block are predicted based on the spatiotemporal adjacent CTU blocks of the current CTU. The spatiotemporal adjacent CTU blocks include temporal adjacent CTUs and spatial adjacent CTUs.

8. A fast size determination device for HEVC inter-frames, characterized in that, include: The module includes a first depth range determination module, a first depth range adjustment module, and a size determination module. The first depth range determination module is used to determine a first depth range based on the texture complexity and spatiotemporal correlation of the current CTU block, combined with the predicted depth value and preliminary depth range of the current CTU block; specifically: calculating the texture complexity of the current CTU block and determining whether the current CTU block conforms to texture characteristics; if it does, obtaining the texture complexity of the current CTU block and determining the first depth range based on the predicted depth value and preliminary depth range of the current CTU block; if it does not, obtaining the temporally co-located CTU block and determining the first depth range based on the depth correlation of the temporally co-located CTU block combined with the predicted depth value and preliminary depth range of the current CTU block; the current CTU block is the current largest coding unit block; wherein, if the current CTU block is a texture complex block or a texture simple block, it is determined that the current CTU block conforms to texture characteristics; the CTU block is a coding tree unit; The first depth range adjustment module is used to adjust the first depth range based on the motion and stillness characteristics, combined with the predicted depth value of the current CTU block and the first depth range; The size determination module is used to divide the size of the current CTU block according to the adjusted first depth range to complete the fast size determination between frames; the texture complexity of the current CTU block is calculated by using the gradient of the current CTU block in the horizontal direction and the gradient in the vertical direction of the current frame to calculate the average gradient of the current CTU block, and the texture complexity of the current CTU block is determined according to the average gradient.

9. A computer device, characterized in that, include: The processor, the communication interface, and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to execute the HEVC inter-frame fast size decision method as described in any one of claims 1-7.

Images