A CTU determination method and device, terminal equipment and storage medium

Abstract

The present invention discloses a CTU determination method, apparatus, terminal equipment and storage medium. First, a preliminary depth traversal range of a target CTU is calculated based on depth data of several encoded CTUs adjacent to the target CTU in spatiotemporal domains. Then, depth prediction data is calculated based on the spatiotemporal domain correlation between the target CTU and several encoded CTUs. Then, texture complexity is calculated based on the pixel data of the target CTU. Finally, based on the preliminary depth traversal range, the depth interval of the target CTU is calculated based on the depth prediction data and the depth data according to CTU types with different texture complexities, thereby improving the accuracy of CTU depth interval calculation, determining the optimal CTU through the obtained CTU depth interval, and performing HEVC encoding through the optimal CTU. This can reduce the complexity of HEVC encoding and thereby improve HEVC encoding efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video coding, and particularly relates to a CTU determination method and device, a terminal device and a storage medium. BACKGROUND

[0002] As a new generation of video coding standard, HEVC (High Efficiency Video Coding) has doubled the compression efficiency compared with the previous generation of video coding standard H.264 / AVC, but at the same time, the coding complexity has also greatly increased. Due to the new technology introduced in HEVC, the coding complexity is extremely high, which makes it impossible to be used in real-time application scenarios such as video conferencing. There is a HEVC inter-frame coding fast mode selection method in the prior art, which estimates the depth prediction range, mode selection and early CU termination of the current CTU (Coding Tree Unit) from the perspective of spatial and temporal correlation to determine the CTU depth, and then determines the setting of the CTU according to the CTU depth, so as to accelerate the HEVC coding process through the set CTU.

[0003] However, the prior art only considers the spatial and temporal correlation, and does not consider the texture features of the current CTU block. Therefore, the prior art has limitations for CTU depth decision and reducing coding complexity, and the CTU depth obtained by calculation has low accuracy, which will affect the coding efficiency of HEVC.

[0004] Therefore, there is an urgent need for a CTU determination strategy to solve the problem of low accuracy of optimal CTU depth calculation. SUMMARY

[0005] The embodiments of the present application provide a CTU determination method and device, a terminal device and a storage medium to improve the accuracy of optimal CTU depth calculation.

[0006] To solve the above problems, an embodiment of the present application provides a CTU determination method, comprising:

[0007] Obtain depth data of a plurality of coded CTUs, and determine a preliminary depth traversal range of a target CTU according to the depth data; wherein the plurality of coded CTUs are CTUs adjacent to the target CTU in space and time domain;

[0008] According to the spatial and temporal correlation between the plurality of coded CTUs and the target CTU, and the depth data, obtain depth prediction data of the target CTU;

[0009] According to the pixel data of the target CTU, obtain the texture complexity of the target CTU; wherein the type of the texture complexity of the target CTU includes one or more of the following: flat block, complex block and ordinary block;

[0010] When the target CTU is a flat block or a complex block, a depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data; when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data.

[0011] A target CTU is selected according to the depth interval of the target CTU, and HEVC encoding is performed based on the selected target CTU.

[0012] As an improvement of the above scheme, the depth data of a plurality of coded CTUs is obtained, and the preliminary depth traversal range of the target CTU is determined according to the depth data, specifically as follows:

[0013] The depth data of a plurality of coded CTUs is read and obtained according to the spatio-temporal neighboring coded CTUs of the target CTU.

[0014] According to the depth data, the maximum depth value and the minimum depth value are selected from the depth values of the plurality of coded CTUs as the preliminary depth traversal range of the target CTU.

[0015] As an improvement of the above scheme, the depth prediction data of the target CTU is obtained according to the spatio-temporal correlation between the plurality of coded CTUs and the target CTU and the depth data, specifically as follows:

[0016] According to the spatio-temporal correlation between the plurality of coded CTUs and the target CTU, CUL on the left side of the target CTU in the spatial domain, CUA on the top side of the target CTU in the spatial domain, CULA on the top-left side of the target CTU in the spatial domain, CURA on the top-right side of the target CTU in the spatial domain, and CUcol in the same position as the target CTU in the temporal domain are determined; wherein the plurality of coded CTUs include CUL, CUA, CULA, CURA and CUcol.

[0017] According to the depth data, the average depth value level1 of the right half block of CUL, the average depth value level2 of the lower half block of CUA, the average depth value level3 of the first coding unit in CULA, the average depth value level4 of the second coding unit in CURA, and the average depth value level5 of CUcol are obtained, and are substituted into a prediction depth calculation formula to obtain the depth prediction data; wherein the distance between the first coding unit and the target CTU is a first preset value among all coding units in CULA; the distance between the second coding unit and the target CTU is a second preset value among all coding units in CURA; the depth data includes level1, level2, level3, level4 and level5; and the prediction depth calculation formula is as follows:

[0018]

[0019] wherein, N = 5, ω i is a weight factor of the spatio-temporal neighboring CTU, level i is a coding depth value of the spatio-temporal neighboring CTU; the weight factor ω i is set according to the correlation between the current CTU and the spatio-temporal neighboring CTU.

[0020] As an improvement of the above scheme, the texture complexity of the target CTU is obtained according to the pixel data of the target CTU, and specifically:

[0021] The average gradient value of the target CTU is calculated by a Roberts gradient operator calculation method according to the pixel data of the target CTU.

[0022] The texture complexity of the target CTU is obtained according to the average gradient value, a preset lower texture complexity threshold and an upper texture complexity threshold: when the average gradient value is less than the lower texture complexity threshold, the target CTU is a flat block; when the average gradient value is greater than the lower texture complexity threshold and less than the upper texture complexity threshold, the target CTU is an ordinary block; and when the average gradient value is greater than the upper texture complexity threshold, the target CTU is a complex block.

[0023] As an improvement of the above scheme, when the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data, and specifically:

[0024] When the target CTU is a complex block, the preliminary depth traversal range and the depth prediction data of the target CTU are obtained.

[0025] When the minimum depth value of the preliminary depth traversal range is less than or equal to a first depth threshold, depth extension is performed, and the depth interval of the target CTU is [1, 3].

[0026] When the value of the depth prediction data is greater than a second depth threshold, the depth interval of the target CTU is [2, 3].

[0027] As an improvement of the above scheme, when the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data, and further includes:

[0028] When the target CTU is a flat block, the preliminary depth traversal range and the depth prediction data of the target CTU are obtained.

[0029] When the minimum depth value of the preliminary depth traversal range is greater than or equal to a third depth threshold, depth clipping is performed, and the depth interval of the target CTU is [0, 2].

[0030] When the value of the depth prediction data is less than the fourth depth threshold, the depth interval of the target CTU is [0, 2].

[0031] As an improvement of the above scheme, when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data, and specifically:

[0032] When the target CTU is a normal block, the CUcol, the CULc and the CUL of the target CTU are determined according to the depth correlation between the plurality of coded CTUs and the target CTU, wherein the plurality of coded CTUs include the CUcol, the CULc and the CUL.

[0033] According to the depth data, the average depth value Depth co of the CUcol, the average depth value Depth co of the CULc and the average depth value Depth co of the CUL are obtained.

[0034] According to Depth co , Depth pre , the depth prediction data Depth co , the depth interval of the target CTU is obtained: when Depth co , Depth pre ≥ 2 and Depth co ≥ 2, the minimum depth value of the depth interval of the target CTU is 2; when Depth co , Depth pre ≥ 1 and Depth co ≥ 1, the minimum depth value of the depth interval of the target CTU is 1; when Depth co , Depth pre ≤ 1 and Depth co ≤ 1, the maximum depth value of the depth interval of the target CTU is 1; when Depth co , Depth pre ≤ 2 and Depth ≤ 2, the maximum depth value of the depth interval of the target CTU is 2.

[0035] Correspondingly, an embodiment of the present application further provides a CTU determination device, comprising a first data acquisition module, a second data acquisition module, a third data acquisition module, a depth interval calculation module and a result generation module.

[0036] The first data acquisition module is configured to acquire depth data of a plurality of coded CTUs, and determine a preliminary depth traversal range of a target CTU according to the depth data, wherein the plurality of coded CTUs are CTUs adjacent to the target CTU in a spatial and temporal domain.

[0037] The second data acquisition module is configured to acquire depth prediction data of the target CTU according to a spatial and temporal domain correlation between the plurality of coded CTUs and the target CTU, and the depth data.

[0038] The third data acquisition module is configured to acquire texture complexity of the target CTU according to pixel data of the target CTU, wherein the texture complexity includes one or more of a flat block, a complex block, and a normal block.

[0039] The depth interval calculation module is configured to acquire a depth interval of the target CTU according to the preliminary depth traversal range and the depth prediction data when the target CTU is a flat block or a complex block, and acquire the depth interval of the target CTU according to the depth prediction data and the depth data when the target CTU is a normal block.

[0040] The result generation module is configured to select the target CTU according to the depth interval of the target CTU, and perform HEVC encoding based on the selected target CTU.

[0041] As an improvement of the above-mentioned scheme, the first data acquisition module comprises a depth data acquisition unit and a preliminary depth traversal range calculation unit.

[0042] The depth data acquisition unit is configured to read and obtain the depth data of the plurality of coded CTUs according to the coded CTUs adjacent to the target CTU in the spatial and temporal domain.

[0043] The preliminary depth traversal range calculation unit is configured to select a maximum depth value and a minimum depth value in depth values of the plurality of coded CTUs as the preliminary depth traversal range of the target CTU according to the depth data.

[0044] As an improvement of the above-mentioned scheme, the second data acquisition module comprises a spatial and temporal domain correlation acquisition unit and a depth prediction data calculation unit.

[0045] The spatial and temporal domain correlation acquisition unit is configured to determine CUL on a left side of the target CTU in a spatial domain, CUA on an upper side of the target CTU in the spatial domain, CULA on a top-left side of the target CTU in the spatial domain, CURA on a top-right side of the target CTU in the spatial domain, and CUcol in a same temporal position as the target CTU according to the spatial and temporal domain correlation between the plurality of coded CTUs and the target CTU, wherein the plurality of coded CTUs comprise CUL, CUA, CULA, CURA, and CUcol.

[0046] The depth prediction data calculation unit is configured to obtain, based on the depth data, an average depth value level 1 of the right half block of CUL, an average depth value level 2 of the lower half block of CUA, an average depth value level 3 of the first coding unit in CULA, an average depth value level 4 of the second coding unit in CURA, and an average depth value level 5 of CUcol, and substitute the obtained depth data into a predicted depth calculation formula for calculation to obtain depth prediction data; wherein, in all coding units of CULA, a distance between the first coding unit and the target CTU is a first preset value; in all coding units of CURA, a distance between the second coding unit and the target CTU is a second preset value; the depth data includes: level 1, level 2, level 3, level 4, and level 5; and the predicted depth calculation formula is as follows:

[0047]

[0048] Where N = 5, ω i is the weight factor of adjacent CTUs in the spatiotemporal domain, level i is the coding depth value of the adjacent CTU in the spatiotemporal domain; the weight factor ω i It is set based on the correlation between the current CTU and its adjacent CTUs in the spatiotemporal domain.

[0049] As an improvement of the above solution, the third data acquisition module includes: an average gradient value calculation unit and a judgment unit;

[0050] The average gradient value calculation unit is configured to calculate an average gradient value of the target CTU using a Roberts gradient operator calculation method based on the pixel data of the target CTU;

[0051] The judgment unit is configured to obtain the texture complexity of the target CTU based on the average gradient value and a preset texture complexity lower threshold and texture complexity upper threshold: when the average gradient value is less than the texture complexity lower threshold, the target CTU is a flat block; when the average gradient value is greater than the texture complexity lower threshold and less than the texture complexity upper threshold, the target CTU is a normal block; and when the average gradient value is greater than the texture complexity upper threshold, the target CTU is a complex block.

[0052] As an improvement to the above solution, when the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data, specifically:

[0053] When the target CTU is a complex block, obtaining a preliminary depth traversal range and depth prediction data of the target CTU;

[0054] When the minimum depth value of the initial depth traversal range is less than or equal to the first depth threshold, depth expansion is performed, and the depth interval of the target CTU is [1,3];

[0055] When the value of the depth prediction data is greater than the second depth threshold, the depth interval of the target CTU is [2, 3].

[0056] As an improvement to the above solution, when the target CTU is a flat block or a complex block, obtaining the depth interval of the target CTU according to the preliminary depth traversal range and the depth prediction data further includes:

[0057] When the target CTU is a flat block, obtaining a preliminary depth traversal range and depth prediction data of the target CTU;

[0058] When the minimum depth value of the initial depth traversal range is greater than or equal to the third depth threshold, depth clipping is performed, and the depth interval of the target CTU is [0, 2];

[0059] When the value of the depth prediction data is less than the fourth depth threshold, the depth range of the target CTU is [0, 2].

[0060] As an improvement to the above solution, when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data, specifically:

[0061] When the target CTU is a common block, determining, based on the depth correlation between the plurality of coded CTUs and the target CTU, a CUcol co-located in the time domain with the target CTU, a CULc to the left of CUcol in the spatial domain, and a CUL to the left of the target CTU in the spatial domain; wherein the plurality of coded CTUs include: CUcol, CULc, and CUL;

[0062] According to the depth data, obtain the average depth value Depth of CUcol co , the average depth value of CULc DepthL co and the average depth value DepthL of CUL;

[0063] According to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , get the depth interval of the target CTU: when DepthL co <Depth co , DepthL≥2 and Depth pre ≥2, the minimum depth value of the depth interval of the target CTU is 2; when DepthL co<Depth co , DepthL≥1 and Depth pre ≥1, then the minimum depth value of the depth interval of the target CTU is 1; when DepthL co >Depth co , DepthL≤1 and Depth pre ≤1, then the maximum depth value of the depth interval of the target CTU is 1; when DepthL co >Depth co , DepthL≤2 and Depth pre ≤2, then the maximum depth value of the depth interval of the target CTU is 2.

[0064] Correspondingly, an embodiment of the present application further provides a computer terminal device, comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the CTU determination method when executing the computer program.

[0065] Correspondingly, an embodiment of the present application further provides a computer readable storage medium, comprising a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the CTU determination method when the computer program runs.

[0066] As can be seen from the above, the present application has the following beneficial effects:

[0067] The present application provides a CTU determination method, which firstly calculates the preliminary depth traversal range of a target CTU according to the depth data of a plurality of coded CTUs adjacent to the target CTU in the space-time domain; then calculates the depth prediction data by combining the space-time domain correlation of the target CTU and the plurality of coded CTUs and the depth data; then calculates the texture complexity according to the pixel data of the target CTU; and finally calculates the depth interval of the target CTU on the basis of the preliminary depth traversal range by combining the depth prediction data and the depth data according to the CTU type of different texture complexity, thereby improving the depth interval calculation accuracy of the CTU, determining the optimal CTU through the obtained CTU depth interval, and reducing the complexity of HEVC encoding through the optimal CTU to execute HEVC encoding, thereby improving the HEVC encoding efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a flowchart of the CTU determination method provided by an embodiment of the present application;

[0069] Figure 2is a structural schematic diagram of a CTU determination device provided by an embodiment of the present application;

[0070] Figure 3 is a position relationship of a plurality of coded CTUs and a target CTU in a space-time domain provided by an embodiment of the present application;

[0071] Figure 4 is a depth correlation between a plurality of coded CTUs and a target CTU in a space domain provided by an embodiment of the present application;

[0072] Figure 5 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0074] Embodiment One

[0075] Reference is made to Figure 1 , Figure 1 is a flowchart of a CTU determination method provided by an embodiment of the present application, as shown in FIG. 1, the present embodiment includes steps 101 to 105, and each step is specifically as follows: Figure 1

[0076] Step 101: Obtain depth data of a plurality of coded CTUs, and determine a preliminary depth traversal range of a target CTU according to the depth data; wherein the plurality of coded CTUs are CTUs adjacent to the target CTU in a space-time domain.

[0077] In the present embodiment, the obtaining of the depth data of the plurality of coded CTUs and the determination of the preliminary depth traversal range of the target CTU according to the depth data are specifically as follows:

[0078] According to the coded CTUs adjacent to the target CTU in a space-time domain, read and obtain the depth data of the plurality of coded CTUs.

[0079] According to the depth data, select a maximum depth value and a minimum depth value in the depth values of the plurality of coded CTUs as the preliminary depth traversal range of the target CTU.

[0080] In a specific embodiment, let the maximum depth value be D max , and the minimum depth value be D min ​, the calculation method of the initial depth traversal range of the target CTU is as follows:

[0081]

[0082] Among them D min and D max Indicates the minimum and maximum depth values ​​of all existing spatiotemporal adjacent CTUs, and DR indicates the preliminary depth traversal range of the current CTU.

[0083] If there are no spatiotemporal adjacent CTUs of the current CTU, the DR is [0, 3].

[0084] In a specific embodiment, to better illustrate the target CTU ( Figure 3 CU0) is a coded CTU that is adjacent in time and space, see Figure 3 ,like Figure 3 As shown, the adjacent coded CTUs (left CTU: CUL, upper CTU: CUA, upper left CTU: CULA, upper right CTU: CURA) in the spatial domain (current frame) and the co-located CTU (CUcol) coded in the temporal domain (forward reference frame) are shown.

[0085] Step 102: Obtain depth prediction data of the target CTU according to the spatiotemporal correlations between the plurality of encoded CTUs and the target CTU, and the depth data.

[0086] In a specific embodiment, to better illustrate the temporal and spatial correlation between several coded CTUs and the target CTU, see Figure 4 ; Among them, the depth value of the left CTU adopts the average depth value of the right half CTU block of the CTU to the left of the target CTU; similarly, the depth value of the upper CTU adopts the average depth value of the lower half CTU block of the CTU above the target CTU; since the upper left and upper right blocks have little correlation with the target CTU, the depth value of the upper left CTU and the depth value of the upper right CTU adopt the depth value of the CU most adjacent to the target CTU.

[0087] In this embodiment, the depth prediction data of the target CTU is obtained according to the spatiotemporal correlation between the plurality of encoded CTUs and the target CTU, and the depth data, specifically:

[0088] Determining, based on the spatial and temporal correlations between the plurality of coded CTUs and the target CTU, a CUL on the spatial left side of the target CTU, a CUA on the spatial upper side of the target CTU, a CULA on the spatial upper left side of the target CTU, a CURA on the spatial upper right side of the target CTU, and a CUcol that is temporally co-located with the target CTU; wherein the plurality of coded CTUs include: CUL, CUA, CULA, CURA, and CUcol;

[0089] According to the depth data, an average depth value level1 of the right half block of CUL, an average depth value level2 of the lower half block of CUA, an average depth value level3 of the first coding unit in CULA, an average depth value level4 of the second coding unit in CURA, and an average depth value level5 of CUcol are obtained, and the average depth values ​​are substituted into a predicted depth calculation formula for calculation to obtain depth prediction data; wherein, in all coding units of CULA, the distance between the first coding unit and the target CTU is a first preset value; in all coding units of CURA, the distance between the second coding unit and the target CTU is a second preset value; the depth data includes: level1, level2, level3, level4, and level5; and the predicted depth calculation formula is as follows:

[0090]

[0091] Where N = 5, ω i is the weight factor of adjacent CTUs in the spatiotemporal domain, level i is the coding depth value of the adjacent CTU in the spatiotemporal domain; the weight factor ω i It is set based on the correlation between the current CTU and its adjacent CTUs in the spatiotemporal domain.

[0092] In a specific embodiment, among all coding units of CULA, a coding unit closest to a target CTU is selected as a first coding unit (i.e., a distance between the first coding unit and the target CTU is a first preset value); among all coding units of CURA, a coding unit closest to the target CTU is selected as a second coding unit (i.e., a distance between the second coding unit and the target CTU is a second preset value);

[0093] Since the left CTU (i.e., CUL) and the upper CTU (CUA) have a greater correlation with the target CTU, ω i are set to 0.25 respectively; the upper left CTU (i.e. CULA) and the upper right CTU (i.e. CURA) have a smaller correlation with the current CTU, ω i are set to 0.2 respectively, and the ω corresponding to the time domain co-located CTU (i.e. CUcol) i Set to 0.1.

[0094] Step 103: Acquire the texture complexity of the target CTU according to the pixel data of the target CTU; wherein the type of the texture complexity of the target CTU includes one or more of the following: flat block, complex block, and ordinary block.

[0095] In the embodiment, the texture complexity of the target CTU is obtained according to the pixel data of the target CTU, and specifically,

[0096] The average gradient value of the target CTU is obtained by a Roberts gradient operator calculation method according to the pixel data of the target CTU.

[0097] The texture complexity of the target CTU is obtained according to the average gradient value, a preset lower threshold of the texture complexity and an upper threshold of the texture complexity: when the average gradient value is less than the lower threshold of the texture complexity, the target CTU is a flat block; when the average gradient value is greater than the lower threshold of the texture complexity and less than the upper threshold of the texture complexity, the target CTU is an ordinary block; and when the average gradient value is greater than the upper threshold of the texture complexity, the target CTU is a complex block.

[0098] In a specific embodiment, the Roberts gradient operator calculation method is as follows:

[0099] g x = f(x+1, y) - f(x, y)

[0100] g y = f(x, y+1) - f(x, y)

[0101] G(x, y) = |g x + g y |

[0102]

[0103] wherein f(x, y) represents a pixel value at position (x, y), g x represents a horizontal direction gradient of (x, y), g y represents a vertical direction gradient of (x, y), G(x, y) represents a gradient at (x, y), and G avg represents an average gradient value of the current CTU; height represents a height of the current CTU, width represents a width of the current CTU, and therefore the range of i is limited to x ~ x+width and the range of j is limited to y ~ y+height.

[0104] In a specific embodiment, the following formula is used for better representation of the texture complexity:

[0105]

[0106] Wherein, TC represents texture complexity of the current CTU, Simple represents that the current CTU is a flat block, Common represents that the current CTU is a common block, Complex represents that the current CTU is a complex block, Thr1 represents a lower threshold of the texture complexity, and Thr2 represents an upper threshold of the texture complexity; in the embodiment, Thr1 is 5, and Thr2 is 10.

[0107] Step 104: when the target CTU is a flat block or a complex block, obtaining a depth interval of the target CTU according to the preliminary depth traversal range and the depth prediction data; when the target CTU is a common block, obtaining the depth interval of the target CTU according to the depth prediction data and the depth data.

[0108] In the embodiment, when the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data, and specifically:

[0109] When the target CTU is a flat block or a complex block, the preliminary depth traversal range and the depth prediction data of the target CTU are obtained.

[0110] When the minimum depth value of the preliminary depth traversal range is less than or equal to the first depth threshold, depth extension is performed, and the depth interval of the target CTU is [1, 3];

[0111] When the value of the depth prediction data is greater than the second depth threshold, the depth interval of the target CTU is [2, 3].

[0112] In a specific embodiment, if the current CTU is a complex block, and the prediction depth data Depth pre is greater than 2 (the second depth threshold), the depth interval of the current CTU is [2, 3], that is, the current CTU skips the calculation of depth 0 and 1; if the current CTU is a complex block, and the minimum depth value in the DR is less than or equal to 1 (the first depth threshold), depth extension is performed, and the depth interval of the current CTU is [1, 3], that is, the current CTU skips the calculation of depth 0.

[0113] In the embodiment, when the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range and the depth prediction data, and further includes:

[0114] When the target CTU is a flat block, the preliminary depth traversal range and the depth prediction data of the target CTU are obtained.

[0115] When the minimum depth value of the preliminary depth traversal range is greater than or equal to the third depth threshold, depth clipping is performed, and the depth interval of the target CTU is [0, 2];

[0116] When the value of the depth prediction data is less than the fourth depth threshold, the depth range of the target CTU is [0, 2].

[0117] In a specific embodiment, if the current CTU is a flat block and its predicted depth data Depth pre If the depth of the current CTU is less than 2 (the fourth depth threshold), the depth interval of the current CTU is [0, 2], that is, the current CTU skips the calculation of depth 3; if the current CTU is a flat block and the maximum depth value in the DR is greater than or equal to 2 (the third depth threshold), depth clipping is performed, and the depth interval of the current CTU is [0, 2], that is, the current CTU skips the calculation of depth 3.

[0118] In this embodiment, when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data, specifically:

[0119] When the target CTU is a common block, determining, based on the depth correlation between the plurality of coded CTUs and the target CTU, a CUcol co-located in the time domain with the target CTU, a CULc to the left of CUcol in the spatial domain, and a CUL to the left of the target CTU in the spatial domain; wherein the plurality of coded CTUs include: CUcol, CULc, and CUL;

[0120] According to the depth data, obtain the average depth value Depth of CUcol co , the average depth value of CULc DepthL co and the average depth value DepthL of CUL;

[0121] According to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , get the depth interval of the target CTU: when DepthL co <Depth co , DepthL≥2 and Depth pre ≥2, the minimum depth value of the depth interval of the target CTU is 2; when DepthL co <Depth co , DepthL≥1 and Depth pre ≥1, the minimum depth value of the depth interval of the target CTU is 1; when DepthL co Depth co , DepthL≤1 and Depth pre ≤1, the maximum depth value of the depth interval of the target CTU is 1; when DepthL co Depth co, DepthL≤2 and Depth pre When DepthL≤2 and Depth

[0122] Step 105: selecting a target CTU according to a depth interval of the target CTU, and performing HEVC encoding based on the selected target CTU.

[0123] The embodiment firstly calculates a preliminary depth traversal range of the target CTU according to depth data of a plurality of coded CTUs adjacent to a spatio-temporal domain of the target CTU; then calculates depth prediction data by combining the depth data and spatio-temporal domain correlation of the target CTU and the plurality of coded CTUs; then calculates texture complexity according to pixel data of the target CTU; and finally calculates a depth interval of the target CTU based on the preliminary depth traversal range, the depth prediction data and the depth data according to CTU types of different texture complexity. The depth interval of the current CTU is determined to accelerate the HEVC encoding process. Compared with the open-source encoder x265, the encoding time complexity is reduced by 8.1% on average under the condition of no loss of average rate-distortion performance in the low delay P (Low delay P) test.

[0124] Embodiment Two

[0125] Referring to Figure 2 , Figure 2 is a structural schematic diagram of a CTU determination device provided by an embodiment of the application, comprising a first data acquisition module 201, a second data acquisition module 202, a third data acquisition module 203, a depth interval calculation module 204 and a result generation module 205.

[0126] The first data acquisition module 201 is configured to acquire depth data of a plurality of coded CTUs, and determine a preliminary depth traversal range of a target CTU according to the depth data; wherein the plurality of coded CTUs are CTUs adjacent to a spatio-temporal domain of the target CTU.

[0127] The second data acquisition module 202 is configured to acquire depth prediction data of the target CTU according to spatio-temporal domain correlation between the plurality of coded CTUs and the target CTU, and the depth data.

[0128] The third data acquisition module 203 is configured to acquire texture complexity of the target CTU according to pixel data of the target CTU; wherein the type of the texture complexity of the target CTU comprises one or more of the following: a flat block, a complex block and a normal block.

[0129] The depth interval calculation module 204 is configured to: when the target CTU is a flat block or a complex block, obtain a depth interval of the target CTU according to the preliminary depth traversal range and the depth prediction data; and when the target CTU is a normal block, obtain the depth interval of the target CTU according to the depth prediction data and the depth data.

[0130] The result generation module 205 is configured to select the target CTU according to the depth interval of the target CTU, and perform HEVC encoding based on the selected target CTU.

[0131] As an improvement of the above-mentioned scheme, the first data acquisition module 201 comprises a depth data acquisition unit and a preliminary depth traversal range calculation unit.

[0132] The depth data acquisition unit is configured to read and obtain depth data of a plurality of coded CTUs according to the spatio-temporal neighboring coded CTUs of the target CTU.

[0133] The preliminary depth traversal range calculation unit is configured to select a maximum depth value and a minimum depth value from the depth values of the plurality of coded CTUs as the preliminary depth traversal range of the target CTU according to the depth data.

[0134] As an improvement of the above-mentioned scheme, the second data acquisition module 202 comprises a spatio-temporal correlation acquisition unit and a depth prediction data calculation unit.

[0135] The spatio-temporal correlation acquisition unit is configured to determine a CUL on the left side of the target CTU in the spatial domain, a CUA on the top side of the target CTU in the spatial domain, a CULA on the top-left side of the target CTU in the spatial domain, a CURA on the top-right side of the target CTU in the spatial domain, and a CUcol in the same position as the target CTU in the temporal domain according to the spatio-temporal correlation between the plurality of coded CTUs and the target CTU; wherein the plurality of coded CTUs comprise the CUL, the CUA, the CULA, the CURA and the CUcol.

[0136] The depth prediction data calculation unit is configured to obtain, according to the depth data, an average depth value level1 of a right half block of the CUL, an average depth value level2 of a lower half block of the CUA, an average depth value level3 of a first coding unit in the CULA, an average depth value level4 of a second coding unit in the CURA, and an average depth value level5 of the CUcol, and substitute the values into a prediction depth calculation formula to obtain depth prediction data; wherein the first coding unit and the target CTU have a first preset distance in all coding units of the CULA; the second coding unit and the target CTU have a second preset distance in all coding units of the CURA; the depth data comprises the level1, the level2, the level3, the level4, and the level5; and the prediction depth calculation formula is as follows:

[0137]

[0138] wherein N=5, ω i is a weight factor of a spatio-temporal neighboring CTU, level i is a coding depth value of the spatio-temporal neighboring CTU; and the weight factor ω i is set according to a correlation between the current CTU and the spatio-temporal neighboring CTU.

[0139] As an improvement of the above scheme, the third data acquisition module 203 comprises an average gradient value calculation unit and a judging unit.

[0140] The average gradient value calculation unit is configured to calculate an average gradient value of the target CTU by a Roberts gradient operator calculation method according to pixel data of the target CTU.

[0141] The judging unit is configured to obtain a texture complexity of the target CTU according to the average gradient value by a preset lower texture complexity threshold and an upper texture complexity threshold; when the average gradient value is less than the lower texture complexity threshold, the target CTU is a flat block; when the average gradient value is greater than the lower texture complexity threshold and less than the upper texture complexity threshold, the target CTU is an ordinary block; and when the average gradient value is greater than the upper texture complexity threshold, the target CTU is a complex block.

[0142] As an improvement of the above scheme, when the target CTU is a flat block or a complex block, a depth interval of the target CTU is obtained according to a preliminary depth traversal range and depth prediction data, and specifically:

[0143] When the target CTU is a complex block, the preliminary depth traversal range and the depth prediction data of the target CTU are obtained.

[0144] When the minimum depth value of the initial depth traversal range is less than or equal to the first depth threshold, depth expansion is performed, and the depth interval of the target CTU is [1,3];

[0145] When the value of the depth prediction data is greater than the second depth threshold, the depth interval of the target CTU is [2, 3].

[0146] As an improvement to the above solution, when the target CTU is a flat block or a complex block, obtaining the depth interval of the target CTU according to the preliminary depth traversal range and the depth prediction data further includes:

[0147] When the target CTU is a flat block, obtaining a preliminary depth traversal range and depth prediction data of the target CTU;

[0148] When the minimum depth value of the initial depth traversal range is greater than or equal to the third depth threshold, depth clipping is performed, and the depth interval of the target CTU is [0, 2];

[0149] When the value of the depth prediction data is less than the fourth depth threshold, the depth range of the target CTU is [0, 2].

[0150] As an improvement to the above solution, when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data, specifically:

[0151] When the target CTU is a common block, determining, based on the depth correlation between the plurality of coded CTUs and the target CTU, a CUcol co-located in the time domain with the target CTU, a CULc to the left of CUcol in the spatial domain, and a CUL to the left of the target CTU in the spatial domain; wherein the plurality of coded CTUs include: CUcol, CULc, and CUL;

[0152] According to the depth data, obtain the average depth value Depth of CUcol co , the average depth value of CULc DepthL co and the average depth value DepthL of CUL;

[0153] According to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , get the depth interval of the target CTU: when DepthL co <Depth co , DepthL≥2 and Depth pre ≥2, the minimum depth value of the depth interval of the target CTU is 2; when DepthL co <Depth co, DepthL>1 and Depth pre When DepthL co >Depth co , DepthL>1 and Depth pre When DepthL co >Depth co , DepthL>1 and Depth pre When DepthL

[0154] The embodiment obtains the preliminary depth traversal range of the target CTU through the first data acquisition module, obtains the depth prediction data of the target CTU through the second data acquisition module, and obtains the texture complexity of the target CTU through the third data acquisition module. The depth prediction data, the preliminary depth traversal range and the texture complexity are input into the depth interval calculation module to calculate the depth interval of the target CTU, and the depth interval is input into the result generation module to select the target CTU, so that the HEVC encoding is performed according to the selected target CTU. The embodiment determines the optimal CTU according to the obtained CTU depth interval, and performs the HEVC encoding through the optimal CTU, so that the complexity of the HEVC encoding is reduced, and the HEVC encoding efficiency is improved.

[0155] Embodiment Three

[0156] Referring to Figure 5 , Figure 5 is a schematic diagram of a terminal device structure provided by an embodiment of the present application.

[0157] The terminal device of the embodiment includes a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. The processor 501 implements the steps of the CTU determination method in the embodiment when executing the computer program, for example, all steps of the CTU determination method shown in FIG. 8. Figure 1 Alternatively, the processor implements the functions of the modules in the device embodiments when executing the computer program, for example, all modules of the CTU determination device shown in FIG. 9. Figure 2 Alternatively, the processor implements the functions of the modules in the device embodiments when executing the computer program, for example, all modules of the CTU determination device shown in FIG. 9.

[0158] In addition, the embodiment of the present application further provides a computer readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer readable storage medium is located performs the CTU determination method according to any one of the above embodiments.

[0159] Those skilled in the art can understand that the schematic diagram is only an example of the terminal device, and does not constitute a limitation on the terminal device, and can include more or fewer components than the diagram, or combine certain components, or different components, for example, the terminal device can also include an input and output device, a network access device, a bus, etc.

[0160] The processor 501 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor 501 is a control center of the terminal device, and is connected with all parts of the terminal device through various interfaces and lines.

[0161] The memory 502 can be used to store computer programs and / or modules, and the processor 501 realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory, and calling data stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to use of the terminal device (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0162] The modules / units integrated in the terminal device, if in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0163] It should be noted that the above-described device embodiments are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. In addition, the connection relationship between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0164] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which are also considered within the scope of protection of the present application.

Claims

1. A method for determining CTU, characterized in that: include: Acquire depth data of a plurality of encoded CTUs, and determine a preliminary depth traversal range of a target CTU based on the depth data; wherein the plurality of encoded CTUs are CTUs that are adjacent to the target CTU in the spatiotemporal domain; Obtaining depth prediction data of the target CTU according to the spatiotemporal correlations between the plurality of encoded CTUs and the target CTU, and the depth data; Acquire, according to the pixel data of the target CTU, a texture complexity of the target CTU; wherein the type of the texture complexity of the target CTU includes one or more of the following: a flat block, a complex block, and a normal block; When the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range or the depth prediction data; when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data; wherein, when the target CTU is a normal block, the CUcol of the target CTU in the time domain, the CULc on the left side of the CUcol in the spatial domain, and the CUL on the left side of the target CTU in the spatial domain are determined according to the depth correlation between the several encoded CTUs and the target CTU; the several encoded CTUs include: CUcol, CULc and CUL; according to the depth data, the average depth value Depth of CUcol is obtained co , the average depth value of CULc DepthL co and the average depth value DepthL of CUL; according to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , obtain the depth interval of the target CTU; A target CTU is selected according to the depth range of the target CTU, and HEVC encoding is performed based on the selected target CTU.

2. The method for determining the CTU according to claim 1, wherein: The obtaining of depth data of a plurality of encoded CTUs and determining a preliminary depth traversal range of a target CTU according to the depth data is specifically as follows: Read and obtain depth data of several encoded CTUs according to the target CTU's temporal and spatial adjacent encoded CTUs; According to the depth data, a maximum depth value and a minimum depth value are selected from the depth values ​​of the encoded CTUs as a preliminary depth traversal range of the target CTU. The preliminary depth traversal range of the target CTU is calculated as follows: Among them D min and D max Indicates the minimum and maximum depth values ​​of all existing spatiotemporal adjacent CTUs, and DR indicates the initial depth traversal range of the current CTU; If there are no spatiotemporal adjacent CTUs of the current CTU, the DR is [0, 3].

3. The method for determining the CTU according to claim 1, wherein: The acquiring, according to the spatiotemporal correlations between the plurality of encoded CTUs and the target CTU, and the depth data, depth prediction data of the target CTU is specifically: Determining, based on the spatial and temporal correlations between the plurality of coded CTUs and the target CTU, a CUL on the spatial left side of the target CTU, a CUA on the spatial upper side of the target CTU, a CULA on the spatial upper left side of the target CTU, a CURA on the spatial upper right side of the target CTU, and a CUcol that is temporally co-located with the target CTU; wherein the plurality of coded CTUs include: CUL, CUA, CULA, CURA, and CUcol; According to the depth data, an average depth value level1 of the right half block of CUL, an average depth value level2 of the lower half block of CUA, an average depth value level3 of the first coding unit in CULA, an average depth value level4 of the second coding unit in CURA, and an average depth value level5 of CUcol are obtained, and the depth prediction data are obtained by substituting the obtained values ​​into a predicted depth calculation formula. Among all the coding units of CULA, a coding unit closest to a target CTU is selected as a first coding unit, and a distance between the first coding unit and the target CTU is a first preset value. Among all the coding units of CURA, a coding unit closest to a target CTU is selected as a second coding unit, and a distance between the second coding unit and the target CTU is a second preset value. The depth data includes: level1, level2, level3, level4, and level5. The predicted depth calculation formula is as follows: Where N = 5, ω i is the weight factor of adjacent CTUs in the spatiotemporal domain, level i is the coding depth value of the adjacent CTU in the spatiotemporal domain; the weight factor ω i It is set based on the correlation between the current CTU and its adjacent CTUs in the spatiotemporal domain.

4. The method for determining CTU according to claim 1, wherein: The acquiring, according to the pixel data of the target CTU, the texture complexity of the target CTU, is specifically: According to the pixel data of the target CTU, the average gradient value of the target CTU is calculated by using the Roberts gradient operator calculation method; According to the average gradient value, the texture complexity of the target CTU is obtained by using a preset texture complexity lower threshold and a texture complexity upper threshold: when the average gradient value is less than the texture complexity lower threshold, the target CTU is a flat block; when the average gradient value is greater than the texture complexity lower threshold and less than the texture complexity upper threshold, the target CTU is a normal block; when the average gradient value is greater than the texture complexity upper threshold, the target CTU is a complex block.

5. The method for determining CTU according to claim 1, wherein: When the target CTU is a flat block or a complex block, the depth interval of the target CTU is obtained according to the preliminary depth traversal range or depth prediction data, specifically: When the target CTU is a complex block, obtaining a preliminary depth traversal range and depth prediction data of the target CTU; When the minimum depth value of the initial depth traversal range is less than or equal to the first depth threshold, depth expansion is performed, and the depth interval of the target CTU is [1,3]; When the value of the depth prediction data is greater than the second depth threshold, the depth interval of the target CTU is [2, 3].

6. The method for determining CTU according to claim 5, wherein: When the target CTU is a flat block or a complex block, obtaining a depth interval of the target CTU according to a preliminary depth traversal range or depth prediction data further includes: When the target CTU is a flat block, obtaining a preliminary depth traversal range and depth prediction data of the target CTU; When the minimum depth value of the initial depth traversal range is greater than or equal to the third depth threshold, depth clipping is performed, and the depth interval of the target CTU is [0, 2]; When the value of the depth prediction data is less than the fourth depth threshold, the depth range of the target CTU is [0, 2].

7. The method for determining CTU according to claim 1, wherein: According to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , get the depth interval of the target CTU, specifically: When DepthL co <Depth co , DepthL≥2 and Depth pre ≥2, the minimum depth value of the depth interval of the target CTU is 2; when DepthL co <Depth co , DepthL≥1 and Depth pre ≥1, the minimum depth value of the depth interval of the target CTU is 1; when DepthL co Depth co , DepthL≤1 and Depth pre ≤1, the maximum depth value of the depth interval of the target CTU is 1; when DepthL co Depth co , DepthL≤2 and Depth pre When ≤2, the maximum depth value of the depth interval of the target CTU is 2.

8. A CTU determination device, characterized in that: include: A first data acquisition module, a second data acquisition module, a third data acquisition module, a depth interval calculation module and a result generation module; The first data acquisition module is configured to acquire depth data of a plurality of encoded CTUs and determine a preliminary depth traversal range of a target CTU based on the depth data; wherein the plurality of encoded CTUs are CTUs that are adjacent to the target CTU in a spatiotemporal domain; The second data acquisition module is configured to acquire depth prediction data of the target CTU based on the spatiotemporal correlation between the plurality of encoded CTUs and the target CTU, and the depth data; The third data acquisition module is configured to acquire a texture complexity of the target CTU based on the pixel data of the target CTU; wherein the texture complexity includes one or more of the following: a flat block, a complex block, and a normal block; The depth interval calculation module is used to obtain the depth interval of the target CTU according to the preliminary depth traversal range or depth prediction data when the target CTU is a flat block or a complex block; when the target CTU is a normal block, the depth interval of the target CTU is obtained according to the depth prediction data and the depth data; wherein, when the target CTU is a normal block, the CUcol of the target CTU in the time domain, the CULc on the left side of the CUcol in the spatial domain, and the CUL on the left side of the target CTU in the spatial domain are determined according to the depth correlation between the several encoded CTUs and the target CTU; the several encoded CTUs include: CUcol, CULc and CUL; according to the depth data, the average depth value Depth of CUcol is obtained co , the average depth value of CULc DepthL co and the average depth value DepthL of CUL; according to Depth co 、DepthL co , DepthL and the depth prediction data Depth pre , obtain the depth interval of the target CTU; The result generating module is configured to select a target CTU according to the depth range of the target CTU, and perform HEVC encoding based on the selected target CTU.

9. A computer terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for determining a CTU according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the computer program controls the device where the computer-readable storage medium is located to perform the CTU determination method according to any one of claims 1 to 7.

Images