Video coding method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video coding method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an initial coding unit corresponding to a video frame of a to-be-coded video, and determining the initial coding unit as a first coding unit; determining spatial perception information of the first coding unit according to the original pixel matrix of the first coding unit, and judging whether the first coding unit is divided or not; under the condition of determining to divide the first coding unit, dividing the first coding unit into four sub-coding units with the same size, and performing recursive analysis on the sub-coding units in sequence; otherwise, traversing other coding units with the same size as the first coding unit and analyzing the other coding units. According to the embodiment of the invention, unnecessary recursive division can be terminated in advance, and the division efficiency of the coding unit and the video coding efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a video encoding method, device, electronic equipment and storage medium. Background Art

[0002] With the development of video coding technology, the popularity of high-definition and ultra-high-definition videos has put forward higher requirements for coding efficiency. The current common High Efficiency Video Coding (HEVC / H.265) standard can reduce the bit rate by about 50% compared to the traditional Advanced Video Coding (AVC) with the same video quality.

[0003] Different from AVC which uses fixed-size macroblocks, HEVC has four coding units (CU) of different sizes and uses a more flexible quadtree recursive partitioning method to divide the coding units, which greatly increases the computational complexity and leads to slower video encoding speed. Summary of the invention

[0004] The embodiments of the present invention provide a video encoding method, device, electronic device and storage medium, which can solve the problem of high computational complexity of the encoding unit division algorithm in the related art, resulting in slow encoding speed.

[0005] On the one hand, an embodiment of the present invention discloses a video encoding method, which is applied to an encoder and includes: Obtaining an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; Determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; In the case where it is determined to divide the first coding unit, the first coding unit is divided into four sub-coding units of the same size, and the sub-coding units are recursively analyzed in sequence; Otherwise, the first coding unit is determined as the target coding unit, and other coding units with the same size as the first coding unit are traversed and analyzed until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

[0006] Optionally, determining the spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit includes: Performing a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; Calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; The spatial perception information of the first coding unit is calculated according to the gradient average.

[0007] Optionally, calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix includes: According to the size of the first coding unit, the horizontal gradient matrix is ​​split into M horizontal sub-pixel matrices, and the vertical gradient matrix is ​​split into M vertical sub-pixel matrices; M is a positive integer; Calculate the average horizontal gradient of each horizontal sub-pixel matrix and the average vertical gradient of each vertical sub-pixel matrix; The gradient average of the original pixel matrix is ​​calculated according to the M horizontal gradient averages and the M vertical gradient averages.

[0008] Optionally, calculating the spatial perception information of the first coding unit according to the gradient average value includes: Calculating a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; Calculate the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; Spatial perception information of the first coding unit is determined according to the first parameter and a size of the first coding unit.

[0009] Optionally, the determining whether to divide the first coding unit includes: Determine a texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent information entropy of a texture area of ​​the first coding unit; Determining whether to divide the first coding unit according to the texture complexity.

[0010] Optionally, the determining whether to divide the first coding unit according to the texture complexity includes: When the texture complexity belongs to the first level, the first coding unit is not divided; In the case where the texture complexity belongs to the second level, respectively determining the texture complexity of four sub-coding units of the first coding unit, and if there is at least one sub-coding unit whose texture complexity belongs to the third level, determining to divide the first coding unit; When the texture complexity belongs to the third level, it is determined to divide the first coding unit.

[0011] Optionally, the method further comprises: Determine at least one group of pictures according to all video frames of the video to be encoded; For each image group, dividing the first frame image in the image group into coding units according to the first coding algorithm, temporarily storing the spatial perception information of the coding units that do not need to be further divided in the first array, and temporarily storing the spatial perception information of the coding units that need to be further divided in the second array; averaging the elements in the first array to obtain a first threshold; averaging the elements in the second array to obtain a second threshold; The level to which the texture complexity of the first coding unit belongs is determined according to the first threshold and the second threshold respectively.

[0012] On the other hand, an embodiment of the present invention discloses a video encoding device, which is applied to an encoder, and includes: An acquisition module, used for acquiring an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; an analysis module, configured to determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; A division module is used to divide the first coding unit into four sub-coding units of the same size and recursively analyze the sub-coding units in turn when it is determined that the first coding unit is to be divided; otherwise, the first coding unit is determined as a target coding unit, other coding units of the same size as the first coding unit are traversed and analyzed, until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

[0013] Optionally, the analysis module includes: A convolution submodule, configured to perform a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; A first calculation submodule, used for calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; A second calculation submodule is used to calculate the spatial perception information of the first coding unit according to the gradient average value.

[0014] Optionally, the first calculation submodule includes: a matrix splitting unit, configured to split the horizontal gradient matrix into M horizontal sub-pixel matrices and split the vertical gradient matrix into M vertical sub-pixel matrices according to the size of the first coding unit; M is a positive integer; A first calculation unit, used to calculate the average horizontal gradient value corresponding to each horizontal sub-pixel matrix and the average vertical gradient value corresponding to each vertical sub-pixel matrix; The second calculation unit is used to calculate the gradient average of the original pixel matrix according to the M horizontal gradient averages and the M vertical gradient averages.

[0015] Optionally, the second calculation submodule includes: A third calculation unit, configured to calculate a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; A fourth calculation unit, used for respectively calculating the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; A fifth calculation unit is used to determine the spatial perception information of the first coding unit according to the first parameter and the size of the first coding unit.

[0016] Optionally, the analysis module includes: A first determining submodule, configured to determine a texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent information entropy of a texture region of the first coding unit; The second determining submodule is used to determine whether to divide the first coding unit according to the texture complexity.

[0017] Optionally, the second determining submodule includes: A first processing unit, configured not to divide the first coding unit when the texture complexity belongs to the first level; A second processing unit is used to determine the texture complexity of each of the four sub-coding units of the first coding unit when the texture complexity belongs to the second level, and determine to divide the first coding unit if there is at least one sub-coding unit whose texture complexity belongs to the third level; The third processing unit is used to determine to divide the first coding unit when the texture complexity belongs to the third level.

[0018] Optionally, the device further comprises: A grouping module, configured to determine at least one image group according to all video frames of the video to be encoded; a temporary storage module, configured to divide the first frame image in each image group into coding units according to a first coding algorithm, temporarily store the spatial perception information of the coding units that do not need to be further divided in a first array, and temporarily store the spatial perception information of the coding units that need to be further divided in a second array; A first threshold calculation module, used for averaging the elements in the first array to obtain a first threshold; A second threshold calculation module, used for averaging the elements in the second array to obtain a second threshold; The level determination module is used to determine the level to which the texture complexity of the first coding unit belongs according to the first threshold and the second threshold respectively.

[0019] On the other hand, an embodiment of the present invention further discloses an electronic device, which includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to execute the aforementioned video encoding method by one or more processors.

[0020] The embodiment of the present invention further discloses a readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the aforementioned video encoding method.

[0021] The embodiments of the present invention include the following advantages: The embodiment of the present invention provides a video encoding method, which can determine spatial perception information according to the original pixel matrix of the first coding unit, use the spatial perception information to judge the texture complexity of the first coding unit, determine whether to divide the first coding unit, and when it is determined that the first coding unit needs to be divided, divide the first coding unit into four sub-coding units of the same size, and recursively analyze the four sub-coding units in turn; otherwise, directly terminate the further division of the first coding unit in advance, determine the first coding unit as the target coding unit, traverse other coding units of the same size as the first coding unit and analyze them until all coding units do not need to be divided. The embodiment of the present invention determines whether to divide the coding unit based on the spatial perception information of the coding unit, and can terminate unnecessary recursive division in advance, thereby improving the division efficiency of the coding unit and the video coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments of the present invention are briefly introduced below.

[0023] Figure 1 is a flow chart of steps of an embodiment of a video encoding method of the present invention; Figure 2It is a schematic diagram of a flow chart of a video encoding method of the present invention; Figure 3 It is a schematic diagram of a pixel matrix of a video frame of the present invention; Figure 4 It is a schematic diagram of a flow chart of calculating the average gradient value by blocks according to the present invention; Figure 5 It is a calculation flow chart of an SSI value of the present invention; Figure 6 is a structural block diagram of an embodiment of a video encoding device of the present invention; Figure 7 It is a structural block diagram of an electronic device provided by an example of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0025] Method Embodiment Reference Figure 1 , shows a flowchart of a video encoding method embodiment of the present invention, the method is applied to an encoder, and the method may specifically include the following steps: Step 101: obtaining an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; Step 102: determining spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determining whether to divide the first coding unit; Step 103: When it is determined to divide the first coding unit, divide the first coding unit into four sub-coding units of the same size, and recursively analyze the sub-coding units in turn; Step 104: Otherwise, determine the first coding unit as the target coding unit, traverse and analyze other coding units of the same size as the first coding unit, and perform encoding based on the target coding unit when all coding units do not need to be divided.

[0026] The video encoding method provided by an embodiment of the present invention can be applied to an encoder to divide a video frame into coding units, use spatial-perceptual information (SI) to measure the texture complexity of the coding unit to determine whether to further divide the coding unit, and terminate unnecessary recursive division in advance when it is determined that the coding unit does not need to be divided, thereby improving the division efficiency of the coding unit and the video encoding efficiency.

[0027] Specifically, in an embodiment of the present invention, the video frame of the video to be encoded is first divided into initial coding units. For example, the video frame can be divided according to the maximum size of the coding unit specified in the HEVC coding algorithm to obtain the initial coding unit of the video frame. It should be noted that in the HEVC coding algorithm, four sizes of coding units are specified, namely: 64×64, 32×32, 16×16 and 8×8, among which the size of the largest size coding unit (LCU) is 64×64. If the video frame is divided into the first round according to the maximum size specified by the HEVC algorithm, the initial coding unit obtained can be recorded as LCU. Of course, in an embodiment of the present invention, the video frame can also be initially divided according to other video coding algorithms to obtain the initial coding unit.

[0028] Next, the initial coding unit is determined as the first coding unit, and recursive analysis of the first coding unit begins: First, the spatial perceptual information of the first coding unit is determined according to the original pixel matrix of the first coding unit. In the embodiment of the present invention, the spatial perceptual information of the first coding unit can be calculated by using the original SI calculation formula, or a simplified spatial perceptual information (Simple Spatial-perceptual Information, SSI) can be used.

[0029] For example, the calculation formula of the original SI can be expressed as: (1) in, Indicates the pixel value of the pixel point with coordinates (i, j) in the original pixel matrix of the first coding unit, Sobel indicates edge detection (Sobel) operation, Ave indicates averaging operation, and N indicates the size of the first coding unit. For example, for a first coding unit with a size of 64×64, N=64; for a first coding unit with a size of 32×32, N=32.

[0030] It is understandable that the method of calculating the spatial perception information SI using formula (1) involves complex calculation operations such as calculating the mean square error and taking the square root, which has a high calculation complexity and will introduce additional calculations for the division of the coding unit, increase the calculation overhead, and affect the efficiency of the division of the coding unit. In order to reduce the calculation complexity, a simplified SI calculation method can be used to replace the original SI calculation method. The embodiment of the present invention provides a simplified calculation formula for the spatial perception information SSI, which can be expressed as: (2) in, is the gradient modulus of the original pixel matrix of the first coding unit, is the gradient average of the original pixel matrix of the first coding unit, and N is used to represent the size of the first coding unit.

[0031] Furthermore, the gradient average of the original pixel matrix of the first coding unit is It can be calculated based on the average horizontal gradient and the average vertical gradient of the original pixel matrix, which can be specifically expressed as: (3) in, is the average value of the horizontal gradient of the original pixel matrix of the first coding unit, is the average value of the vertical gradient of the original pixel matrix of the first coding unit.

[0032] The horizontal gradient of the original pixel matrix of the first coding unit can be expressed as: (4) The vertical gradient of the original pixel matrix of the first coding unit can be expressed as: (5) in, Represents the pixel value of the pixel with coordinates (i, j) in the original pixel matrix of the first coding unit.

[0033] Gradient Mode , which is equal to the horizontal gradient of the pixel with coordinates (i, j) in the original pixel matrix of the first coding unit The modulus of the vertical gradient of the pixel with coordinates (i, j) in the original pixel matrix of the first coding unit The sum of the moduli can be specifically expressed as: (6) In the embodiment of the present invention, the simplified spatial perception information is used to calculate the spatial perception information of the first coding unit according to the above formula (2), so that complex calculation operations such as mean square error and square root extraction do not need to be introduced, thereby reducing the calculation complexity, which is beneficial to improving the division efficiency of the coding unit, and further improving the video coding efficiency.

[0034] After obtaining the spatial perception information of the first coding unit, it can be determined whether to divide the first coding unit based on the spatial perception information. It can be understood that the spatial perception information is used to characterize the amount of spatial detail of a frame of an image. The more spatially complex the scene, the higher the SI value or SSI value. In an embodiment of the present invention, the spatial perception information can be used to characterize the texture complexity of the coding unit. Specifically, the higher the SI value or SSI value of the first coding unit, the higher the texture complexity of the first coding unit can be determined, and the coding unit needs to be further encoded to reflect more detailed texture information in the video frame. Conversely, the lower the SI value or SSI value of the first coding unit, the lower its texture complexity, and it is not necessary to continue encoding the coding unit.

[0035] For the first coding unit with low texture complexity, the recursive division can be terminated, that is, the first coding unit is not divided; for the first coding unit with high texture complexity, it can continue to be divided, that is, the first coding unit is divided into four sub-coding units of the same size, and the four sub-coding units are recursively analyzed in turn.

[0036] As an example, the texture complexity of the first coding unit can be determined based on the size relationship between the SI value or SSI value of the first coding unit and a preset threshold. For example, the SI value or SSI value of the first coding unit is compared with the preset threshold. If the SI value or SSI value is less than or equal to the preset threshold, it can be considered that the texture complexity of the first coding unit is low, and the first coding unit is not divided; if the SI value or SSI value is greater than the preset threshold, it can be considered that the texture complexity of the first coding unit is high, and the first coding unit needs to be divided. It can be understood that the preset threshold corresponding to the SI value and the preset threshold corresponding to the SSI value can be the same or different, and the specific value of the preset threshold can be determined based on experimental data analysis and coding requirements, and the embodiments of the present invention do not limit this.

[0037] It should be noted that the spatial perception information in the embodiment of the present invention may be an SI value or an SSI value. Optionally, in order to further reduce the computational complexity and improve the efficiency of dividing the coding unit, the SSI value may be used to determine the texture complexity of the first coding unit.

[0038] Reference Figure 2 , shows a schematic flow chart of a video encoding method provided by an embodiment of the present invention. Figure 2As shown, taking the initial coding unit of the video frame as the largest coding unit LCU (64×64) in the HEVC coding algorithm as an example, traversal starts from the LCU. In the first round of analysis, any LCU in the video frame is determined as the first coding unit, for example, the first LCU in the upper left corner of the video frame is determined as the first coding unit. According to the original pixel matrix of the first coding unit, the spatial perception information of the first coding unit is determined, for example, the SSI value of the first coding unit is calculated according to the above formula (2).

[0039] It should be noted that in the HEVC encoding algorithm, the minimum size of a coding unit is 8×8. Therefore, if the size of a coding unit is already 8×8, the coding unit will not be further divided and will be directly determined as the target coding unit.

[0040] like Figure 2 As shown, before comparing the SSI value of the first coding unit with the preset threshold, it can be determined whether the size of the first coding unit is greater than 8×8. If the size of the first coding unit is less than or equal to 8×8, the division of the first coding unit is directly terminated, and the first coding unit is determined as the target coding unit. If the size of the first coding unit is greater than 8×8, it is determined whether the SSI value of the first coding unit is less than the preset threshold. If the SSI value of the first coding unit is less than or equal to the preset threshold, it is determined not to divide the first coding unit, and the first coding unit is determined as the target coding unit. Other coding units with the same size as the first coding unit are traversed, that is, other coding units in the parent node that are at the same division depth as the first coding unit are traversed to perform a new round of recursive analysis.

[0041] If the SSI value of the first coding unit is greater than the preset threshold, it means that the texture complexity of the first coding unit is relatively high, and the first coding unit needs to be analyzed. Specifically, the first coding unit is divided into four sub-coding units of the same size. Next, these four sub-coding units are determined as new first coding units in turn, and recursive analysis is performed. For example, for a 64×64 LCU, it is divided into four 32×32 sub-coding units. For each 32×32 sub-coding unit, repeat the previous analysis steps: calculate the SSI value. At this time, the size of the first coding unit is 32×32, so continue to compare the size relationship between the SSI value and the preset threshold. If the SSI value is less than or equal to the preset threshold, it means that the corresponding texture information is relatively simple, and the 32×32 first coding unit is no longer further divided. It is determined as the target coding unit, and the other 32×32 sub-coding units corresponding to the LCU are traversed and analyzed. If the SSI value of a sub-coding unit is greater than the preset threshold, it means that the corresponding texture information is relatively complex, and it is necessary to continue to divide the 32×32 first coding unit into 4 16×16 sub-coding units, and determine these 4 16×16 sub-coding units as new first coding units in turn, and perform a new round of recursive analysis until the 8×8 target coding unit is used as the recursive exit, and then analyze the 16×16 sub-coding unit as the parent node, and repeat this cycle. After all sub-coding units are analyzed, start analyzing the next first coding unit until all coding units corresponding to the video frame do not need to be divided, and the video frame is encoded based on each target coding unit.

[0042] In the standard HEVC Coding Unit (CU) division process, the encoder traverses and searches all sub-CU divisions of the LCU (maximum size CU, 64x64), and obtains the final CU division by comparing the rate distortion cost (Rate DistortionCost, RD_Cost) of the current CU and the sub-CU. The CU division process can be divided into a top-down CU division process and a bottom-up merging process. The CU division process mainly completes the selection of the optimal intra-frame prediction mode and the calculation of the rate distortion cost under the optimal prediction mode. The CU merging process compares the sum of the rate distortion costs of the four sub-CUs with the current depth CU rate distortion cost from bottom to top to determine whether the current CU is divided into four sub-CUs. After the CU division is completed, the encoder performs the final entropy coding according to the division method and optimal prediction mode selected for each layer of CU. This process is very time-consuming.

[0043] Compared with the division process of the standard HEVC coding unit, the video coding method provided by the embodiment of the present invention can determine the spatial perception information according to the original pixel matrix of the first coding unit, use the spatial perception information to judge the texture complexity of the first coding unit, determine whether to divide the first coding unit, and when it is determined that the first coding unit needs to be divided, divide the first coding unit into four sub-coding units of the same size, and recursively analyze the four sub-coding units in turn; otherwise, directly terminate the further division of the first coding unit in advance, determine the first coding unit as the target coding unit, traverse and analyze other coding units of the same size as the first coding unit until all coding units do not need to be divided. The embodiment of the present invention determines whether to divide the coding unit based on its spatial perception information, and can terminate unnecessary recursive division in advance, thereby improving the division efficiency of the coding unit and the video coding efficiency.

[0044] In an optional embodiment of the present invention, the step 102 of determining the spatial perception information of the first coding unit according to the original pixel matrix of the first coding unit includes: Step S11, performing a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; Step S12, calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; Step S13: Calculate spatial perception information of the first coding unit according to the gradient average value.

[0045] In the embodiment of the present invention, a simplified version of the spatial perception information calculation method may be used. Specifically, an edge detection operator may be used to perform a convolution operation on the original pixel matrix of the first coding unit to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix.

[0046] Then, the gradient average of the original pixel matrix is ​​calculated based on the horizontal gradient matrix and the vertical gradient matrix.

[0047] After determining the gradient average value of the original pixel matrix of the first coding unit, the simplified spatial perception information SSI value of the first coding unit is calculated according to the gradient average value. The specific calculation method can refer to the above formula (2).

[0048] Optionally, calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix includes: Step S21, according to the size of the first coding unit, split the horizontal gradient matrix into M horizontal sub-pixel matrices, and split the vertical gradient matrix into M vertical sub-pixel matrices; M is a positive integer; Step S22, respectively calculating the average horizontal gradient value corresponding to each horizontal sub-pixel matrix and the average vertical gradient value corresponding to each vertical sub-pixel matrix; Step S23: Calculate the gradient average of the original pixel matrix according to the M horizontal gradient averages and the M vertical gradient averages.

[0049] In the embodiment of the present invention, the original pixel matrix of the first encoding unit may be divided into a plurality of sub-pixel matrices, and the gradient average value of the original pixel matrix may be calculated by adopting a block calculation method.

[0050] For example, if the size of the original pixel matrix of the first coding unit is N×N, the original pixel matrix can be equally divided into M sub-pixel matrices of size 4×4, where M= The horizontal gradient average and the vertical gradient average are calculated for these sub-pixel matrices separately, and then the gradient average of the first coding unit is calculated based on the M horizontal gradient averages and the M vertical gradient averages.

[0051] As another example, in the embodiment of the present invention, after step S11, the horizontal gradient matrix and the vertical gradient matrix corresponding to the original pixel matrix have been obtained, and the horizontal gradient matrix can be directly split into M horizontal sub-pixel matrices, and the vertical gradient matrix can be split into M vertical sub-pixel matrices, M= .

[0052] It can be understood that the sizes of the horizontal gradient matrix and the vertical gradient matrix are the same as the size of the original pixel matrix of the first coding unit. For example, the size of the original pixel matrix of the first coding unit with a size of N×N is N×N, and the size of the corresponding horizontal gradient matrix and the vertical gradient matrix are also N×N.

[0053] Then, the horizontal gradient average of each horizontal sub-pixel matrix and the vertical gradient average of each vertical sub-pixel matrix are calculated by block calculation method. The horizontal gradient averages of M horizontal sub-pixel matrices and the vertical gradient averages of M vertical sub-pixel matrices are accumulated and averaged to obtain the gradient average of the original pixel matrix.

[0054] Exemplarily, the horizontal gradient average of the original pixel matrix of the first coding unit is It can be expressed as: (7) The vertical gradient average of the original pixel matrix of the first coding unit It can be expressed as: (8) in, represents the horizontal gradient of the pixel with coordinates (i, j) in the original pixel matrix of the first coding unit, represents the vertical gradient of the pixel point with coordinates (i, j) in the original pixel matrix of the first coding unit, t represents the relative horizontal coordinate of the sub-pixel matrix in the original pixel matrix, k represents the relative vertical coordinate of the sub-pixel matrix in the original pixel matrix, and N represents the size of the first coding unit.

[0055] The horizontal gradient average of the original pixel matrix The vertical gradient average By adding them together, we can get the average gradient, as shown in the above formula (3).

[0056] Optionally, calculating the spatial perception information of the first coding unit according to the gradient average value includes: Step S31, calculating a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; Step S32, calculating the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; Step S33: Determine spatial perception information of the first coding unit according to the first parameter and the size of the first coding unit.

[0057] In the embodiment of the present invention, the gradient modulus matrix corresponding to the first coding unit may be calculated according to the horizontal gradient matrix and the vertical gradient matrix of the first coding unit.

[0058] Specifically, the gradient modulus at coordinate (i, j) in the gradient modulus matrix , which is equal to the horizontal gradient of the first coding unit with coordinates (i, j) in the horizontal gradient matrix The modulus of the vertical gradient matrix of the first coding unit with the coordinate (i, j) The sum of the moduli can be specifically expressed as: (6) Then, according to the gradient average value of the original pixel matrix calculated in the previous step, the mean difference of each element in the gradient modulus matrix is ​​calculated to obtain the first parameter. By calculating the ratio of the first parameter to the size (N×N) of the first coding unit, the simplified spatial perception information SSI value of the first coding unit can be obtained, which can be specifically expressed as: (2) in, is the gradient modulus of the original pixel matrix of the first coding unit, is the gradient average of the original pixel matrix of the first coding unit, and N is used to represent the size of the first coding unit.

[0059] Reference Figure 3 , shows a schematic diagram of a pixel matrix of a video frame. Figure 3 As shown, the 64×64 LCU pixel matrix in the upper left corner of the video frame is extracted. In order to conveniently represent the calculation process, an 8×8 CU pixel matrix is ​​further extracted from the LCU pixel matrix for subsequent display.

[0060] Reference Figure 4 , shows a schematic diagram of a flow chart of calculating the average gradient value by blocks according to an embodiment of the present invention. Figure 4 As shown, first, according to formula (4), the Sobel calculation is performed on the 8×8 CU pixel matrix in the horizontal direction, that is, the Sobel operator is used to perform a convolution operation on the 8×8 CU pixel matrix in the horizontal direction to obtain the horizontal gradient matrix of the CU pixel matrix; according to formula (5), the Sobel calculation is performed on the 8×8 CU pixel matrix in the vertical direction to obtain the vertical gradient matrix of the CU pixel matrix.

[0061] Then, the horizontal gradient matrix is ​​equally divided into 4 horizontal sub-pixel matrices of size 4×4, and the vertical gradient matrix is ​​equally divided into 4 vertical sub-pixel matrices of size 4×4. The average gradient value of each sub-pixel matrix is ​​calculated using the block calculation method, and then the horizontal and vertical gradient matrices of the 8×8 CU are formed.

[0062] Reference Figure 5 , shows a flow chart of calculating an SSI value in an embodiment of the present invention. Figure 5 As shown, according to the horizontal horizontal gradient matrix and the average vertical gradient matrix, the average gradient of the current 8×8 CU is calculated according to the above formula (3), that is, Figure 5 The average gradient norm in .

[0063] According to the horizontal gradient matrix and vertical gradient matrix of 8×8 CU, the above formula (6) is used to calculate Figure 5 The gradient norm matrix in .

[0064] Finally, the SSI value of the 8×8 CU is calculated using the gradient mean value and gradient modulus matrix of the 8×8 CU according to the above formula (2).

[0065] The embodiment of the present invention adopts a block calculation method to determine the spatial perception information of the first coding unit, which can avoid some repeated calculations, facilitate parallel calculations, improve the calculation rate, and help further improve the division efficiency of the coding units.

[0066] In an optional embodiment of the present invention, the determining whether to divide the first coding unit includes: Step S41, determining the texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent the information entropy of the texture area of ​​the first coding unit; Step S42: Determine whether to divide the first coding unit according to the texture complexity.

[0067] In an embodiment of the present invention, spatial perception information can be used to characterize the texture complexity of a coding unit. Specifically, the higher the SI value or SSI value of the first coding unit, the higher the texture complexity of the first coding unit can be determined, indicating that the information entropy of the texture area of ​​the first coding unit is greater, the distribution of pixel values ​​is more random, the difference in pixel values ​​is greater, and the compression complexity is higher. Further recursive analysis is required to divide the first coding unit into smaller sub-coding units and reduce the number of pixels contained in a single coding unit, so as to reduce the information entropy of the texture area within the coding unit and reduce the compression complexity. Conversely, the lower the SI value or SSI value of the first coding unit, the lower its texture complexity, indicating that the information entropy of the texture area of ​​the first coding unit is lower, the distribution of pixel values ​​is relatively regular, the difference in pixel values ​​is smaller, and the compression complexity is lower, that is, there is no need to further recursively analyze the coding unit and divide it again.

[0068] For the first coding unit with low texture complexity, the recursive division can be terminated, that is, the first coding unit is not divided; for the first coding unit with high texture complexity, it can continue to be divided into four sub-coding units of the same size, and the sub-coding unit is determined as the new first coding unit, and the four sub-coding units are recursively analyzed in turn.

[0069] Optionally, the determining whether to divide the first coding unit according to the texture complexity includes: Step S51: when the texture complexity belongs to the first level, the first coding unit is not divided; Step S52: when the texture complexity belongs to the second level, determining the texture complexity of each of the four sub-coding units of the first coding unit respectively; if there is at least one sub-coding unit whose texture complexity belongs to the third level, determining to divide the first coding unit; Step S53: When the texture complexity belongs to the third level, determine to divide the first coding unit.

[0070] In the embodiment of the present invention, the texture complexity of the first coding unit may be divided into three levels according to the spatial perception information of the first coding unit, and the corresponding texture complexities are: first level < second level < third level.

[0071] For the first coding unit of the first level, it can be considered that its texture complexity is low and it does not need to be further divided. The division of the first coding unit can be terminated directly.

[0072] For the first coding unit of the second level, it can be considered that its texture complexity is medium. In this case, it can be determined whether to divide the first coding unit by further judging the texture complexity of the four sub-coding units corresponding to the first coding unit. Specifically, if the texture complexity of at least one sub-coding unit among the four sub-coding units of the first coding unit belongs to the third level, it means that the texture complexity of the sub-coding unit of the first coding unit is large, that is, the texture is still relatively complex, and the first coding unit needs to be further divided, and the four sub-coding units of the first coding unit continue to be recursively analyzed. Specifically, the four sub-coding units of the first coding unit are determined as new first coding units in turn, and according to the steps 102 to 105 above, it is determined whether to divide the new first coding unit (sub-coding unit). If the texture complexity of the four sub-coding units of the first coding unit is the first level or the second level, it means that the texture complexity of the four sub-coding units of the first coding unit is small, that is, the texture is relatively simple, and there is no need to divide the first coding unit.

[0073] For the first coding unit of the third level, its texture is relatively complex and needs to be further divided. The first coding unit can be directly divided into four sub-coding units of the same size, and the four sub-coding units are recursively analyzed in turn.

[0074] It should be noted that in the HEVC encoding algorithm, the minimum size of the coding unit is 8×8. Therefore, if the size of a coding unit is already 8×8, it is considered that the size of the 8×8 coding unit is small enough, and there should be no relatively complex texture inside, and the corresponding texture features are simple enough, that is, the coding unit is no longer divided further, and the coding unit is directly determined as the target coding unit.

[0075] Optionally, determining the texture complexity of the first coding unit according to the spatial perception information includes: Step S61: When the spatial perception information is less than a first threshold, determining that the texture complexity of the first coding unit belongs to a first level; Step S62: When the spatial perception information is greater than or equal to the first threshold and less than a second threshold, determine that the texture complexity of the first coding unit belongs to the second level; Step S63: When the spatial perception information is greater than or equal to the second threshold, determine that the texture complexity of the first coding unit belongs to the third level.

[0076] In the embodiment of the present invention, the levels of texture complexity of the coding unit may be divided according to the first threshold and the second threshold.

[0077] Exemplarily, according to the SSI value of the CU and the first threshold , the second threshold (where subscript Represents the CU size, N={8|16|32|64}), the CU can be divided into the following three categories:

[0078] Among them, Simple corresponds to the first level in the embodiment of the present invention, indicating that the texture complexity of the coding unit is low, and the division of the Simple class coding unit can be terminated in advance. Medium corresponds to the second level in the embodiment of the present invention. For the Medium class coding unit, it means that the texture complexity of the coding unit is high, and it is necessary to calculate the SSI values ​​of the four sub-CUs of the current CU, and determine the texture complexity of the sub-CU according to the SSI values ​​of the four sub-CUs. If at least one of the four sub-CUs is a Medium class coding unit, it means that there is at least one sub-CU with high texture complexity, and the current CU needs to be divided. Complex corresponds to the third level in the embodiment of the present invention, indicating that the texture complexity of the coding unit is high. For the Complex class coding unit, it is directly divided.

[0079] It is understandable that the first threshold , the second threshold This is determined based on experimental data analysis and coding requirements, and is not limited in the embodiments of the present invention.

[0080] As an example, a reference frame may be selected from a video frame of the video to be encoded, and the first threshold value may be determined based on the SSI value of each coding unit of the reference frame. and the second threshold .

[0081] Optionally, the method further comprises: Step S71, determining at least one image group according to all video frames of the video to be encoded; Step S72: for each image group, divide the first frame image in the image group into coding units according to the first coding algorithm, temporarily store the spatial perception information of the coding units that do not need to be further divided in the first array, and temporarily store the spatial perception information of the coding units that need to be further divided in the second array; Step S73, averaging the elements in the first array to obtain a first threshold; Step S74, averaging the elements in the second array to obtain a second threshold; Step S75: Determine the level to which the texture complexity of the first coding unit belongs according to the first threshold and the second threshold. In an embodiment of the present invention, all video frames of the video to be encoded can be grouped into at least one group of pictures (Group of Pictures, GOP), and a reference frame is selected from each group of pictures. According to the spatial perception information of each coding unit of the reference frame, a first threshold and a second threshold corresponding to the group of pictures are determined, and then the level of texture complexity of the coding units of other video frames in the group of pictures is determined according to the first threshold and the second threshold. Whether to divide the coding units is determined based on the level, and the details can be referred to the aforementioned steps S51 to S53.

[0082] As an example, when determining the image group, at least one continuous video frame may be divided into an image group according to the sequence of the video frames in the video to be encoded, and the number of video frames included in each image group may be the same or different. For example, assuming that the video frame to be encoded includes 100 video frames, starting from the first video frame, 10 continuous video frames may be divided into an image group, thereby obtaining 10 image groups.

[0083] As another example, the image groups may be divided according to the feature information of the video frames in the video to be encoded. For example, the video frames with the same and continuous scenes in the video to be encoded are divided into one image group. For example, assuming that after scene recognition, the order of appearance of the scenes in the video to be encoded is: scene 1→scene 2 (recorded as scene 2.1)→scene 3→scene 2 (recorded as scene 2.2), the video frames corresponding to scene 1 may be divided into one image group, the video frames corresponding to scene 2.1 may be divided into one image group, scene 3 may be divided into one image group, and the video frames corresponding to the second group of scene 2.2 may be divided into one image group. Alternatively, the video frames containing the same object in the video to be encoded may be divided into one image group. For example, assuming that after object recognition, the order of appearance of the objects in the video to be encoded is: child→puppy→child and puppy, the video frames containing only the child may be divided into one image group, the video frames containing only the puppy may be divided into one image group, and the video frames containing both the child and the puppy may be divided into one image group.

[0084] It can be understood that, in an embodiment of the present invention, an image group can correspond to a set of thresholds. For the reference frames in the image group, the first encoding algorithm can be directly used to divide the encoding units. Exemplarily, the first frame image in each image group can be used as a reference frame, and the first frame image can be divided into encoding units according to the first encoding algorithm. Among them, the first encoding algorithm can be a standard HEVC encoding algorithm, or an AVC encoding algorithm, etc., which is not limited in the embodiment of the present invention.

[0085] In the process of dividing the coding units of the first frame image in the image group, the spatial perception information of the coding units that do not need to be further divided is temporarily stored in the first array, and the spatial perception information of the coding units that need to be further divided is temporarily stored in the second array. Then, the elements in the first array are averaged to obtain the first threshold value; the elements in the second array are averaged to obtain the second threshold value.

[0086] After obtaining the first threshold and the second threshold, the level to which the texture complexity of the coding units of other video frames in the image group belongs can be determined according to the first threshold and the second threshold, and whether to divide the coding units is determined based on the level.

[0087] It should be noted that, for an image group, the reference frame therein, such as the first frame image, can be divided into coding units according to the first coding algorithm; for other video frames in the image group, the coding units can be divided according to the method provided in the embodiment of the present invention.

[0088] For example, assume that the video to be encoded contains a total of 20 video frames. According to the order of the video frames, these 20 video frames are divided into two image groups. The first 10 video frames P1-P10 are recorded as image group A, and the last 10 video frames P11-P20 are recorded as image group B. For image groups A and B, take their respective first frame images as reference frames, and calculate the first threshold and the second threshold respectively. Specifically, for image group A, take the first frame image, that is, the first video frame P1 of the video to be encoded, and follow the methods of the aforementioned steps S71 to S74 to obtain a set of thresholds, which are recorded as the first threshold A1 and the second threshold A2. For image group B, also take the first frame image, that is, the video frame P11 of the video to be encoded, and follow the methods of the aforementioned steps S71 to S74 to obtain a set of thresholds, which are recorded as the first threshold B1 and the second threshold B2.

[0089] It is understandable that the first frame image P1 in image group A and the first frame image P11 in image group B have completed the division of coding units according to the first coding algorithm during the process of calculating the first threshold and the second threshold, and therefore no further analysis is required.

[0090] For other video frames in the group of images A and the group of images B, the coding units may be divided according to the methods provided in the embodiments of the present invention: For the video frames P2-P10 in the image group A, the initial coding units corresponding to the respective video frames are first determined respectively. For example, the video frames are initially divided according to the HEVC coding algorithm to obtain the initial coding units LCU. Then, the initial coding unit is determined as the first coding unit, and the first round of recursive analysis is started: according to the original pixel matrix of the first coding unit, the simplified spatial perception information of the first coding unit, such as the SSI value, is determined, and then according to the relationship between the SSI value and the first threshold value A1 and the second threshold value A2, the level of the texture complexity of the first coding unit is determined, that is, it is determined whether the first coding unit belongs to the first level, the second level and the third level. After determining the level of the first coding unit, it can be further determined whether to divide the first coding unit according to the method of the aforementioned steps S51 to S53.

[0091] Similarly, for video frames P12-P20 in image group B, the same method is used to determine the SSI value of each first coding unit in each video frame, and then determine the level of texture complexity of the first coding unit based on the relationship between the SSI value and the first threshold B1 and the second threshold B2, and then determine whether to divide the first coding unit based on the level.

[0092] In an embodiment of the present invention, the video frames of the video to be encoded are divided into image groups. For each image group, a first threshold and a second threshold are calculated based on the first frame image in the image group. The video frames in the image group often have a certain correlation in texture features, and the calculated threshold is more in line with the texture features of each video frame in the image group; the texture complexity levels of other video frames in the image group are divided according to the first threshold and the second threshold, and the level division results are more accurate, which is beneficial to improving the division accuracy of the encoding unit.

[0093] In summary, an embodiment of the present invention provides a video encoding method, which can determine spatial perception information based on the original pixel matrix of the first coding unit, use the spatial perception information to judge the texture complexity of the first coding unit, determine whether to divide the first coding unit, and when it is determined that the first coding unit needs to be divided, divide the first coding unit into four sub-coding units of the same size, and recursively analyze the four sub-coding units in turn; otherwise, directly terminate further division of the first coding unit in advance, determine the first coding unit as the target coding unit, traverse other coding units of the same size as the first coding unit and analyze them until all coding units do not need to be divided. The embodiment of the present invention determines whether to divide the coding unit based on its spatial perception information, and can terminate unnecessary recursive division in advance, thereby improving the division efficiency of the coding unit and the video encoding efficiency.

[0094] Device Embodiment Reference Figure 6 , shows a structural block diagram of an embodiment of a video encoding device of the present invention, the device is applied to an encoder, and the device may specifically include: An acquisition module 201 is used to acquire an initial coding unit corresponding to a video frame of a video to be encoded, and determine the initial coding unit as a first coding unit; An analysis module 202 is used to determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; The division module 203 is used to divide the first coding unit into four sub-coding units of the same size and recursively analyze the sub-coding units in turn when it is determined that the first coding unit is to be divided; otherwise, the first coding unit is determined as a target coding unit, and other coding units of the same size as the first coding unit are traversed and analyzed until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

[0095] Optionally, the analysis module includes: A convolution submodule, configured to perform a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; A first calculation submodule, used for calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; A second calculation submodule is used to calculate the spatial perception information of the first coding unit according to the gradient average value.

[0096] Optionally, the first calculation submodule includes: a matrix splitting unit, configured to split the horizontal gradient matrix into M horizontal sub-pixel matrices and split the vertical gradient matrix into M vertical sub-pixel matrices according to the size of the first coding unit; M is a positive integer; A first calculation unit, used to calculate the average horizontal gradient value corresponding to each horizontal sub-pixel matrix and the average vertical gradient value corresponding to each vertical sub-pixel matrix; The second calculation unit is used to calculate the gradient average of the original pixel matrix according to the M horizontal gradient averages and the M vertical gradient averages.

[0097] Optionally, the second calculation submodule includes: A third calculation unit, configured to calculate a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; A fourth calculation unit, used for respectively calculating the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; A fifth calculation unit is used to determine the spatial perception information of the first coding unit according to the first parameter and the size of the first coding unit.

[0098] Optionally, the analysis module includes: A first determining submodule, configured to determine a texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent information entropy of a texture region of the first coding unit; The second determining submodule is used to determine whether to divide the first coding unit according to the texture complexity.

[0099] Optionally, the second determining submodule includes: A first processing unit, configured to terminate the division of the first coding unit when the texture complexity belongs to the first level; A second processing unit is used to determine the texture complexity of each of the four sub-coding units of the first coding unit when the texture complexity belongs to the second level, and determine to divide the first coding unit if the texture complexity of at least one sub-coding unit among the four sub-coding units belongs to the third level; The third processing unit is used to determine to divide the first coding unit when the texture complexity belongs to the third level.

[0100] Optionally, the device further comprises: A grouping module, configured to determine at least one image group according to all video frames of the video to be encoded; a temporary storage module, configured to divide the first frame image in each image group into coding units according to a first coding algorithm, temporarily store the spatial perception information of the coding units that do not need to be further divided in a first array, and temporarily store the spatial perception information of the coding units that need to be further divided in a second array; A first threshold calculation module, used for averaging the elements in the first array to obtain a first threshold; A second threshold calculation module, used for averaging the elements in the second array to obtain a second threshold; The level determination module is used to determine the level to which the texture complexity of the first coding unit belongs according to the first threshold and the second threshold respectively.

[0101] In summary, an embodiment of the present invention provides a video encoding device, which can determine spatial perception information based on the original pixel matrix of the first coding unit, use the spatial perception information to judge the texture complexity of the first coding unit, determine whether to divide the first coding unit, and when it is determined that the first coding unit needs to be divided, divide the first coding unit into four sub-coding units of the same size, and recursively analyze the four sub-coding units in turn; otherwise, directly terminate the further division of the first coding unit in advance, determine the first coding unit as the target coding unit, traverse other coding units of the same size as the first coding unit and analyze them until all coding units do not need to be divided. The embodiment of the present invention determines whether to divide the coding unit based on its spatial perception information, and can terminate unnecessary recursive division in advance, thereby improving the division efficiency of the coding unit and the video coding efficiency.

[0102] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0104] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0105] An embodiment of the present invention provides an electronic device for video encoding, the electronic device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: Obtaining an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; Determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; In the case where it is determined to divide the first coding unit, the first coding unit is divided into four sub-coding units of the same size, and the sub-coding units are recursively analyzed in turn; Otherwise, the first coding unit is determined as the target coding unit, and other coding units with the same size as the first coding unit are traversed and analyzed until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

[0106] Figure 7 1 is a block diagram of an electronic device 600 for video encoding according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0107] Reference Figure 7 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0108] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0109] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0111] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0112] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operation mode, such as a call mode, a recording mode, and a voice information processing mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0113] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0114] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0115] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0116] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0117] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0118] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device (server or terminal), enables the processor to perform Figure 1 The video encoding method shown.

[0119] The video encoding method, device, electronic device and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A video encoding method, characterized in that: Applied to an encoder, the method comprises: Obtaining an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; Determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; In the case where it is determined to divide the first coding unit, the first coding unit is divided into four sub-coding units of the same size, and the sub-coding units are recursively analyzed in sequence; Otherwise, the first coding unit is determined as the target coding unit, and other coding units with the same size as the first coding unit are traversed and analyzed until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

2. The method according to claim 1, characterized in that The determining, according to the original pixel matrix of the first coding unit, the spatial perception information of the first coding unit includes: Performing a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; Calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; The spatial perception information of the first coding unit is calculated according to the gradient average.

3. The method according to claim 2, characterized in that The step of calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix comprises: According to the size of the first coding unit, the horizontal gradient matrix is ​​split into M horizontal sub-pixel matrices, and the vertical gradient matrix is ​​split into M vertical sub-pixel matrices; M is a positive integer; Calculate the average horizontal gradient of each horizontal sub-pixel matrix and the average vertical gradient of each vertical sub-pixel matrix; The gradient average of the original pixel matrix is ​​calculated according to the M horizontal gradient averages and the M vertical gradient averages.

4. The method according to claim 2, characterized in that: The calculating the spatial perception information of the first coding unit according to the gradient average value includes: Calculating a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; Calculate the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; Spatial perception information of the first coding unit is determined according to the first parameter and a size of the first coding unit.

5. The method according to claim 1, characterized in that The determining whether to divide the first coding unit includes: Determine a texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent information entropy of a texture area of ​​the first coding unit; Determining whether to divide the first coding unit according to the texture complexity.

6. The method according to claim 5, characterized in that The determining whether to divide the first coding unit according to the texture complexity includes: When the texture complexity belongs to the first level, the first coding unit is not divided; In the case where the texture complexity belongs to the second level, respectively determining the texture complexity of four sub-coding units of the first coding unit, and if there is at least one sub-coding unit whose texture complexity belongs to the third level, determining to divide the first coding unit; When the texture complexity belongs to the third level, it is determined to divide the first coding unit.

7. The method according to claim 6, characterized in that The method further comprises: Determine at least one group of pictures according to all video frames of the video to be encoded; For each image group, dividing the first frame image in the image group into coding units according to the first coding algorithm, temporarily storing the spatial perception information of the coding units that do not need to be further divided in the first array, and temporarily storing the spatial perception information of the coding units that need to be further divided in the second array; averaging the elements in the first array to obtain a first threshold; averaging the elements in the second array to obtain a second threshold; The level to which the texture complexity of the first coding unit belongs is determined according to the first threshold and the second threshold respectively.

8. A video encoding device, characterized in that: The device is applied to an encoder, and the device comprises: An acquisition module, used for acquiring an initial coding unit corresponding to a video frame of a video to be encoded, and determining the initial coding unit as a first coding unit; an analysis module, configured to determine spatial perception information of the first coding unit according to an original pixel matrix of the first coding unit, and determine whether to divide the first coding unit; A division module is used to divide the first coding unit into four sub-coding units of the same size and recursively analyze the sub-coding units in turn when it is determined that the first coding unit is to be divided; otherwise, the first coding unit is determined as a target coding unit, and other coding units of the same size as the first coding unit are traversed and analyzed until all coding units do not need to be divided, and encoding is performed based on the target coding unit.

9. The device according to claim 8, characterized in that The analysis module comprises: A convolution submodule, configured to perform a convolution operation on the original pixel matrix of the first coding unit using an edge detection operator to obtain a horizontal gradient matrix and a vertical gradient matrix corresponding to the original pixel matrix; A first calculation submodule, used for calculating the gradient average of the original pixel matrix according to the horizontal gradient matrix and the vertical gradient matrix; A second calculation submodule is used to calculate the spatial perception information of the first coding unit according to the gradient average value.

10. The device according to claim 9, characterized in that The first computing submodule includes: a matrix splitting unit, configured to split the horizontal gradient matrix into M horizontal sub-pixel matrices and split the vertical gradient matrix into M vertical sub-pixel matrices according to the size of the first coding unit; M is a positive integer; A first calculation unit, used to calculate the average horizontal gradient value corresponding to each horizontal sub-pixel matrix and the average vertical gradient value corresponding to each vertical sub-pixel matrix; The second calculation unit is used to calculate the gradient average of the original pixel matrix according to the M horizontal gradient averages and the M vertical gradient averages.

11. The device according to claim 9, characterized in that The second computing submodule includes: A third calculation unit, configured to calculate a gradient modulus matrix corresponding to the first coding unit according to the horizontal gradient matrix and the vertical gradient matrix; A fourth calculation unit, used for respectively calculating the mean difference of each element in the gradient modulus matrix according to the gradient average value to obtain a first parameter; A fifth calculation unit is used to determine the spatial perception information of the first coding unit according to the first parameter and the size of the first coding unit.

12. The device according to claim 8, characterized in that The analysis module comprises: A first determining submodule, configured to determine a texture complexity of the first coding unit according to the spatial perception information; the texture complexity is used to represent information entropy of a texture region of the first coding unit; The second determining submodule is used to determine whether to divide the first coding unit according to the texture complexity.

13. The device according to claim 12, characterized in that The second determining submodule includes: A first processing unit, configured not to divide the first coding unit when the texture complexity belongs to the first level; A second processing unit is used to determine the texture complexity of each of the four sub-coding units of the first coding unit when the texture complexity belongs to the second level, and determine to divide the first coding unit if there is at least one sub-coding unit whose texture complexity belongs to the third level; The third processing unit is used to determine to divide the first coding unit when the texture complexity belongs to the third level.

14. An electronic device, characterized in that: The electronic device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to execute the video encoding method according to any one of claims 1 to 7 by one or more processors.

15. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the processor is enabled to execute the video encoding method as claimed in any one of claims 1 to 7.

Images