Intra-frame angle prediction method and device, electronic equipment and computer program product

Abstract

The invention relates to an intra-frame angle prediction method and device, electronic equipment and a computer program product, and belongs to the technical field of video encoding and decoding. The method comprises the following steps: dividing an angle mode of intra-frame angle prediction into a plurality of angle mode types, and respectively providing a corresponding assembly function for each angle mode type; acquiring a to-be-predicted pixel in a to-be-predicted pixel block, and acquiring a reference pixel of the to-be-predicted pixel from a pixel mapping table corresponding to an angle mode type through an assembly function corresponding to the angle mode type of the to-be-predicted pixel; obtaining an interpolation coefficient of the reference pixel according to an interpolation coefficient mapping table corresponding to the angle mode type; and obtaining a predicted pixel value of the to-be-predicted pixel according to the pixel value of each reference pixel and the interpolation coefficient. The angle prediction calculation complexity can be reduced, and the decoding efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video coding, in particular to an intra angular prediction method, an intra angular prediction device, an electronic device and a computer program product. BACKGROUND

[0002] VVC (Versatile Video Coding) is the latest generation of international video coding standard. Intra prediction as an important module in the video hybrid coding framework, its core principle is to use the already coded pixels of the same frame image to predict the current pixels, thereby greatly reducing the spatial redundancy existing in the video, and is one of the core methods to improve the compression rate.

[0003] In the intra prediction module, the VVC intra angular prediction algorithm is the most basic algorithm, and its core principle is to use the already coded reference pixels from a certain angle direction to predict the current pixels, wherein the angle is specified in the standard. More complex intra angular prediction not only improves the compression rate, but also increases the complexity of client decoding, affecting the user experience.

[0004] Therefore, there is an urgent need in the art for an intra angular prediction method that can reduce the computational complexity of intra angular prediction and improve decoding efficiency.

[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present disclosure is to provide an intra angular prediction method, an intra angular prediction device, an electronic device and a computer program product, which can at least reduce the computational complexity of intra angular prediction and improve decoding efficiency to some extent.

[0007] According to a first aspect of the present disclosure, an intra angular prediction method is provided, comprising:

[0008] dividing the angle mode of the intra angular prediction into a plurality of angle mode types, and providing a corresponding assembly function for each angle mode type;

[0009] obtaining a to-be-predicted pixel in a to-be-predicted pixel block, and obtaining a reference pixel of the to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel;

[0010] obtaining an interpolation coefficient of the reference pixel according to an interpolation coefficient mapping table corresponding to the angle mode type;

[0011] According to the pixel value of each reference pixel and the interpolation coefficient, a prediction pixel value of the to-be-predicted pixel is obtained.

[0012] In an example embodiment of the present disclosure, the obtaining of the to-be-predicted pixel in the to-be-predicted pixel block comprises:

[0013] Meanwhile, a plurality of continuous to-be-predicted pixels in the to-be-predicted pixel block are obtained, and a plurality of to-be-referenced pixels are obtained according to the plurality of continuous to-be-predicted pixels;

[0014] The reference pixel index value corresponding to each to-be-predicted pixel is obtained from the pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel.

[0015] The plurality of to-be-referenced pixels are sorted according to the reference pixel index value, and the reference pixel corresponding to each to-be-predicted pixel is obtained.

[0016] In an example embodiment of the present disclosure, the sorting of the plurality of to-be-referenced pixels according to the reference pixel index value, and the obtaining of the reference pixel corresponding to each to-be-predicted pixel comprises:

[0017] The plurality of to-be-referenced pixels are sorted according to the reference pixel index value through a sorting instruction in the assembly function, and the reference pixel corresponding to each to-be-predicted pixel is obtained.

[0018] In an example embodiment of the present disclosure, the angle mode type comprises a wide angle type, and the obtaining of the plurality of continuous to-be-predicted pixels in the to-be-predicted pixel block comprises:

[0019] A to-be-predicted pixel sub-block in the to-be-predicted pixel block is determined, and a plurality of to-be-predicted pixels in the to-be-predicted pixel sub-block are obtained.

[0020] In an example embodiment of the present disclosure, the obtaining of the interpolation coefficient of the reference pixel according to the interpolation coefficient mapping table corresponding to the angle mode type comprises:

[0021] The interpolation coefficient index value of the reference pixel is obtained from the interpolation coefficient mapping table corresponding to the angle mode type.

[0022] The interpolation coefficient of the reference pixel is obtained from an interpolation coefficient query table corresponding to a preset interpolation coefficient type according to the interpolation coefficient index value of the reference pixel and the interpolation coefficient type.

[0023] In an example embodiment of the present disclosure, the type of the interpolation coefficient includes a cubic interpolation coefficient, and the cubic interpolation coefficient includes a positive interpolation coefficient and a negative interpolation coefficient; and the obtaining the predicted pixel value of the to-be-predicted pixel according to the pixel value of each of the reference pixels and the interpolation coefficient includes:

[0024] a first pixel weighted sum is obtained according to the positive interpolation coefficient and the pixel value of the corresponding reference pixel, and a second pixel weighted sum is obtained according to the negative interpolation coefficient and the pixel value of the corresponding reference pixel;

[0025] the predicted pixel value of the to-be-predicted pixel is obtained according to the difference between the first pixel weighted sum and the second pixel weighted sum.

[0026] In an example embodiment of the present disclosure, the obtaining the second pixel weighted sum according to the negative interpolation coefficient and the pixel value of the corresponding reference pixel includes:

[0027] the second pixel weighted sum is obtained according to the absolute value of the negative interpolation coefficient and the pixel value of the corresponding reference pixel.

[0028] According to a second aspect of the present disclosure, an intra angular prediction device is provided, including:

[0029] an angle mode division module configured to divide an angle mode of intra angular prediction into a plurality of angle mode types, and provide a corresponding assembly function for each of the angle mode types;

[0030] a reference pixel acquisition module configured to acquire a to-be-predicted pixel in a to-be-predicted pixel block, and acquire reference pixels of the to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel;

[0031] an interpolation coefficient acquisition module configured to acquire interpolation coefficients of the reference pixels according to an interpolation coefficient mapping table corresponding to the angle mode type;

[0032] a predicted pixel determination module configured to obtain a predicted pixel value of the to-be-predicted pixel according to the pixel value of each of the reference pixels and the interpolation coefficient.

[0033] In an example embodiment of the present disclosure, the reference pixel acquisition module includes:

[0034] a to-be-predicted pixel acquisition unit configured to simultaneously acquire a plurality of continuous to-be-predicted pixels in the to-be-predicted pixel block, and obtain a plurality of to-be-referenced pixels according to the plurality of continuous to-be-predicted pixels;

[0035] The reference pixel index value obtaining unit is configured to execute an assembly function corresponding to the angle mode type of the to-be-predicted pixel, and obtain a reference pixel index value corresponding to each to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type.

[0036] The reference pixel obtaining unit is configured to execute sorting of the plurality of to-be-referenced pixels according to the reference pixel index values, and obtain a reference pixel corresponding to each to-be-predicted pixel.

[0037] In an exemplary embodiment of the present disclosure, the reference pixel obtaining unit comprises:

[0038] The to-be-referenced pixel sorting unit is configured to execute sorting of the plurality of to-be-referenced pixels according to the reference pixel index values by means of a sorting instruction in the assembly function, and obtain a reference pixel corresponding to each to-be-predicted pixel.

[0039] In an exemplary embodiment of the present disclosure, the angle mode type comprises a wide angle type, and the to-be-predicted pixel obtaining unit comprises:

[0040] The to-be-predicted pixel sub-block obtaining unit is configured to execute determination of a to-be-predicted pixel sub-block in the to-be-predicted pixel block, and obtain a plurality of to-be-predicted pixels in the to-be-predicted pixel sub-block.

[0041] In an exemplary embodiment of the present disclosure, the interpolation coefficient obtaining module comprises:

[0042] The interpolation coefficient index value obtaining unit is configured to execute obtaining of an interpolation coefficient index value of the reference pixel from an interpolation coefficient mapping table corresponding to the angle mode type.

[0043] The interpolation coefficient obtaining unit is configured to execute obtaining of an interpolation coefficient of the reference pixel from an interpolation coefficient query table corresponding to a preset interpolation coefficient type according to the interpolation coefficient index value of the reference pixel and the interpolation coefficient type.

[0044] In an exemplary embodiment of the present disclosure, the type of the interpolation coefficient comprises a cubic interpolation coefficient, and the cubic interpolation coefficient comprises a positive interpolation coefficient and a negative interpolation coefficient; and the prediction pixel determining module comprises:

[0045] The pixel weighted sum determining unit is configured to execute obtaining of a first pixel weighted sum according to the positive interpolation coefficient and a pixel value of the corresponding reference pixel, and obtaining of a second pixel weighted sum according to the negative interpolation coefficient and the pixel value of the corresponding reference pixel.

[0046] The prediction pixel value determination unit is configured to determine the prediction pixel value of the to-be-predicted pixel according to a difference between the first pixel weight sum and the second pixel weight sum.

[0047] In an exemplary embodiment of the present disclosure, the pixel weight sum determination unit comprises:

[0048] The second pixel weight sum determination unit is configured to determine the second pixel weight sum according to an absolute value of the negative interpolation coefficient and a pixel value of the reference pixel corresponding to the negative interpolation coefficient.

[0049] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the intra angular prediction method according to any one of the preceding aspects.

[0050] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the intra angular prediction method according to any one of the preceding aspects.

[0051] The exemplary embodiments of the present disclosure can have the following beneficial effects:

[0052] In the intra angular prediction method of the exemplary embodiments of the present disclosure, on one hand, the angle mode is classified based on the angle mapping relationship, and a corresponding assembly function is provided for each angle mode type, which greatly reduces the number of assembly functions required for intra angular prediction, thereby reducing the maintenance cost and the size of the decoder package; on the other hand, the reference pixel of the to-be-predicted pixel and the interpolation coefficient corresponding to the reference pixel are obtained by combining the instruction set in the assembly function in the form of table lookup, which effectively avoids the transpose operation in the horizontal direction angle prediction, greatly reduces the calculation complexity, and has the significant advantages of reducing the consumption of computing resources and improving the decoding efficiency of the client.

[0053] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0055] Figure 1 A schematic diagram of an intra prediction angle mode in a related embodiment of the present disclosure is shown.

[0056] Figure 2 A schematic diagram of intra angular prediction in one related embodiment of the present disclosure is shown;

[0057] Figure 3 A schematic diagram of horizontal directional angular prediction in one related embodiment of the present disclosure is shown;

[0058] Figure 4 A flowchart of an intra angular prediction method in an example embodiment of the present disclosure is shown;

[0059] Figure 5 A flowchart of obtaining reference pixels of a pixel to be predicted in an example embodiment of the present disclosure is shown;

[0060] Figure 6 A schematic diagram of obtaining a predicted pixel based on table lookup and sorting in one specific embodiment of the present disclosure is shown;

[0061] Figure 7 A schematic diagram of wide angle prediction based on multi-line parallel in one specific embodiment of the present disclosure is shown;

[0062] Figure 8 A flowchart of obtaining a predicted pixel value based on cubic interpolation coefficients in an example embodiment of the present disclosure is shown;

[0063] Figure 9 A block diagram of an intra angular prediction apparatus in an example embodiment of the present disclosure is shown;

[0064] Figure 10 A structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] In order to make the ordinary person skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0067] The example implementations described herein can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects of the example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more implementations. In the following description, numerous specific details are provided to give a thorough understanding of implementations of the disclosure. One skilled in the relevant art will recognize, however, that the implementations of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures have not been described in detail so as not to obscure aspects of the disclosure.

[0068] Furthermore, the accompanying drawings are merely ideal representations and are not necessarily drawn to scale. The same reference numerals in different drawings represent the same or similar structure and thus their repetitive description will be omitted. Some of the block diagrams in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0069] For intra angular prediction algorithm, VVC has many improvements compared with HEVC (High Efficiency Video Coding), one of which is to increase the preset angle direction from 35 to 94 (including 67 regular angles and 27 wide angles), as shown in Figure 1 At the same time, the interpolation algorithm used for prediction pixels is also improved from 2-tap interpolation to 4-tap interpolation. More complex intra angular prediction not only improves the compression rate, but also increases the client decoding complexity, affecting the user experience.

[0070] In some related embodiments, in order to reduce the decoding complexity, the SIMD (Single Instruction Multiple Data) ARMv8 (the 8th version of ARM architecture) instruction set can be used for acceleration. However, the exponential increase of VVC angle direction, the introduction of wide angle and the more complex interpolation algorithm all bring difficulties to the use of SIMD, and it is difficult to achieve efficient acceleration.

[0071] Figure 2A schematic diagram of intra angular prediction in one related embodiment of the present disclosure is shown. Take angle mode number 30 as an example, the output is a to-be-predicted block with a width of W and a height of H, and the input is a reference pixel with a length of 2*W+2*H+1. For any one prediction pixel, its prediction value is obtained in the following manner:

[0072]

[0073] wherein iIdx is a position offset value of the reference pixel compared with the prediction pixel, is the continuous 4 reference pixels in the corresponding angle direction, is the interpolation coefficient related to the corresponding angle mapping position, and the interpolation coefficients used in VVC can include Gaussian interpolation and cubic interpolation.

[0074] If implemented in C code, the complete WxH times need to be looped through, and 4 reference pixels are read and 4 interpolation coefficients are calculated each time to predict one prediction pixel through the above formula.

[0075] If the parallel operation between pixels is implemented in the form of ARMv8 SIMD, for each angle, a special ARMv8 assembly function can be designed according to the mapping relationship of this angle. Simply speaking, n reference pixels are read at a time by using the vector register of ARMv8 to realize single instruction set multi-data stream operation and improve the operation speed. However, this method will have the following problems:

[0076] 1. The number of ARMv8 assembly functions is too large, the subsequent maintenance and optimization cost is high, and the decoder package size is greatly increased. Each size of the coding block of each angle needs to be designed and written into a special assembly function. For example, for 94 angles, 5 block sizes and 2 types of interpolation coefficients, a total of 94x5x2=940 assembly functions are needed.

[0077] 2. The linear interpolation calculation of prediction is particularly complex. When optimized by using the ARMv8 SIMD instruction set, the 8-bit pixel input, and the intermediate result reaches 32 bits, which reduces the overall throughput.

[0078] 3. The horizontal direction angle (-14~33) needs to be transposed for the whole block, which brings a lot of redundant calculation.

[0079] The angle in the horizontal direction needs to be transposed because the corresponding reference pixels of the continuous prediction pixels in the same row are not continuous. Figure 3A schematic diagram illustrating horizontal angle prediction in a related embodiment of the present disclosure is shown. Taking angle 6 as an example, for consecutive predicted pixels P0 to P7, their corresponding reference pixels are R0, R1, R1, R2, R3, R3, R4, and R5, respectively. Since this problem does not exist for vertical angles, the horizontal angle is processed by first converting the horizontal direction to the vertical direction, switching from row-based processing to column-based processing, and then converting back to the horizontal direction using a matrix transpose.

[0080] 4. The distance between reference pixels at wide angles is far, memory access efficiency is low, and it is difficult to optimize using ARMv8 assembly.

[0081] Based on the above problems, this exemplary embodiment first provides an intra-frame angle prediction method. Figure 4 As shown, the above intra-frame angle prediction method may include the following steps:

[0082] Step S410: Divide the angle modes of intra-frame angle prediction into multiple angle mode types, and provide a corresponding assembly function for each angle mode type.

[0083] Step S420: Obtain the pixel to be predicted in the pixel block to be predicted, and obtain the reference pixel of the pixel to be predicted from the pixel mapping table corresponding to the angle mode type through the assembly function corresponding to the angle mode type of the pixel to be predicted.

[0084] Step S430: Obtain the interpolation coefficient of the reference pixel according to the interpolation coefficient mapping table corresponding to the angle mode type.

[0085] Step S440: Obtain the predicted pixel value of the pixel to be predicted according to the pixel value and interpolation coefficient of each reference pixel.

[0086] In the intra-frame angle prediction method of the example implementation method of the present disclosure, on the one hand, by classifying angle modes based on the angle mapping relationship and providing corresponding assembly functions for each angle mode type, the number of assembly functions required for intra-frame angle prediction is greatly reduced, thereby reducing maintenance costs and decoder packet size; on the other hand, by obtaining the reference pixels of the pixels to be predicted and the interpolation coefficients corresponding to the reference pixels through table lookup combined with the instruction set in the assembly function, the transposition operation in the horizontal angle prediction is effectively avoided, the computational complexity is greatly reduced, and it has the significant advantages of reducing computing resource consumption and improving client decoding efficiency.

[0087] Next, combine Figures 5 to 8 The above steps of this exemplary embodiment are described in more detail.

[0088] In step S410, the angle mode of the intra angular prediction is divided into multiple angle mode types, and a corresponding assembly function is provided for each angle mode type.

[0089] In the example embodiment, all angle modes of the intra angular prediction can be divided into multiple angle mode types based on the angle mapping relationship, and the same assembly function is used for the same angle mode type. The assembly function refers to a sequence of machine instructions optimized for a specific angle type, which can be written in ARMv8 SIMD instruction set, and each function contains pixel loading and calculation instructions common to all angle modes of the type. At the same time, one assembly function can support all block size cases.

[0090] Specifically, the 94 angles shown in FIG. 1 can be divided into the following 7 categories: Figure 1

[0091] (1) Horizontal mode: 18;

[0092] (2) Vertical mode: 50;

[0093] (3) Diagonal mode: 34;

[0094] (4) Horizontal wide angle: -14 to 1;

[0095] (5) Left lower horizontal angle: 3 to 17;

[0096] (6) Left upper horizontal angle: 19 to 33;

[0097] (7) Vertical angle: 35 to 80 (except 50).

[0098] In this way, the number of assemblies can be reduced from 940 (94x5x2) to 14 (7x2), greatly reducing the maintenance cost and package size, and solving the problem of excessive maintenance cost and package size caused by designing a separate ARMv8 SIMD assembly function for each angle and each block size.

[0099] In the video decoding process, when processing a pixel block to be predicted, first identify the angle mode type to which it belongs. Then select the pre-generated assembly function according to the type, which contains the relevant operation instructions suitable for all angle modes of the type.

[0100] In step S420, the pixel to be predicted in the pixel block to be predicted is obtained, and the reference pixel of the pixel to be predicted is obtained from the pixel mapping table corresponding to the angle mode type of the pixel to be predicted through the assembly function corresponding to the angle mode type of the pixel to be predicted.

[0101] ​In the example embodiment, two special mapping tables, offset table (pixel mapping table) and fraction table (interpolation coefficient mapping table), can be designed for each angle, and the required values can be obtained from the tables according to the angle mode value of each angle as an index. The pixel mapping table refers to a lookup table that stores the mapping relationship between the predicted pixel and the reference pixel, and a two-dimensional array structure can be used to store the index offset of the reference pixel under different angle modes, and the overhead of real-time position calculation can be eliminated by pre-computation.

[0102] In the example embodiment, as shown in Figure 5 the predicted pixel in the predicted pixel block is obtained, and the reference pixel of the predicted pixel is obtained from the pixel mapping table corresponding to the angle mode type of the predicted pixel through an assembly function corresponding to the angle mode type of the predicted pixel, which can include the following steps:

[0103] Step S510. A plurality of continuous predicted pixels in the predicted pixel block are obtained, and a plurality of reference pixels are obtained according to the plurality of continuous predicted pixels.

[0104] In the example embodiment, the parallel data loading capability of the SIMD instruction set can be used to simultaneously obtain a plurality of continuous predicted pixels. The reference pixel refers to a plurality of candidate reference pixels. Figure 6 An example of obtaining a predicted pixel based on table lookup and sorting in one specific embodiment of the present disclosure is shown, taking Figure 6 for example, a plurality of continuous predicted pixels, such as P0 to P7, and the corresponding reference pixels are R0 to R7.

[0105] Step S520. The reference pixel index value corresponding to each predicted pixel is obtained from the pixel mapping table corresponding to the angle mode type of the predicted pixel through an assembly function corresponding to the angle mode type of the predicted pixel.

[0106] The reference pixel index value refers to a numerical position identifier established in advance according to the angle mode type, which can be stored in the form of an integer array, and each index value corresponds to the storage position of the reference pixel in the mapping table, and the sorting basis is established through numerical comparison.

[0107] Through the corresponding assembly function, the reference pixel index value corresponding to each predicted pixel can be obtained from the pixel mapping table corresponding to the angle mode type. For example Figure 6 the predicted pixels P0 to P7 are R0, R1, R1, R2, R3, R3, R4, and R5, and the index values in the offset table are 0, 1, 1, 2, 3, 3, and 4.

[0108] Step S530. Sort the plurality of reference pixels according to the reference pixel index value, to obtain the reference pixel corresponding to each to-be-predicted pixel.

[0109] In the example embodiment, the plurality of to-be-referenced pixels can be sorted according to the reference pixel index value by using a sorting instruction in the assembly function, to obtain the reference pixel corresponding to each to-be-predicted pixel.

[0110] The sorting according to the reference pixel index value refers to position recombination of the pixel data by using a hardware-accelerated vector rearrangement instruction. Specifically, the plurality of to-be-referenced pixels can be sorted according to the read offset table by using a sorting instruction in ARMv8 SIMD, to directly read the reference pixel corresponding to each to-be-predicted pixel, such as R0, R1, R1, R2, R3, R3, R4, and R5 in Figure 6 .

[0111] In the example embodiment, the reference pixel reading method based on the table lookup and sorting can be used to read all the reference pixels required by a series of continuous prediction pixels at one time by using the table lookup and sorting, to convert the column-by-column operation into row-by-row operation, avoid the transposition operation, and greatly reduce the calculation complexity, thereby solving the problem of transposition required for the horizontal direction angle implementation.

[0112] In the example embodiment, if the angle mode type is the wide-angle type, when the plurality of continuous to-be-predicted pixels in the to-be-predicted pixel block are obtained, a to-be-predicted pixel sub-block in the to-be-predicted pixel block can be determined, and the plurality of to-be-predicted pixels in the to-be-predicted pixel sub-block can be obtained.

[0113] The to-be-predicted pixel sub-block refers to dividing the to-be-predicted pixel block into a plurality of pixel regions with continuous spatial positions, which can be implemented by using a fixed-size rectangular region division manner, for example, dividing the pixel block into 4x4 or 8x8 sub-blocks. The division manner limits the pixel access range, so that the long-distance reference pixel access in the wide-angle mode is constrained within the sub-block, thereby improving the data locality.

[0114] Figure 7 FIG. 1 shows a schematic diagram of wide-angle prediction based on multi-row parallelism in one specific embodiment of the present disclosure. Taking wide-angle 7 as an example, the reference pixels corresponding to a row of continuous prediction pixels are far apart, so the input reference pixels cannot be read at one time by using the above method, and the memory access efficiency is very low.

[0115] In the example embodiment, by providing an implementation scheme of multi-line parallel processing, for example, the prediction pixels of one line P00-P015 can be processed at a time, and the prediction pixels of a 4x4 block P00-P33 can be processed at a time. For the prediction pixels in the 4x4 block, the distances between the mapped reference pixels are relatively concentrated. On this basis, the offset and fraction tables corresponding to the 4x4 parallel processing block are also designed, so as to realize fast parallel processing and greatly improve the throughput of wide-angle calculation.

[0116] By means of multi-line parallel processing, the reference pixels of wide-angle parallel processing are mapped to more concentrated positions, so that they can be optimized by SIMD according to the same scheme as the conventional horizontal direction angle, and the memory access efficiency is greatly improved, solving the problems of low memory access efficiency and difficulty in ARMv8 SIMD assembly optimization due to the long distance between the mapped reference pixels of wide-angle.

[0117] With reference to the foregoing description, in step S430, the interpolation coefficient of the reference pixel is obtained according to the interpolation coefficient mapping table corresponding to the angle mode type. Figure 4

[0118] In the example embodiment, the interpolation coefficient index value of the reference pixel can be obtained from the interpolation coefficient mapping table corresponding to the angle mode type, and the interpolation coefficient of the reference pixel is obtained from the interpolation coefficient query table corresponding to the preset interpolation coefficient type according to the interpolation coefficient index value of the reference pixel and the interpolation coefficient type.

[0119] The interpolation coefficient mapping table refers to a lookup table for storing the mapping relationship between the prediction pixel and the interpolation coefficient index value corresponding to the prediction pixel. The interpolation coefficient participating in the final operation is read from the standard specified Gaussian interpolation coefficient table or cubic interpolation coefficient table through the interpolation coefficient index value.

[0120] In step S440, the prediction pixel value of the to-be-predicted pixel is obtained according to the pixel value and the interpolation coefficient of each reference pixel.

[0121] In the example embodiment, after obtaining the pixel value and the interpolation coefficient of each reference pixel of the prediction pixel, the prediction pixel value can be generated by weighted summation calculation.

[0122] In the example embodiment, the type of the interpolation coefficient includes a cubic interpolation coefficient, and the cubic interpolation coefficient includes a positive interpolation coefficient and a negative interpolation coefficient. The cubic interpolation coefficient refers to a weight parameter used in a cubic polynomial interpolation algorithm, and can be implemented by using a fixed value stored in a mapping table in advance, for weighted calculation of multiple reference pixels.

[0123] ​The reason why the intermediate value of the calculation of the cubic interpolation coefficient reaches 32 bits is that the coefficient has a negative number, so the calculation process needs to use signed number storage data. Through analysis, it can be seen that the cubic interpolation coefficient has the following rules:

[0124] (1) f1+f2+f3+f4=64

[0125] (2) f1, f4 are negative numbers, and f2, f3 are positive numbers

[0126] (3) For 8-bit pixel values, the value range is 0-255, so (abs(f1)*R1+abs(f4)*R4) and (f2*R2+f3*R3) will not exceed 16 bits.

[0127] Therefore, based on the above analysis, the calculation process can be disassembled to ensure that the intermediate value of the entire calculation process does not exceed 16 bits.

[0128] In the example embodiment, when the type of the interpolation coefficient is a cubic interpolation coefficient, as shown in Figure 8 , the predicted pixel value of the pixel to be predicted is obtained according to the pixel value of each reference pixel and the interpolation coefficient, which can specifically include the following steps:

[0129] Step S810. Obtain a first pixel weighted sum according to the positive interpolation coefficient and the pixel value of the corresponding reference pixel, and obtain a second pixel weighted sum according to the negative interpolation coefficient and the pixel value of the corresponding reference pixel.

[0130] The first pixel weighted sum refers to the sum of the product of all positive interpolation coefficients and the corresponding reference pixels, which can be implemented by a parallel multiplication accumulation instruction for fast calculation of positive weighted components. The second pixel weighted sum refers to the sum of the product of the absolute value of all negative interpolation coefficients and the corresponding reference pixels, which can be implemented by a multiplication operation that separates the sign and the value, for independent processing of negative weighted components. The second pixel weighted sum can be obtained according to the absolute value of the negative interpolation coefficient and the pixel value of the reference pixel corresponding to the negative interpolation coefficient.

[0131] Specifically, in calculating the predicted pixel value, the cubic interpolation coefficient is first divided into two categories: positive and negative. For the positive interpolation coefficient, it is directly multiplied by the corresponding reference pixel value item by item and accumulated to obtain the first pixel weighted sum. For the negative interpolation coefficient, its absolute value is first extracted, and then multiplied by the corresponding reference pixel value and accumulated to form the second pixel weighted sum.

[0132] Step S820. Obtain the predicted pixel value of the pixel to be predicted according to the difference between the first pixel weighted sum and the second pixel weighted sum.

[0133] The disassembled calculation formula is as follows:

[0134]

[0135] wherein, >> 6 represents binary right shift 6 bits. For the problem of low throughput caused by the intermediate value reaching 32 bits, the intermediate value is strictly controlled within 16 bits through a mathematically equivalent transformation, effectively improving the computing throughput by nearly one time.

[0136] In the example embodiment, the implementation method of VVC intra angular prediction mode acceleration by using ARMv8 SIMD instruction set on the client effectively reduces the computational complexity of VVC intra angular prediction, solves the problem of low SIMD acceleration efficiency caused by multi-angle mode, and can accelerate by 70% under the same test conditions compared with the advanced open source implementation, significantly improving the real-time decoding performance of high-resolution video.

[0137] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0138] Further, the present disclosure also provides an intra angular prediction device. Referring to Figure 9 As shown, the intra angular prediction device can include an angle mode division module 910, a reference pixel acquisition module 920, an interpolation coefficient acquisition module 930, and a predicted pixel determination module 940. Wherein:

[0139] The angle mode division module 910 is configured to divide the angle mode of intra angular prediction into a plurality of angle mode types, and provide a corresponding assembly function for each angle mode type;

[0140] The reference pixel acquisition module 920 is configured to acquire the to-be-predicted pixels in the to-be-predicted pixel block, and acquire the reference pixels of the to-be-predicted pixels from the pixel mapping table corresponding to the angle mode type of the to-be-predicted pixels through the assembly function corresponding to the angle mode type of the to-be-predicted pixels;

[0141] The interpolation coefficient acquisition module 930 is configured to acquire the interpolation coefficients of the reference pixels according to the interpolation coefficient mapping table corresponding to the angle mode type;

[0142] The predicted pixel determination module 940 is configured to obtain the predicted pixel value of the to-be-predicted pixel according to the pixel value and the interpolation coefficient of each reference pixel.

[0143] In some example embodiments of the present disclosure, the reference pixel obtaining module 920 can include a to-be-predicted pixel obtaining unit, a reference pixel index value obtaining unit, and a reference pixel obtaining unit. Wherein:

[0144] The to-be-predicted pixel obtaining unit is configured to simultaneously obtain a plurality of continuous to-be-predicted pixels in a to-be-predicted pixel block, and obtain a plurality of to-be-referenced pixels according to the plurality of continuous to-be-predicted pixels.

[0145] The reference pixel index value obtaining unit is configured to obtain a reference pixel index value corresponding to each to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type through an assembly function corresponding to the angle mode type of the to-be-predicted pixel.

[0146] The reference pixel obtaining unit is configured to sort the plurality of to-be-referenced pixels according to the reference pixel index value, and obtain a reference pixel corresponding to each to-be-predicted pixel.

[0147] In some example embodiments of the present disclosure, the reference pixel obtaining unit can include a to-be-referenced pixel sorting unit configured to sort the plurality of to-be-referenced pixels according to the reference pixel index value through a sorting instruction in the assembly function, and obtain a reference pixel corresponding to each to-be-predicted pixel.

[0148] In some example embodiments of the present disclosure, the to-be-predicted pixel obtaining unit can include a to-be-predicted pixel sub-block obtaining unit configured to determine a to-be-predicted pixel sub-block in the to-be-predicted pixel block, and obtain a plurality of to-be-predicted pixels in the to-be-predicted pixel sub-block.

[0149] In some example embodiments of the present disclosure, the interpolation coefficient obtaining module 930 can include an interpolation coefficient index value obtaining unit and an interpolation coefficient obtaining unit. Wherein:

[0150] The interpolation coefficient index value obtaining unit is configured to obtain an interpolation coefficient index value of the reference pixel from an interpolation coefficient mapping table corresponding to the angle mode type.

[0151] The interpolation coefficient obtaining unit is configured to obtain the interpolation coefficient of the reference pixel from an interpolation coefficient query table corresponding to an interpolation coefficient type according to the interpolation coefficient index value of the reference pixel and the preset interpolation coefficient type.

[0152] In some example embodiments of the present disclosure, the predicted pixel determining module 940 can include a pixel weight sum determining unit and a predicted pixel value determining unit. Wherein:

[0153] The pixel weighted sum determination unit is configured to perform a first pixel weighted sum according to the positive interpolation coefficient and pixel values of the corresponding reference pixels, and a second pixel weighted sum according to the negative interpolation coefficient and pixel values of the corresponding reference pixels.

[0154] The predicted pixel value determination unit is configured to perform a predicted pixel value of the pixel to be predicted according to a difference between the first pixel weighted sum and the second pixel weighted sum.

[0155] In some exemplary embodiments of the present disclosure, the pixel weighted sum determination unit can include a second pixel weighted sum determination unit configured to perform a second pixel weighted sum according to an absolute value of the negative interpolation coefficient and pixel values of the corresponding reference pixels.

[0156] The specific details of the modules / units in the above-mentioned intra angular prediction device have been described in detail in the corresponding method embodiment part, and will not be repeated here.

[0157] Figure 10 A structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present disclosure is shown.

[0158] It should be noted that, Figure 10 The computer system 1000 of the electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0159] As Figure 10 shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage portion 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0160] The following components are connected to the I / O interface 1005: an input part 1006 including a keyboard, a mouse, etc.; an output part 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1008 including a hard disk, etc.; and a communication part 1009 including a network interface card such as a LAN card, a modem, etc. The communication part 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1010 as necessary, so that a computer program read out therefrom is installed in the storage part 1008 as necessary.

[0161] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the system of the present disclosure are executed.

[0162] An exemplary embodiment of the present disclosure also provides a computer program product. The computer program product includes a computer program which, when executed by a processor, implements the above-described intra angular prediction method.

[0163] In an embodiment, the computer program product can be a tangible product containing the computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, etc. signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid state disk (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing the computer program, such as a read-only memory, a Nand flash, etc.

[0164] In an embodiment, the computer program product can be an intangible product containing the computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, an installation package, etc. digital file storing the computer program.

[0165] The code of the computer program can be written in any form of programming language. Programming languages such as C, Java, C++, etc. The program code can execute entirely on the user's computing device, or partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, such as a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0166] The computer program can be carried by or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. An electronic device can convert the signal carrying the computer program into a digital signal, and then run the computer program. When the computer program is run on the electronic device, its code is used to make the electronic device perform (more specifically, can make the processor of the electronic device perform) the method steps of various exemplary embodiments of the present disclosure, such as the intra-angle prediction method described above.

[0167] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0168] It should be noted that although several modules of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into embodied by several modules.

[0169] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the disclosure be construed as including any paterns, uses, or adaptations of the features disclosed herein and including modifications apparent to those skilled in the art with the ordinary knowledge of the art to which the features disclosed herein pertain.

[0170] It is to be understood that the disclosure is not limited to the precise construction hereindescribed and illustrated and that various modifications and changes can be made by those skilled in the art without departing from the scope of the disclosure. The scope of the disclosure is limited only by the claims appended hereto.

Claims

1. An intra angular prediction method, characterized by, The application relates to a method for predicting a pixel block, comprising the following steps: dividing an angle mode of intra-angle prediction into multiple angle mode types, and providing corresponding assembly functions for each angle mode type; obtaining a to-be-predicted pixel in a to-be-predicted pixel block, and obtaining a reference pixel of the to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel; obtaining an interpolation coefficient of the reference pixel according to an interpolation coefficient mapping table corresponding to the angle mode type; obtaining a predicted pixel value of the to-be-predicted pixel according to a pixel value of each reference pixel and the interpolation coefficient.

2. The intra angular prediction method of claim 1, wherein, The step of obtaining a to-be-predicted pixel in a to-be-predicted pixel block, and obtaining a reference pixel of the to-be-predicted pixel from a pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel, comprises the following steps: simultaneously obtaining multiple continuous to-be-predicted pixels in the to-be-predicted pixel block, and obtaining multiple to-be-referenced pixels according to the multiple continuous to-be-predicted pixels; obtaining reference pixel index values corresponding to the multiple to-be-predicted pixels respectively from a pixel mapping table corresponding to the angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type of the to-be-predicted pixel; sorting the multiple to-be-referenced pixels according to the reference pixel index values to obtain reference pixels corresponding to the multiple to-be-predicted pixels respectively.

3. The intra angular prediction method of claim 2, wherein, The step of sorting the multiple to-be-referenced pixels according to the reference pixel index values to obtain reference pixels corresponding to the multiple to-be-predicted pixels respectively, comprises the following step: sorting the multiple to-be-referenced pixels according to the reference pixel index values through a sorting instruction in the assembly function to obtain reference pixels corresponding to the multiple to-be-predicted pixels respectively.

4. The method of intra angular prediction of claim 2, wherein, The angle mode type comprises a wide-angle type, and the step of simultaneously obtaining multiple continuous to-be-predicted pixels in the to-be-predicted pixel block, comprises the following steps: determining a to-be-predicted pixel sub-block in the to-be-predicted pixel block, and obtaining multiple to-be-predicted pixels in the to-be-predicted pixel sub-block.

5. The method of intra angular prediction according to claim 1, wherein, The step of obtaining an interpolation coefficient of the reference pixel according to an interpolation coefficient mapping table corresponding to the angle mode type, comprises the following steps: obtaining an interpolation coefficient index value of the reference pixel from an interpolation coefficient mapping table corresponding to the angle mode type; obtaining the interpolation coefficient of the reference pixel from an interpolation coefficient query table corresponding to an interpolation coefficient type according to the interpolation coefficient index value of the reference pixel and the preset interpolation coefficient type.

6. The method of intra angular prediction according to claim 1, wherein, The type of the interpolation coefficient comprises a cubic interpolation coefficient, the cubic interpolation coefficient comprises a positive number interpolation coefficient and a negative number interpolation coefficient, and the step of obtaining a predicted pixel value of the to-be-predicted pixel according to a pixel value of each reference pixel and the interpolation coefficient, comprises the following steps: obtaining a first pixel weighted sum according to the positive number interpolation coefficient and the pixel value of the corresponding reference pixel, and obtaining a second pixel weighted sum according to the negative number interpolation coefficient and the pixel value of the corresponding reference pixel; obtaining the predicted pixel value of the to-be-predicted pixel according to a difference value of the first pixel weighted sum and the second pixel weighted sum.

7. The method of intra angular prediction according to claim 6, wherein, The second pixel weighted sum is obtained according to the negative interpolation coefficient and a pixel value of the corresponding reference pixel, and the method comprises the steps of: The second pixel weighted sum is obtained according to an absolute value of the negative interpolation coefficient and a pixel value of the corresponding reference pixel.

8. An apparatus for intra angular prediction, the apparatus comprising: The method comprises the steps of: An angle mode division module is configured to divide angle modes of intra angle prediction into multiple angle mode types, and provide corresponding assembly functions for each angle mode type respectively; A reference pixel acquisition module is configured to acquire a to-be-predicted pixel in a to-be-predicted pixel block, and acquire a reference pixel of the to-be-predicted pixel from a pixel mapping table corresponding to an angle mode type of the to-be-predicted pixel through an assembly function corresponding to the angle mode type; An interpolation coefficient acquisition module is configured to acquire an interpolation coefficient of the reference pixel according to an interpolation coefficient mapping table corresponding to the angle mode type; A predicted pixel determination module is configured to obtain a predicted pixel value of the to-be-predicted pixel according to a pixel value of each reference pixel and the interpolation coefficient.

9. An electronic device, comprising: The method comprises the steps of: A processor; A memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the intra angle prediction method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the intra angle prediction method according to any one of claims 1 to 7.

Images