Intra prediction search method, apparatus and computer program product

Abstract

The application relates to an intra-frame prediction search method, device and computer program product. The intra-frame prediction search method is applied to a hardware encoder, the length of a pipeline of the hardware encoder is M coding units, the hardware encoder can perform parallel processing on intra-frame prediction of the M coding units, a target prediction mode is determined, and M is a positive integer; the method comprises the following steps: when the width of an image is greater than the length of the pipeline, for each current coding unit in the image which is located at a non-boundary position, the upper reference pixel of the current coding unit is determined as the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding unit is determined as the original pixel of the left pixel; and intra-frame prediction search is performed based on the upper reference pixel and the left reference pixel. The technical scheme of the application can remove the dependent relationship between the coarse search and the fine search of the intra-frame prediction, so that the coarse search and the fine search can be performed in parallel, and the speed of the hardware encoder can be improved.

Description

Technical Field

[0001] This application relates to the field of video coding technology, specifically to an intra-frame prediction search method, apparatus, and computer program product. Background Technology

[0002] In related technologies, intra-frame prediction in video coding refers to using the spatially adjacent reconstructed pixels of the current block to be encoded to predict the current block to be encoded, thereby obtaining predicted pixels. Then, the difference between the original pixels and the predicted pixels of the current block to be encoded is encoded to achieve the purpose of removing information redundancy.

[0003] With the continuous development of video technology, intra-frame prediction technology has become increasingly complex, mainly manifested in: (1) the number of intra-frame prediction modes is increasing; and (2) the size of intra-frame prediction blocks is becoming more and more diverse.

[0004] Table 1 shows the changes in intra-frame prediction techniques in the video coding standards H.264, H.265, and H.266.

[0005] Table 1

[0006] [Table 1_sm_0001]

[0007] like Figure 1 As shown, region F is a block to be encoded with a width of W and a height of H. Regions A, B, C, D, and E are the reconstructed pixels that are spatially adjacent to the block to be encoded. We call them reference pixels. Using these reference pixels, we can obtain the predicted pixels under different prediction modes of the current block to be encoded.

[0008] Each prediction mode requires W×H predicted pixels. Obtaining one predicted pixel requires two multiplication operations and two addition operations. To encode 4K or even 8K video in real time, it is often necessary to predict several pixels in parallel.

[0009] Taking the H.265 video coding standard as an example, the maximum size of a CTU (Coding Tree Unit) in H.265 is 64×64 pixels. When encoding a CTU, intra-frame prediction in 35 modes is required for each of the 256 4×4 blocks, 64 8×8 blocks, 16 16×16 blocks, 4 32×32 blocks, and 1 64×64 block within that CTU. The total number of intra-frame predictions required is shown in Table 2 below.

[0010] Table 2

[0011] [Table 1_sm_0002]

[0012] Performing intra-frame prediction in all modes for all coded blocks of all sizes to meet the real-time encoding requirements of 4K or even 8K video would result in a massive computational burden and correspondingly enormous hardware resource consumption. Therefore, it is necessary to optimize the intra-frame prediction algorithm to reduce computational complexity and hardware resource consumption.

[0013] The optimal intra-frame prediction mode can generally be selected using three cost criteria: SATD (Sum of Absolute Transformed Difference), SAD (Sum of Absolute Difference), and RDO (Rate Distortion Optimization). Of these three criteria, SAD is the simplest and has the lowest computational cost. SATD adds a Hadamard transform, making it slightly more complex but also slightly improving mode selection accuracy. RDO is the most complex, with a significantly higher computational cost, but it also offers the highest mode selection accuracy.

[0014] If mode selection is performed based on RDO cost for all intra-prediction modes of all block sizes, the computational load would be enormous. For the reasons mentioned above, most encoders in related technologies divide intra-prediction into two steps: First step: coarse search, which uses SAD or SATD cost to select N most likely candidate modes from 35 modes. N is generally less than 9, but not limited to this. Second step: fine search, which uses RDO cost to precisely filter the N modes obtained in the first step to obtain the final optimal prediction mode.

[0015] While the above approach can reduce the complexity and computational cost of intra-frame prediction to some extent, it still has the following problems: (1) The coarse search uses SAD or SATD costs for mode selection, which only reduces the computational cost during mode selection. It still requires prediction of all modes for all sizes of blocks, and this computational cost is still very high. (2) As mentioned above, the reference pixels for intra-frame prediction come from the reconstructed pixels of adjacent blocks. The reconstructed pixels of adjacent blocks can only be obtained after the adjacent blocks determine the optimal prediction mode. This leads to a data dependency between the coarse search and the fine search, and the two cannot be processed in parallel, affecting the speed of the hardware encoder. For example, as Figure 2 As shown, when performing intra-frame prediction on a 4*4 coded block Blk1, the rightmost column of the 4*4 coded block Blk0 is used to reconstruct pixels P1. Summary of the Invention Technical issues

[0016] The technical problem solved by this application is to provide an intra-frame prediction search method, apparatus and computer program product to remove the dependency between coarse search and fine search in intra-frame prediction, so that coarse search and fine search can be carried out in parallel, thereby improving the speed of hardware encoder. Technical solution

[0017] According to a first aspect of the embodiments of this application, an intra-frame prediction search method is provided, applied to a hardware encoder, wherein the pipeline length of the hardware encoder is M coding units, and it is capable of processing intra-frame prediction of M coding units in parallel to determine a target prediction mode, where M is a positive integer; the method includes: when the width of the image is greater than the length of the pipeline, for each current coding unit in the image at a non-boundary position, determining that the upper reference pixel of the current coding unit is the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding unit is the original pixel of the left pixel; and performing intra-frame prediction search based on the upper reference pixel and the left reference pixel.

[0018] In one implementation, the intra-frame prediction search based on the upper reference pixel and the left reference pixel may include: for the current coding block located in the first row of the current coding unit, determining that the upper reference pixel of the current coding block is the reconstructed pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; the current coding block in the first row is adjacent to the upper pixel; for the current coding blocks in the remaining rows of the current coding unit other than the first row, determining that the upper reference pixel of the current coding block is the original pixel of the upper pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; for each current coding block in the current coding unit, performing an intra-frame prediction search based on the upper reference pixel and the left reference pixel to determine N candidate intra-frame prediction modes for the current coding block, where N is a positive integer greater than 1; generating a candidate mode list based on the N candidate intra-frame prediction modes, the candidate mode list including the mode identifiers of the N candidate intra-frame prediction modes.

[0019] In one implementation, for each current coding block in the current coding unit, intra-prediction search is performed based on the upper reference pixel and the left reference pixel to determine N candidate intra-prediction modes for the current coding block. This may include: for each current coding block, calculating the first prediction cost corresponding to the specified L intra-prediction modes; where L is an integer greater than N; and determining the intra-prediction mode corresponding to the N first prediction costs with the smallest value among the L first prediction costs as the N candidate intra-prediction modes.

[0020] In one implementation, the current coding unit is divided into first coding blocks of a first size according to a first size, and the current coding unit is further divided into second coding blocks of a second size according to a second size; the first size is smaller than the second size; several adjacent first coding blocks can form the second coding block; the current coding block is the second coding block; the step of calculating the first prediction cost corresponding to the specified L intra-frame prediction modes for each current coding block may include: for each current coding block, for each intra-frame prediction mode, calculating the second prediction cost of all first coding blocks in the current coding block; calculating the sum of the second prediction costs of all first coding blocks in the current coding block to obtain the first prediction cost corresponding to the intra-frame prediction mode.

[0021] In one implementation, after determining the N candidate intra-prediction modes of the current coding block, the method may further include: determining the rate-distortion cost corresponding to the N candidate intra-prediction modes; and determining the intra-prediction mode corresponding to the smallest rate-distortion cost among the N rate-distortion costs as the target prediction mode.

[0022] According to a second aspect of the embodiments of this application, an intra-frame prediction search apparatus is provided for implementing the above-described intra-frame prediction search method. The apparatus includes: a reconstructed pixel storage module, an original pixel storage module, a reference pixel generation module, and a search module. The original pixel storage module is used to store the original pixels of pixels in an image. The reconstructed pixel storage module is used to store the reconstructed pixels of pixels in an image. The reference pixel generation module is used to, when the width of the image is greater than the length of the pipeline, for each current coding unit located at a non-boundary position in the image, obtain the upper reference pixel from the reconstructed pixel storage module when the upper reference pixel of the current coding unit is a reconstructed pixel, obtain the upper reference pixel from the original pixel storage module when the upper reference pixel of the current coding unit is an original pixel, and obtain the left reference pixel from the original pixel storage module when the left reference pixel of the current coding unit is an original pixel. The search module is used to perform intra-frame prediction search based on the upper reference pixel and the left reference pixel.

[0023] In one implementation, the search module can further be used to: for a current coding block located in the first row of the current coding unit, determine that the upper reference pixel of the current coding block is the reconstructed pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; the first row of the current coding block is adjacent to the upper pixel; for the current coding blocks in the remaining rows of the current coding unit other than the first row, determine that the upper reference pixel of the current coding block is the original pixel of the upper pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; for each current coding block in the current coding unit, perform intra-frame prediction search based on the upper reference pixel and the left reference pixel to determine N candidate intra-frame prediction modes of the current coding block, where N is a positive integer greater than 1; generate a first candidate mode list based on the N candidate intra-frame prediction modes, the first candidate mode list including the mode identifiers of the N candidate intra-frame prediction modes.

[0024] In one implementation, the search module can also be used to: calculate the first prediction cost corresponding to the specified L intra-prediction modes for each current coding block; L is an integer greater than N; and determine the intra-prediction modes corresponding to the N first prediction costs with the smallest value among the L first prediction costs as N candidate intra-prediction modes.

[0025] In one embodiment, the current coding unit is divided into a first coding block of a first size according to a first size; the current coding block is the first coding block; the search module may include: K intra-prediction modules, K first cost calculation modules, and a first list generation module; K is an integer greater than 1 and less than L; the K intra-prediction modules are used to perform intra-prediction on K intra-prediction modes for the same first coding block to obtain K prediction blocks, until the intra-prediction of L intra-prediction modes for the first coding block is completed to obtain L prediction blocks; the K first cost calculation modules are connected one-to-one with the K intra-prediction modules, and each first cost calculation module is used to calculate a second prediction cost based on the prediction block of the first coding block and the corresponding original block, until the second prediction cost of L intra-prediction modes is obtained; the first list generation module is used to determine the intra-prediction modes corresponding to the N smallest second prediction costs among the L second prediction costs as N candidate intra-prediction modes, and to generate a second candidate mode list based on the N candidate intra-prediction modes.

[0026] In one embodiment, the current coding unit is further divided into second coding blocks of a second size according to a second size; the first size is smaller than the second size; several adjacent first coding blocks can form the second coding block; the current coding block is the second coding block; the search module further includes: a second cost calculation module and a second list generation module; the second cost calculation module is used to calculate the sum of the second prediction costs of all first coding blocks in the second coding block for each intra-prediction mode, to obtain the first prediction cost of the second coding block; the second list generation module is used to determine the intra-prediction modes corresponding to the N first prediction costs with the smallest value among the L first prediction costs as N candidate intra-prediction modes, and to generate a first candidate mode list based on the N candidate intra-prediction modes.

[0027] In one embodiment, all first coding blocks in the current coding unit are input to the intra-prediction search device in a serial manner. The input order of the first coding blocks is the same as the intra-prediction order. The input order of the first coding blocks satisfies the following condition: the first prediction cost of the corresponding second coding block can be obtained by continuously accumulating the second prediction cost of several adjacent first coding blocks.

[0028] In one implementation, the reference pixel generation module, intra-frame prediction module, first cost calculation module, first list generation module, second cost calculation module, and second list generation module can be processed in parallel to form the pipeline.

[0029] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0030] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, can implement the above-described method.

[0031] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method. Invention Overview Beneficial effects

[0032] Compared with existing technologies, the beneficial effects of this application are as follows: When the width of the image is greater than the length of the hardware encoder pipeline, for each current coding unit located at a non-boundary position in the image, the upper reference pixel of the current coding unit is determined as the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding unit is determined as the original pixel of the left pixel, rather than the reconstructed pixel. Then, intra-frame prediction search is performed based on the upper reference pixel and the left reference pixel. That is, when determining the upper reference pixel and the left reference pixel of the current coding unit, the left reference pixel can use the original pixel of the left pixel, instead of using the reconstructed pixel of the left pixel as the left reference pixel after generating the reconstructed pixel of the left pixel. Therefore, the dependency between the coarse search and fine search of intra-frame prediction can be removed, allowing the coarse search and fine search to be performed in parallel, thereby improving the speed of the hardware encoder.

[0033] For the current coding block in the remaining rows of the current coding unit, excluding the first row, the upper reference pixel of the current coding block is determined as the original pixel of the upper pixel, and the left reference pixel of the current coding block is determined as the original pixel of the left pixel. That is, when determining the upper reference pixel and the left reference pixel of the current coding unit, the upper reference pixel and the left reference pixel can use the original pixel instead of the reconstructed pixel. Therefore, the dependency between the coarse search and fine search of intra-frame prediction can be removed, so that the coarse search and fine search can be carried out in parallel, which can improve the speed of the hardware encoder.

[0034] The current coding unit is divided into a first coding block of the first size according to a first size. The current coding unit is also divided into a second coding block of the second size according to a second size. The first size is smaller than the second size. Several adjacent first coding blocks can form a second coding block. The current coding block is the second coding block. For each current coding block and for each intra-frame prediction mode, the second prediction cost of all first coding blocks in the current coding block is calculated. Then, the sum of the second prediction costs of all first coding blocks in the current coding block is calculated to obtain the first prediction cost corresponding to the intra-frame prediction mode. Instead of performing intra-frame prediction on the second coding block to obtain the first prediction cost, this reduces the computational load of intra-frame prediction. Industrial applicability

[0035] The intra-frame prediction search method, apparatus, and computer program product according to this application can be used in display device optimization to remove the dependency between coarse and fine searches in intra-frame prediction, so that coarse and fine searches can be performed in parallel, thereby improving the speed of the hardware encoder. Attached Figure Description

[0036] Figure 1 This is a schematic diagram showing a block to be encoded and a reference pixel, based on relevant technologies.

[0037] Figure 2 This is a schematic diagram illustrating another block to be encoded and a reference pixel, based on relevant technologies.

[0038] Figure 3 This is a flowchart illustrating an intra-frame prediction search method according to an exemplary embodiment.

[0039] Figure 4 This is a schematic diagram illustrating a current coding unit and a reference pixel according to an exemplary embodiment.

[0040] Figure 5 This is a flowchart illustrating an intra-frame prediction search method according to another exemplary embodiment.

[0041] Figure 6 This is a flowchart illustrating an intra-frame prediction search method according to another exemplary embodiment.

[0042] Figure 7 This is a schematic diagram illustrating, according to an exemplary embodiment, that a current coding unit can be divided into a first coding block according to a first size.

[0043] Figure 8 This is a schematic diagram illustrating, according to an exemplary embodiment, that a current coding unit can be divided into a second coding block according to a second size.

[0044] Figure 9 This is a flowchart illustrating an intra-frame prediction search method according to another exemplary embodiment.

[0045] Figure 10 is a schematic diagram of intra-frame prediction for a 32×32 pixel coded block 70 according to an exemplary embodiment.

[0046] Figure 11 is a schematic diagram of an intra-frame prediction search device according to an exemplary embodiment.

[0047] Figure 12 is a schematic diagram illustrating the input order of a first 4×4 pixel coding block in a CTU according to an exemplary embodiment.

[0048] Figure 13 is a schematic diagram illustrating a method of continuously accumulating the second prediction cost of a first coding block to obtain the first prediction cost of a second coding block according to an exemplary embodiment.

[0049] Figure 14 is a schematic diagram illustrating a method for obtaining a third prediction cost of a third coding block by continuously accumulating a first prediction cost of a second coding block according to an exemplary embodiment.

[0050] Figure 15 is a schematic diagram illustrating, according to an exemplary embodiment, how to continuously accumulate the third prediction cost of the third coding block to obtain the fourth prediction cost of the fourth coding block, and how to continuously accumulate the fourth prediction cost of the fourth coding block to obtain the fifth prediction cost of the fifth coding block.

[0051] Figure 16 illustrates the processing timing of various modules in an intra-frame prediction search apparatus according to an exemplary embodiment.

[0052] Figure 17 is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0053] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. Specific embodiments of this application will be described below in conjunction with the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. Without departing from the spirit and scope of this application, those skilled in the art can modify and substitute the embodiments of this application, and the resulting embodiments are also within the protection scope of this application.

[0054] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0055] One embodiment of this application provides an intra-frame prediction search method. This intra-frame prediction search method can be applied to a hardware encoder and an electronic device using a hardware encoder. The hardware encoder pipeline has a length of M coding units, capable of processing intra-frame predictions of M coding units in parallel to determine a target prediction mode. The target prediction mode can be the optimal prediction mode, where M is a positive integer. In this embodiment, M is 1 or 2, but is not limited thereto. In this embodiment, the coding unit can be a CTU (Coding Tree Unit), with a size of 64×64 pixels, which is the largest coding unit, but is not limited thereto.

[0056] Please see Figure 3 The intra-frame prediction search method may include the following steps S301 to S302:

[0057] Step S301: When the width of the image is greater than the length of the pipeline, for each current coding unit in the image that is in a non-boundary position, determine that the upper reference pixel of the current coding unit is the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding unit is the original pixel of the left pixel.

[0058] In practical applications, the width of an image is generally much larger than the length of the pipeline. For example, a 1080p image is 17 CTUs wide, and a 4k image is 64 CTUs wide. However, the pipeline of a hardware encoder is generally within 1 to 2 CTUs. This means that as long as the image width is greater than 2 CTUs, there is no longer a pipeline-level data dependency between the CTU to be encoded and the CTU above it.

[0059] In this step, such as Figure 4 As shown, when the width of the image is greater than the length of the pipeline, for each current coding unit 41 located at a non-boundary position in the image, since the pixel above the current coding unit 41 has already completed intra-frame prediction and obtained the reconstructed pixel of the pixel above, it is not necessary to wait and directly determine the upper reference pixel 42 of the current coding unit 41 as the reconstructed pixel of the pixel above. However, the pixel to the left of the current coding unit 41 has not yet completed intra-frame prediction and has not generated a reconstructed pixel. In order to improve the speed of the hardware encoder, it is no longer necessary to wait for the intra-frame prediction result of the left pixel of the current coding unit 41, and directly determine the left reference pixel 43 of the current coding unit 41 as the original pixel of the left pixel.

[0060] Step S302: Perform intra-frame prediction search based on the upper reference pixel and the left reference pixel.

[0061] In this step, intra-frame prediction search is performed on the current coding unit 41 based on the upper reference pixel and the left reference pixel.

[0062] In this embodiment, when determining the upper reference pixel and the left reference pixel of the current coding unit, the left reference pixel can use the original pixel of the left pixel, instead of using the reconstructed pixel of the left pixel as the left reference pixel after generating the reconstructed pixel of the left pixel. Therefore, the dependency between the coarse search and fine search of intra-frame prediction can be removed, so that the coarse search and fine search can be carried out in parallel, which can improve the speed of the hardware encoder to meet the real-time encoding speed requirements of 4K or even 8K video.

[0063] In one embodiment, the current encoding unit may include an array of encoding blocks. For example, the current encoding unit may include an array of 4×4 pixel encoding blocks, or an array of 8×8 pixel encoding blocks, or an array of 16×16 pixel encoding blocks, or an array of 32×32 pixel encoding blocks. When the current encoding unit may include an array of encoding blocks, such as... Figure 5 As shown, step S302 may include the following steps S501 to S504.

[0064] Step S501: For the current coding block located in the first row of the current coding unit, determine that the upper reference pixel of the current coding block is the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; the current coding block in the first row is adjacent to the upper pixel.

[0065] like Figure 4 As shown, in the current coding unit 41, the first row of coding block 411 is adjacent to the pixel above. The pixel above the first row of coding block 411 has completed intra-frame prediction. When the current coding block is the first row of coding block 411, the upper reference pixel of the current coding block is determined to be the reconstructed pixel of the upper pixel. However, the left pixel has not yet completed intra-frame prediction. In order to improve the speed of the hardware encoder, it no longer waits for the intra-frame prediction result of the left pixel of the current coding block, and directly determines the left reference pixel of the current coding block to be the original pixel of the left pixel.

[0066] Step S502: For the current coding block in the remaining rows of the current coding unit excluding the first row, determine that the upper reference pixel of the current coding block is the original pixel of the upper pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel.

[0067] like Figure 4 As shown, when the current coding block is coding block 412 of the remaining rows in the current coding unit 41 excluding the first row, since the upper and left pixels of the current coding block have not completed intra-frame prediction, in order to improve the speed of the hardware encoder, it no longer waits for the intra-frame prediction results of the upper and left pixels of the current coding block, and directly determines the upper reference pixel of the current coding block as the original pixel of the upper pixel, and the left reference pixel of the current coding block as the original pixel of the left pixel.

[0068] For the current coding block in the remaining rows of the current coding unit (excluding the first row), the upper reference pixel of the current coding block is determined as the original pixel of the upper pixel, and the left reference pixel of the current coding block is determined as the original pixel of the left pixel. That is, when determining the upper and left reference pixels of the current coding unit, the upper and left reference pixels can use the original pixels instead of reconstructed pixels. Therefore, the dependency between the coarse search and fine search of intra-frame prediction can be removed, allowing the coarse search and fine search to be carried out in parallel. This can improve the speed of the hardware encoder to meet the real-time encoding speed requirements of 4K or even 8K video.

[0069] Step S503: For each current coding block in the current coding unit, perform intra-frame prediction search based on the upper reference pixel and the left reference pixel to determine N candidate intra-frame prediction modes for the current coding block, where N is a positive integer greater than 1.

[0070] In this step, for each current coding block in the current coding unit, an intra-prediction search is performed based on the upper reference pixel and the left reference pixel to determine N candidate intra-prediction modes for the current coding block, where N is a positive integer greater than 1.

[0071] In one embodiment, N can be 2, 3, 4, 5, 6, 7 or 8, but is not limited thereto.

[0072] like Figure 6 As shown, step S503 may include the following steps S601 to S602.

[0073] Step S601: For each current coding block, calculate the first prediction cost corresponding to the specified L intra-frame prediction modes; L is an integer greater than N.

[0074] In this step, for each current coding block, the first prediction cost corresponding to the specified L intra-frame prediction modes is calculated; L is an integer greater than N. For example, L is 35, but it is not limited to this. The first prediction cost can be SAD (Sum of Absolute Difference) or SATD (Sum of Absolute Transformed Difference), but it is not limited to this.

[0075] Step S602: Determine the intra-prediction modes corresponding to the N smallest first prediction costs among the L first prediction costs as N candidate intra-prediction modes.

[0076] In this step, for each current coding block, the L first prediction costs are compared, the N smallest first prediction costs are selected, and the intra prediction modes corresponding to the N smallest first prediction costs are determined as N candidate intra prediction modes.

[0077] In one embodiment, L is 35 and N is 8. For each current coding block, the first prediction cost corresponding to the specified 35 intra-prediction modes is calculated, and the intra-prediction modes corresponding to the 8 smallest first prediction costs among the 35 first prediction costs are determined as 8 candidate intra-prediction modes.

[0078] Step S504: Generate a candidate mode list based on N candidate intra-frame prediction modes. The candidate mode list includes the mode identifiers of the N candidate intra-frame prediction modes.

[0079] In this step, for each current coding block, a candidate mode list is generated based on N candidate intra-prediction modes. The candidate mode list includes the mode identifiers of the N candidate intra-prediction modes.

[0080] In one embodiment, after step S504, the rate-distortion cost corresponding to N candidate intra-prediction modes is determined. Then, the intra-prediction mode corresponding to the smallest rate-distortion cost among the N rate-distortion costs is determined as the target prediction mode, which is the optimal prediction mode.

[0081] In one embodiment, the current coding unit can be divided into a first coding block of a first size according to a first size, and the current coding unit can also be divided into a second coding block of a second size according to a second size, where the first size is smaller than the second size. Several adjacent first coding blocks can form a second coding block, and the current coding block is the second coding block.

[0082] For example, the first size is 4×4 pixels, and the second size is 16×16 pixels. Figures 7-8 As shown, the first coding block 71 is a 4×4 pixel coding block, and the second coding block 72 is a 16×16 pixel coding block. Four × four adjacent first coding blocks 71 can form a corresponding second coding block 72.

[0083] like Figure 9 As shown, step S601 may include the following steps S901 to S902:

[0084] Step S901: For each current coding block and for each intra-frame prediction mode, calculate the second prediction cost for all first coding blocks in the current coding block.

[0085] In this embodiment, as Figures 7-8 As shown, when the current coding block is a 16×16 pixel second coding block 72, for each current coding block and for each intra-frame prediction mode, the second prediction cost for all first coding blocks 71 in the current coding block is calculated. The category of the second prediction cost is the same as the category of the first prediction cost. When the first prediction cost can be SAD, the second prediction cost is also SAD. When the first prediction cost can be SATD, the second prediction cost is also SATD.

[0086] Step S902: Calculate the sum of the second prediction costs of all first coding blocks in the current coding block to obtain the first prediction cost corresponding to the intra-frame prediction mode.

[0087] In this embodiment, for each current coding block, the sum of the second prediction costs of all first coding blocks in the current coding block is calculated to obtain the first prediction cost corresponding to the intra-frame prediction mode. For example, for each second coding block 72, the sum of the second prediction costs of the 16 first coding blocks 71 in the second coding block 72 is calculated to obtain the first prediction cost corresponding to the intra-frame prediction mode.

[0088] It should be noted that the so-called prediction mode in intra-frame prediction actually represents the texture direction of the image. Different prediction modes represent different texture directions. The closer the selected prediction mode is to the real texture direction, the more accurate the prediction will be.

[0089] For image textures, the following rule applies: if the texture of a large coding block has a certain directional feature, then all 4×4 small coding blocks within it have the same directional feature.

[0090] For example, such as Figures 7-8 As shown, a 16×16 second coding block 72 has the texture direction feature indicated by arrow 73. Therefore, we assume that the 16 4×4 pixel first coding blocks 71 within the second coding block 72 also have the same texture direction feature.

[0091] Considering that the coarse search (used to determine the intra-prediction modes of N candidate frames) does not require high search accuracy, we can assume that the sum of the prediction costs of the first coded block 71 of 16 4×4 pixels is equivalent to the prediction cost of the second coded block 72 of 16×16 pixels itself.

[0092] Of course, in other embodiments, the second size may also be 8×8 pixels, 32×32 pixels, or 64×64 pixels.

[0093] Based on the above pattern, it is only necessary to first perform intra-frame prediction on the first 4×4 pixel coding block 71, calculate the second prediction cost of different prediction modes, and then accumulate the second prediction costs of a certain intra-frame prediction mode of the first 4×4 pixel coding block 71 at different positions. This sum can be used to represent the prediction cost of the same intra-frame prediction mode of the corresponding larger coding block, without having to perform intra-frame prediction on the larger coding block and calculate the prediction cost based on the intra-frame prediction results, thus reducing the amount of computation.

[0094] In this embodiment, the current coding unit is divided into a first coding block of the first size according to a first size, and the current coding unit is also divided into a second coding block of the second size according to a second size. The first size is smaller than the second size. Several adjacent first coding blocks can form a second coding block. When the current coding block is a second coding block, for each current coding block and for each intra-frame prediction mode, the second prediction cost of all first coding blocks in the current coding block is calculated. Then, the sum of the second prediction costs of all first coding blocks in the current coding block is calculated to obtain the first prediction cost corresponding to the intra-frame prediction mode. Instead of performing intra-frame prediction on the second coding block to obtain the first prediction cost, the computational amount of intra-frame prediction can be reduced.

[0095] Figure 10 is a schematic diagram of intra-frame prediction for a 32×32 pixel coded block 70. The middle image in Figure 10 shows the first coded block 71, which is divided into 64 4×4 pixel blocks. The left image shows the third coded block 74, which is divided into 16 8×8 pixel blocks. The right image shows the fourth coded block 75, which is divided into 4 16×16 pixel blocks.

[0096] For the i-th intra-frame prediction mode Mode i Intra-frame prediction is performed on the 64 4×4 pixel first coding blocks 71 within coding block 70 to obtain the second prediction cost for each first coding block 71. The value of i ranges from 1 to 35.

[0097] When it is necessary to calculate the prediction cost of one of the third coding blocks 74, the prediction costs of the four 4×4 pixel first coding blocks 71 at the corresponding position are summed to obtain the prediction cost of the third coding block 74.

[0098] When it is necessary to calculate the prediction cost of one of the fourth coding blocks 75, the prediction costs of the eight 4×4 pixel first coding blocks 71 at the corresponding position are summed to obtain the prediction cost of the fourth coding block 75.

[0099] When it is necessary to calculate the prediction cost of coded block 70, the prediction cost of coded block 70 can be obtained by summing up the second prediction costs of the first coded block 71 of 64 4×4 pixels.

[0100] As mentioned earlier, the computational cost of intra-frame prediction is comparable between 256 4×4 pixel coding blocks, 64 8×8 pixel coding blocks, 16 16×16 pixel coding blocks, 4 32×32 pixel coding blocks, and 1 64×64 pixel coding block within a CTU.

[0101] Using the above intra-prediction coarse search algorithm, intra-prediction only needs to be performed for 256 4×4 pixel coding blocks. Then, the prediction costs of the 4×4 pixel coding blocks are combined to generate the prediction costs of 8×8 pixel coding blocks, 16×16 pixel coding blocks, 32×32 pixel coding blocks, and 64×64 pixel coding blocks. The overall computational cost is directly reduced to 1 / 5 of the original, which greatly reduces the computational cost.

[0102] Another embodiment of this application provides an intra-frame prediction search apparatus. This intra-frame prediction search apparatus is used to implement the above-described intra-frame prediction search method. Referring to FIG11, the intra-frame prediction search apparatus may include a reconstructed pixel storage module 1101, an original pixel storage module 1102, a reference pixel generation module 1103, and a search module 1104.

[0103] The raw pixel storage module 1102 is used to store the raw pixels of pixels in an image. The raw pixels are the pixels before intra-frame prediction.

[0104] The reconstructed pixel storage module 1101 is used to store reconstructed pixels of pixels in an image. In video encoding, after the original pixels undergo steps such as prediction, residual calculation, transformation, quantization, and entropy coding, they are "reconstructed" by the encoder and decoder respectively to obtain reconstructed pixels. In this application, the reconstructed pixels can be the pixels reconstructed by the encoder.

[0105] The reference pixel generation module 1103 is used to, when the width of the image is greater than the length of the pipeline, for each current coding unit in the image that is at a non-boundary position, obtain the corresponding upper reference pixel from the reconstructed pixel storage module when the upper reference pixel of the current coding unit is a reconstructed pixel, obtain the corresponding upper reference pixel from the original pixel storage module when the upper reference pixel of the current coding unit is an original pixel, and obtain the left reference pixel from the original pixel storage module when the left reference pixel of the current coding unit is an original pixel.

[0106] Search module 1104 is used to perform intra-frame prediction search based on the upper reference pixel and the left reference pixel.

[0107] In one embodiment, the reference pixel generation module 1103 is used to generate reference pixels for a 4×4 pixel coded block. As mentioned earlier, the reference pixels are determined to be either original pixels or reconstructed pixels depending on the position of the 4×4 pixel coded block.

[0108] In one embodiment, the search module 1104 is further configured to perform the following steps: for the current coding block located in the first row of the current coding unit, determine that the upper reference pixel of the current coding block is the reconstructed pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; the current coding block in the first row is adjacent to the upper pixel; for the current coding blocks in the remaining rows of the current coding unit other than the first row, determine that the upper reference pixel of the current coding block is the original pixel of the upper pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; for each current coding block in the current coding unit, perform intra-frame prediction search based on its upper reference pixel and left reference pixel to determine N candidate intra-frame prediction modes of the current coding block, where N is a positive integer greater than 1, for example, N is 8, but not limited to this; generate a first candidate mode list based on the N candidate intra-frame prediction modes, the first candidate mode list including the mode identifiers of the N candidate intra-frame prediction modes. The technical solution in this embodiment is similar to... Figure 4 and Figure 5 The technical solutions shown are the same, and will not be described again here.

[0109] In one embodiment, the search module 1104 is further configured to perform the following steps: for each current coding block, calculate the first prediction cost corresponding to the specified L intra-prediction modes; L is an integer greater than N, for example, L is 35, but not limited thereto; determine the intra-prediction modes corresponding to the N first prediction costs with the smallest values ​​among the L first prediction costs as N candidate intra-prediction modes. The technical solution in this embodiment is similar to... Figure 6 The technical solutions shown are the same, and will not be described again here.

[0110] In one embodiment, the current coding unit is divided into a first coding block of a first size according to a first size. The current coding block is the first coding block, and the first size is 4×4 pixels. As shown in FIG11, the search module 1104 may include: K intra-frame prediction modules 141, K first cost calculation modules 142 and a first list generation module 143. K can be an integer greater than 1 and less than L. For example, when L is 35, K can be 9, 12 or 18, but is not limited thereto.

[0111] K intra-frame prediction modules 141 are used to perform intra-frame prediction on K intra-frame prediction modes for the same first coding block to obtain K prediction blocks, until intra-frame prediction of L intra-frame prediction modes is completed for the first coding block to obtain L prediction blocks.

[0112] For example, when L is 35 and K is 12, in the first time period, 12 intra-prediction modules 141 perform intra-prediction on 12 intra-prediction modes for the same first coding block, resulting in 12 prediction blocks. In the second time period, the 12 intra-prediction modules 141 perform intra-prediction on another 12 intra-prediction modes for the same first coding block, resulting in another 12 prediction blocks. In the third time period, 11 intra-prediction modules 141 perform intra-prediction on the remaining 11 intra-prediction modes for the same first coding block, resulting in 11 prediction blocks, for a total of 35 prediction blocks. In other words, by using 12 intra-prediction modules 141 to calculate three times consecutively, intra-prediction of 35 intra-prediction modes for the same first coding block is completed. The first, second, and third time periods are arranged chronologically.

[0113] K first cost calculation modules 142 are connected one-to-one with K intra-frame prediction modules 141. Each first cost calculation module 142 is used to calculate the second prediction cost based on the predicted block of the first coded block calculated by the intra-frame prediction module 141 to which it is connected and the corresponding original block, until the second prediction cost of L intra-frame prediction modes is obtained. The original block refers to the first coded block composed of original pixels.

[0114] For example, when L is 35 and K is 12, in the fourth time period, the 12 first cost calculation modules 142 calculate 12 second predicted costs; in the fifth time period, the 12 first cost calculation modules 142 calculate another 12 second predicted costs; and in the sixth time period, the 11 first cost calculation modules 142 calculate the remaining 11 second predicted costs, for a total of 35 second predicted costs. The fourth, fifth, and sixth time periods are arranged chronologically.

[0115] The first list generation module 143 is used to determine the intra-prediction modes corresponding to the N second prediction costs with the smallest value among the L second prediction costs as N candidate intra-prediction modes, and to generate a list of second candidate modes based on the N candidate intra-prediction modes.

[0116] For example, when L is 35 and N is 8, the first list generation module 143 is used to determine the intra-prediction modes corresponding to the 8 smallest second prediction costs among the 35 second prediction costs as 8 candidate intra-prediction modes, and to generate a list of second candidate modes based on the 8 candidate intra-prediction modes.

[0117] In one embodiment, the current coding unit is further divided into a second coding block of a second size, where the first size is smaller than the second size. In this embodiment, the second size is 8×8 pixels. Several adjacent first coding blocks can form a second coding block; the current coding block is the second coding block. As shown in FIG11, the search module 1104 may further include: a second cost calculation module 144 and a second list generation module 145.

[0118] The second cost calculation module 144 is used to calculate the sum of the second prediction costs of all first coding blocks within the second coding block for each intra-frame prediction mode, thereby obtaining the first prediction cost of the second coding block. The method by which the second cost calculation module 144 calculates the first prediction cost of the second coding block for each intra-frame prediction mode is the same as described above. Figure 7-9 The technical solutions of the embodiments shown are similar and will not be described again here.

[0119] For example, for each 8×8 pixel second coding block, the second cost calculation module 144 calculates the sum of the second prediction costs of the four 4×4 pixel first coding blocks within the second coding block for each intra-frame prediction mode, and obtains the first prediction cost of the second coding block.

[0120] The second list generation module 145 is used to determine the intra-prediction modes corresponding to the N first prediction costs with the smallest value among the L first prediction costs as N candidate intra-prediction modes, and to generate a first candidate mode list based on the N candidate intra-prediction modes.

[0121] For example, when L is 35 and N is 8, the second list generation module 145 is used to determine the intra-prediction modes corresponding to the 8 smallest first prediction costs among the 35 first prediction costs calculated by the second cost calculation module 144 as 8 candidate intra-prediction modes, and to generate a first candidate mode list for the current coding unit based on the 8 candidate intra-prediction modes.

[0122] In one embodiment, the current coding unit is further divided into a third coding block of a third size, where the second size is smaller than the third size. In this embodiment, the third size is 16×16 pixels. Several adjacent second coding blocks can form a third coding block; the current coding block is the third coding block. As shown in FIG11, the search module 1104 may further include: a third cost calculation module 146 and a third list generation module 147.

[0123] The third cost calculation module 146 is used to calculate the sum of the first prediction costs of all second coding blocks within the third coding block for each intra-frame prediction mode, thereby obtaining the third prediction cost of the third coding block. The method by which the third cost calculation module 146 calculates the third prediction cost of the third coding block for each intra-frame prediction mode is similar to the method by which the second cost calculation module 144 calculates the first prediction cost of the second coding block, and will not be described again here.

[0124] For example, for each 16×16 pixel third coding block, the third cost calculation module 146 calculates the sum of the first prediction costs of the four 8×8 pixel second coding blocks within the third coding block for each intra-frame prediction mode, and obtains the third prediction cost of the third coding block.

[0125] The third list generation module 147 is used to determine the intra-prediction modes corresponding to the N smallest third prediction costs among the L third prediction costs as N candidate intra-prediction modes, and to generate a third candidate mode list based on the N candidate intra-prediction modes.

[0126] For example, when L is 35 and N is 8, the third list generation module 147 is used to determine the intra-prediction modes corresponding to the 8 smallest third prediction costs among the 35 third prediction costs calculated by the third cost calculation module 146 as 8 candidate intra-prediction modes, and to generate a list of third candidate modes for the current coding unit based on the 8 candidate intra-prediction modes.

[0127] In one embodiment, the current coding unit is further divided into a fourth coding block of a fourth size, where the third size is smaller than the fourth size. In this embodiment, the fourth size is 32×32 pixels. Several adjacent third coding blocks can form a fourth coding block; the current coding block is the fourth coding block. As shown in FIG11, the search module 1104 may further include: a fourth cost calculation module 148 and a fourth list generation module 149.

[0128] The fourth cost calculation module 148 is used to calculate the sum of the third prediction costs of all third coding blocks within the fourth coding block for each intra-frame prediction mode, thereby obtaining the fourth prediction cost of the fourth coding block. The method by which the fourth cost calculation module 148 calculates the fourth prediction cost of the fourth coding block is similar to the method by which the third cost calculation module 146 calculates the third prediction cost of the third coding block, and will not be described again here.

[0129] For example, for each 32×32 pixel fourth coding block, the fourth cost calculation module 148 calculates the sum of the third prediction costs of the four 16×16 pixel third coding blocks within the fourth coding block for each intra-frame prediction mode, and obtains the fourth prediction cost of the fourth coding block.

[0130] The fourth list generation module 149 is used to determine the intra-prediction mode corresponding to the N fourth prediction costs with the smallest value among the L fourth prediction costs as N candidate intra-prediction modes, and to generate a fourth candidate mode list based on the N candidate intra-prediction modes.

[0131] In one embodiment, the current coding unit is further divided into a fifth coding block of the fifth size, where the fourth size is smaller than the fifth size. In this embodiment, the fifth size is 64×64 pixels. Several adjacent fourth coding blocks can form a fifth coding block; the current coding block is the fifth coding block. As shown in FIG11, the search module 1104 may further include: a fifth cost calculation module 151 and a fifth list generation module 152.

[0132] The fifth cost calculation module 151 is used to calculate the sum of the fourth prediction costs of all fourth coding blocks within the fifth coding block for each intra-frame prediction mode, thereby obtaining the fifth prediction cost of the fifth coding block. The method by which the fifth cost calculation module 151 calculates the fifth prediction cost of the fifth coding block is similar to the method by which the fourth cost calculation module 148 calculates the fourth prediction cost of the fourth coding block, and will not be described again here.

[0133] For example, for each 64×64 pixel fifth coding block, the fifth cost calculation module 151 calculates the sum of the fourth prediction costs of the four 32×32 pixel fourth coding blocks within the fifth coding block for each intra-frame prediction mode, and obtains the fifth prediction cost of the fifth coding block.

[0134] The fifth list generation module 152 is used to determine the intra-prediction modes corresponding to the N fifth prediction costs with the smallest value among the L fifth prediction costs as N candidate intra-prediction modes, and to generate a fifth candidate mode list based on the N candidate intra-prediction modes.

[0135] In one embodiment, the current coding unit is divided into first coding blocks of a first size according to a first size, and the current coding unit is further divided into second coding blocks of a second size according to a second size, where the first size is smaller than the second size. Several adjacent first coding blocks can form a second coding block; the size of the first coding block is 4×4 pixels, and the size of the second coding block is 8×8 pixels. The current coding unit is a second coding block. All first coding blocks in the current coding unit are input to the aforementioned intra-frame prediction search device in a serial manner, and the input order of the first coding blocks is the same as the intra-frame prediction order, that is, the intra-frame prediction search device performs intra-frame prediction on the first coding blocks according to the input order of the first coding blocks. The input order of the first coding blocks satisfies the following condition: the first prediction cost of the corresponding second coding block can be obtained by continuously accumulating the second prediction costs of several adjacent first coding blocks.

[0136] For example, if the input order of the 16 first coding blocks is 0-15, the second prediction cost of the first coding blocks with the input order of 0-3 can be continuously accumulated to obtain the first prediction cost of the corresponding second coding block. The second prediction cost of the first coding blocks with the input order of 4-7 can be continuously accumulated to obtain the first prediction cost of the corresponding second coding block. The second prediction cost of the first coding blocks with the input order of 8-11 can be continuously accumulated to obtain the first prediction cost of the corresponding second coding block. The second prediction cost of the first coding blocks with the input order of 11-15 can be continuously accumulated to obtain the first prediction cost of the corresponding second coding block.

[0137] In another embodiment, the current coding unit is divided into a first coding block of the first size according to a first size, the current coding unit is further divided into a second coding block of the second size according to a second size, the current coding unit is further divided into a third coding block of the third size according to a third size, the current coding unit is further divided into a fourth coding block of the fourth size according to a fourth size, and the current coding unit is further divided into a fifth coding block of the fifth size according to a fifth size, with the first, second, third, fourth, and fifth sizes increasing sequentially. Several adjacent first coding blocks can form a second coding block, several adjacent second coding blocks can form a third coding block, several adjacent third coding blocks can form a fourth coding block, and several adjacent fourth coding blocks can form a fifth coding block. The current coding block is the fifth coding block. The size of the first coding block is 4×4 pixels, the size of the second coding block is 8×8 pixels, the size of the third coding block is 16×16 pixels, the size of the fourth coding block is 32×32 pixels, and the size of the fifth coding block is 64×64 pixels.

[0138] The current coding unit is the fifth coding block. All first coding blocks in the current coding unit are input serially to the aforementioned intra-prediction search device. The input order of the first coding blocks is the same as the intra-prediction order; that is, the intra-prediction search device performs intra-prediction on the first coding blocks according to their input order. The input order of the first coding blocks satisfies the following conditions: accumulating the second prediction costs of several adjacent first coding blocks consecutively yields the first prediction cost of the corresponding second coding block; accumulating the first prediction costs of several adjacent second coding blocks consecutively yields the third prediction cost of the corresponding third coding block; accumulating the third prediction costs of several adjacent third coding blocks consecutively yields the fourth prediction cost of the corresponding fourth coding block; and accumulating the fourth prediction costs of several adjacent fourth coding blocks consecutively yields the fifth prediction cost of the corresponding fifth coding block. Thus, after calculating the second prediction cost of the first coding block, the prediction costs of the second, third, fourth, and fifth coding blocks can be obtained through continuous accumulation, making the calculation convenient.

[0139] As shown in Figure 12, the fifth coding block 1201 is a 64×64 pixel coding block, which is a CTU, including 256 4×4 pixel first coding blocks 71. The input order of the 256 first coding blocks 71 is the number marked at the position of the first coding block 71. The input order of the 256 first coding blocks is 0-255. The first prediction cost of the corresponding second coding block can be obtained by continuously accumulating the second prediction cost of the first coding blocks with the input order 0-3, the second prediction cost of the corresponding second coding block with the input order 4-7, the second prediction cost of the corresponding second coding block with the input order 8-11, and the second prediction cost of the corresponding second coding block with the input order 11-15.

[0140] As shown in Figure 13, based on the encoding order shown in Figure 12, in the second prediction cost calculation module 144, the second prediction cost of the first encoding block 71 for each prediction mode is continuously accumulated four times to obtain the first prediction cost of the corresponding prediction mode for an 8×8 pixel second encoding block 72. The left side of Figure 13 is a schematic diagram of the input order of 256 first encoding blocks 71, and the right side is a schematic diagram of the input order of 63 second encoding blocks 72.

[0141] After obtaining the first prediction cost of a certain prediction mode of 8×8 pixels, the first prediction cost is input into the second list generation module 145 to select the first candidate mode list of the second coding block 72 corresponding to 8×8 pixels.

[0142] On the other hand, the first prediction cost is input into the third prediction cost calculation module 146. In the third prediction cost calculation module 146, as shown in FIG14, the first prediction cost of the second coding block 72 for each prediction mode is continuously accumulated 4 times to obtain the third prediction cost of the corresponding prediction mode of a 16×16 pixel third coding block 74.

[0143] Similarly, as shown in Figure 15, after obtaining the third prediction cost of a certain prediction mode of a 16×16 pixel third coding block 74, it is directly input into the fourth prediction cost calculation module 148. In the fourth prediction cost calculation module 148, the third prediction cost of a certain prediction mode of the third coding block 74 for each prediction mode is continuously accumulated 4 times to obtain the fourth prediction cost of the corresponding prediction mode of a 32×32 pixel fourth coding block 75.

[0144] After obtaining the fourth prediction cost of a certain prediction mode of a 32×32 pixel fourth coding block 75, it is directly input into the fifth prediction cost calculation module 151. In the fifth prediction cost calculation module 151, the fourth prediction cost of the fourth coding block 75 for each prediction mode is continuously accumulated 4 times to obtain the fifth prediction cost of the corresponding prediction mode of a 64×64 pixel fifth coding block 1201.

[0145] In one embodiment, the reference pixel generation module 1103, the intra-frame prediction module 141, the first cost calculation module 142, the first list generation module 143, the second cost calculation module 144, the second list generation module 145, the third cost calculation module 146, the third list generation module 147, the fourth cost calculation module 148, the fourth list generation module 149, the fifth cost calculation module 151, and the fifth list generation module 152 can be processed in parallel to form the pipeline described above.

[0146] The processing timing of the reference pixel generation module 1103, intra-frame prediction module 141, first cost calculation module 142, first list generation module 143, second cost calculation module 144, second list generation module 145, third cost calculation module 146, third list generation module 147, fourth cost calculation module 148, fourth list generation module 149, fifth cost calculation module 151, and fifth list generation module 152 is shown in Figure 16. Time periods T0-T9 are arranged sequentially according to time.

[0147] As shown in Figure 16, the reference pixel generation module 1103 is used to generate reference pixels for coding blocks 1601, 1602, and 1603 in time periods T0, T1, and T2, respectively. Coding blocks 1601, 1602, and 1603 are the Nth, N+1th, and N+2th first coding blocks 71, respectively.

[0148] As shown in Figure 16, the intra-frame prediction module 141 performs intra-frame prediction on coding blocks 1601, 1602, and 1603 in time periods T1, T2, and T3, respectively. In time period T1, the intra-frame prediction module 141 processes the data in parallel with the reference pixel generation module 1103.

[0149] As shown in Figure 16, the first cost calculation module 142 calculates the second prediction costs of coding blocks 1601, 1602, and 1603 in time periods T2, T3, and T4, respectively. In time period T2, the reference pixel generation module 1103, the intra-frame prediction module 141, and the first cost calculation module 142 process in parallel.

[0150] As shown in Figure 16, the first list generation module 143 generates second candidate mode lists for coding blocks 1601, 1602, and 1603 in time periods T3, T4, and T5, respectively. In time period T3, the reference pixel generation module 1103, the intra-frame prediction module 141, the first cost calculation module 142, and the first list generation module 143 process in parallel.

[0151] Similarly, as shown in Figure 16, in time period T3, the reference pixel generation module 1103, intra-frame prediction module 141, first cost calculation module 142, first list generation module 143, and second cost calculation module 144 are processed in parallel. In time period T4, the reference pixel generation module 1103, intra-frame prediction module 141, first cost calculation module 142, first list generation module 143, second cost calculation module 144, and second list generation module 145 are processed in parallel, ... In time period T9, the reference pixel generation module 1103, intra-frame prediction module 141, first cost calculation module 142, first list generation module 143, second cost calculation module 144, second list generation module 145, third cost calculation module 146, third list generation module 147, fourth cost calculation module 148, fourth list generation module 149, fifth cost calculation module 151, and fifth list generation module 152 are processed in parallel.

[0152] For example, during time period T3, the second cost calculation module 144 processes the second prediction cost of the Nth first coding block 71 to obtain the first prediction cost, the first list generation module 143 generates the second candidate mode list of the Nth first coding block 71 in parallel, the first cost calculation module 142 calculates the second prediction cost of the N+1th first coding block 71 in parallel, and the intra-frame prediction module 141 calculates the prediction block of the N+2th first coding block 71 in parallel.

[0153] The technical solution provided in this application has the following advantages when applied to HEVC (High Efficiency Video Coding, also known as H.265): Since only intra-prediction coarse search needs to be performed on 4×4 pixel coding blocks, compared with intra-prediction in all modes for coding blocks of all sizes, this application is expected to reduce the intra-prediction coarse search computation time by 60% while keeping the hardware resources basically unchanged, and is expected to reduce the intra-prediction coarse search resource overhead by 60% while keeping the coding time basically unchanged.

[0154] The technical solution provided in this application can be implemented based on an H.265 encoder. With certain modifications, the technical solution provided in this application can also be applied to an H.266 encoder.

[0155] The method and apparatus embodiments of this application can complement each other.

[0156] Embodiments of this application also propose an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the intra-frame prediction search method of any of the above embodiments.

[0157] Embodiments of this application also propose a computer-readable storage medium that, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the intra-frame prediction search method of any of the above embodiments.

[0158] Embodiments of this application also propose a computer program product, including a computer program that, when executed by a processor, implements a driving method for the universal serial bus interface of any of the above embodiments.

[0159] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0160] An embodiment of this application also provides an electronic device 1700, as shown in FIG17. The electronic device 1700 includes a memory 1701 and a processor 1702. The memory 1701 is used to store a computer program executable by the processor 1702. The processor 1702 is used to execute the computer program in the memory 1701 to implement the intra-frame prediction search method provided in any of the above embodiments.

[0161] Electronic device 1700 also includes communication interface 1703. Processor 1702, memory 1701 and communication interface 1703 are connected through a communication bus and communicate with each other.

[0162] Processor 1702 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0163] The communication interface 1703 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0164] The memory 1701 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0165] In this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0166] The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. An intra-frame prediction search method, characterized in that, The method is applied to a hardware encoder, wherein the hardware encoder has a pipeline length of M coding units, and the intra-frame prediction of the M coding units is processed in parallel to determine the target prediction mode, where M is a positive integer; the method includes: When the width of the image is greater than the length of the pipeline, for each current coding unit in the image that is in a non-boundary position, the upper reference pixel of the current coding unit is determined as the reconstructed pixel of the upper pixel, and the left reference pixel of the current coding unit is determined as the original pixel of the left pixel. Intra-frame prediction search is performed based on the upper reference pixel and the left reference pixel.

2. The intra-frame prediction search method as described in claim 1, characterized in that, The current coding unit includes coding blocks arranged in an array; The intra-frame prediction search based on the upper reference pixel and the left reference pixel includes: For the current coding block located in the first row of the current coding unit, the upper reference pixel of the current coding block is determined as the reconstructed pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; the current coding block in the first row is adjacent to the upper pixel; For the current coding blocks in the remaining rows of the current coding unit other than the first row, the upper reference pixel of the current coding block is determined as the original pixel of the upper pixel, and the left reference pixel of the current coding block is determined as the original pixel of the left pixel; For each current coding block in the current coding unit, an intra-frame prediction search is performed based on the upper reference pixel and the left reference pixel to determine N candidate intra-frame prediction modes for the current coding block, where N is a positive integer greater than 1. A candidate mode list is generated based on N candidate intra-prediction modes. The candidate mode list includes the mode identifiers of the N candidate intra-prediction modes.

3. The intra-frame prediction search method as described in claim 2, characterized in that, For each current coding block in the current coding unit, an intra-prediction search is performed based on the upper reference pixel and the left reference pixel to determine N candidate intra-prediction modes for the current coding block, including: For each current coding block, calculate the first prediction cost corresponding to the specified L intra-frame prediction modes; L is an integer greater than N; The intra-prediction modes corresponding to the N smallest first prediction costs among the L first prediction costs are determined as N candidate intra-prediction modes.

4. The intra-frame prediction search method as described in claim 3, characterized in that, The current coding unit is divided into a first coding block of the first size according to a first size, and the current coding unit is further divided into a second coding block of the second size according to a second size; the first size is smaller than the second size; a plurality of adjacent first coding blocks constitute the second coding block; The current encoded block is the second encoded block; The step of calculating the first prediction cost corresponding to the specified L intra-frame prediction modes for each current coding block includes: For each current coding block, and for each intra-frame prediction mode, calculate the second prediction cost for all first coding blocks in the current coding block; The sum of the second prediction costs of all first coding blocks in the current coding block is calculated to obtain the first prediction cost corresponding to the intra-frame prediction mode.

5. The intra-frame prediction search method as described in claim 1, characterized in that, After determining the N candidate intra-prediction modes for the current coding block, the method further includes: Determine the rate-distortion cost corresponding to N candidate intra-frame prediction modes; The intra-frame prediction mode corresponding to the smallest rate-distortion cost among N rate-distortion costs is determined as the target prediction mode.

6. An intra-frame prediction search device, characterized in that, The apparatus for implementing the intra-frame prediction search method as described in claim 1 includes: a reconstructed pixel storage module, an original pixel storage module, a reference pixel generation module, and a search module; The original pixel storage module is used to store the original pixels of the pixels in the image; The reconstructed pixel storage module is used to store the reconstructed pixels of the pixels in the image; The reference pixel generation module is configured to, when the width of the image is greater than the length of the pipeline, for each current coding unit in the image at a non-boundary position, obtain the upper reference pixel from the reconstructed pixel storage module when the upper reference pixel of the current coding unit is a reconstructed pixel, obtain the upper reference pixel from the original pixel storage module when the upper reference pixel of the current coding unit is an original pixel, and obtain the left reference pixel from the original pixel storage module when the left reference pixel of the current coding unit is an original pixel. The search module is used to perform intra-frame prediction search based on the upper reference pixel and the left reference pixel.

7. The intra-frame prediction search apparatus as described in claim 6, characterized in that, The search module is also used for: For the current coding block located in the first row of the current coding unit, the upper reference pixel of the current coding block is determined as the reconstructed pixel, and the left reference pixel of the current coding block is the original pixel of the left pixel; The current encoding block in the first row is adjacent to the pixel above it; For the current coding blocks in the remaining rows of the current coding unit other than the first row, the upper reference pixel of the current coding block is determined as the original pixel of the upper pixel, and the left reference pixel of the current coding block is determined as the original pixel of the left pixel; For each current coding block in the current coding unit, an intra-frame prediction search is performed based on the upper reference pixel and the left reference pixel to determine N candidate intra-frame prediction modes for the current coding block, where N is a positive integer greater than 1. A first candidate mode list is generated based on N candidate intra-frame prediction modes. The first candidate mode list includes the mode identifiers of the N candidate intra-frame prediction modes.

8. The intra-frame prediction search apparatus as described in claim 7, characterized in that, The search module is also used for: For each current coding block, calculate the first prediction cost corresponding to the specified L intra-frame prediction modes; L is an integer greater than N; The intra-prediction modes corresponding to the N smallest first prediction costs among the L first prediction costs are determined as N candidate intra-prediction modes.

9. The intra-frame prediction search apparatus as described in claim 8, characterized in that, The current coding unit is divided into a first coding block of the first size according to a first size; the current coding block is the first coding block; The search module includes: K intra-frame prediction modules, K first cost calculation modules, and a first list generation module; K is an integer greater than 1 and less than L; K intra-frame prediction modules are used to perform intra-frame prediction on K intra-frame prediction modes for the same first coding block to obtain K prediction blocks, until intra-frame prediction of L intra-frame prediction modes is completed for the first coding block to obtain L prediction blocks. K first cost calculation modules are connected one-to-one with K intra-frame prediction modules. Each first cost calculation module is used to calculate the second prediction cost based on the prediction block of the first coding block and the corresponding original block, until the second prediction cost of L intra-frame prediction modes is obtained. The first list generation module is used to determine the intra-prediction modes corresponding to the N smallest second prediction costs among the L second prediction costs as N candidate intra-prediction modes, and to generate a list of second candidate modes based on the N candidate intra-prediction modes.

10. The intra-frame prediction search apparatus as described in claim 9, characterized in that, The current coding unit is further divided into second coding blocks of a second size according to a second size; the first size is smaller than the second size; several adjacent first coding blocks can form the second coding block; The current encoded block is the second encoded block; The search module further includes: a second cost calculation module and a second list generation module; The second cost calculation module is used to calculate the sum of the second prediction costs of all the first coding blocks in the second coding block for each of the intra-frame prediction modes, so as to obtain the first prediction cost of the second coding block. The second list generation module is used to determine the intra-prediction modes corresponding to the N first prediction costs with the smallest value among the L first prediction costs as N candidate intra-prediction modes, and to generate a first candidate mode list based on the N candidate intra-prediction modes.

11. The intra-frame prediction search apparatus as described in claim 10, characterized in that, All first coding blocks in the current coding unit are input to the intra-prediction search device in a serial manner. The input order of the first coding blocks is the same as the intra-prediction order. The input order of the first coding blocks satisfies the following condition: the first prediction cost of the corresponding second coding block is obtained by continuously accumulating the second prediction costs of several adjacent first coding blocks.

12. The intra-frame prediction search apparatus as described in claim 11, characterized in that, The reference pixel generation module, intra-frame prediction module, first cost calculation module, first list generation module, second cost calculation module, and second list generation module are processed in parallel to form the pipeline.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in claim 1.

Images