Intra-frame prediction mode selection method, device, equipment and readable storage medium

Abstract

An embodiment of the present application provides a method for selecting an intra-frame prediction mode, wherein a first device selects a block to be encoded of a size of N*N in an image to be encoded, and determines the availability of the adjacent blocks based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks. If the adjacent blocks are available, the first device uses the first pixel to calculate the predicted values ​​of the N*N pixels of the block to be encoded under different intra-frame prediction modes. Then, the first device calculates the residuals of the N*N pixels of the block to be encoded under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the block to be encoded and the original values ​​of the N*N pixels of the block to be encoded, and selects the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals under the different intra-frame prediction modes.

Description

Technical Field

[0001] The present application relates to the technical field of video coding, and in particular to a method for selecting an intra-frame prediction mode, related devices, equipment, and a computer-readable storage medium. Background Art

[0002] The existing international standards for video compression coding basically adopt the hybrid encoder framework of "prediction + transform". Starting from H.264, intra-frame prediction technology has been introduced in the predictive coding link to further improve compression efficiency. As a key link in predictive coding, intra-frame prediction mainly uses the strong correlation between adjacent pixels. Based on a certain prediction relationship, the prediction value of the current block is derived from the reference pixels at adjacent positions in units of blocks. During encoding, only the difference between the predicted value and the true value (i.e., the prediction error or residual) needs to be transmitted.

[0003] Intra-frame coding includes two parts: intra-frame decision and final prediction. The final prediction must strictly follow the protocol to encode a code stream that meets the specifications. Intra-frame decision is the algorithm part, including the division depth decision and intra-frame mode selection. Different algorithms will produce different decision results, which in turn determine the complexity of implementation and the final compression rate. The existing intra-frame mode selection algorithms are based on the protocol and use the reconstructed pixels of the coded blocks adjacent to the current block to predict the current block. However, this method, when implemented in hardware, needs to wait for the encoding of the adjacent blocks of the current block to be completed, the timing design is strict, and the encoding time is often long. In addition, the pixels of the upper row and the left column of the current block need to be saved as references for the current block; for different CTU rows, the data of the previous row of CTU of the current row of Collect Transfer Unit (CTU) also needs to be saved. Especially when the image resolution is large, the required memory overhead is very large. Summary of the invention

[0004] The present application provides a method for selecting an intra-frame prediction mode, which solves the problems of long waiting time and high consumption of storage resources in the process of selecting an intra-frame prediction mode by an encoder, and improves the working performance of the encoder.

[0005] In a first aspect, an embodiment of the present application provides a method for selecting an intra-frame prediction mode, characterized by comprising:

[0006] The first device selects a block to be encoded with a size of N*N in the image to be encoded;

[0007] The first device determines availability of the adjacent block based on a position of the block to be encoded in the image to be encoded and an encoding order of the block to be encoded and its adjacent blocks;

[0008] If the neighboring block is available, the first device uses the first pixel to calculate the predicted values ​​of N*N pixels of the block to be encoded under different intra-frame prediction modes; wherein the first pixel is a pixel adjacent to the neighboring block;

[0009] The first device calculates the residual of the N*N pixels of the to-be-encoded block under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the to-be-encoded block and the original values ​​of the N*N pixels of the to-be-encoded block;

[0010] The first device selects the intra-frame prediction mode with the smallest intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of N*N pixels of the to-be-encoded block under the different intra-frame prediction modes.

[0011] In the embodiment of the present application, the first device uses the pixel of the block to be encoded in the image to be encoded as the first pixel, and uses the first pixel to perform intra-frame prediction, and then selects the best intra-frame prediction mode, which reduces the waiting time and improves the encoding speed while ensuring the encoding quality, meeting the needs of low-latency devices. At the same time, since the encoder stores less information about the pixels, it saves the encoder's storage resources to a certain extent and improves the encoder's working performance.

[0012] In a possible implementation manner, the first pixel includes:

[0013] If the available adjacent blocks include a first adjacent block, the first adjacent block is N pixels distributed in a vertical direction and adjacent to the block to be encoded in a first horizontal direction; then, the first pixel includes N pixels adjacent to the first adjacent block in the block to be encoded;

[0014] If the available adjacent blocks include a second adjacent block, the second adjacent block is N pixels arranged in a horizontal direction and adjacent to the block to be encoded in a vertical first direction; then, the first pixel includes N pixels in the block to be encoded that are adjacent to the second adjacent block;

[0015] If the available adjacent blocks include a third adjacent block, the third adjacent block is N pixels arranged in a vertical direction and adjacent to the first adjacent block in a second vertical direction; then, the first pixel includes N pixels distributed in a vertical direction and adjacent to the third adjacent block in a second horizontal direction;

[0016] If the available adjacent blocks include a fourth adjacent block, the fourth adjacent block is N pixels distributed in the horizontal direction and adjacent to the second adjacent block in the horizontal second direction; then, the first pixel includes N pixels distributed in the horizontal direction and adjacent to the fourth adjacent block in the vertical second direction;

[0017] If the available adjacent blocks include a fifth adjacent block, the fifth adjacent block is a pixel adjacent to the second adjacent block in a first horizontal direction and distributed in a vertical direction; then the first pixel includes a pixel in the coding block that is adjacent to the fifth adjacent block;

[0018] The opposite direction of the first direction is the second direction.

[0019] In the embodiment of the present application, the first pixel is divided in this way so that the first pixel includes at most 4*N-1 pixels, and the encoder stores the pixel values ​​of at most N*N+2*N pixels (wherein, N*N pixels are the pixels of the block to be encoded, and 2*N pixels are the N pixels adjacent to the third adjacent block and the N pixels adjacent to the fourth adjacent block in the original image), which saves the storage resources of the encoder to a certain extent.

[0020] In a possible implementation manner, the first device determines availability of the adjacent block based on a position of the block to be encoded in the image to be encoded and an encoding order of the block to be encoded and its adjacent blocks, including:

[0021] The first device determines whether the adjacent block exists in the image to be encoded;

[0022] If the adjacent block exists in the image to be encoded, the first device obtains the encoding order of the block to be encoded and its adjacent blocks;

[0023] If the encoding order of the neighboring blocks indicates that the neighboring block is encoded before the block to be encoded, the first device determines that the neighboring block is an available neighboring block.

[0024] In the embodiment of the present application, the first device determines the availability of the adjacent blocks, which is conducive to the first device determining the first pixel based on the available adjacent blocks, and using the first pixel for intra-frame prediction, and then selecting the best intra-frame prediction mode, which reduces the waiting time and improves the efficiency of intra-frame prediction mode selection under the premise of ensuring the encoding quality. At the same time, since the encoder stores less information about pixels, the encoder's storage resources are saved to a certain extent, and the encoder's working performance is improved.

[0025] In a possible implementation manner, the first device uses the first pixel to calculate the prediction values ​​of N*N pixels in the to-be-encoded block in different intra-frame prediction modes, including:

[0026] After determining the availability of the neighboring blocks, if there is an unavailable neighboring block, the first device fills the pixel value of the unavailable neighboring block with the first pixel;

[0027] The first device calculates the prediction values ​​of N*N pixels in the block to be encoded based on the calculation formula of the prediction values ​​in different intra-frame prediction modes and uses the corresponding reference pixels; wherein the reference pixels include the first pixels and the pixel values ​​after filling the unavailable adjacent blocks.

[0028] In an embodiment of the present application, the encoder uses the first pixel to calculate the pixel value of the unavailable adjacent block, so that when the encoder calculates the prediction value of the to-be-encoded block under different intra-frame prediction modes, it can use the corresponding reference pixel for calculation, without having to wait for the available adjacent block to be encoded before calculating the prediction value, thereby saving the time of the encoder in selecting the intra-frame prediction mode.

[0029] In a possible implementation manner, the first device uses the first pixel to calculate the prediction values ​​of N*N pixels in the to-be-encoded block in different intra-frame prediction modes, including:

[0030] After determining the availability of the neighboring blocks, if the available neighboring blocks do not exist, the first device fills the pixels of the unavailable neighboring blocks with a fixed value; wherein the fixed value is obtained based on the bit depth of the image to be encoded;

[0031] The first device calculates the prediction values ​​of N*N pixels in the block to be encoded based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and uses the corresponding reference pixels; wherein the reference pixels are the pixel values ​​after filling the unavailable adjacent blocks.

[0032] In a possible implementation, the first device selects the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of the N*N pixels of the to-be-coded block under the different intra-frame prediction modes, including:

[0033] The first device obtains a distortion value based on the residuals in the different intra-frame prediction modes;

[0034] The first device calculates the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes according to the formula J=D+λ*R;

[0035] The first device selects the intra-frame prediction mode with the smallest rate-distortion cost as the optimal intra-frame prediction mode;

[0036] Among them, J is the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes; D is the distortion value, and D is used to indicate the error between the predicted value of the different intra-frame prediction modes and the original value of the pixel of the block to be encoded; R is the number of bits required to encode the image to be encoded under the different intra-frame prediction modes; λ is the Lagrange multiplier.

[0037] In a possible implementation manner, the first device uses the first pixel to fill the pixel value of the unavailable adjacent block, including:

[0038] The first device selects, in the first pixel, pixels adjacent to the unavailable neighboring block;

[0039] The first device fills pixels of the unavailable neighboring block with the adjacent pixels.

[0040] In a second aspect, an embodiment of the present application provides a device for selecting an intra-frame prediction mode, characterized by comprising:

[0041] A block selection unit to be encoded, used to select a block to be encoded with a size of N*N in the image to be encoded;

[0042] an availability judgment unit, configured to judge the availability of the adjacent block based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks;

[0043] A prediction value calculation unit, configured for the first device to calculate prediction values ​​of N*N pixels of the to-be-encoded block under different intra-frame prediction modes using the first pixel;

[0044] A residual calculation unit, configured to calculate the residuals of the N*N pixels of the to-be-encoded block under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the to-be-encoded block and the original values ​​of the N*N pixels of the to-be-encoded block;

[0045] The intra-frame prediction mode selection unit is used to select the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of the N*N pixels of the to-be-coded block under the different intra-frame prediction modes.

[0046] In a possible implementation, the availability determination unit includes:

[0047] A first judging unit, configured to judge whether the adjacent block exists in the image to be encoded;

[0048] a coding order obtaining unit, configured to obtain a coding order of the block to be coded and its adjacent blocks if the first determining unit determines that the block to be coded is yes;

[0049] The second judgment unit is configured to judge that the neighboring block is an available neighboring block if the encoding order of the neighboring block indicates that the neighboring block is encoded before the block to be encoded.

[0050] In a possible implementation, the prediction value calculation unit includes:

[0051] A first pixel filling unit, used to fill the pixel value of the unavailable adjacent block with the first pixel;

[0052] The first calculation unit is used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and using the corresponding reference pixels.

[0053] In a possible implementation, the prediction value calculation unit further includes:

[0054] A second pixel filling unit is used to fill the pixels of the unavailable adjacent block with a fixed value if there is no available adjacent block after the availability judgment unit judges the availability of the adjacent block;

[0055] The second calculation unit is used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and using the corresponding reference pixels.

[0056] In a possible implementation, the intra-frame prediction mode selection unit includes:

[0057] A distortion value generating unit, configured to obtain a distortion value based on the residuals in the different intra-frame prediction modes;

[0058] The rate-distortion cost calculation unit is used to calculate the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes according to the formula J=D+λ*R.

[0059] The selection unit is used to select the intra-frame prediction mode with the smallest rate-distortion cost as the optimal intra-frame prediction mode.

[0060] In a possible implementation manner, the first pixel filling unit includes:

[0061] A first selection unit, configured to select, from among the first pixels, pixels adjacent to the unavailable adjacent block;

[0062] The first filling unit is configured to fill the pixels of the unavailable adjacent block with the adjacent pixels.

[0063] In a third aspect, an embodiment of the present application provides a device for selecting an intra-frame prediction mode, characterized by comprising: a memory and a processor;

[0064] The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to execute the method for selecting the intra-frame prediction mode in the above-mentioned first aspect and various possible implementations thereof.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the first device implements the method for selecting an intra-frame prediction mode in the above-mentioned first aspect and its various possible implementation methods.

[0066] In a fifth aspect, an embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a computer, a first device can execute the process executed by the first device in the above-mentioned first aspect and its various possible implementation methods.

[0067] In a sixth aspect, an embodiment of the present application provides a chip system, which includes a processor for supporting a first device to implement the functions involved in the method in the first aspect and its various possible modes.

[0068] In a possible design, the chip system further includes a memory, and the memory is used to store program instructions and data necessary for the first device. The chip system can be composed of a chip, or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a schematic diagram of a flow chart of intra-frame prediction mode selection provided by an embodiment of the present application;

[0070] Figure 2 This is a schematic diagram of intra-frame prediction of adjacent block pixels provided by an embodiment of the present application;

[0071] Figure 3 It is a structural schematic diagram of a device for selecting an intra-frame prediction mode provided in an embodiment of the present application;

[0072] Figure 4 It is a structural diagram of a device for selecting an intra-frame prediction mode provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0074] Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present embodiment application. The appearance of this phrase in various positions in the specification is not necessarily the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It can be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0075] The terms "first", "second", "third", etc. in the specification and claims of the present application and the drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a series of steps or units are included, or optionally, steps or units not listed are included, or optionally, other steps or units inherent to these processes, methods, products or devices are included.

[0076] Only the part relevant to the present application but not all content is shown in the accompanying drawings.Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processing or methods described as flow charts.Although flow charts describe various operations (or steps) as sequential processing, many operations therein can be implemented in parallel, concurrently or simultaneously.In addition, the order of various operations can be rearranged.Described processing can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings.Described processing can correspond to method, function, procedure, subroutine, subprogram etc.

[0077] As used in this specification, the terms "component", "module", "system", "unit", etc. are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate through local and / or remote processes, for example, based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network. For example, the Internet interacts with other systems via signals).

[0078] The first device described in the claims and embodiments of the present application is a device with encoding function, such as an encoder.

[0079] See also Figure 1 , Figure 1 1 is a flowchart of selecting an intra-frame prediction mode provided by an embodiment of the present application. Since pixels adjacent to the adjacent blocks in the block to be encoded have a high degree of similarity with the pixels of the adjacent blocks, it is feasible to use the pixels of the block to be encoded as intra-frame prediction and then select the best intra-frame prediction mode. This embodiment takes the first device as an encoder as an example. Figure 1 As shown, the process of the encoder selecting the intra prediction mode is as follows:

[0080] Step S101: the encoder selects a block to be encoded with a size of N*N in the image to be encoded.

[0081] Specifically, the compression coding of the image is based on the correlation between pixels. In the image, only pixels in a small neighborhood have a high correlation. Therefore, the encoder divides the image to be encoded into multiple blocks to be encoded of size N*N. The block to be encoded is a neighborhood, which is conducive to the encoder estimating the prediction value and residual of the block to be encoded based on the correlation of pixels in the neighborhood. Then, the encoder calculates the rate-distortion cost of the different intra-frame prediction modes based on the residuals of the different intra-frame prediction modes, thereby selecting the best intra-frame prediction mode.

[0082] Step S102: the encoder determines the availability of the adjacent blocks based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks.

[0083] Specifically, for a block to be encoded with a size of N*N, if its neighboring blocks exist, the neighboring blocks of the reference pixels of the block to be encoded are at most 5. The encoder determines the availability of neighboring blocks based on the following two principles:

[0084] The first principle is that the encoder determines the available adjacent blocks based on the position of the block to be encoded in the image to be encoded. When the block to be encoded is located at the edge of the image to be encoded, some of the adjacent blocks of the block to be encoded do not exist, and the adjacent blocks that do not exist are naturally unavailable.

[0085] The second principle is that in the presence of adjacent blocks, the encoder determines the availability of adjacent blocks based on the encoding order of the adjacent blocks and the encoding order of the block to be encoded; the available adjacent blocks are defined as adjacent blocks that are to be encoded prior to the block to be encoded; therefore, if the encoding order of the adjacent blocks indicates that the adjacent blocks are encoded prior to the block to be encoded, the adjacent blocks are available; if the encoding order of the adjacent blocks does not indicate that the adjacent blocks are encoded prior to the block to be encoded, the adjacent blocks are unavailable; wherein, the encoding order is used to indicate the order of block encoding in the image to be encoded.

[0086] See also Figure 2 , Figure 2 Schematic diagram of an intra-frame prediction of adjacent block pixels provided by an embodiment of the present application. Figure 2 In the example, the size of the block to be coded is N*N, that is, the block to be coded has N*N pixels (pixel P 11 ~Pixel P N,N ), the block to be encoded has at most 5 adjacent blocks that can be used as reference pixels for calculating the predicted values ​​of N*N pixels of the block to be encoded under different intra-frame prediction modes; Figure 2 As shown, the five adjacent blocks of the block to be encoded are adjacent blocks A (pixel R 0,N+1 ~Pixel R 0,2N ), adjacent block B (pixel R 0,1 ~Pixel R 0,N ), adjacent block C (pixel R 0,0 ), adjacent block D (pixel R 1,0 ~Pixel R N,0 ) and adjacent blocks E(R N+1,0 ~R 2N,0 ). After the encoder scans the block position in the to-be-encoded image, the encoder obtains the encoding order of the to-be-encoded block and its five neighboring blocks; for example, if the encoding order of the neighboring block B is earlier than the encoding order of the to-be-encoded block, the neighboring block B is an available neighboring block; if the encoding of the five neighboring blocks is later than the to-be-encoded block, the five neighboring blocks are all unavailable neighboring blocks.

[0087] Step S103: If the neighboring block is available, the encoder uses the first pixel to calculate prediction values ​​of N*N pixels of the to-be-encoded block under different intra-frame prediction modes.

[0088] Specifically, after the encoder determines the availability of its neighboring blocks, if there are available neighboring blocks, the encoder searches for the first pixel corresponding to the available neighboring blocks, and the first pixel is defined as follows: if the available neighboring blocks include a first neighboring block, the first neighboring block is N pixels adjacent to the block to be encoded in the horizontal first direction and arranged in the vertical direction; then, the first pixel includes the N pixels adjacent to the first neighboring block in the block to be encoded. If the available neighboring blocks include a second neighboring block, the second neighboring block is N pixels adjacent to the block to be encoded in the vertical first direction and arranged in the horizontal direction; then, the first pixel includes the N pixels adjacent to the second neighboring block in the block to be encoded. If the available neighboring blocks include a third neighboring block, the third neighboring block is N pixels adjacent to the first neighboring block in the vertical second direction and arranged in the vertical direction; then, the first pixel includes the N pixels adjacent to the third neighboring block in the horizontal second direction and distributed in the vertical direction. If the available adjacent blocks include a fourth adjacent block, the fourth adjacent block is N pixels distributed in the horizontal direction and adjacent to the second adjacent block in the horizontal second direction; then, the first pixel includes N pixels distributed in the horizontal direction and adjacent to the fourth adjacent block in the vertical second direction. If the available adjacent blocks include a fifth adjacent block, the fifth adjacent block is a pixel distributed in the vertical direction and adjacent to the second adjacent block in the horizontal first direction; then, the first pixel includes a pixel in the coding block that is adjacent to the fifth adjacent block; wherein the opposite direction of the first direction is the second direction.

[0089] exist Figure 2 In the example, if the horizontal first direction is the left, the horizontal second direction is the right, the vertical first direction is the top, and the vertical second direction is the bottom, then: the first adjacent block is the adjacent block B adjacent to the block to be encoded on the left of the block to be encoded; the second adjacent block is the adjacent block D adjacent to the block to be encoded below the block to be encoded; the third adjacent block is the adjacent block A adjacent to the adjacent block B below the adjacent block B; the fourth adjacent block is the adjacent block E adjacent to the adjacent block D on the right of the adjacent block D; the fifth adjacent block is the adjacent block D adjacent to the adjacent block D on the left of the adjacent block D. According to the above definition of the first pixel, if the adjacent block B is available, the pixel P 1,1 ~Pixel P 1,N is the first pixel; if the adjacent block D is available, the pixel P1,1 ~Pixel P N,1 is the first pixel; if the adjacent block A is available, then pixel M 1,1 ~Pixel M 1,N is the first pixel; if the adjacent block E is available, then pixel Q 1,1 ~Pixel QN,1 is the first pixel; if the adjacent block C is available, then pixel P 1,1 As a result, we can know that for a block to be encoded with a size of N*N, if there are available adjacent blocks, the maximum number of first pixels of the block to be encoded is 4*N-1 (when adjacent blocks A, B, C, D, E or adjacent blocks A, B, D, E are available, the first pixel includes pixel P 1,1 ~Pixel P 1,N , P 2,1 ~Pixel P N,1 , Pixel M 1,1 ~Pixel M 1,N and Pixel Q 1,1 ~Pixel Q N,1 ).

[0090] In one possible implementation, if the number of available adjacent blocks is less than 5, the encoder uses the first pixel to fill the pixel value of the unavailable pixel, and the filling rule is: the encoder uses the pixel adjacent to the unavailable adjacent block in the first pixel to fill a pixel in the unavailable adjacent block adjacent to the pixel, and then fills the pixels adjacent to the pixel in the unavailable adjacent block based on the pixel in the filled unavailable adjacent block, until all the pixels in the unavailable adjacent block are filled, to obtain the filled adjacent block, the pixels of the filled adjacent block and the first pixel are reference pixels, and the reference pixels are used to calculate the predicted values ​​of N*N pixels of the block to be encoded under different intra-frame prediction modes; through the above method, the encoder can fill all the unavailable adjacent blocks to obtain reference pixels. Then, the encoder calculates the predicted values ​​of all pixels in the block to be encoded using the corresponding reference pixels according to the formula for calculating the predicted values ​​in different intra-frame prediction modes. Figure 2 If only the neighboring block B is available, then pixel P 1,1 ~Pixel P 1,N For the first pixel, the encoder uses pixel P 1,1 Fill pixel R 0,0 and Pixel R 1,0 , then the encoder uses pixel R 1,0 To fill pixel R 2,0 Then use pixel R 2,0 Fill in the adjacent block D with pixels R 2,0 The next adjacent pixel is filled until all the pixels of the adjacent block D are filled; the encoder uses the filled pixel R in the same way N,0 To fill pixel R N+1,0 , until all pixels of the adjacent block E are filled; similarly, the encoder uses the same method, using pixel P 1,NFill all pixels in the adjacent block A. The filled pixels and the first pixel are reference pixels. The encoder uses the corresponding reference pixels to calculate the predicted values ​​of N*N pixels in the block to be encoded based on the calculation formulas of the predicted values ​​of different intra-frame prediction modes. It should be noted that if the pixels required by the prediction value calculation formula are pixels of the available adjacent blocks, the encoder uses the first pixel as the reference pixel to replace the pixels of the available adjacent blocks; for example, Figure 2 In the example, if there is an intra-frame prediction mode, the calculation formula of its prediction value needs to use the values ​​of adjacent blocks A, adjacent blocks B and adjacent blocks C, and the adjacent block B is an available adjacent block. The encoder calculates the first pixel and the pixel values ​​of the filled adjacent blocks A, adjacent blocks B and adjacent blocks C through the calculation formula of the prediction value to obtain the prediction value of the block to be encoded.

[0091] In one possible implementation, if the number of available adjacent blocks is less than 5, the encoder uses the first pixel to fill the pixel value of the unavailable pixel, and the filling rule may also be: the encoder selects pixels adjacent to the unavailable adjacent block in the first pixel as filling pixels, and uses the filling pixels to fill the pixels of the unavailable adjacent block to obtain the filled adjacent block; the pixels of the filled adjacent block and the first pixel are reference pixels, and the reference pixels are used to calculate the predicted values ​​of N*N pixels of the block to be encoded in different intra-frame prediction modes; in the above manner, the encoder can fill all the unavailable adjacent blocks to obtain reference pixels. Then, the encoder calculates the predicted values ​​of all pixels in the block to be encoded using the corresponding reference pixels according to the formula for calculating the predicted values ​​in different intra-frame prediction modes. For example, in Figure 2 If only the neighboring block B is available, then pixel P 1,1 ~Pixel P 1,N is a first pixel, and the encoder selects adjacent pixels P of adjacent blocks C, adjacent blocks D and adjacent blocks E from the first pixel. 1,1 is a filling pixel, and uses the filling pixel P 1,1 Fill the pixels in the adjacent blocks C, D and E; similarly, the encoder uses the pixel P adjacent to the adjacent block A in the first pixel. 1,N is a filling pixel, and uses the filling pixel P 1,N Fill the adjacent block A. The filled pixels and the first pixel are reference pixels, and the encoder uses the corresponding reference pixels to calculate the predicted values ​​of N*N pixels in the block to be encoded based on the calculation formulas of the predicted values ​​of different intra-frame prediction modes. It should be noted that if the pixels required by the prediction value calculation formula are pixels of the available adjacent blocks, the encoder uses the first pixel as the reference pixel to replace the pixels of the available adjacent blocks; for example, Figure 2In the example, if there is a certain intra-frame prediction mode, the calculation formula of its prediction value needs to use the values ​​of adjacent blocks A, adjacent blocks B and adjacent blocks C, and the adjacent block B is an available adjacent block, the encoder calculates the pixel values ​​of the first pixel and the filled adjacent blocks A, adjacent blocks B and adjacent blocks C through the calculation formula of the prediction value to obtain the prediction value of the block to be encoded. In a possible implementation, if there is no available adjacent block, the encoder selects a fixed value to fill the pixel values ​​of its 5 adjacent blocks, and the fixed value is 2 n-1 , where n is the bit depth. For example, for a 10-bit image, 512 is used to fill the pixels of all unavailable adjacent blocks, and the filled pixels are reference pixels. Then, the encoder uses the corresponding reference pixels to calculate the prediction values ​​of N*N pixels in the block to be encoded based on the calculation formula of the prediction values ​​of different intra-frame prediction modes.

[0092] Step S104: the encoder calculates residuals of the N*N pixels of the block to be encoded under different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the block to be encoded and the original values ​​of the N*N pixels.

[0093] Specifically, the encoder subtracts the predicted value of each pixel in the block to be encoded from the original value of each pixel in the block to be encoded to obtain the residual of N*N pixels in the block to be encoded.

[0094] Step S105: selecting the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of the N*N pixels of the to-be-coded block under the different intra-frame prediction modes.

[0095] Specifically, the encoder can calculate the cost of intra-frame coding based on the residual, and then select the intra-frame prediction mode with the minimum cost as the optimal intra-frame prediction mode. The embodiment of the present application takes the rate-distortion cost as an example for explanation. The calculation formula of the rate-distortion cost is: J = D + λ * R

[0096] Among them, J is the rate distortion cost of a certain intra-frame mode, D is the distortion value obtained by the residual matrix after Hadamard transform or other transformations, which can reflect the error between the pixel value of the block to be coded and the predicted value to a certain extent; R is the number of bits required to encode the intra-frame prediction mode, that is, the estimated value of the mode rate, which does not need to go through the complete "prediction-change-quantization-entropy coding" process. Generally, λ and R can use empirical values, so it takes less time. Among them, the selection of λ empirical value is related to the encoding QP value, and the selection of R empirical value is related to the MPM (Most Probable Mode) list. Whether it is in the list and the order in the list are different, the R value is different.

[0097] In an embodiment of the present application, the encoder selects a first pixel based on an available adjacent block, and then calculates the pixel value of the unavailable adjacent block through the first pixel. The first pixel and the pixel value of the unavailable adjacent block obtained by filling the first pixel are reference pixels, and the reference pixels are used to calculate the predicted values ​​of all pixels of the block to be encoded under different intra-frame prediction modes. After the encoder calculates the predicted values ​​and residuals of all pixels in the block to be encoded under different intra-frame prediction modes, the intra-frame prediction mode with the minimum intra-frame coding cost is selected as the optimal prediction mode based on the residual. In the process of selecting the intra-frame prediction mode by the encoder in the above-mentioned embodiment, when the encoder uses the reference pixel to calculate the predicted value of the block to be encoded, it is not necessary to wait for the pixel encoding of the available adjacent block to be completed before calculating the predicted value, thereby greatly saving the time for the encoder to select the intra-frame prediction mode, increasing the degree of freedom of the actual processing order of the coding block when selecting the intra-frame prediction mode, and improving the working efficiency of the encoder. In addition, the encoder only needs to store the pixel values ​​of all pixels in the block to be encoded and the pixel values ​​of the first pixel that are not the block to be encoded, which greatly saves the storage space of the encoder.

[0098] The method of the embodiment of the present application is described in detail above, and the related devices and equipment of the embodiment are provided below.

[0099] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of an intra-frame prediction mode selection device provided in an embodiment of the present application. The intra-frame prediction mode selection device 3 may be the first device in the above method embodiment. The intra-frame prediction mode selection device 3 may include a to-be-encoded block selection unit 301, an availability judgment unit 302, a prediction value calculation unit 303, a residual calculation unit 304, and an intra-frame prediction mode selection unit 305, wherein each unit is described in detail as follows:

[0100] The to-be-encoded block selection unit 301 is used to select a to-be-encoded block of size N*N in the to-be-encoded image;

[0101] The availability judgment unit 302 judges the availability of the adjacent block based on the position of the block to be coded in the image to be coded and the coding order of the block to be coded and its adjacent blocks;

[0102] A prediction value calculation unit 303, configured for the first device to calculate prediction values ​​of N*N pixels of the to-be-encoded block under different intra-frame prediction modes using the first pixel;

[0103] The residual calculation unit 304 is used to calculate the residual of the N*N pixels of the block to be encoded under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the block to be encoded and the original values ​​of the N*N pixels of the block to be encoded;

[0104] The intra-frame prediction mode selection unit 305 is configured to select the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of the N*N pixels of the to-be-coded block under the different intra-frame prediction modes.

[0105] In a possible implementation, the availability determination unit 302 includes:

[0106] A first judging unit, configured to judge whether the adjacent block exists in the image to be encoded;

[0107] a coding order obtaining unit, configured to obtain a coding order of the block to be coded and its adjacent blocks if the first determining unit determines that the block to be coded is yes;

[0108] The second judgment unit is configured to judge that the neighboring block is an available neighboring block if the encoding order of the neighboring block indicates that the neighboring block is encoded before the block to be encoded.

[0109] In a possible implementation, the prediction value calculation unit 303 includes:

[0110] A first pixel filling unit, used to fill the pixel value of the unavailable adjacent block with the first pixel;

[0111] The first calculation unit is used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and using the corresponding reference pixels.

[0112] In a possible implementation, the prediction value calculation unit 303 further includes:

[0113] A second pixel filling unit is used to fill the pixels of the unavailable adjacent block with a fixed value if there is no available adjacent block after the availability judgment unit judges the availability of the adjacent block;

[0114] The second calculation unit is used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and using the corresponding reference pixels.

[0115] In a possible implementation, the intra-frame prediction mode selection unit 305 includes:

[0116] A distortion value generating unit, configured to obtain a distortion value based on the residuals in the different intra-frame prediction modes;

[0117] The rate-distortion cost calculation unit is used to calculate the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes according to the formula J=D+λ*R.

[0118] The selection unit is used to select the intra-frame prediction mode with the smallest rate-distortion cost as the optimal intra-frame prediction mode.

[0119] In a possible implementation manner, the first pixel filling unit includes:

[0120] A first selection unit, configured to select, from among the first pixels, pixels adjacent to the unavailable adjacent block;

[0121] The first filling unit is configured to fill the pixels of the unavailable adjacent block with the adjacent pixels.

[0122] See also Figure 4 , Figure 4 4 is a schematic diagram of the structure of a device for selecting an intra-frame prediction mode provided in an embodiment of the present application. The device for selecting the intra-frame prediction mode 4 may be the first device in the above method embodiment. The device for selecting the intra-frame prediction mode 4 may include a memory 401 and a processor 402, wherein each unit is described in detail as follows:

[0123] The memory 401 is used to store program codes.

[0124] The processor 402 is used to call the program code stored in the memory to execute the following steps:

[0125] Select a block to be encoded with a size of N*N in the image to be encoded;

[0126] Based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks, determining the availability of the adjacent blocks;

[0127] If the neighboring block is available, using the first pixel to calculate the predicted values ​​of N*N pixels of the to-be-encoded block under different intra-frame prediction modes;

[0128] Calculate the residuals of the N*N pixels of the to-be-encoded block under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the to-be-encoded block and the original values ​​of the N*N pixels of the to-be-encoded block;

[0129] Based on the residuals of N*N pixels of the to-be-encoded block under the different intra-frame prediction modes, the intra-frame prediction mode with the minimum intra-frame encoding cost is selected as the optimal intra-frame prediction mode.

[0130] In a possible implementation manner, the processor 402 determines the availability of the adjacent block based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks, including:

[0131] Based on the position of the block to be encoded in the image to be encoded, determining whether the adjacent block exists in the image to be encoded;

[0132] If the adjacent block exists in the to-be-encoded image, obtaining the encoding order of the to-be-encoded block and its adjacent blocks;

[0133] If the encoding order of the neighboring blocks indicates that the neighboring block is encoded before the block to be encoded, the neighboring block is determined to be an available neighboring block.

[0134] In a possible implementation, the processor 402 uses the first pixel to calculate the prediction values ​​of N*N pixels in the to-be-encoded block in different intra-frame prediction modes, including:

[0135] After determining the availability of the adjacent blocks, if there is an unavailable adjacent block, use the first pixel to fill the pixel value of the unavailable adjacent block;

[0136] Based on the calculation formulas of prediction values ​​in different intra-frame prediction modes, corresponding reference pixels are used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block.

[0137] In a possible implementation, the processor 402 uses the first pixel to calculate the prediction values ​​of N*N pixels in the to-be-encoded block in different intra-frame prediction modes, including:

[0138] After determining the availability of the neighboring blocks, if the available neighboring blocks do not exist, fill the pixels of the unavailable neighboring blocks with fixed values;

[0139] Based on the calculation formulas of prediction values ​​in different intra-frame prediction modes, corresponding reference pixels are used to calculate the prediction values ​​of N*N pixels in the to-be-encoded block.

[0140] In a possible implementation manner, the processor 402 selects the intra-frame prediction mode with the minimum intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals in the different intra-frame prediction modes, including:

[0141] Obtaining distortion values ​​based on the residuals in the different intra-frame prediction modes;

[0142] Calculating the rate distortion cost of intra-frame coding under the different intra-frame prediction modes according to the formula J=D+λ*R;

[0143] The first device selects the intra-frame prediction mode with the smallest rate-distortion cost as the optimal intra-frame prediction mode.

[0144] In a possible implementation manner, the processor 402 uses the first pixel to fill the pixel value of the unavailable adjacent block, including:

[0145] Selecting pixels adjacent to the unavailable neighboring block among the first pixels;

[0146] The pixels of the unavailable neighboring block are filled with the adjacent pixels.

[0147] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for selecting an intra-frame prediction mode in the above embodiment and various possible implementations thereof is implemented.

[0148] An embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a computer, a first device can execute the process executed by the first device in the above embodiment and various possible implementation methods thereof.

[0149] An embodiment of the present application provides a chip system, which includes a processor for supporting a first device to implement the functions involved in the methods in the above embodiments and various possible modes thereof.

[0150] In a possible design, the chip system further includes a memory, and the memory is used to store program instructions and data necessary for the first device. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0151] It is to be understood that those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the various embodiments of the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] Those skilled in the art will appreciate that the functions described in the various illustrative logic blocks, modules, and algorithm steps disclosed in the various embodiments of the present application can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or includes any communication media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this way, computer-readable media can generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in the present application. A computer program product may include computer-readable media.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0154] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0157] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0158] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for selecting an intra-frame prediction mode, characterized in that: include: The first device selects a block to be encoded with a size of N*N in the image to be encoded; The first device determines availability of the adjacent block based on a position of the block to be encoded in the image to be encoded and an encoding order of the block to be encoded and its adjacent blocks; If the neighboring block is available, the first device uses the first pixel to calculate the predicted values ​​of N*N pixels of the block to be encoded under different intra-frame prediction modes; wherein the first pixel is a pixel adjacent to the neighboring block; The first device calculates the residual of the N*N pixels of the to-be-encoded block under the different intra-frame prediction modes based on the predicted values ​​of the N*N pixels of the to-be-encoded block and the original values ​​of the N*N pixels of the to-be-encoded block; The first device selects the intra-frame prediction mode with the smallest intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals of N*N pixels of the to-be-encoded block under the different intra-frame prediction modes; The first pixel comprises: If the available adjacent blocks include a first adjacent block, the first adjacent block is N pixels distributed in a vertical direction and adjacent to the block to be encoded in a first horizontal direction; then, the first pixel includes N pixels adjacent to the first adjacent block in the block to be encoded; If the available adjacent blocks include a second adjacent block, the second adjacent block is N pixels arranged in a horizontal direction and adjacent to the block to be encoded in a vertical first direction; then, the first pixel includes N pixels in the block to be encoded that are adjacent to the second adjacent block; If the available adjacent blocks include a third adjacent block, the third adjacent block is N pixels arranged in a vertical direction and adjacent to the first adjacent block in a second vertical direction; then, the first pixel includes N pixels distributed in a vertical direction and adjacent to the third adjacent block in a second horizontal direction; If the available adjacent blocks include a fourth adjacent block, the fourth adjacent block is N pixels distributed in the horizontal direction and adjacent to the second adjacent block in the horizontal second direction; then, the first pixel includes N pixels distributed in the horizontal direction and adjacent to the fourth adjacent block in the vertical second direction; If the available adjacent blocks include a fifth adjacent block, the fifth adjacent block is a pixel adjacent to the second adjacent block in a first horizontal direction and distributed in a vertical direction; then the first pixel includes a pixel in the coding block that is adjacent to the fifth adjacent block; The opposite direction of the first direction is the second direction.

2. The method according to claim 1, characterized in that The first device determines the availability of the adjacent block based on the position of the block to be encoded in the image to be encoded and the encoding order of the block to be encoded and its adjacent blocks, including: The first device determines whether the adjacent block exists in the image to be encoded based on the position of the block to be encoded in the image to be encoded; If the adjacent block exists in the image to be encoded, the first device obtains the encoding order of the block to be encoded and its adjacent blocks; If the encoding order of the neighboring blocks indicates that the neighboring block is encoded before the block to be encoded, the first device determines that the neighboring block is an available neighboring block.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: If there is an unavailable adjacent block, the first device uses the first pixel to fill the pixel value of the unavailable adjacent block; The first device calculates the prediction values ​​of N*N pixels in the block to be encoded based on the calculation formula of the prediction values ​​in the different intra-frame prediction modes and uses the corresponding reference pixels; wherein the reference pixels include the first pixels and the pixels filled with the unavailable adjacent blocks.

4. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: If the available neighboring block does not exist, the first device fills the pixels of the unavailable neighboring block with a fixed value; wherein the fixed value is obtained based on the bit depth of the image to be encoded; The first device calculates the prediction values ​​of N*N pixels in the block to be encoded based on the calculation formulas of the prediction values ​​in different intra-frame prediction modes and uses the corresponding reference pixels; wherein the reference pixels are the pixels filled with the unavailable adjacent blocks.

5. The method according to any one of claims 1 to 2, characterized in that: The first device selects the intra-frame prediction mode with the smallest intra-frame coding cost as the optimal intra-frame prediction mode based on the residuals under the different intra-frame prediction modes, including: The first device obtains a distortion value based on the residuals in the different intra-frame prediction modes; The first device calculates the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes according to the formula J=D+λ*R; The first device selects the intra-frame prediction mode with the smallest rate-distortion cost as the optimal intra-frame prediction mode; Among them, J is the rate-distortion cost of intra-frame coding under the different intra-frame prediction modes; D is the distortion value, and D is used to indicate the error between the predicted value of the different intra-frame prediction modes and the original value of the pixel of the block to be encoded; R is the number of bits required to encode the image to be encoded under the different intra-frame prediction modes; λ is the Lagrange multiplier.

6. The method according to claim 3, characterized in that The first device uses the first pixel to fill the pixel value of the unavailable adjacent block, including: The first device selects, in the first pixel, pixels adjacent to the unavailable neighboring block; The first device fills pixels of the unavailable neighboring block with the adjacent pixels.

7. A device for selecting an intra-frame prediction mode, characterized in that: The intra-frame prediction mode selection device is a first device, comprising a unit for executing the intra-frame prediction mode selection method as described in any one of claims 1-6.

8. A device for selecting an intra-frame prediction mode, characterized in that: include: Memory and processor; The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images