A Fast Intra-Frame and Inter-Frame Mode Selection Method and Related Devices

Abstract

The present invention provides a fast intra-frame and inter-frame mode selection method and related devices, which are applied to an H.264 encoder. By reading multiple frames of images to be encoded in the video sequence to be processed in the H.264 encoder; dividing each frame of the image to be encoded respectively to obtain a plurality of macroblocks; judging the type of each macroblock by using the coding information of the encoded macroblocks; when the current macroblock is an I-block, attempting to encode by using the partitioning mode used by the macroblocks around the current macroblock, and selecting the mode with the smallest coding cost as the intra-frame coding mode of the current macroblock; when the current macroblock is a P-block, selecting the optimal partitioning mode from the partitioning modes of the encoded macroblocks around the current macroblock to attempt encoding, and taking the partitioning mode with the smallest coding cost in the optimal partitioning mode as the inter-frame coding mode of the current macroblock; realizing that the encoder reduces the number of attempts for encoding during mode selection, achieving the purpose of fast encoding and reducing time overhead.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding and decoding, and in particular to a fast intra-frame and inter-frame mode selection method and related equipment. Background Art

[0002] Digital video consists of continuous image frames, each of which contains a large amount of pixel data. The amount of uncompressed raw video data is very large and difficult to transmit efficiently in limited bandwidth. Therefore, video encoding technology was introduced to reduce the size of video files and ensure more efficient use of bandwidth during transmission. Decoding is the reverse process of encoding, which means restoring the compressed data to a playable video sequence when playing the video.

[0003] In the H.264 standard, encoding time, video quality and compression efficiency are called the "impossible triangle". If the compression efficiency is higher under the condition of a certain encoding time, the video quality is lower, and vice versa. Therefore, in order to take into account the encoding time, video quality and compression efficiency, video frames are divided into three types: I frames, P frames and B frames. I frames are independently encoded frames and can be used as the starting point for decoding, but because they contain information about the entire frame of the image, the encoding complexity is high and the data volume is large; P frames rely on previous reference frames through predictive encoding, and have high encoding efficiency, but they need to rely on reference frames for decoding; B frames use more reference information through bidirectional predictive encoding, so they have the highest epicenter encoding efficiency among all types, but when decoding, they need to decode the previous and next reference frames at the same time, and the decoding complexity is high, which is not suitable for real-time video applications that are sensitive to delay. Therefore, in scenarios with high real-time requirements, video frames will only be divided into two types: I frames and P frames. For I frames, since they are independently encoded frames and do not use other frames as references, the encoder will only use intra-frame compression coding, try all division modes and prediction modes, and select the mode with the lowest cost; for P frames, in order to improve compression efficiency, the encoder will use inter-frame compression coding. In addition, after finding the optimal inter-frame mode, the encoder will also execute the intra-frame optimal mode search algorithm, and compare the optimal mode obtained by the intra-frame and inter-frame algorithms, and select the mode with the lowest cost. This is because in extreme cases such as violent motion and complex textures, inter-frame coding is not necessarily more efficient than intra-frame coding. Therefore, choosing intra-frame coding in these specific scenarios can achieve better coding efficiency. In this process, since a complete encoding is required every time the cost is calculated, especially for P frames, it is necessary to try all the intra-frame and inter-frame coding modes, which requires a lot of time overhead.

[0004] In response to this situation, some researchers have proposed using machine learning methods to pre - determine whether to use the intra - frame coding mode or the inter - frame coding mode for some specific situations. This method saves 19 - 25% of the coding time without significantly degrading the video quality. In addition, some researchers have proposed a method that combines a real - motion tracking algorithm to select the intra - frame and inter - frame coding modes. The simulation results show that this method improves the quality of the image sequence. In addition, for the optimization of the time overhead of intra - frame coding, some researchers have proposed a progressive coarse - mode search based on the Hadamard transform and early termination of unnecessary CU partitioning based on the H.265 coding standard. The simulation results show that this method saves 60% of the coding time at the cost of a 1.0% increase in bitrate. However, all of the above methods need to try all the intra - frame and inter - frame coding modes, resulting in a relatively large overall time overhead. Summary of the Invention

[0005] The present invention provides a fast intra - frame and inter - frame mode selection method and related devices, aiming to reduce the number of encoding attempts during mode selection by the encoder, achieving the purpose of fast encoding and reducing time overhead.

[0006] To achieve the above purpose, the present invention provides a fast intra - frame and inter - frame mode selection method, which is applied to an H.264 encoder and includes:

[0007] Step 1: Read multiple frames of images to be encoded in the video sequence to be processed in the H.264 encoder;

[0008] Step 2: For each frame of the image to be encoded, divide the image to be encoded to obtain multiple macro - blocks;

[0009] Step 3: For each macro - block, use the coding information of the already - encoded macro - blocks to determine the type of the current macro - block;

[0010] If the current macro - block is an I - block, determine the partitioning modes used by the already - encoded macro - blocks on the left, above, upper - left, and upper - right of the current macro - block, and use the determined partitioning mode to perform trial encoding on the current macro - block, calculate the encoding cost, search for the remaining modes, and select the mode with the minimum encoding cost as the intra - frame coding mode of the current macro - block;

[0011] If the current macro - block is a P - block, select several partitioning modes with the highest selection probability from the partitioning modes of the already - encoded macro - blocks around the current macro - block as the optimal partitioning modes for trial encoding, calculate the encoding costs of these several partitioning modes in the optimal partitioning modes, and use the partitioning mode with the minimum encoding cost in the optimal partitioning modes as the inter - frame coding mode of the current macro - block.

[0012] Furthermore, Step 1 includes:

[0013] Input the video sequence to be processed into the encoder;

[0014] The encoder locates the starting position of each image to be encoded in the video sequence to be processed, and obtains the resolution and encoding format information of the video sequence to be processed;

[0015] The encoder calculates the data size of each image to be encoded according to the starting position, resolution and encoding format information of each image to be encoded;

[0016] Read the information of each image to be encoded continuously in the video sequence to be processed to obtain multiple images to be encoded.

[0017] Furthermore, step 2 includes:

[0018] For each image to be encoded, divide the image to be encoded in units of 16 pixels × 16 pixels to obtain a plurality of macroblocks.

[0019] Furthermore, use the encoding information of the encoded macroblocks to determine the type of the current macroblock, including:

[0020] When the macroblocks on the left, above, top-left or top-right of the current macroblock are all unencoded macroblocks, return to step 2;

[0021] When there are encoded macroblocks among the macroblocks on the left, above, top-left or top-right of the current macroblock, then determine the type of the current macroblock according to the encoding information of the encoded macroblocks in different cases.

[0022] Furthermore, when there are encoded macroblocks among the macroblocks on the left, above, top-left or top-right of the current macroblock, then determine the type of the current macroblock according to the encoding information of the encoded macroblocks in different cases, including:

[0023] If the macroblocks on the left, above, top-left or top-right of the current macroblock are all I blocks, calculate the average cost of all I blocks. When the average cost is less than the first cost threshold, set the current macroblock as an I block, otherwise, return to step 2;

[0024] If the macroblocks on the left, above, top-left or top-right of the current macroblock are all P blocks, calculate the average cost of all P blocks. When the average cost is less than the second cost threshold, set the current macroblock as a P block, otherwise, return to step 2;

[0025] If there are both I blocks and P blocks among the macroblocks on the left, above, top-left or top-right of the current macroblock, then return to step 2.

[0026] Furthermore, when the current macroblock is an I-block, determine the partitioning modes used by the macroblocks that have been encoded to the left, above, top-left, and top-right of the current macroblock, and calculate the encoding cost after attempting to encode the current macroblock using the determined partitioning modes. Search for the remaining modes and select the mode with the minimum encoding cost as the intra-frame encoding mode of the current macroblock, including:

[0027] When the current macroblock is an I-block, select a 16×16 partitioning mode for the current macroblock, determine the partitioning mode with the minimum encoding cost for attempted encoding;

[0028] When the encoding cost is less than the third cost threshold, perform prediction in three directions adjacent to the intra-frame direction represented by the partitioning mode with the minimum encoding cost in the 4×4 prediction mode to obtain the optimal direction and encoding cost, and select the mode with the minimum encoding cost among the 16×16 partitioning mode and the 4×4 prediction mode as the intra-frame encoding mode;

[0029] When the encoding cost is greater than the third cost threshold, attempt to encode all partitioning modes and prediction modes and calculate the encoding cost, and use the mode with the minimum encoding cost as the intra-frame encoding mode.

[0030] Furthermore, when the current macroblock is a P-block, select several partitioning modes with the highest selection probabilities from the partitioning modes of the macroblocks that have been encoded around the current macroblock as the optimal partitioning modes for attempted encoding, calculate the encoding costs of the several partitioning modes in the optimal partitioning modes, and use the partitioning mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock, including:

[0031] When the current macroblock is a P-block and all the surrounding macroblocks use the same macroblock partitioning mode:

[0032] If the same macroblock partitioning mode is 16×16, then the current macroblock selects three partitioning modes, namely 16×16, 16×8, and 8×16, as the optimal partitioning modes for attempted encoding, calculate the encoding costs of each partitioning mode in the optimal partitioning modes, and use the mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock;

[0033] If the same macroblock partitioning mode is 4×4, then the current macroblock selects six partitioning modes, namely 4×4, 4×8, 8×4, 8×8, 8×16, and 16×8, as the optimal partitioning modes for attempted encoding, calculate the encoding costs of each partitioning mode in the optimal partitioning modes, and use the mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock;

[0034] When the current macroblock is a P block and all surrounding macroblocks use different macroblock partitioning modes, the current macroblock selects all partitioning modes to attempt encoding and calculates the encoding cost. It calculates the encoding cost for each partitioning mode and selects the mode with the minimum encoding cost as the inter-frame encoding mode of the current macroblock.

[0035] Furthermore, the encoding information includes:

[0036] Encoding method, partitioning mode, prediction mode, encoding cost.

[0037] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the fast intra-frame and inter-frame mode selection method.

[0038] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the fast intra-frame and inter-frame mode selection method.

[0039] The above solution of the present invention has the following beneficial effects:

[0040] The present invention is applied to an H.264 encoder. By reading multiple frames of images to be encoded in the video sequence to be processed in the H.264 encoder; partitioning each frame of the image to be encoded respectively to obtain multiple macroblocks; for each macroblock respectively, using the encoding information of the encoded macroblocks to determine the type of the current macroblock; if the current macroblock is an I block, determine the partitioning modes used by the macroblocks that have been encoded on the left, above, top-left, and top-right of the current macroblock, and use the determined partitioning mode to attempt encoding the current macroblock and calculate the encoding cost, search for the remaining modes, and select the mode with the minimum encoding cost as the intra-frame encoding mode of the current macroblock; if the current macroblock is a P block, select several partitioning modes with the highest selection probability from the partitioning modes of the encoded macroblocks around the current macroblock as the optimal partitioning modes to attempt encoding, calculate the encoding costs of these several partitioning modes in the optimal partitioning modes, and select the partitioning mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock; compared with the prior art, the present invention utilizes the correlation between different encoding modes and the correlation of partitioning mode selection between adjacent macroblocks, realizes that the encoder reduces the number of attempts for encoding during mode selection, and achieves the purpose of fast encoding and reducing time overhead.

[0041] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation part. Description of the Drawings

[0042] Figure 1 It is a schematic flowchart of an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the partitioning mode in the embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the structure of the terminal device in the embodiment of the present invention. Detailed implementation manners

[0045] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0047] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0048] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] The present invention provides a fast intra-frame and inter-frame mode selection method and related devices for solving existing problems.

[0050] As Figure 1 shown, an embodiment of the present invention provides a fast intra-frame and inter-frame mode selection method, which is applied to an H.264 encoder and includes:

[0051] Step 1, read multiple frames of images to be encoded in the video sequence to be processed in the H.264 encoder;

[0052] Step 2: For each frame of the image to be encoded, divide the image to be encoded to obtain a plurality of macroblocks;

[0053] Step 3: For each macroblock, use the coding information of the encoded macroblocks to determine the type of the current macroblock;

[0054] When the current macroblock is an I-block, determine the partitioning modes used by the encoded macroblocks on the left, above, top-left, and top-right of the current macroblock, and use the determined partitioning modes to perform trial encoding on the current macroblock, calculate the encoding cost, search for the remaining modes, and select the mode with the minimum encoding cost as the intra-frame encoding mode of the current macroblock;

[0055] When the current macroblock is a P-block, select several partitioning modes with the highest selection probability from the partitioning modes of the encoded macroblocks around the current macroblock as the optimal partitioning modes for trial encoding, calculate the encoding costs of the several partitioning modes in the optimal partitioning modes, and use the partitioning mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock.

[0056] In the embodiments of the present invention, the video sequence to be processed is a video sequence in YUV format, because the video sequence in YUV format can be directly used by the encoder, or it can be other common video formats, such as MP4, MOV, etc. Video sequences can also be used as input, but they will still be transcoded into video sequences in YUV format when being read by the encoder.

[0057] Specifically, Step 1 includes:

[0058] Input the video sequence to be processed into the encoder;

[0059] The encoder locates the starting position of each frame of the image to be encoded in the video sequence to be processed, and obtains the resolution and coding format information of the video sequence to be processed;

[0060] The encoder calculates the data size of each frame of the image to be encoded according to the starting position, resolution, and coding format information of each frame of the image to be encoded;

[0061] Continuously read the information of each frame of the image to be encoded in the video sequence to be processed to obtain multiple frames of images to be encoded.

[0062] Specifically, Step 2 includes:

[0063] For each frame of the image to be encoded, divide the image to be encoded in units of 16 pixels × 16 pixels to obtain a plurality of macroblocks.

[0064] Most preferably, using the coding information of the encoded macroblocks to determine the type of the current macroblock includes:

[0065] When the macroblocks on the left, above, top-left, or top-right of the current macroblock are all uncoded macroblocks, return to step 2;

[0066] When there are coded macroblocks among the macroblocks on the left, above, top-left, or top-right of the current macroblock, determine the type of the current macroblock according to the coding information of the coded macroblocks in different cases.

[0067] Most preferably, when there are coded macroblocks among the macroblocks on the left, above, top-left, or top-right of the current macroblock, determine the type of the current macroblock according to the coding information of the coded macroblocks in different cases, including:

[0068] If the macroblocks on the left, above, top-left, or top-right of the current macroblock are all I blocks, calculate the average cost of all I blocks. When the average cost is less than the first cost threshold, set the current macroblock as an I block; otherwise, return to step 2;

[0069] If the macroblocks on the left, above, top-left, or top-right of the current macroblock are all P blocks, calculate the average cost of all P blocks. When the average cost is less than the second cost threshold, set the current macroblock as a P block; otherwise, return to step 2;

[0070] If there are both I blocks and P blocks among the macroblocks on the left, above, top-left, or top-right of the current macroblock, return to step 2.

[0071] It should be noted that during the existing standard H.264 encoding, for the macroblocks in the P Slice, the encoder will try all the intra-frame and inter-frame modes and select the mode with the minimum cost; while in the embodiment of the present invention, the one-time discrimination method is used, and the H.264 encoder will determine whether to use intra-frame encoding or inter-frame encoding before encoding and only perform intra-frame or inter-frame encoding in the subsequent encoding.

[0072] In the embodiment of the present invention, when there are coded macroblocks among the macroblocks on the left, above, top-left, or top-right of the current macroblock, determining the type of the current macroblock according to the coding information of the coded macroblocks in different cases further includes:

[0073] If the current macroblock does not meet the conditions for using the fast algorithm, traverse all the coding combinations one by one to ensure that a better mode can be selected.

[0074] Specifically, traversing all the coding combinations one by one to ensure that a better mode can be selected includes the following steps:

[0075] For the inter-frame mode, it is necessary to recursively traverse all the partitioning modes of all macroblocks. There are seven partitioning modes in total: 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, 4×4, as Figure 2As shown; for each partitioning mode, motion estimation and motion compensation need to be performed on it, and its cost is calculated. After traversing all macroblock partitioning modes, the mode with the minimum coding cost is selected as the inter-frame coding mode;

[0076] For the intra-frame mode, all macroblock partitioning modes and prediction modes need to be traversed; there are two macroblock partitioning modes, 16×16 and 4×4. Among them, there are four prediction modes for 16×16, namely V, H, P, and DC, and nine prediction modes for 4×4, namely V, H, DC, DDL, DDR, VR, HD, VL, and HU; for each macroblock, a total of 13 traversals are required, and its cost is calculated. After completing all traversals, the mode with the minimum coding cost is selected as the intra-frame coding mode;

[0077] Determine whether the current segment is an inter-frame segment (P segment). If so, select the mode with the smaller coding cost from the inter-frame coding mode and the intra-frame coding mode as the final result; if not, directly select the intra-frame coding mode as the final result.

[0078] Most preferably, if the current macroblock is an I block, determine the partitioning modes used by the macroblocks that have been encoded on the left, above, top-left, and top-right of the current macroblock, and calculate the coding cost after attempting to encode the current macroblock using the determined partitioning mode. Search for the remaining modes, and finally select the mode with the minimum coding cost as the intra-frame coding mode of the current macroblock; the specific steps include:

[0079] If the current macroblock is an I block, select the 16×16 partitioning mode for the current macroblock, determine the partitioning mode with the minimum coding cost for attempt encoding and calculate the coding cost, which is used to narrow the traversal range of the subsequent partitioning modes;

[0080] When the coding cost is less than the third cost threshold, perform prediction in three directions adjacent to the intra-frame direction represented by the partitioning mode with the minimum coding cost among the 4×4 prediction modes to obtain the optimal direction and coding cost, and select the mode with the minimum coding cost from the 16×16 partitioning mode and the 4×4 prediction mode as the intra-frame coding mode;

[0081] When the coding cost is greater than the third cost threshold, perform attempt encoding on all partitioning modes and prediction modes and calculate the coding cost, and select the mode with the minimum coding cost as the intra-frame coding mode.

[0082] It should be noted that when encoding in the existing standard H.264, all macroblock partitioning modes and intra-frame coding modes will be traversed to obtain the mode with the minimum cost among them as the final mode; while in the embodiment of the present invention, a binary search method is used to find the optimal intra-frame mode, and the H.264 encoder will use a variable step size search method to find the optimal mode, so as to avoid attempting some modes with low possibilities and achieve the goal of reducing time overhead.

[0083] Most preferably, when the current macroblock is a P block, several partition modes with the highest selection probabilities are selected from the partition modes of the already encoded macroblocks around the current macroblock as the optimal partition modes for trial encoding, and the encoding costs of several partition modes in the optimal partition modes are calculated. The partition mode with the minimum encoding cost in the optimal partition modes is used as the inter-frame encoding mode of the current macroblock, including:

[0084] When the current macroblock is a P block and all the surrounding macroblocks use the same macroblock partition mode:

[0085] If the same macroblock partition mode is 16×16, the current macroblock selects three partition modes of 16×16, 16×8, and 8×16 as the optimal partition modes for trial encoding, and the encoding costs of each partition mode in the optimal partition modes are calculated. The mode with the minimum encoding cost in the optimal partition modes is used as the inter-frame encoding mode of the current macroblock;

[0086] If the same macroblock partition mode is 4×4, the current macroblock selects six partition modes of 4×4, 4×8, 8×4, 8×8, 8×16, and 16×8 as the optimal partition modes for trial encoding, and the encoding costs of each partition mode in the optimal partition modes are calculated. The mode with the minimum encoding cost in the optimal partition modes is used as the inter-frame encoding mode of the current macroblock;

[0087] When the current macroblock is a P block and all the surrounding macroblocks use different macroblock partition modes, the current macroblock selects all partition modes for trial encoding and calculates the encoding costs, and calculates the encoding costs of each partition mode. The mode with the minimum encoding cost is used as the inter-frame encoding mode of the current macroblock.

[0088] It should be noted that during the existing H.264 encoding of the standard, all partition modes will be tried once to find the partition mode with the minimum cost; while in the embodiment of the present invention, the encoding information of the surrounding macroblocks is used to assist in judgment to find the optimal inter-frame prediction partition mode. The H.264 encoder will combine the partition situation of the surrounding macroblocks to determine whether the current macroblock can directly determine the partition mode, so that in some specific cases, the encoder does not need to traverse all partition modes.

[0089] Since the distortion and compression efficiency in the encoding process are relative concepts, the higher the compression efficiency, the higher the distortion, and vice versa; in the embodiment of the present invention, in order to determine that a certain mode can obtain a higher compression efficiency and does not cause excessive distortion, the concept of encoding cost is proposed. The encoding cost comprehensively considers the trade-off relationship between compression efficiency and distortion, and its calculation expression is:

[0090]

[0091] Among them, represents the distortion degree of the reconstructed image compared with the original image, usually expressed by the sum of absolute differences (SAD) or the sum of squared differences (SSD); represents the output bitrate, which is used to reflect the compression efficiency; represents the Lagrange constant, which is determined by the quantization parameter.

[0092] In the embodiment of the present invention, after step 3, the encoding information of the current macroblock is saved in the slicer handle of the H.264 encoder for use in subsequent macroblock encoding.

[0093] The above steps mainly save the encoding information required in the previous steps, including the encoding mode, partition mode, prediction mode, and encoding cost. These information will not be saved in the H.264 encoder, so it is necessary to save them for subsequent use.

[0094] The embodiment of the present invention conducts a comparative experiment on the provided method for implementing fast intra-frame and inter-frame mode selection with the original encoder. The experimental results are shown in Table 1 below:

[0095] Table 1 Comparative experimental results of the method for implementing fast intra-frame and inter-frame mode selection and the original encoder

[0096]

[0097] As can be seen from Table 1 above, when the peak signal-to-noise ratio (PSNR) and the bitrate change little, (FPS, Frames Per Second): refers to the number of frames encoded per second. The larger the value, the faster the encoding speed. Compared with the two methods, the method provided by the embodiment of the present invention improves the encoding speed by about 8%.

[0098] The embodiments of the present invention are applied to an H.264 encoder. By reading multiple frames of images to be encoded in the video sequence to be processed within the H.264 encoder; dividing each frame of the image to be encoded to obtain multiple macroblocks; for each macroblock, using the encoding information of the encoded macroblocks to determine the type of the current macroblock; if the current macroblock is an I-block, determining the partitioning modes used by the macroblocks that have been encoded to the left, above, top-left, and top-right of the current macroblock, and using the determined partitioning modes to perform trial encoding on the current macroblock and then calculating the encoding cost, searching for the remaining modes, and selecting the mode with the minimum encoding cost as the intra-frame encoding mode of the current macroblock; if the current macroblock is a P-block, selecting several partitioning modes with the highest selection probabilities from the partitioning modes of the encoded macroblocks around the current macroblock as the optimal partitioning modes for trial encoding, and calculating the encoding costs of these partitioning modes in the optimal partitioning modes, and taking the partitioning mode with the minimum encoding cost in the optimal partitioning modes as the inter-frame encoding mode of the current macroblock; compared with the prior art, the embodiments of the present invention utilize the correlation between different encoding modes and the correlation of partitioning mode selection between adjacent macroblocks, achieving the reduction of the number of trial encodings during mode selection by the encoder, and achieving the purpose of fast encoding and reducing time overhead.

[0099] The embodiments of the present invention also provide a terminal device, as Figure 3 shown. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, it implements the above-mentioned method for fast intra-frame and inter-frame mode selection.

[0100] The terminal device D10 may be a computing device such as a desktop computer, a notebook, a palm computer, a server, a server cluster, and a cloud server. The terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art can understand that Figure 3 this is only an example of the terminal device D10, and does not constitute a limitation on the terminal device D10. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0101] The so-called processor D100 may be a central processing unit (CPU, Central Processing Unit), and this processor D100 may also be other general-purpose processors, digital signal processors (DSPs, Digital Signal Processors), application-specific integrated circuits (ASICs, Application Specific Integrated Circuits), field-programmable gate arrays (FPGAs, Field-Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0102] The memory D101 may be an internal storage unit of the terminal device D10 in some embodiments, such as the hard disk or memory of the terminal device D10. The memory D101 may also be an external storage device of the terminal device D10 in other embodiments, such as a plug-in hard disk, a smart media card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc., equipped on the terminal device D10. Further, the memory D101 may also include both the internal storage unit and the external storage device of the terminal device D10. The memory D101 is used to store an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or will be output.

[0103] It should be noted that, for the content such as information interaction and execution process between the above-mentioned device / unit, since it is based on the same concept as the method embodiment of the present application, for its specific functions and the technical effects brought, reference may be specifically made to the method embodiment part, and details are not described herein again.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments and will not be elaborated here.

[0105] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the fast intra-frame and inter-frame mode selection method.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the construction device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc.

[0107] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A fast intra-frame and inter-frame mode selection method, characterized in that: Applicable to H.264 encoders, including: Step 1, reading multiple frames of images to be encoded from the video sequence to be processed in the H.264 encoder; Step 2, for each frame of the image to be encoded, dividing the image to be encoded to obtain multiple macroblocks; Step 3, for each macroblock, using the encoding information of the encoded macroblock, determine the type of the current macroblock; If the current macroblock is an I block, determine the division mode used by the macroblocks that have been encoded on the left, above, upper left, and upper right of the current macroblock, and use the determined division mode to try to encode the current macroblock and then calculate the encoding cost, search for the remaining modes, and select the mode with the smallest encoding cost as the intra-frame encoding mode of the current macroblock, including: If the current macroblock is an I block, a 16×16 partition mode is selected for the current macroblock, and the partition mode with the lowest encoding cost is determined for trial encoding; When the coding cost is less than the third cost threshold, predict three directions adjacent to the intra-frame direction represented by the partition mode with the smallest coding cost in the 4×4 prediction mode to obtain the optimal direction and coding cost, and select the mode with the smallest coding cost as the intra-frame coding mode in the 16×16 partition mode and the 4×4 prediction mode; When the coding cost is greater than the third cost threshold, all partition modes and prediction modes are tried to be coded and the coding costs are calculated, and the mode with the minimum coding cost is used as the intra-frame coding mode; If the current macroblock is a P block, the division modes with the highest probability are selected from the division modes of the encoded macroblocks around the current macroblock as the optimal division modes for attempted encoding, and the encoding costs of several division modes in the optimal division modes are calculated, and the division mode with the smallest encoding cost in the optimal division modes is used as the inter-frame coding mode of the current macroblock.

2. The fast intra-frame and inter-frame mode selection method according to claim 1, characterized in that: The step 1 comprises: inputting a video sequence to be processed into an encoder; The encoder locates the starting position of each frame of the to-be-encoded image in the to-be-processed video sequence, and obtains the resolution and encoding format information of the to-be-processed video sequence; The encoder calculates the data size of each frame of the image to be encoded according to the starting position, resolution and encoding format information of each frame of the image to be encoded; The information of each frame of the image to be encoded is continuously read in the video sequence to be processed to obtain multiple frames of the image to be encoded.

3. The fast intra-frame and inter-frame mode selection method according to claim 2, characterized in that: The step 2 comprises: For each frame of the image to be encoded, the image to be encoded is divided into units of 16 pixels×16 pixels to obtain a plurality of macroblocks.

4. The fast intra-frame and inter-frame mode selection method according to claim 3, characterized in that: Using the encoding information of the encoded macroblock, determine the type of the current macroblock, including: When the macroblocks on the left, above, left-upper or right-upper of the current macroblock are all uncoded macroblocks, return to step 2; When there is an encoded macroblock on the left, above, upper left or upper right of the current macroblock, the type of the current macroblock is determined based on the encoding information of the encoded macroblock.

5. The fast intra-frame and inter-frame mode selection method according to claim 4, characterized in that: When there is an encoded macroblock on the left, above, upper left or upper right of the current macroblock, the type of the current macroblock is determined according to the encoding information of the encoded macroblock, including: If the macroblocks to the left, above, left-upper or right-upper of the current macroblock are all I blocks, calculate the average cost of all I blocks, and when the average cost is less than the first cost threshold, set the current macroblock as an I block, otherwise, return to step 2; If the macroblocks to the left, above, left-upper or right-upper of the current macroblock are all P blocks, calculate the average cost of all P blocks, and when the average cost is less than the second cost threshold, set the current macroblock as a P block, otherwise, return to step 2; If the macroblocks to the left, above, left above or right above the current macroblock have both I blocks and P blocks, return to step 2.

6. The fast intra-frame and inter-frame mode selection method according to claim 5, characterized in that: If the current macroblock is a P block, several division modes with the highest probability are selected from the division modes of the coded macroblocks around the current macroblock as the optimal division modes for trial coding, and the coding costs of several division modes in the optimal division modes are calculated, and the division mode with the lowest coding cost in the optimal division modes is used as the inter-frame coding mode of the current macroblock, including: When the current macroblock is a P block and the surrounding macroblocks all use the same macroblock partitioning mode: If the same macroblock partition mode is 16×16, the current macroblock selects 16×16, 16×8, and 8×16 partition modes as the optimal partition modes for trial encoding, and calculates the encoding cost of each partition mode in the optimal partition mode, and uses the mode with the smallest encoding cost in the optimal partition mode as the inter-frame coding mode of the current macroblock; If the same macroblock partition mode is 4×4, the current macroblock selects six partition modes of 4×4, 4×8, 8×4, 8×8, 8×16, and 16×8 as the optimal partition modes for trial encoding, and calculates the encoding cost of each partition mode in the optimal partition mode, and uses the mode with the smallest encoding cost in the optimal partition mode as the inter-frame coding mode of the current macroblock; When the current macroblock is a P block and the surrounding macroblocks use different macroblock partitioning modes, the current macroblock selects all partitioning modes to try encoding and calculate the encoding cost, and calculates the encoding cost of each partitioning mode, and takes the mode with the smallest encoding cost as the inter-frame coding mode of the current macroblock.

7. The fast intra-frame and inter-frame mode selection method according to claim 6, characterized in that: The coding information includes: Coding method, partitioning mode, prediction mode, and coding cost.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the fast intra-frame inter-frame mode selection method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the fast intra-frame and inter-frame mode selection method according to any one of claims 1 to 7 is implemented.

Images