Video Processing Method, Apparatus, Storage Medium, and Electronic Device

Abstract

The present disclosure provides a video processing method, apparatus, storage medium, and electronic device, which relate to the field of computer technologies. In the embodiments of the present disclosure, any frame of video image in the video to be processed can be first encoded and divided to determine a processed image block with a first number of pixels. According to each pixel included in the processed image block, the residual of each pixel in the processed image block is calculated. Based on the residuals of each pixel in the processed image block, a target image block whose residual information meets a preset residual condition is selected. At least one prediction mode is used to calculate the loss of the target image block, and the prediction mode whose loss value meets a preset loss condition is selected as the target mode, and the target mode is used to encode the video to be processed. In this way, compared with the existing encoding methods, the present disclosure can, to a certain extent, ensure the video encoding quality while reducing the computational amount for determining the target image block, thereby reducing the time required for video encoding and improving the efficiency of video encoding.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to a video processing method, apparatus, storage medium, and electronic device. Background Art

[0002] This section aims to provide background or context for the embodiments of the present disclosure stated in the claims, and the description herein is not admitted to be prior art merely because it is included in this section.

[0003] With the development of video coding technology, the wide application of various video services has brought great convenience to people's production and life. Video coding generally needs to use different prediction modes for coding, and due to different video contents, the computational amount of video coding using different prediction modes is different.

[0004] In related technologies, people often directly use the x265 coding method to encode videos, resulting in a large computational amount and low coding efficiency. Summary of the Invention

[0005] To overcome the problems in related technologies, the present disclosure provides a video processing method, apparatus, storage medium, and electronic device.

[0006] According to a first aspect of the present disclosure, a video processing method is provided, and the method includes:

[0007] Performing encoding division on any frame of video image in a video to be processed to determine a processed image block with a first number of pixels;

[0008] Calculating the residuals of each pixel in the processed image block according to each pixel included in the processed image block;

[0009] Based on the residuals of each pixel in the processed image block, selecting a target image block whose residual information meets a preset residual condition; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block;

[0010] Calculating the loss of the target image block using at least one prediction mode, and selecting a prediction mode whose loss value meets a preset loss condition as a target mode; the target mode is used to encode the video to be processed.

[0011] Optionally, the calculating the residuals of each pixel in the processed image block according to each pixel included in the processed image block includes:

[0012] For each pixel included in the processed image block, select P pixels as target pixels; P is less than or equal to the number of the first pixels.

[0013] Calculate the target pixels using a preset residual calculation mode, and use the calculation result as the residual of each pixel in the processed image block.

[0014] Optionally, calculating the residual of each pixel in the processed image block according to each pixel included in the processed image block includes:

[0015] Determine the corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processed image block in the video image;

[0016] Calculate the residual of each pixel in the processed image block according to the pixel values of each pixel in the processed image block and the pixel values of each pixel in the corresponding image.

[0017] Optionally, selecting a target image block whose residual information meets a preset residual condition based on the residuals of each pixel in the processed image block includes:

[0018] Successively perform the following operations to determine the target image block:

[0019] Select the number of the second pixels from the target interval; the maximum value in the target interval is the number of the first pixels, and the minimum value is the number of the third pixels;

[0020] Determine an alternative image block based on each pixel included in the processed image block; the number of pixels included in the alternative image block is the number of the second pixels;

[0021] Calculate the residual information of the alternative image block according to the residuals of each pixel in the alternative image block;

[0022] Judge whether the residual information is less than a preset residual threshold;

[0023] When the residual information is less than the preset residual threshold, then use the alternative image block as the target image block and stop performing the operation of selecting the number of the second pixels.

[0024] Optionally, calculating the residual information of the alternative image block according to the residuals of each pixel in the alternative image block includes:

[0025] Divide the alternative image block according to the number of the third pixels to obtain a plurality of sub-image blocks;

[0026] Calculate the residual value of the sub-image block based on the residuals of each pixel in the sub-image block;

[0027] Determine the residual information of the candidate image block according to the residual values of the sub-image blocks.

[0028] Optionally, the method further includes:

[0029] When the residual information is not less than the preset residual threshold, re-execute the operation of selecting the second number of pixels until the residual information is less than the preset residual threshold.

[0030] Optionally, the loss value is one of the RD loss value and the SATD loss value.

[0031] Optionally, the process of encoding and partitioning any frame of video image in the video to be processed to determine the processed image block with the first number of pixels includes:

[0032] Extract the arbitrary frame of video image from the video to be processed;

[0033] Perform recursive partitioning on the video image according to the preset encoding settings to obtain a plurality of image blocks;

[0034] Select the image block with the number of pixels being the first number of pixels as the processed image block.

[0035] According to the second aspect of the present disclosure, a video processing apparatus is provided, and the apparatus includes:

[0036] A first determination module, configured to perform encoding and partitioning on any frame of video image in the video to be processed, and determine a processed image block with the first number of pixels;

[0037] A calculation module, configured to calculate the residuals of each pixel in the processed image block according to each pixel included in the processed image block;

[0038] A first selection module, configured to select a target image block whose residual information meets a preset residual condition based on the residuals of each pixel in the processed image block; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block;

[0039] A second selection module, configured to calculate the loss of the target image block by using at least one prediction mode, and select the prediction mode whose loss value meets a preset loss condition as the target mode; the target mode is used to encode the video to be processed.

[0040] Optionally, the calculation module is further specifically configured to:

[0041] For each pixel included in the processed image block, select P pixels as target pixels; P is less than or equal to the first number of pixels;

[0042] Calculate the target pixel using a preset residual calculation mode, and use the calculation result as the residual of each pixel in the processed image block.

[0043] Optionally, the calculation module is further specifically configured to:

[0044] Determine the corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processed image block in the video image;

[0045] Calculate the residual of each pixel in the processed image block according to the pixel values of each pixel in the processed image block and the pixel values of each pixel in the corresponding image.

[0046] Optionally, the first selection module is further specifically configured to:

[0047] Successively perform the following operations to determine the target image block:

[0048] Select the second number of pixels from the target interval; the maximum value in the target interval is the first number of pixels, and the minimum value is the third number of pixels;

[0049] Based on each pixel included in the processed image block, determine an alternative image block; the number of pixels included in the alternative image block is the second number of pixels;

[0050] Calculate the residual information of the alternative image block according to the residuals of each pixel in the alternative image block;

[0051] Judge whether the residual information is less than a preset residual threshold;

[0052] When the residual information is less than the preset residual threshold, then use the alternative image block as the target image block and stop performing the operation of selecting the second number of pixels.

[0053] Optionally, the first selection module is further specifically configured to:

[0054] Divide the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks;

[0055] Calculate the residual value of the sub-image block based on the residuals of each pixel in the sub-image block;

[0056] Determine the residual information of the alternative image block according to the residual values of the sub-image blocks.

[0057] Optionally, the device further includes:

[0058] An execution module, configured to, when the residual information is not less than the preset residual threshold, re-execute the operation of selecting the second number of pixels until the residual information is less than the preset residual threshold.

[0059] Optionally, the loss value is one of an RD loss value and an SATD loss value.

[0060] Optionally, the first determination module is further specifically configured to;

[0061] Extract any frame of video image from the video to be processed;

[0062] Perform recursive partitioning on the video image according to a preset coding setting to obtain a plurality of image blocks;

[0063] Select an image block containing the first number of pixels as the processed image block.

[0064] According to one aspect of the present disclosure, there is provided a storage medium having stored thereon a computer program, which, when executed by a processor, implements the above video processing method.

[0065] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0066] A processor; and

[0067] A memory for storing executable instructions of the processor;

[0068] Wherein, the processor is configured to execute the video processing method according to any one of the above by executing the executable instructions.

[0069] In summary, the video processing method provided by the embodiments of the present disclosure can first perform coding and partitioning on any frame of video image in the video to be processed to determine a processed image block with the first number of pixels, then calculate the residuals of each pixel in the processed image block according to each pixel included in the processed image block, and based on the residuals of each pixel in the processed image block, select a target image block whose residual information meets the preset residual condition, the number of pixels included in the target image block is not greater than the first number of pixels, and the residual information is determined according to the residuals of each pixel in the target image block. Finally, at least one prediction mode is used to calculate the loss of the target image block, and the prediction mode whose loss value meets the preset loss condition is selected as the target mode, and the target mode is used to encode the video to be processed. In this way, compared with the existing coding method, the present disclosure can ensure the video coding quality to a certain extent while reducing the calculation amount for determining the target image block, thereby reducing the time required for video coding and improving the efficiency of video coding. Description of the Drawings

[0070] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the drawings, several embodiments of the present disclosure are shown by way of illustration and not limitation, wherein:

[0071] Figure 1 Schematically showing a flowchart of steps of a video processing method provided by an embodiment of the present disclosure;

[0072] Figure 2 Schematically showing a flowchart of a video processing method provided by an embodiment of the present disclosure;

[0073] Figure 3 Schematically showing a flowchart of another video processing method provided by an embodiment of the present disclosure;

[0074] Figure 4 Schematically showing a flowchart of another video processing method provided by an embodiment of the present disclosure;

[0075] Figure 5 Schematically showing a flowchart of another video processing method provided by an embodiment of the present disclosure;

[0076] Figure 6 Schematically showing a block diagram of a video processing apparatus according to an embodiment of the present disclosure;

[0077] Figure 7 Schematically showing a schematic diagram of a storage medium according to an embodiment of the present disclosure; and

[0078] Figure 8 Schematically showing a block diagram of an electronic device according to an embodiment of the invention.

[0079] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. Detailed Embodiments

[0080] The principles and spirit of the present disclosure will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and thereby implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to be able to fully convey the scope of the present disclosure to those skilled in the art.

[0081] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0082] According to an embodiment of the present disclosure, there is provided a video processing method, a video processing apparatus, a storage medium, and an electronic device.

[0083] In this document, the number of any element in the drawings is for illustration rather than limitation, and any naming is only for distinction without any limiting meaning.

[0084] Next, with reference to several representative embodiments of the present disclosure, the principles and spirit of the present disclosure will be elaborated in detail.

[0085] Figure 1 A schematic flowchart of a video processing method provided by an embodiment of the present disclosure is shown, as Figure 1 shown, the method may include:

[0086] Step S101: Perform encoding division on any frame of video image in the video to be processed, and determine a processed image block with a first number of pixels.

[0087] In an embodiment of the present disclosure, the video to be processed may be a video that needs to be encoded. The video may be a video obtained by shooting or a video downloaded from the Internet. The present disclosure does not limit this. Performing encoding division on any frame of video image in the video to be processed and determining a processed image block with a first number of pixels may be when processing the video images of the video to be processed frame by frame, randomly selecting a frame of video image from multiple frames of video images, performing encoding division processing on the video image, obtaining multiple image blocks, and selecting an image block with a first number of pixels from the multiple image blocks as the processed image block.

[0088] In an embodiment of the present disclosure, the content displayed in the video may be divided into a shooting object and a background image other than the shooting object. In continuous multiple frames of video images, usually the position of the shooting object changes in multiple frames of video images, while the background image changes less in multiple frames of video images. Therefore, in order to encode the video and compress the storage volume of the video, by dividing the video image into multiple image blocks, and determining the correlation between adjacent frame video images of the video according to the changes of the image blocks at the same position in different frames, and then encoding according to the correlation between the video images to reduce the storage volume of the video. When performing encoding division on the video to be processed, the encoding division method used may be a tree encoding method, a quadtree encoding method, or a Huffman tree encoding method. The present disclosure does not limit this.

[0089] In the embodiments of the present disclosure, the number of first pixels may be the number of pixels preset according to the actual situation. For example, the number of first pixels may be 64×64 pixels, or may be 32×32 pixels, and so on. Specifically, when the image content in the video to be processed is relatively simple, that is, the color change in the image is single and the noise distribution is relatively uniform, a larger number may be set for the number of first pixels. For example, the number of first pixels may be set to 64×64; when the image content in the video to be processed is relatively rich, that is, the color change in the image is large and the edges and textures are relatively rich, a smaller number may be set for the number of first pixels. For example, the number of first pixels may be set to 16×16. It should be noted that since the processing image block is obtained by dividing the video image for encoding, the number of first pixels is less than the total number of pixels of the video image.

[0090] Step S102: Calculate the residuals of the respective pixels in the processing image block according to the respective pixels included in the processing image block.

[0091] In the embodiments of the present disclosure, the encoding of the video may be to record the change amount between the original frame and the encoded frame, and this change amount may be expressed as the difference amount between the pixels in the original frame and the encoded frame. For example, the change amount may be expressed as the residual of the pixels. Calculating the residuals of the respective pixels in the processing image block may be to calculate the residual between this frame of video image and the previous frame or the next frame of video image, so as to obtain the residual of the processing image block.

[0092] Specifically, when calculating the residual of the processing image block, a prediction mode may be used for calculation. For example, the inter (INTER) mode may be used, or the merge (MERGE) mode may be used, or the skip (SKIP) mode may be used, and so on. It is also possible to directly calculate the pixel value difference between the processing image block and the previous frame or the next frame.

[0093] Step S103: Select a target image block whose residual information meets the preset residual condition based on the residuals of the respective pixels in the processing image block; the number of pixels included in the target image block is not greater than the number of first pixels; the residual information is determined according to the residuals of the respective pixels in the target image block.

[0094] In the embodiments of the present disclosure, it is possible to calculate the residual information of the candidate image block based on the residuals of each pixel in the processed image block, and use the candidate image block whose residual information meets the preset residual condition as the target image block. Among them, the number of pixels included in the candidate image block may be no greater than the number of pixels included in the processed image block, that is, the number of pixels included in the target image block is no greater than the first number of pixels. For example, if the first number of pixels of the processed image block is 64×64, the number of pixels included in the candidate image block may be 64×64, or 32×32, or 16×16, etc., that is, the number of pixels included in the target image block may also be 64×64, or 32×32, or 16×16, etc.

[0095] In the embodiments of the present disclosure, the preset residual condition may be to determine whether the residual information is less than a preset residual threshold. If it is less, it can be determined that the residual information meets the preset residual condition. If it is not less, it can be determined that the residual information does not meet the preset residual condition. Among them, the preset residual threshold may be a value preset according to the actual situation, and the present disclosure does not make any restrictions.

[0096] Step S104: Calculate the loss of the target image block using at least one prediction mode, and select the prediction mode whose loss value meets the preset loss condition as the target mode; the target mode is used to encode the video to be processed.

[0097] In the embodiments of the present disclosure, since during video encoding, it is mainly to transmit the amount of motion change of the objects in the video image between different frames, where the amount of motion change can be determined based on Motion Estimation (ME), Motion Compensation (MC), and Motion Vector (MV), and based on Motion Estimation (ME), Motion Compensation (MC), and Motion Vector (MV), multiple prediction modes for inter-frame prediction can be obtained. The prediction mode can be an intra-frame prediction mode or an inter-frame prediction mode. Among them, the inter-frame prediction mode can be an Advanced Motion Vector Prediction (AMVP) mode or a High Efficiency Video Coding (HEVC) mode. The prediction mode can also be a transform mode in the HEVC mode. For example, it can be the INTER mode in the HEVC mode or the Merge mode in the HEVC mode, etc.

[0098] In the embodiments of the present disclosure, calculating the loss of a target image block using at least one prediction mode may involve selecting at least one prediction mode, processing the target image block using the prediction mode, and calculating the loss value in this prediction mode based on the processing result. Among them, calculating the loss value in this prediction mode may involve calculating the loss value using rate-distortion optimization (RDO), or calculating the loss value using sum of absolute transformed difference (SATD), or calculating the loss value using other methods. The present disclosure places no restrictions on this.

[0099] In the embodiments of the present disclosure, selecting the prediction mode whose loss value meets the preset loss condition as the target mode may involve, among the loss values of different prediction modes, selecting the prediction mode with the smallest loss value as the target mode, or taking the prediction mode whose loss value is less than the preset loss threshold as the target mode. Among them, the preset loss threshold can be determined in advance according to the actual situation. Since the determined target mode is the prediction mode with the smallest image loss value when processing the target image block, and the target image block is the image block with the smallest residual change in the video image, therefore, encoding the video to be processed using the target mode can, to a certain extent, ensure that the residual loss of the video image is small, thus guaranteeing the quality of video encoding. For example, in an actual application scenario, compared with the existing encoding method, using the method of the present disclosure to determine the target mode can increase the encoding speed of the video by approximately 30%, while the quality of the encoded video may decrease by less than 1%.

[0100] In summary, the video processing method provided by the embodiments of the present disclosure may first perform encoding division on any frame of video image in the video to be processed to determine a processed image block with a first number of pixels, then calculate the residuals of each pixel in the processed image block based on each pixel included in the processed image block, and based on the residuals of each pixel in the processed image block, select a target image block whose residual information meets the preset residual condition. The number of pixels included in the target image block is not greater than the first number of pixels, and this residual information is determined according to the residuals of each pixel in the target image block. Finally, calculate the loss of the target image block using at least one prediction mode, and select the prediction mode whose loss value meets the preset loss condition as the target mode. This target mode is used to encode the video to be processed. In this way, compared with the existing encoding method, the present disclosure can, to a certain extent, guarantee the quality of video encoding while reducing the computational amount for determining the target image block, thereby reducing the time required for video encoding and improving the efficiency of video encoding.

[0101] Optionally, in the embodiments of the present disclosure, the operation of performing encoding division on any frame of video image in the video to be processed to determine a processed image block with a first number of pixels, such as Figure 2As shown, it may specifically include;

[0102] Step S1011: Extract any one frame of video image from the video to be processed.

[0103] In the embodiments of the present disclosure, since a video is composed of consecutive frames and each frame corresponds to a video image, therefore, to extract any one frame of video image from the video to be processed, it can be to first convert the video to be processed into multiple frames of video images, and then randomly extract one frame of video image from the multiple frames of video images, or it can be to randomly select a playback time point according to the playback duration of the video to be processed, and use the frame video image corresponding to this playback time point as the extracted frame of video image. The present disclosure does not limit this.

[0104] Step S1012: Recursively divide the video image according to a preset coding setting to obtain multiple image blocks.

[0105] In the embodiments of the present disclosure, taking the Coding Tree Unit (CTU) as an example, the specific coding and dividing steps are as follows: First, determine the coding setting. This coding setting can be pre-determined and can be used to determine the size of the coding tree block. For example, the coding setting can be 64×64, that is, the size of the coding tree block can be 64×64; divide the video image according to the coding setting to obtain multiple coding tree blocks; perform recursive splitting on each coding tree block. The quadtree structure can be used. By slicing and tiling, a coding tree block can be divided into sub-regions of 32×32, 16×16, and 8×8 to obtain a coding tree block group, that is, multiple image blocks can be obtained.

[0106] It should be noted that in the process of dividing the video image into image blocks in the embodiments of the present disclosure, it is often based on the pixels included in the video image as the unit for division, that is, the video image can be divided according to the number of pixels. Further, it can be divided according to different numbers of pixels. Specifically, it can be divided in the form of 2N×2N, where N can be a positive integer. For example, it can be divided into 64×64, 32×32, and 16×16 respectively, or it can be divided in the form of N×M, where M can be a positive integer. For example, it can be divided into 128×64, 32×16, and 8×16 respectively, or it can be divided according to other size shapes. The present disclosure does not limit this.

[0107] Step S1013: Select the image block containing the number of pixels as the first number of pixels as the processed image block.

[0108] In the embodiments of the present disclosure, a tree coding block with a size of the first number of pixels may be selected from the multiple tree coding blocks obtained by partitioning as the processing image block. For example, the first number of pixels may be 64×64, and among the multiple image blocks obtained by partitioning, the sizes of the image blocks may be 64×64, 32×32, 16×16, and 8×8. Therefore, a tree coding block with a size of the first number of pixels 64×64 may be selected from the multiple image blocks, and the 64×64 image block may be used as the processing image block.

[0109] Optionally, in one implementation, in the embodiments of the present disclosure, the operation of calculating the residuals of each pixel in the processing image block according to each pixel included in the processing image block, as Figure 3 shown, may specifically include:

[0110] Step S1021: Select P pixels as target pixels from each pixel included in the processing image block; P is less than or equal to the first number of pixels.

[0111] For example, the number of pixels included in the processing image block may be 64×64, and P may take a value of 64×64, that is, P is equal to the first number of pixels, then all the pixels included in the processing image block may be used as target pixels; P may also take a value of 32×32, that is, P is less than the first number of pixels, then 32×32 pixels may be selected from the pixels included in the processing image block as target pixels.

[0112] Step S1022: Calculate the target pixels using a preset residual calculation mode, and use the calculation result as the residuals of each pixel in the processing image block.

[0113] In the embodiments of the present disclosure, the residual calculation mode may be preset according to the actual situation. For example, the residual calculation mode may be the SKIP mode. Each target pixel is calculated using the SKIP mode, and the calculation result of each target pixel may be used as the residual of the corresponding pixel.

[0114] It should be noted that when P is equal to the first number of pixels, the calculation result of the target pixel may be directly used as the residual of the target pixel. When P is less than the first number of pixels, for the target pixel, the calculation result of the target pixel may be used as the residual of the target pixel, and for the non-target pixel, the average value of the calculation results of the adjacent target pixels may be used as the residual of the pixel.

[0115] Optionally, in another implementation, in the embodiments of the present disclosure, the operation of calculating the residuals of each pixel in the processing image block according to each pixel included in the processing image block, as Figure 4 shown, may specifically include:

[0116] Step S1023: Determine the corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processed image block in the video image.

[0117] Exemplarily, if the relative position of the processed image block in the video image is the 64×64 image block in the lower left corner, then the 64×64 image block in the lower left corner of the previous frame of video image can be used as the corresponding image block at the same relative position as the processed image block.

[0118] Step S1024: Calculate the residuals of each pixel in the processed image block according to the pixel values of each pixel in the processed image block and the pixel values of each pixel in the corresponding image.

[0119] In the embodiments of the present disclosure, it may be to first determine the corresponding pixel of each pixel in the processed image block at the same relative position on the corresponding image block, and subtract the pixel value of the pixel on the processed image block from the pixel value of the corresponding pixel of the pixel, so as to obtain the residual of the pixel. For example, if the pixel value of the first pixel in the upper left corner of the processed image block is 155, and the pixel value of the first pixel in the upper left corner of the corresponding image block is 143, then the residual of the first pixel in the upper left corner of the processed image block can be calculated as 12.

[0120] Optionally, in the embodiments of the present disclosure, the operation of selecting the target image block based on the residuals of each pixel in the processed image block, as Figure 5 shown, may sequentially perform the following operations to determine the target image block:

[0121] Step S1031: Select the second number of pixels from the target interval; the maximum value in the target interval is the first number of pixels, and the minimum value is the third number of pixels.

[0122] In the embodiments of the present disclosure, the target interval may be a closed interval of values. Among them, the maximum value in the target interval may be the first number of pixels, and the minimum value may be the third number of pixels. Randomly select a number of pixels as the second number of pixels in the target interval, that is, it can be obtained that the third number of pixels ≤ the second number of pixels ≤ the first number of pixels. For example, the first number of pixels can be taken as 64×64, the third number of pixels can be taken as 8×8, then the target interval can be [8×8, 64×64], and select the second number of pixels from the target interval. The second number of pixels can be 64×64, or 32×32, or 16×16, or 8×8, and so on.

[0123] Step S1032: Determine the candidate image block based on each pixel included in the processed image block; the number of pixels included in the candidate image block is the second number of pixels.

[0124] In an embodiment of the present disclosure, since the number of pixels included in the processed image block is the first number of pixels, it may be divided in the processed image block based on the second number of pixels, and the divided image block is used as an alternative image block, and the number of pixels included in the alternative image block is the second number of pixels. For example, the second number of pixels may be 32×32. In a processed image block of 64×64, an image block with 32×32 pixels may be divided and used as the alternative image block.

[0125] Step S1033: Calculate the residual information of the alternative image block according to the residuals of each pixel in the alternative image block.

[0126] In an embodiment of the present disclosure, it may be to first determine the residuals of each pixel in the alternative image block according to the residuals of each pixel in the processed image block, and then calculate the residual information of the alternative image block based on the residuals of each pixel in the alternative image block. Among them, the residual information may be, for each pixel of the alternative image block, taking the result of subtracting the minimum value from the maximum value of the pixel residuals as the residual information of the alternative image block, or dividing the alternative image block into units and taking the result of subtracting the minimum value from the maximum value of the unit residuals as the residual information of the alternative image block. It should be noted that the absolute value may also be taken for the residual information. For example, the residual information may be: ∣y max -y min ∣, where y max may be the maximum value of the residuals in the processed image block, and y min may be the minimum value of the residuals in the processed image block.

[0127] Step S1034: Determine whether the residual information is less than a preset residual threshold.

[0128] In an embodiment of the present disclosure, it may be to compare the value of the residual information with the value of the preset threshold. For example, the preset residual threshold may be x, and the residual information may be: ∣y max -y min ∣. It may be to compare ∣y max -y min ∣ with x to determine whether the residual information is less than the preset residual threshold.

[0129] Step S1035: When the residual information is less than the preset residual threshold, use the alternative image block as the target image block and stop performing the operation of selecting the second number of pixels.

[0130] In the embodiments of the present disclosure, it may be determined whether the residual information is less than a preset residual threshold. When the residual information is less than the preset residual threshold, the alternative image block with the residual information may be used as the target image block. The preset residual threshold may be set in advance according to the actual situation. For example, the preset residual threshold may be a quantization parameter (QP), and the quantization parameter (QP) may be a fixed value determined according to the discrete cosine transform. The present disclosure places no restrictions on this.

[0131] Optionally, in the embodiments of the present disclosure, the operation of calculating the residual information of the alternative image block according to the residuals of each pixel in the alternative image block may specifically include:

[0132] Dividing the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks; calculating the residual values of the sub-image blocks based on the residuals of each pixel in the sub-image blocks; and determining the residual information of the alternative image block according to the residual values of the sub-image blocks.

[0133] In the embodiments of the present disclosure, the third number of pixels may be 8×8, and the number of pixels of the alternative image block may be 32×32. Then, dividing the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks may be to divide the 32×32 alternative image block into 8×8 sub-image blocks, thereby obtaining a plurality of 8×8 sub-image blocks.

[0134] In the embodiments of the present disclosure, based on the residuals of each pixel in the alternative image block, it may be possible to first determine the pixels included in the sub-image block and the residuals corresponding to the pixels to obtain the residuals of each pixel in the sub-image block, and then calculate the residual value of the sub-image block. The residual value may be the sum of the residuals of each pixel in the sub-image block or the average value of the residuals of each pixel.

[0135] In the embodiments of the present disclosure, determining the residual information of the alternative image block according to the residual values of the sub-image blocks may be to use the absolute value of the difference between the maximum value and the minimum value of the residual values among the plurality of sub-image blocks as the residual information of the alternative image block, or the square difference between the maximum value and the minimum value of the residual values as the residual information of the alternative image block.

[0136] Optionally, the embodiments of the present disclosure may further specifically include: when the residual information is not less than the preset residual threshold, the operation of selecting the second number of pixels is re-executed until the residual information is less than the preset residual threshold.

[0137] In the embodiments of the present disclosure, when the residual information of the candidate image block is greater than or equal to the preset residual threshold, the operation of selecting the second number of pixels can be re-executed, that is, the operations from step S1031 to step S1034 can be re-executed until the residual information of the selected candidate image block is less than the preset residual threshold.

[0138] For example, the first number of pixels for processing the image block can be 64×64, and the third number of pixels can be 8×8. Then, the target interval can be determined as [8×8, 64×64]. The second number of pixels selected from the target interval can be 64×64, that is, the candidate image block 64×64 is obtained. According to the sub-image block 8×8, the residual information 1 of the candidate image block 64×64 is calculated, and it is judged whether the residual information 1 is less than the preset residual threshold. If the residual information 1 is less than the preset residual threshold, the candidate image block 64×64 is used as the target image block; if the residual information 1 is not less than the preset residual threshold, the second number of pixels can be re-selected as 32×32, that is, the candidate image block 32×32 is obtained. According to the sub-image block 8×8, the residual information 2 of the candidate image block 32×32 is calculated, and it is judged whether the residual information 2 is less than the preset residual threshold. If the residual information 2 is less than the preset residual threshold, the candidate image block 32×32 is used as the target image block; if the residual information 2 is not less than the preset residual threshold, the second number of pixels can be re-selected as 16×16, and so on, which will not be elaborated here.

[0139] Optionally, in the embodiments of the present disclosure, the operation of calculating the loss of the target image block by using at least one prediction mode and selecting the prediction mode whose loss value meets the preset loss condition as the target mode may specifically include:

[0140] According to the preset image loss formula, calculate each loss value of the target image block under the at least one prediction mode; select the prediction mode with the smallest loss value as the target mode.

[0141] In the embodiments of the present disclosure, the image loss formula may be a preset formula for calculating the loss value of the target image block under different prediction modes. The image loss formula may be a rate-distortion cost (RDcost) formula or a sum of absolute transform differences cost (SATDcost) formula. For example, using the rate-distortion cost (RDcost) formula, the specific formula may be as follows:

[0142] J(mode) = SAD + λ * R(ref, mode, mv)

[0143] Wherein, J(mode) may be the loss value of the target image block in this prediction mode (mode), SAD may be the residual value of the target image block, λ may be a weight value. For example, λ may be a Lagrange multiplier, R(*) may be the actual bitstream of the target image block in this prediction mode (mode). The specific calculation content is not limited in this disclosure, and existing rate-distortion cost formulas can be used for calculation. ref may be a reference frame, which may be the frame used when calculating the residual. mode may be a prediction mode, and mv may be the motion vector of the target image block on the reference frame.

[0144] In the embodiments of this disclosure, it may be to select the prediction mode with the smallest loss value from the loss values calculated under different prediction modes, and use this prediction mode as the target mode. For example, the loss values of the target image block in the MERGE mode, INTEL mode, and INTRA mode can be calculated respectively, and these three loss values can be compared, and the prediction mode with the smallest loss value is used as the target mode.

[0145] It should be noted that for the video processing method provided in the embodiments of this disclosure, the execution subject may be a video processing device, or a control module in the video processing device for executing the loaded video processing method. In the embodiments of this disclosure, the video processing method is taken as an example in which the video processing device executes the loaded video processing method to illustrate the video processing method provided in the embodiments of this disclosure. Next, refer to Figure 6 A description is given of the video processing device according to the exemplary embodiments of this disclosure.

[0146] Figure 6 A block diagram schematically showing a video processing device according to an embodiment of this disclosure is shown as Figure 6 shown. The video processing device 60 may include:

[0147] A first determination module 601, configured to perform coding division on any frame of video image in the video to be processed, and determine a processed image block with a first number of pixels;

[0148] A calculation module 602, configured to calculate the residuals of each pixel in the processed image block according to each pixel included in the processed image block;

[0149] A first selection module 603, configured to select a target image block whose residual information meets a preset residual condition based on the residuals of each pixel in the processed image block; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block;

[0150] A second selection module 604, configured to calculate a loss of the target image block by using at least one prediction mode, and select a prediction mode whose loss value meets a preset loss condition as a target mode; the target mode is used to encode the video to be processed.

[0151] In summary, the video processing device provided in the embodiments of the present disclosure can first perform encoding division on any frame of video image in the video to be processed to determine a processing image block with a first number of pixels, and then calculate the residuals of each pixel in the processing image block according to each pixel included in the processing image block, and based on the residuals of each pixel in the processing image block, select a target image block whose residual information meets a preset residual condition, where the number of pixels included in the target image block is not greater than the first number of pixels, and the residual information is determined according to the residuals of each pixel in the target image block. Finally, calculate the loss of the target image block by using at least one prediction mode, and select a prediction mode whose loss value meets a preset loss condition as a target mode, and the target mode is used to encode the video to be processed. In this way, compared with the existing encoding method, the present disclosure can ensure the video encoding quality to a certain extent while reducing the calculation amount for determining the target image block, thereby reducing the time required for video encoding and improving the efficiency of video encoding.

[0152] Optionally, the calculation module 602 is further specifically configured to:

[0153] Select P pixels as target pixels for each pixel included in the processing image block; P is less than or equal to the first number of pixels;

[0154] Calculate the target pixels by using a preset residual calculation mode, and use the calculation result as the residual of each pixel in the processing image block.

[0155] Optionally, the calculation module 602 is further specifically configured to:

[0156] Determine a corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processing image block in the video image;

[0157] Calculate the residuals of each pixel in the processing image block according to the pixel values of each pixel in the processing image block and the pixel values of each pixel in the corresponding image.

[0158] Optionally, the first selection module 603 is further specifically configured to:

[0159] Sequentially perform the following operations to determine the target image block:

[0160] Select a second number of pixels from the target interval; the maximum value in the target interval is the first number of pixels, and the minimum value is the third number of pixels;

[0161] Based on each pixel included in the processed image block, determine an alternative image block; the number of pixels included in the alternative image block is the second number of pixels;

[0162] Calculate residual information of the alternative image block according to the residuals of each pixel in the alternative image block;

[0163] Determine whether the residual information is less than a preset residual threshold;

[0164] When the residual information is less than the preset residual threshold, use the alternative image block as the target image block and stop performing the operation of selecting the second number of pixels.

[0165] Optionally, the first selection module 603 is further specifically configured to:

[0166] Divide the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks;

[0167] Calculate the residual value of the sub-image block based on the residuals of each pixel in the sub-image block;

[0168] Determine the residual information of the alternative image block according to the residual value of the sub-image block.

[0169] Optionally, the apparatus 60 further includes:

[0170] An execution module, configured to when the residual information is not less than the preset residual threshold, re-perform the operation of selecting the second number of pixels until the residual information is less than the preset residual threshold.

[0171] Optionally, the loss value is one of an RD loss value and an SATD loss value.

[0172] Optionally, the first determination module 601 is further specifically configured to;

[0173] Extract any frame of video image from the video to be processed;

[0174] Perform recursive partitioning on the video image according to a preset coding setting to obtain a plurality of image blocks;

[0175] Select an image block including the first number of pixels as the processed image block.

[0176] After introducing the video processing method and apparatus of the exemplary embodiments of the present disclosure, next, refer to Figure 7 Describe the storage medium of the exemplary embodiments of the present disclosure.

[0177] Refer toFigure 7 As shown, a storage medium 700 for implementing the above method according to an embodiment of the present disclosure is described. It can be a portable compact disc read-only memory (CD-ROM), include program code, and can run on a device such as a personal computer. However, the program product of the present disclosure is not limited to this. In this document, a readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component.

[0178] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0179] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or component.

[0180] The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0181] The program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0182] After introducing the storage medium of the exemplary embodiments of the present disclosure, next, reference will be made to Figure 7 describe the electronic device of the exemplary embodiments of the present disclosure.

[0183] Figure 8 The electronic device 800 shown in

[0184] is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure. Figure 8 As shown in

[0185] the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one of the above-mentioned processing units 810, at least one of the above-mentioned storage units 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0186] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification above. For example, the processing unit 810 can execute step S101, perform coding division on any frame of video image in the video to be processed, and determine a processed image block with a first number of pixels; step S102, calculate the residuals of each pixel in the processed image block according to the respective pixels included in the processed image block; step S103, based on the residuals of each pixel in the processed image block, select a target image block whose residual information meets a preset residual condition; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block; step S104, perform loss calculation on the target image block using at least one prediction mode, and select a prediction mode whose loss value meets a preset loss condition as the target mode; the target mode is used to encode the video to be processed.

[0186] The storage unit 820 may include a volatile storage unit, such as a random access storage unit (RAM) 8201 and / or a cache storage unit 8202, and may further include a read-only storage unit (ROM) 8203.

[0187] The storage unit 820 may further include a program / utilities 8204 having a set (at least one) of program modules 8205. Such program modules 8205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0188] The bus 830 may include a data bus, an address bus, and a control bus.

[0189] The electronic device 800 may also communicate with one or more external devices 70 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and such communication may be performed through the input / output (I / O) interface 850. The electronic device 800 further includes a display unit 840, which is connected to the input / output (I / O) interface 850 for display. Moreover, the electronic device 800 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 860. As shown in the figure, the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0190] It should be noted that although several modules or sub-modules of the audio playback device and the audio sharing device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided and embodied by multiple units / modules.

[0191] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present disclosure is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0192] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to enable a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present disclosure.

[0193] The embodiments of the present disclosure have been described above with reference to the accompanying drawings. However, the present disclosure is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present disclosure, those of ordinary skill in the art can also make many forms without departing from the purpose of the present disclosure and the scope protected by the claims, and all of them fall within the protection scope of the present disclosure.

Claims

1. A video processing method, characterized in that, The method includes: Performing encoding division on any frame of video image in the video to be processed, and determining a processed image block with a first number of pixels; Calculating the residuals of each pixel in the processed image block according to each pixel included in the processed image block; Based on the residuals of each pixel in the processed image block, selecting a target image block whose residual information meets a preset residual condition; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block, and the preset residual condition is to judge whether the residual information is less than a preset residual threshold. If it is less, it is determined that the residual information meets the preset residual condition. If it is not less, it is determined that the residual information does not meet the preset residual condition; Calculating the loss of the target image block using at least one prediction mode, and selecting the prediction mode whose loss value meets the preset loss condition as the target mode; the target mode is used for encoding the video to be processed.

2. The method according to claim 1, characterized in that, The calculating the residuals of each pixel in the processed image block according to each pixel included in the processed image block includes: For each pixel included in the processed image block, selecting P pixels as target pixels; P is less than or equal to the first number of pixels; Calculating the target pixels using a preset residual calculation mode, and taking the calculation result as the residuals of each pixel in the processed image block.

3. The method according to claim 1, wherein The calculating the residuals of each pixel in the processed image block according to each pixel included in the processed image block includes: Determining a corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processed image block in the video image; Calculating the residuals of each pixel in the processed image block according to the pixel values of each pixel in the processed image block and the pixel values of each pixel in the corresponding image.

4. The method according to claim 1, characterized in that, The selecting a target image block whose residual information meets a preset residual condition based on the residuals of each pixel in the processed image block includes: Sequentially performing the following operations to determine the target image block: Selecting a second number of pixels from the target interval; the maximum value in the target interval is the first number of pixels, and the minimum value is the third number of pixels; Determining an alternative image block based on each pixel included in the processed image block; the number of pixels included in the alternative image block is the second number of pixels; Calculating the residual information of the alternative image block according to the residuals of each pixel in the alternative image block; Judging whether the residual information is less than a preset residual threshold; When the residual information is less than the preset residual threshold, taking the alternative image block as the target image block and stopping the operation of selecting the second number of pixels.

5. The method according to claim 4, characterized in that The calculating the residual information of the alternative image block according to the residuals of each pixel in the alternative image block includes: Dividing the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks; Calculating the residual values of the sub-image blocks based on the residuals of each pixel in the sub-image blocks; Determining the residual information of the alternative image block according to the residual values of the sub-image blocks.

6. The method according to claim 4, characterized in that, The method further includes: When the residual information is not less than the preset residual threshold, the operation of selecting the second number of pixels is re-executed until the residual information is less than the preset residual threshold.

7. The method according to claim 1, wherein The loss value is one of the RD loss value and the SATD loss value.

8. The method according to claim 1, wherein The processing of encoding and partitioning any frame of video image in the video to be processed to determine the number of first pixels of the image blocks includes: Extracting the any frame of video image from the video to be processed; Performing recursive partitioning on the video image according to the preset encoding settings to obtain a plurality of image blocks; Selecting the image blocks with the number of pixels being the first number of pixels as the processing image blocks.

9. A video processing device, characterized in that, The apparatus includes: A first determination module, configured to perform encoding and partitioning on any frame of video image in the video to be processed to determine the number of first pixels of the image blocks; A calculation module, configured to calculate the residuals of each pixel in the processing image block according to each pixel included in the processing image block; A first selection module, configured to select, based on the residuals of each pixel in the processing image block, a target image block whose residual information meets the preset residual condition; the number of pixels included in the target image block is not greater than the first number of pixels; the residual information is determined according to the residuals of each pixel in the target image block, and the preset residual condition is to determine whether the residual information is less than the preset residual threshold. If it is less, it is determined that the residual information meets the preset residual condition. If it is not less, it is determined that the residual information does not meet the preset residual condition; A second selection module, configured to perform loss calculation on the target image block by using at least one prediction mode, and select the prediction mode whose loss value meets the preset loss condition as the target mode; the target mode is used to encode the video to be processed.

10. The device according to claim 9, characterized in that, The calculation module is further specifically configured to: Select P pixels as target pixels for each pixel included in the processing image block; P is less than or equal to the first number of pixels; Calculating the target pixels by using a preset residual calculation mode, and using the calculation result as the residuals of each pixel in the processing image block.

11. The device according to claim 9, characterized in that, The calculation module is further specifically configured to: Determine the corresponding image block at the same relative position in the previous frame of video image according to the relative position of the processing image block in the video image; Calculating the residuals of each pixel in the processing image block according to the pixel values of each pixel in the processing image block and the pixel values of each pixel in the corresponding image.

12. The device according to claim 9, wherein The first selection module is further specifically configured to: Sequentially perform the following operations to determine the target image block: Select the second number of pixels from the target interval; the maximum value in the target interval is the first number of pixels, and the minimum value is the third number of pixels; Based on each pixel included in the processing image block, determine the candidate image blocks; the number of pixels included in the candidate image blocks is the second number of pixels; Calculating the residual information of the candidate image blocks according to the residuals of each pixel in the candidate image blocks; Determining whether the residual information is less than the preset residual threshold; When the residual information is less than the preset residual threshold, the alternative image block is used as the target image block, and the operation of selecting the second number of pixels is stopped.

13. The device according to claim 12, characterized in that, The first selection module is further specifically configured to: Divide the alternative image block according to the third number of pixels to obtain a plurality of sub-image blocks; Calculate the residual value of the sub-image block based on the residuals of the pixels in the sub-image block; Determine the residual information of the alternative image block according to the residual value of the sub-image block.

14. The device according to claim 12, characterized in that, The apparatus further includes: An execution module, configured to, when the residual information is not less than the preset residual threshold, re-execute the operation of selecting the second number of pixels until the residual information is less than the preset residual threshold.

15. The device according to claim 9, characterized in that, The loss value is one of the RD loss value and the SATD loss value.

16. The device according to claim 9, characterized in that, The first determination module is further specifically configured to; Extract any frame of video image from the video to be processed; Perform recursive partitioning on the video image according to a preset coding setting to obtain a plurality of image blocks; Select an image block containing the first number of pixels as the processed image block.

17. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the video processing method according to any one of claims 1 to 8.

18. An electronic device, characterized in that, Including: A processor; And A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the video processing method according to any one of claims 1 to 8 by executing the executable instructions.

Images