A method and apparatus for video coding

Abstract

The application discloses a kind of method and device of video coding, it is related to computer technical field.The specific embodiment of the method includes: sequentially obtaining the at least one ordered image frame included in the video to be encoded;For each image frame, calculate the motion characteristic value of the image frame according to the pixel data of the image frame;According to the motion characteristic value of the image frame and the motion characteristic value of the at least one image frame before the image frame, the motion characteristic difference value corresponding to the image frame is obtained;According to the motion characteristic difference value, the next image frame of the image frame is quantized parameter adjusted, and video coding is carried out according to the adjusted quantized parameter.This embodiment carries out video coding in the scene of motion object detection through the motion characteristic difference value of image frame, can effectively solve the video coding problem caused by the color similarity of the object and background, and improves the video quality.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for video encoding. Background Technology

[0002] With the development of multimedia technology, video encoding and decoding, as a core component, has also achieved leapfrog development. The main function of video encoding is to compress video pixel data into a video bitstream, thereby reducing the amount of video data. Current video encoding methods typically perform object detection on the original image to generate object information, then convert it into the location region of the video image. By calculating the confidence level, the quantization parameters of the encoding are adjusted to perform video encoding.

[0003] In the process of realizing this invention, the inventors discovered the following problems in the prior art:

[0004] Although existing target detection technologies improve accuracy by appropriately expanding the target range, in scenes where the background color and the color of the detected object are similar, the accuracy of target detection will be affected, which in turn affects the quality of video encoding and video transmission. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a video encoding method and apparatus. Based on the acquired ordered image frames, a motion feature difference between the image frames is obtained according to the motion feature values ​​of the image frames calculated from pixel data and the motion feature values ​​of at least one previous image frame. The quantization parameters of the current image frame and the next image frame are adjusted according to the motion feature difference. The present invention performs video encoding in moving object detection scenarios by using the motion feature difference of image frames, which can effectively solve the video encoding problem caused by the similarity of color between the captured object and the background, and improve video quality.

[0006] To achieve the aforementioned objective, according to one aspect of the present invention, a video encoding method is provided, comprising:

[0007] Sequentially acquire at least one ordered image frame from the video to be encoded;

[0008] For each image frame, calculate the motion feature value of the image frame based on the pixel data of the image frame;

[0009] The motion feature difference value corresponding to the image frame is obtained based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame.

[0010] The quantization parameters of the image frame and the next image frame are adjusted based on the motion feature difference, and video encoding is performed based on the adjusted quantization parameters.

[0011] Optionally, before obtaining the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame, the method further includes: determining whether the image frame is the first frame of the video to be encoded; and, if the image frame is the first frame of the video to be encoded, using the motion feature value of the image frame as the motion feature difference corresponding to the image frame.

[0012] Optionally, obtaining the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame includes: calculating the average motion feature value of at least one image frame preceding the image frame based on the motion feature value of each first image frame in at least one image frame preceding the image frame, wherein the motion feature value of each first image frame is calculated based on the pixel data of each first image frame; and obtaining the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the average motion feature value of at least one image frame preceding the image frame.

[0013] Optionally, calculating the motion feature value of the image frame based on the pixel data of the image frame includes: calculating the motion feature value of the image frame based on the luminance pixels and chrominance pixels of the image frame.

[0014] Optionally, calculating the motion feature value of the image frame based on the pixel data of the image frame includes: dividing the image frame into blocks according to a set image block size to obtain at least one block image; calculating the motion feature value of the block image based on the pixel data of each block image; taking the average of the motion feature values ​​of each block image as the motion feature value of the image frame; obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame includes: calculating the motion feature difference value of each block image included in the image frame based on the motion feature value of the block image and the motion feature value of at least one image frame preceding the image frame; taking the average of the motion feature difference values ​​of each block image as the motion feature difference value corresponding to the image frame; and adjusting the quantization parameters of the image frame based on the motion feature difference value includes: adjusting the quantization parameters of each block image included in the image frame based on the motion feature difference value of the block image to adjust the quantization parameters of the image frame.

[0015] Optionally, adjusting the quantization parameters of the block image based on the motion feature difference of the block image includes: comparing the motion feature difference of the block image with a preset block parameter threshold, the block parameter threshold including a first threshold, a second threshold, and a third threshold; performing a first adjustment to the quantization parameters of the block image when the motion feature difference of the block image is not greater than the first threshold; performing a second adjustment to the quantization parameters of the block image when the motion feature difference of the image is greater than the first threshold but not greater than the second threshold; and performing a third adjustment to the quantization parameters of the block image when the motion feature difference of the image is greater than the second threshold but not greater than the third threshold.

[0016] Optionally, the first adjustment includes decreasing the value according to a preset first threshold adjustment value; the second adjustment includes increasing the value according to a preset second threshold adjustment value; and the third adjustment includes increasing the value according to a preset third threshold adjustment value.

[0017] Optionally, before adjusting the quantization parameters of the next image frame based on the motion feature difference, the method further includes: determining that the image frame is not the last frame of the video to be encoded.

[0018] Optionally, adjusting the quantization parameters of the next image frame based on the motion feature difference includes: comparing the motion feature difference of the image frame with a preset frame parameter threshold, the frame parameter threshold including a fourth threshold, a fifth threshold, and a sixth threshold; performing a fourth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is not greater than the fourth threshold; performing a fifth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fourth threshold but not greater than the fifth threshold; and performing a sixth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fifth threshold but not greater than the sixth threshold.

[0019] Optionally, the fourth adjustment includes decreasing the value according to a preset fourth threshold adjustment value; the fifth adjustment includes decreasing the value according to a preset fifth threshold adjustment value; and the sixth adjustment includes increasing the value according to a preset sixth threshold adjustment value.

[0020] According to a second aspect of the present invention, a video encoding apparatus is provided, comprising:

[0021] The image frame acquisition module is used to sequentially acquire at least one ordered image frame from the video to be encoded.

[0022] The motion feature value acquisition module is used to calculate the motion feature value of each image frame based on the pixel data of the image frame.

[0023] The motion feature difference acquisition module is used to obtain the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame.

[0024] The quantization parameter adjustment module is used to adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference, and to perform video encoding based on the adjusted quantization parameters.

[0025] According to a third aspect of the present invention, a video encoding electronic device is provided, characterized in that it comprises:

[0026] One or more processors;

[0027] Storage device for storing one or more programs.

[0028] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiments of the present invention.

[0029] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0030] One embodiment of the invention has the following advantages or beneficial effects: by sequentially acquiring at least one ordered image frame from the video to be encoded; for each image frame, calculating the motion feature value of the image frame based on the pixel data of the image frame; obtaining the motion feature difference of the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame; adjusting the quantization parameters of the image frame and the next image frame based on the motion feature difference, and performing video encoding based on the adjusted quantization parameters, this invention achieves video encoding in moving object detection scenarios using the motion feature difference of image frames. This effectively solves the video encoding problem caused by the similarity in color between the captured object and the background, thus improving video quality. Attached Figure Description

[0031] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0032] Figure 1 This is a schematic diagram of the main flow of a video encoding method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the principle of video encoding according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the main modules of a video encoding apparatus according to an embodiment of the present invention;

[0035] Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0036] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0037] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this invention comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0038] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0039] While current video coding methods improve the accuracy of object detection by appropriately expanding the target range, this accuracy can still be affected in scenes where the background color and the detected object color are similar. Consequently, this impacts video coding and video quality, failing to adequately meet practical application requirements.

[0040] To address the aforementioned problems in existing technologies, this invention proposes a video encoding method. Based on acquired ordered image frames, the method calculates motion feature differences between image frames using motion feature values ​​calculated from pixel data and motion feature values ​​from at least one preceding image frame. Quantization parameters are then adjusted for the current and next image frames based on these motion feature differences. This invention utilizes motion feature differences between image frames for video encoding in moving object detection scenarios, effectively solving the video encoding problem caused by color similarity between the captured object and the background, thus improving video quality.

[0041] In the description of the embodiments of the present invention, the terms involved and their meanings are as follows:

[0042] YUV: refers to a pixel format in which luminance and chrominance parameters are represented separately. "Y" represents luminance (Luminance or Luma), which is the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which are used to describe the color and saturation of the image and to specify the color of the pixel.

[0043] Figure 1 This is a schematic diagram of the main flow of a video encoding method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the video encoding method of this embodiment includes the following steps S101 to S104.

[0044] Step S101: Sequentially acquire at least one ordered image frame from the video to be encoded.

[0045] Specifically, an image frame is the basic unit of video. The video acquisition device preprocesses the acquired image frames and sends them to the video encoding module in an orderly manner for video encoding. The video encoding module sorts the received image frames according to the encoding order to obtain at least one image frame to be encoded in sequence.

[0046] Step S102: For each image frame, calculate the motion feature value of the image frame based on the pixel data of the image frame.

[0047] According to one embodiment of the present invention, calculating the motion feature value of the image frame based on the pixel data of the image frame includes: calculating the motion feature value of the image frame based on the luminance pixels and chrominance pixels of the image frame.

[0048] Specifically, considering the motion attributes of objects in the video captured images, the difference between the YUV color space and chromaticity is used for moving object detection. YUV uses the luminance (Y) and chromaticity (UV) in the color space to replace the traditional three primary colors to compress the image. For at least one ordered image frame in the above encoding module, the motion feature value of each image frame needs to be calculated. Based on the relevant calculation principles of the comprehensive quality evaluation index of the YUV color space, the luminance and chromaticity pixels of each image frame are obtained to calculate the motion feature value of the image frame.

[0049] Step S103: Obtain the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one previous image frame.

[0050] According to one embodiment of the present invention, before obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame, the method further includes: determining whether the image frame is the first frame of the video to be encoded; and, if the image frame is the first frame of the video to be encoded, using the motion feature value of the image frame as the motion feature difference value corresponding to the image frame.

[0051] Specifically, color space-based difference matching is obtained by comparing the motion feature values ​​of an image frame with the motion feature values ​​of previous image frames. For the first frame in the image frame sequence, since there are no image frames before the first frame, the motion feature difference corresponding to the image frame is the motion feature value of this image frame itself when the image frame is the first frame of the video to be encoded.

[0052] According to another embodiment of the present invention, obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame includes: calculating the average motion feature value of at least one image frame preceding the image frame based on the motion feature value of each first image frame in at least one image frame preceding the image frame, wherein the motion feature value of each first image frame is calculated based on the pixel data of each first image frame; and obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the average motion feature value of at least one image frame preceding the image frame.

[0053] Specifically, taking the current image frame as the baseline, the image frames preceding the current image frame are defined as the first image frames. The first m first image frames can be selected from the current image frame. Using the method described above for calculating the motion feature values ​​of image frames, the motion feature value of each first image frame is obtained. Based on the motion feature values ​​of each of the first m first image frames, the average motion feature value of the first m first image frames is calculated. For example, the calculation of the average motion feature value Avg_preFrame of the first m first image frames is as follows:

[0054]

[0055] Where Dyn_preFrame represents the motion feature value of the first image frame, the difference between the motion feature value of the current image frame and the average motion feature value of the previous m first image frames is then calculated to obtain the motion feature difference value corresponding to the current image frame. Alternatively, the previous image frame can be selected, and the difference between the motion feature value of the current image frame and the motion feature value of the previous image frame can be calculated to obtain the motion feature difference value corresponding to the current image frame. In the embodiments of the present invention, m is a positive integer greater than or equal to 1. It should be noted that if the number of first image frames before the current image frame is less than m, then it is only necessary to calculate the average motion feature value of all first image frames before the current image frame. For example, if m = 2, and the current image frame is the 2nd frame of the video, then there is only 1 first image frame before the current image frame. In this case, it is only necessary to combine this first image frame to calculate the average motion feature value (i.e., the motion feature value of the first image frame).

[0056] According to another embodiment of the present invention, calculating the motion feature value of the image frame based on the pixel data of the image frame includes: dividing the image frame into blocks according to a set image block size to obtain at least one block image; calculating the motion feature value of the block image based on the pixel data of each block image; taking the average value of the motion feature values ​​of each block image as the motion feature value of the image frame; obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame includes: calculating the motion feature difference value of each block image included in the image frame based on the motion feature value of the block image and the motion feature value of at least one image frame preceding the image frame; taking the average value of the motion feature difference value of each block image as the motion feature difference value corresponding to the image frame.

[0057] Specifically, in actual video encoding, it is usually necessary to divide image frames into blocks, dividing an image into several blocks. This not only enables parallel processing but also reduces the amount of computational data and pressure. The current image frame is divided into blocks according to a set block size to obtain at least one block image. In this embodiment, a block size of 16*16 is selected, so a 1280x720 resolution image can be divided into 3600 blocks. For each block image in the image frame, similar to the method for calculating motion feature values ​​described above, based on the relevant calculation principles of the YUV color space quality evaluation comprehensive index, the motion feature value of the block image is calculated using the luminance and chrominance pixels. The specific calculation of the block image's motion feature value (minisValue) is as follows:

[0058]

[0059] Where y_pixel represents the luminance pixels of the block image, and u_pixel and v_pixel represent the chrominance pixels of the block image; the average value of the motion feature values ​​of each block image in the image frame is used as the motion feature value of the image frame.

[0060] Accordingly, when calculating the motion feature difference corresponding to the image frame, at least one image frame preceding the current image frame is first divided into blocks. Then, following the method described above for calculating minisValue, the motion feature value minisValue_preFrame of each block of at least one image frame preceding the current image frame is calculated. Subsequently, the mean motion feature value Avg_minisValue_preFrame of an image frame is calculated. The specific calculation is as follows:

[0061]

[0062] Where `block_num` represents the number of block images. If only the previous image frame is selected, the average motion feature value `Avg_minisValue_preFrame` is the motion feature value of the image frames before the current image frame. If the previous m image frames are selected, the `Avg_minisValue_preFrame` of each of the previous m image frames needs to be averaged again. The average of the average motion feature values ​​of the previous m image frames is then used as the motion feature value of the image frames before the current image frame. The difference between the `minisValue` of the block image in the current image frame and the motion feature value of at least one image frame before the current image frame is then calculated to obtain the motion feature difference of the block image. For example, when only the previous image frame is selected, the calculation of the motion feature difference `diff_curblock` is as follows:

[0063] diff_curblock=minisValue-Avg_minisValue_preFrame;

[0064] Finally, the average motion feature difference of each block image in the current image frame is calculated. For example, based on the motion feature difference diff_curblock mentioned above, the average motion feature difference of each block image in the current image frame, minisValue_nextFrame, is calculated as follows:

[0065]

[0066] Where block_num is the number of block images, and the average value of the motion feature difference of each block image is used as the motion feature difference of the current image frame.

[0067] It should be noted that in the embodiments of the present invention, when the mean of multiple motion feature values ​​is involved, the mean of the multiple motion feature values ​​is directly obtained by averaging the multiple motion feature values. In the specific implementation process, the multiple motion feature values ​​can also be weighted and then averaged according to the application scenario. The present invention does not limit this.

[0068] Step S104: Adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference, and perform video encoding based on the adjusted quantization parameters.

[0069] According to one embodiment of the present invention, adjusting the quantization parameters of the image frame based on the motion feature difference includes: adjusting the quantization parameters of each block image included in the image frame based on the motion feature difference of the block image, so as to adjust the quantization parameters of the image frame.

[0070] Specifically, in this embodiment of the invention, the quantization parameters of the current image frame and the next image frame are adjusted by motion feature difference. For the quantization parameter adjustment of the image frame, after the current image frame is processed into blocks, the corresponding quantization parameter adjustment of the image frame is mainly adjusted by the motion feature difference of the block images, such as the diff_curblock mentioned above.

[0071] According to another embodiment of the present invention, adjusting the quantization parameters of the block image based on the motion feature difference of the block image includes: comparing the motion feature difference of the block image with a preset block parameter threshold, the block parameter threshold including a first threshold, a second threshold, and a third threshold; performing a first adjustment on the quantization parameters of the block image when the motion feature difference of the block image is not greater than the first threshold; performing a second adjustment on the quantization parameters of the block image when the motion feature difference of the image is greater than the first threshold but not greater than the second threshold; and performing a third adjustment on the quantization parameters of the block image when the motion feature difference of the image is greater than the second threshold but not greater than the third threshold.

[0072] According to another embodiment of the present invention, the first adjustment includes a numerical decrease operation according to a preset first threshold adjustment value; the second adjustment includes a numerical increase operation according to a preset second threshold adjustment value; and the third adjustment includes a numerical increase operation according to a preset third threshold adjustment value.

[0073] Specifically, the motion feature difference of the current image frame's block image is compared with a preset block parameter threshold. The block parameter threshold is obtained from empirical analysis and usually includes three thresholds. Using the three thresholds and actual data characteristics, three numerical regions can be determined. The numerical region into which the motion feature difference of the block image falls is judged. The preset block parameter threshold can usually be expressed as [a, b, c]. The corresponding three numerical regions are values ​​not greater than a, values ​​greater than a and not greater than b, and values ​​greater than b and not greater than c, respectively. In this embodiment of the invention, the preset block parameter threshold is [a = 0, b = 0.5, c = 1.0]. Based on the motion intensity represented by the numerical region into which the motion feature difference of the block image falls, different adjustments to the block image quantization parameters are determined. In this embodiment of the invention, if the motion feature difference of the current block image is in a value range no greater than 'a', the current block image is considered a non-motion region, i.e., a region sensitive to the human eye. The quantization parameter corresponding to the current block image is then reduced by a factor of 0.5 to improve quantization precision and enhance image quality. If the motion feature difference of the current block image is in a value range greater than 'a' and no greater than 'b', the current block image is considered a moderately motion region, i.e., a region weakly sensitive to the human eye. The quantization parameter corresponding to the current block image is then increased by a factor of 0.5 to reduce quantization precision without being perceived by the human eye. If the motion feature difference of the current block image is in a value range greater than 'b' and no greater than 'c', the current block image is considered a strongly motion region, i.e., a region insensitive to the human eye. The quantization parameter corresponding to the current block image is then increased by a factor of 0.75 to further reduce quantization precision without being perceived by the human eye.

[0074] According to another embodiment of the present invention, before adjusting the quantization parameters of the next image frame of the image frame based on the motion feature difference, the method further includes: determining that the image frame is not the last frame of the video to be encoded.

[0075] Specifically, for the last frame in the image frame sequence, since there is no next image frame after the last image, the quantization parameter adjustment for the next image frame needs to confirm that the current image frame is not the last frame of the video to be encoded. Only image frames that are not the last frame can have their quantization parameters adjusted for the next image frame.

[0076] According to another embodiment of the present invention, adjusting the quantization parameters of the next image frame based on the motion feature difference includes: comparing the motion feature difference of the image frame with a preset frame parameter threshold, wherein the frame parameter threshold includes a fourth threshold, a fifth threshold, and a sixth threshold; performing a fourth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is not greater than the fourth threshold; performing a fifth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fourth threshold but not greater than the fifth threshold; and performing a sixth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fifth threshold but not greater than the sixth threshold.

[0077] According to another embodiment of the present invention, the fourth adjustment includes a numerical decrease operation according to a preset fourth threshold adjustment value; the fifth adjustment includes a numerical decrease operation according to a preset fifth threshold adjustment value; and the sixth adjustment includes a numerical increase operation according to a preset sixth threshold adjustment value.

[0078] Specifically, the calculated motion feature difference of the image frame, such as minisValue_nextFrame mentioned above, is used to adjust the quantization parameters of the next image frame. Based on the adjustment experience value of the next image frame, the preset frame parameter threshold includes three thresholds. Using the three thresholds and actual data characteristics, three numerical regions can be determined. The numerical region into which the motion feature difference of the image frame falls is judged. Usually, the preset block parameter threshold can be represented as [e, f, g]. The corresponding three numerical regions are values ​​not greater than e, values ​​greater than e and not greater than f, and values ​​greater than f and not greater than g, respectively. In this embodiment of the invention, the preset block parameter threshold is [e = 0, f = 0.01, g = 0.05]. The quantization parameters of the next image frame are adjusted according to the numerical region into which the motion feature difference of the image frame falls, ensuring that the quality of the next image frame is purposefully and adaptively adjusted while controlling the bitrate of the video encoding and improving the stability of bitrate fluctuations. In this embodiment of the invention, the motion feature difference of the current image frame is likely to fall within a numerical region no greater than e and a numerical region greater than e but no greater than f, with a higher probability of falling within the region greater than e but no greater than f. To improve the image quality of the next image frame and enhance its quality as a reference frame for subsequent image frames, when the motion feature difference of the current image frame is in the region no greater than e, the quantization parameter corresponding to the next image frame is reduced by a value of 0.5 to improve the quality of the next image frame and increase the bit rate. Correspondingly, when the motion feature difference of the current image frame is in the region greater than e but no greater than f, i.e., a high-probability event, the quantization parameter corresponding to the next image frame is reduced by a value of 0.75 to further improve the quality of the next image frame and increase the bit rate. When the probability of the motion feature difference of the current image frame falling within the region greater than f but no greater than g is very small, to balance the bit rate control and ensure smooth and stable bit rate fluctuations, the value of the next image frame in this case is increased by a value of 1.0 to reduce the overall stability of the image encoding bit rate.

[0079] By adaptively adjusting the quantization parameters based on the difference in motion features, the detection of moving objects using the differences in the YUV color space is achieved, and the encoding quantization parameters are adjusted according to the quantization values ​​of the differences. This overcomes the video encoding problem caused by the similarity in color between the captured object and the background, improves video quality, and optimizes subjective quality.

[0080] Figure 2This is a schematic diagram illustrating the principle of video encoding in an embodiment of the present invention. The video system sends the acquired image frames 1 to n to the video encoding module. The video encoding module sorts the received n image frames according to the encoding order, obtaining an ordered sequence of image frames to be encoded: image frame 1, image frame 2, image frame 3, image frame 4, ..., image frame n. Taking image frame 3 as an example, image frame 3 is the current image frame. The previous image frame (image frame 2) of image frame 3 is selected as at least one image frame before the current image frame in this embodiment of the present invention. The image frames are divided into blocks according to a set image block size, and the motion feature value minisVal of the block image of the current image frame 3 is calculated. The motion feature value of the block image in image frame 3 is obtained by subtracting the motion feature value of the block image from the motion feature value of the previous image frame (Avg_minisValue_preFrame). Then, the average value of the motion feature difference of each block image in image frame 3 is used as the motion feature difference value corresponding to image frame 3. Finally, the quantization parameters of this block image are adjusted according to the motion feature difference value of the block image in image frame 3. The quantization parameters of the next image frame 4 are adjusted according to the motion feature difference value corresponding to image frame 3 to encode the video image and perform subsequent decoding and output.

[0081] Figure 3 This is a schematic diagram of the main modules of a video encoding apparatus according to an embodiment of the present invention. Figure 3 As shown, the video encoding device 300 mainly includes an image frame acquisition module 301, a motion feature value acquisition module 302, a motion feature difference acquisition module 303, and a snapshot module 304.

[0082] The image frame acquisition module 301 is used to sequentially acquire at least one ordered image frame from the video to be encoded.

[0083] The motion feature value acquisition module 302 is used to calculate the motion feature value of each image frame based on the pixel data of the image frame.

[0084] The motion feature difference acquisition module 303 is used to obtain the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame.

[0085] The quantization parameter adjustment module 304 is used to adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference, and to perform video encoding based on the adjusted quantization parameters.

[0086] According to an embodiment of the present invention, the video encoding apparatus 300 further includes a first frame determination module (not shown in the figure), configured to: determine whether the image frame is the first frame of the video to be encoded before obtaining the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame; and, if the image frame is the first frame of the video to be encoded, use the motion feature value of the image frame as the motion feature difference corresponding to the image frame.

[0087] According to another embodiment of the present invention, the motion feature difference acquisition module 303 is further configured to: calculate the average motion feature of at least one image frame before the image frame based on the motion feature value of each first image frame in at least one image frame before the image frame, wherein the motion feature value of each first image frame is calculated based on the pixel data of each first image frame; and obtain the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the average motion feature of at least one image frame before the image frame.

[0088] According to another embodiment of the present invention, the motion feature value acquisition module 302 is used to: calculate the motion feature value of the image frame based on the luminance pixels and chrominance pixels of the image frame.

[0089] According to another embodiment of the present invention, the motion feature value acquisition module 302 is further configured to: divide the image frame into blocks according to a set image block size to obtain at least one block image; calculate the motion feature value of the block image based on the pixel data of the block image for each block image; and use the average value of the motion feature values ​​of each block image as the motion feature value of the image frame; correspondingly, the motion feature difference acquisition module 303 is further configured to: calculate the motion feature difference of the block image for each block image included in the image frame based on the motion feature value of the block image and the motion feature value of at least one image frame preceding the image frame; and use the average value of the motion feature difference of each block image as the motion feature difference corresponding to the image frame; correspondingly, the quantization parameter adjustment module 304 is further configured to: adjust the quantization parameters of the block image for each block image included in the image frame based on the motion feature difference of the block image, so as to adjust the quantization parameters of the image frame.

[0090] According to another embodiment of the present invention, the quantization parameter adjustment module 304 is further configured to: compare the motion feature difference of the block image with a preset block parameter threshold, the block parameter threshold including a first threshold, a second threshold and a third threshold; perform a first adjustment on the quantization parameter of the block image when the motion feature difference of the block image is not greater than the first threshold; perform a second adjustment on the quantization parameter of the block image when the motion feature difference of the image is greater than the first threshold and not greater than the second threshold; and perform a third adjustment on the quantization parameter of the block image when the motion feature difference of the image is greater than the second threshold and not greater than the third threshold.

[0091] According to another embodiment of the present invention, the first adjustment includes a numerical decrease operation according to a preset first threshold adjustment value; the second adjustment includes a numerical increase operation according to a preset second threshold adjustment value; and the third adjustment includes a numerical increase operation according to a preset third threshold adjustment value.

[0092] According to another embodiment of the present invention, the video encoding apparatus 300 further includes a last frame determination module (not shown in the figure), configured to: determine that the image frame is not the last frame of the video to be encoded before adjusting the quantization parameters of the next image frame of the image frame based on the motion feature difference.

[0093] According to another embodiment of the present invention, the quantization parameter adjustment module 304 is further configured to: compare the motion feature difference of the image frame with a preset frame parameter threshold, the frame parameter threshold including a fourth threshold, a fifth threshold and a sixth threshold; perform a fourth adjustment on the quantization parameter of the next image frame when the motion feature difference of the image frame is not greater than the fourth threshold; perform a fifth adjustment on the quantization parameter of the next image frame when the motion feature difference of the image frame is greater than the fourth threshold but not greater than the fifth threshold; and perform a sixth adjustment on the quantization parameter of the next image frame when the motion feature difference of the image frame is greater than the fifth threshold but not greater than the sixth threshold.

[0094] According to another embodiment of the present invention, the fourth adjustment includes a numerical decrease operation according to a preset fourth threshold adjustment value; the fifth adjustment includes a numerical decrease operation according to a preset fifth threshold adjustment value; and the sixth adjustment includes a numerical increase operation according to a preset sixth threshold adjustment value.

[0095] Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied.

[0096] like Figure 4As shown, system architecture 400 may include terminal devices 401, 402, and 403, a network 404, and a server 405. Network 404 serves as the medium for providing communication links between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0097] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications, such as video encoding applications, can be installed on terminal devices 401, 402, and 403 (for example only).

[0098] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0099] Server 405 can be a server providing various services, such as a backend management server supporting video encoding performed by users using terminal devices 401, 402, and 403 (for example only). The backend management server can sequentially acquire at least one ordered image frame from the video to be encoded; for each image frame, calculate the motion feature value of the image frame based on its pixel data; obtain the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame; adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference; perform video encoding and other processing based on the adjusted quantization parameters; and feed back the processing results (e.g., encoded data – for example only) to the terminal device.

[0100] It should be noted that the video encoding method provided in the embodiments of the present invention is generally executed by server 405, and correspondingly, the video encoding device is generally located in server 405.

[0101] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0102] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing terminal devices or servers of the present invention. Figure 5 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0103] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0104] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0105] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0106] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] The units described in the embodiments of the present invention can be implemented in software or in hardware. The described units can also be housed in a processor; for example, a processor can be described as including: an image frame acquisition module, a motion feature value acquisition module, a motion feature difference acquisition module, and a quantization parameter adjustment module.

[0109] In some cases, the names of these modules do not constitute a limitation on the module itself. For example, the image frame acquisition module can also be described as "a module for sequentially acquiring at least one ordered image frame from the video to be encoded".

[0110] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to: sequentially acquire at least one ordered image frame comprising a video to be encoded; for each image frame, calculate a motion feature value of the image frame based on the pixel data of the image frame; obtain a motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame; adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference, and perform video encoding based on the adjusted quantization parameters.

[0111] According to the technical solution of the present invention, the following advantages or beneficial effects are achieved: by sequentially acquiring at least one ordered image frame including the video to be encoded; calculating the motion feature value of each image frame based on the pixel data of the image frame; obtaining the motion feature difference of the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame; adjusting the quantization parameters of the image frame and the next image frame based on the motion feature difference; and performing video encoding based on the adjusted quantization parameters. This invention realizes that based on the acquired ordered image frames, the motion feature value of the image frame calculated from the pixel data and the motion feature value of at least one image frame preceding the image frame are used to obtain the motion feature difference of the image frames. The quantization parameters of the current image frame and the next image frame are adjusted based on the motion feature difference. The present invention performs video encoding in moving object detection scenarios by using the motion feature difference of image frames, which can effectively solve the video encoding problem caused by the similarity of color between the captured object and the background, and improve video quality.

[0112] The specific embodiments described herein do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video encoding method, characterized in that, include: Sequentially acquire at least one ordered image frame from the video to be encoded; For each image frame, calculate the motion feature value of the image frame based on the pixel data of the image frame; The motion feature difference value corresponding to the image frame is obtained based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame. The quantization parameters of the image frame and the next image frame are adjusted based on the motion feature difference, and video encoding is performed based on the adjusted quantization parameters, including: The motion feature difference of the image frame is compared with preset frame parameter thresholds, including a fourth threshold, a fifth threshold, and a sixth threshold. If the motion feature difference of the image frame is not greater than the fourth threshold, a fourth adjustment is made to the quantization parameters of the next image frame. If the motion feature difference of the image frame is greater than the fourth threshold but not greater than the fifth threshold, a fifth adjustment is made to the quantization parameters of the next image frame. If the motion feature difference of the image frame is greater than the fifth threshold but not greater than the sixth threshold, a sixth adjustment is made to the quantization parameters of the next image frame. The fourth adjustment includes decreasing the value according to a preset fourth threshold adjustment value; the fifth adjustment includes decreasing the value according to a preset fifth threshold adjustment value; and the sixth adjustment includes increasing the value according to a preset sixth threshold adjustment value.

2. The method according to claim 1, characterized in that, Before obtaining the motion feature difference value corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame, the method further includes: Determine whether the image frame is the first frame of the video to be encoded; Furthermore, when the image frame is the first frame of the video to be encoded, the motion feature value of the image frame is used as the motion feature difference corresponding to the image frame.

3. The method according to claim 1, characterized in that, The motion feature difference value corresponding to the image frame is obtained based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame, including: Based on the motion feature values ​​of each first image frame in at least one image frame preceding the image frame, calculate the average motion feature value of at least one image frame preceding the image frame, wherein the motion feature value of each first image frame is calculated based on the pixel data of each first image frame; The motion feature difference value corresponding to the image frame is obtained based on the motion feature value of the image frame and the average motion feature value of at least one previous image frame.

4. The method according to claim 1, characterized in that, Calculating the motion feature values ​​of the image frame based on the pixel data of the image frame includes: The motion feature values ​​of the image frame are calculated based on the luminance and chrominance pixels of the image frame.

5. The method according to claim 1, characterized in that, Calculating the motion feature values ​​of the image frame based on the pixel data of the image frame includes: The image frame is divided into blocks according to the set image block size to obtain at least one block image; For each block image, calculate the motion feature value of the block image based on the pixel data of the block image; The average value of the motion feature values ​​of each block image is used as the motion feature value of the image frame. The motion feature difference value corresponding to the image frame is obtained based on the motion feature value of the image frame and the motion feature value of at least one preceding image frame, including: For each block image included in the image frame, a motion feature difference of the block image is calculated based on the motion feature value of the block image and the motion feature value of at least one image frame preceding the image frame; The average value of the motion feature difference of each block image is used as the motion feature difference corresponding to the image frame; Adjusting the quantization parameters of the image frame based on the motion feature difference includes: For each block image included in the image frame, the quantization parameters of the block image are adjusted according to the motion feature difference of the block image, so as to adjust the quantization parameters of the image frame.

6. The method according to claim 5, characterized in that, The quantization parameters of the block image are adjusted based on the motion feature difference of the block image, including: The motion feature difference of the block image is compared with a preset block parameter threshold, which includes a first threshold, a second threshold, and a third threshold. If the difference in motion features of the block image is not greater than the first threshold, the quantization parameters of the block image are adjusted for the first time. If the motion feature difference of the image is greater than the first threshold but not greater than the second threshold, the quantization parameters of the block image are adjusted in a second way. If the difference in motion features of the image is greater than the second threshold but not greater than the third threshold, the quantization parameters of the block image are adjusted in a third way.

7. The method according to claim 6, characterized in that, The first adjustment includes reducing the value according to a preset first threshold adjustment value; The second adjustment includes increasing the value according to a preset second threshold adjustment value; The third adjustment includes increasing the value according to a preset third threshold adjustment value.

8. The method according to claim 1, characterized in that, Before adjusting the quantization parameters of the next image frame based on the motion feature difference, the method further includes: It is determined that the image frame is not the last frame of the video to be encoded.

9. A video encoding apparatus, characterized in that, include: The image frame acquisition module is used to sequentially acquire at least one ordered image frame from the video to be encoded. The motion feature value acquisition module is used to calculate the motion feature value of each image frame based on the pixel data of the image frame. The motion feature difference acquisition module is used to obtain the motion feature difference corresponding to the image frame based on the motion feature value of the image frame and the motion feature value of at least one image frame preceding the image frame. A quantization parameter adjustment module is used to adjust the quantization parameters of the image frame and the next image frame based on the motion feature difference, and to perform video encoding based on the adjusted quantization parameters. The module includes: comparing the motion feature difference of the image frame with preset frame parameter thresholds, where the frame parameter thresholds include a fourth threshold, a fifth threshold, and a sixth threshold; performing a fourth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is not greater than the fourth threshold; performing a fifth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fourth threshold but not greater than the fifth threshold; and performing a sixth adjustment on the quantization parameters of the next image frame when the motion feature difference of the image frame is greater than the fifth threshold but not greater than the sixth threshold. The fourth adjustment includes decreasing the value according to a preset fourth threshold adjustment value; the fifth adjustment includes decreasing the value according to a preset fifth threshold adjustment value; and the sixth adjustment includes increasing the value according to a preset sixth threshold adjustment value.

10. A mobile electronic device terminal, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

Images