Video encoding methods, apparatus, electronic devices and storage media
By performing scene switching detection and motion scene scaling factor calculation on the video, the estimated quantization parameters of the video encoding are adjusted, solving the problem of untimely bitrate control in the existing technology and improving the real-time performance and quality of video encoding.
Patent Information
- Application Number
- CN202310487481.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-04-28
AI Technical Summary
Existing video coding algorithms often fail to control bitrate promptly when switching from simple or static scenes to complex or motion-dependent scenes, resulting in poor coding performance.
By performing scene switching detection on the video to be encoded, the complexity and scene type of the initial encoded image are obtained, the motion scene scaling factor is calculated, and the bit rate factor is adjusted based on the estimated quantization parameters until the bit consumption deviation is within the preset range, thus achieving differentiated encoding.
It improves the real-time performance of video encoding and the accuracy of the overall bitrate, ensuring the stability of video quality.
Smart Images

Figure CN118869996B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more specifically, to a video encoding method, apparatus, electronic device, and storage medium. Background Technology
[0002] In recent years, with the rapid development of the internet, the demand for online live video streaming has increased significantly. For users, the video quality directly impacts their experience. Therefore, choosing a suitable video encoding algorithm is crucial for ensuring video quality.
[0003] Existing video encoding algorithms mainly control the bitrate while maintaining image quality by controlling the overall file size after encoding and decoding. However, this algorithm has a significant problem: when switching from simple or static scenes to complex or dynamic scenes, the bitrate control based on user-set encoding parameters and encoding modes may not be timely, resulting in poor encoding performance. Summary of the Invention
[0004] In view of this, embodiments of this application propose a video encoding method apparatus, electronic device, and storage medium, which can improve the encoding performance of videos with multiple scenes.
[0005] In a first aspect, embodiments of this application provide a video encoding method, the method comprising: acquiring an initial encoded image obtained by encoding each image frame in a video to be encoded, and initial encoding information corresponding to each initial encoded image, wherein the initial encoding information includes initial complexity, initial quantization parameters, and initial bitrate factor; performing scene switching detection on the video to be encoded to obtain scene segments included in the video to be encoded and scene types of each image frame included in each scene segment; obtaining motion scene scaling coefficients corresponding to each image frame based on the initial complexity and scene type corresponding to each image frame in the scene segment; and obtaining the initial encoded image based on the initial quantization parameters and initial bitrate factor corresponding to each initial encoded image. The estimated quantization parameters are obtained; based on the motion scene scaling factor corresponding to each of the image frames and the estimated quantization parameters of each of the initial encoded images, the bit consumption deviation is obtained. The bit consumption deviation is used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each of the initial encoded images; if the bit consumption deviation is not within the preset deviation range, the initial bit rate factor is updated, and the process is returned to execute the estimated quantization parameters of the initial encoded images based on the initial quantization parameters and the initial bit rate factor corresponding to each of the initial encoded images, until the bit consumption deviation is within the preset deviation range, and the initial encoded images are encoded based on the estimated quantization parameters of the initial encoded images.
[0006] Secondly, embodiments of this application provide a video encoding apparatus, comprising: a data acquisition module, configured to acquire initial encoded images obtained by encoding each image frame in a video to be encoded, and initial encoding information corresponding to each initial encoded image, wherein the initial encoding information includes initial complexity, initial quantization parameters, and initial bitrate factor; a scaling factor acquisition module, configured to perform scene switching detection on the video to be encoded, obtain scene segments included in the video to be encoded and scene types of each image frame included in each scene segment, and obtain motion scene scaling factors corresponding to each image frame based on the initial complexity and scene type corresponding to each image frame in the scene segment; and a quantization parameter acquisition module, configured to... The initial quantization parameters and initial bit rate factor corresponding to each initial encoded image are used to obtain the estimated quantization parameters of the initial encoded image; the deviation acquisition module is used to obtain the bit consumption deviation based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image, the bit consumption deviation being used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each initial encoded image; the encoding module is used to update the initial bit rate factor if the bit consumption deviation is not within a preset deviation range; if the bit consumption deviation is within the preset deviation range, the initial encoded image is encoded based on the estimated quantization parameters of the initial encoded image.
[0007] In one possible implementation, the scaling factor acquisition module includes: a scene detection submodule, a motion analysis submodule, a complexity calculation submodule, and a scaling factor calculation submodule. The scene detection submodule is used to perform scene switching detection on the video to be encoded, obtaining multiple scene segments included in the video to be encoded; the motion analysis submodule is used to perform motion analysis on each image frame within each scene segment, obtaining the scene type of each image frame, where the scene type includes a slow motion type or a non-slow motion type; the complexity calculation submodule is used to obtain a first complexity based on the initial complexity corresponding to each image frame corresponding to the slow motion type within the same scene segment, and a second complexity based on the initial complexity corresponding to the image frames corresponding to the non-slow motion type within the same scene segment; the scaling factor calculation submodule is used to obtain the motion scene scaling factor corresponding to each image frame in the scene segment based on the first complexity and the second complexity within the same scene segment.
[0008] In one possible implementation, the scaling factor calculation submodule is further configured to perform exponential calculation using the first complexity as the base and the second complexity as the exponent to obtain a calculation result. If the calculation result is between a first preset value and a second preset value, the calculation result is determined as the motion scene scaling factor. If the calculation result is greater than the first preset value, the first preset value is determined as the motion scene scaling factor. If the calculation result is less than the second preset value, the second preset value is determined as the motion scene scaling factor, wherein the first preset value is greater than the second preset value.
[0009] In one possible implementation, the encoding method used to encode each image frame in the video to be encoded to obtain the initial encoded image is YUV encoding. Each image frame in the video to be encoded has a color component, a luminance component, or a chrominance component. The initial encoding information also includes the complexity corresponding to each component and the initial bit consumption corresponding to each component. The device further includes: a complexity ratio calculation module, a bit consumption ratio calculation submodule, and a parameter offset calculation submodule. The complexity ratio calculation module is used to obtain the complexity ratio of each component in the same scene segment based on the complexity corresponding to each component in each image frame. The bit consumption ratio calculation submodule is used to obtain the bit consumption ratio of each component in the same scene segment based on the initial bit consumption corresponding to each component in the initial encoded image corresponding to the same scene segment. The parameter offset calculation submodule is used to obtain the quantization parameter offset of each initial encoded image in each component of the same scene segment based on the complexity ratio and bit consumption ratio of each component. The encoding module is also used to encode the initial encoded image based on the estimated quantization parameters of the initial encoded image and the quantization parameter offset of the initial encoded image in each component.
[0010] In one possible implementation, the parameter offset calculation submodule is further configured to, for each target component, if the complexity ratio of the luminance component within the same scene segment is greater than a third preset value, and the bit consumption ratio of the target component within the scene segment is less than a fourth preset value, then obtain the quantization parameter offset of the target component within the scene segment based on the bit consumption ratio of the target component within the scene segment, wherein the target component is any one of the color component, luminance component, or chrominance component in the initial encoded image; if the complexity ratio of the luminance component within the same scene segment is not greater than the third preset value, or the bit consumption ratio of the target component within the scene segment is not less than the fourth preset value, then obtain a fifth preset value as the quantization parameter offset of each component within the scene segment.
[0011] In one possible implementation, the initial encoding information further includes the initial bits and the intra-frame estimated optimal block ratio of the initial encoded image. The quantization parameter acquisition module includes a complexity acquisition submodule and a quantization parameter acquisition submodule. The complexity acquisition submodule is used to obtain the estimated complexity of each initial encoded image in each scene segment based on the initial quantization parameters, initial complexity, initial bits, and intra-frame estimated optimal block ratio of each initial encoded image in the same scene segment. The quantization parameter acquisition submodule is used to obtain the estimated quantization parameters of each initial encoded image in each scene segment based on the estimated complexity and initial bitrate factor of each initial encoded image in the same scene segment.
[0012] In one possible implementation, a complexity calculation submodule is used to obtain the average quantization parameter of each video to be encoded in the scene segment based on the initial quantization parameter corresponding to each initial encoded image in each scene segment; multiply the average quantization parameter by the square of the difference between the preset parameter and the intra-frame estimated optimal block ratio corresponding to each initial encoded image, and the initial bit corresponding to the initial encoded image to obtain the complexity calculation value corresponding to each initial encoded image; for a target initial encoded image, using the target initial encoded image as the Gaussian distribution center, obtain the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images; and according to the complexity calculation value corresponding to each initial encoded image, and the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images, obtain the estimated complexity of the target initial encoded image, wherein the target initial encoded image is any one of multiple initial encoded images.
[0013] In one possible implementation, the initial quantization parameters further include initial bits. The deviation acquisition module includes a predicted bit acquisition submodule, an actual bit acquisition submodule, and a deviation acquisition submodule. The predicted bit acquisition submodule is used to obtain the predicted bit consumption of the video to be encoded based on the initial quantization parameters, the predicted quantization parameters, and the initial bits corresponding to each of the initial encoded images. The actual bit acquisition submodule is used to obtain the actual bit consumption of the video to be encoded based on the target bitrate, the video frame rate of the video to be encoded, the number of image frames in the video to be encoded, and the motion scene scaling factor corresponding to each image frame. The deviation acquisition submodule is used to obtain the bit consumption deviation based on the actual bit consumption and the predicted bit consumption.
[0014] In one possible implementation, the estimated bit acquisition submodule is further configured to obtain the predicted bits corresponding to the initial encoded image based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to the initial encoded image; and to accumulate the predicted bits corresponding to each initial encoded image to obtain the estimated bit consumption of the video to be encoded.
[0015] In one possible implementation, the bit estimation submodule is further configured to obtain a first bit of the initial encoded image based on the initial quantization parameters and initial complexity of the initial encoded image; obtain a second bit of the initial encoded image based on the initial quantization parameters, the estimated quantization parameters, and the initial bit; and select the larger bit between the first bit and the second bit of the initial encoded image as the prediction bit of the initial encoded image.
[0016] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above-described method.
[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run by a processor.
[0018] Fifthly, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device retrieves the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described above.
[0019] This application provides a video encoding method, apparatus, electronic device, and storage medium. The method includes: performing scene switching detection on the video to be encoded; obtaining motion scene scaling factors corresponding to each image frame based on the initial complexity and scene type of each image frame in the detected scene segment; obtaining estimated quantization parameters of each initial encoded image based on the initial quantization parameters corresponding to each initial encoded image; obtaining bit consumption deviation based on the motion scene scaling factors and estimated quantization parameters corresponding to each image frame; if the bit consumption deviation is not within a preset deviation range, updating the initial bitrate factor and returning the estimated quantization parameters of the deviated initial encoded image obtained based on the initial bitrate factor corresponding to each initial encoded image, until the bit consumption deviation is within the preset deviation range, and encoding the initial encoded image based on the estimated quantization parameters of the initial encoded image. By adopting the above method, motion scene scaling factors can be set for image frames in each scene included in the video to be encoded. Based on the motion scene scaling factors, more accurate estimated quantization parameters can be configured for each initial encoded image in the video to be encoded. This enables differentiated encoding when encoding based on the estimated quantization parameters of each initial encoded image, ensuring video quality. In addition, the estimated quantization parameters used in the encoding process are within a preset deviation range when the bit consumption deviation is within the preset deviation range. Encoding the initial encoded image based on the estimated quantization parameters of the initial encoded image can effectively improve the accuracy of the overall bit rate and the real-time performance of the encoding. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This illustration shows an application scenario diagram of a video encoding method provided in an embodiment of this application;
[0022] Figure 2 A flowchart illustrating a video encoding method provided in an embodiment of this application is shown;
[0023] Figure 3 A schematic diagram of a scene switching detection method provided in an embodiment of this application is shown;
[0024] Figure 4 It shows Figure 2 A flowchart illustrating step S120;
[0025] Figure 5 It shows Figure 2 A flowchart illustrating step S130;
[0026] Figure 6 It shows Figure 2 A flowchart illustrating step S140;
[0027] Figure 7 This paper illustrates another flowchart of a video encoding method provided in an embodiment of this application;
[0028] Figure 8 A flowchart of a video encoding method provided in an embodiment of this application is shown;
[0029] Figure 9 This diagram shows a connection block diagram of a video encoding apparatus provided in an embodiment of this application;
[0030] Figure 10 A structural block diagram of an electronic device for performing the methods of embodiments of this application is shown. Detailed Implementation
[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.
[0032] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0033] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0034] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0035] It should be noted that "multiple" as mentioned in this article refers to two or more.
[0036] Before proceeding with specific explanations, the terms used in this application are explained as follows:
[0037] The principle of multi-pass encoding: Multi-pass encoding involves encoding the video multiple times. Before each encoding pass, information from the previous pass, such as bit consumption, quantization parameters, and video complexity (SATD), is used to pre-calculate the bit consumption for each frame. Correction operations are then performed during the actual encoding process. Multi-pass encoding makes the bitrate control model more accurate, improving the compression ratio of the original video while minimizing bitrate control errors. Since multi-pass encoding can be 2-pass or 3-pass, more passes result in a more even distribution of bits across the video, approaching the optimal level. However, the extra time required is unacceptable. In practical applications, 2-pass encoding is generally used.
[0038] The first encoding process in a two-pass encoding method can be the same as ABR (Average Bitrate Control) encoding. Simply put, it requires two encoding passes. The first pass scans the entire file, recording statistical information. The second pass uses this statistical information for encoding, which improves encoding quality. Specifically, the first pass writes key information from the encoding process into files (such as bit consumption, quantization parameters, and complexity (SATD) into the initial encoded images obtained after the first pass) for reference in the second pass.
[0039] The quantizer parameter (QP) reflects the degree of spatial detail compression. A smaller value results in finer quantization, higher image quality, and a longer bitstream. A small QP preserves most of the details; a larger QP leads to some detail loss, lower bitrate, but also increased image distortion and reduced quality.
[0040] The bitrate factor is the encoder's quality (and bitrate control) setting. It can range from 0 to 51; lower values result in better quality but also larger output files, while higher values mean a higher compression ratio.
[0041] Complexity represents the level of complexity of an image frame. The image complexity parameter is a parameter in the bitrate control model used to calculate the quantization step size of an image frame. A larger image complexity parameter indicates more complex image texture and other information, requiring a higher bitrate for transmission. The complexity update value is used to update the image complexity parameter dynamically.
[0042] Bitrate refers to the number of bits of data transmitted per unit of time during data transmission, used for encoding and decoding each video frame.
[0043] Frame rate: A measure of the number of frames displayed. The unit of measurement is frames per second (FPS) or "Hertz" (Hz). Generally, FPS is used to describe how many frames are played per second in video, digital graphics, or games, while Hertz is used to describe how many times the screen refreshes its image per second.
[0044] The following describes exemplary applications of the device provided in the embodiments of the present invention for performing the above-described video encoding method. The video encoding method provided in the embodiments of the present invention can be applied to, for example... Figure 1 In the server within the application environment shown.
[0045] Figure 1 This is a diagram illustrating the application environment of a video encoding method in one embodiment. (Refer to...) Figure 1This video encoding method is applied to a video encoding system. The video encoding system includes a first terminal 10, a video distribution server 20, and a second terminal 30. The first terminal 10 and the video distribution server 20 are connected via a network, and the video distribution server 20 is also connected to the second terminal 30 via a network. The first terminal 10 and the second terminal 30 can be desktop terminals or mobile terminals; the mobile terminal can be at least one of a mobile phone, tablet computer, or laptop computer. The video distribution server 20 can be implemented using a standalone server or a server cluster consisting of multiple servers. The video distribution server 20 can also be a cloud server providing basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN. The first terminal 10 can upload a video stream to the video distribution server 20. The video encoder in the video distribution server 20 is used to encode the video stream uploaded by the first terminal 10. The specific encoding process can be as follows: obtaining the initial encoded image obtained by encoding each image frame in the video to be encoded, and the initial encoding information corresponding to each initial encoded image. The initial encoding information includes the initial complexity, initial quantization parameters, and initial bitrate factor; performing scene switching detection on the video to be encoded to obtain the scene segments included in the video to be encoded and the scene type of each image frame included in each scene segment; obtaining the motion scene scaling factor corresponding to each image frame based on the initial complexity and scene type corresponding to each image frame in the scene segment; and obtaining the motion scene scaling factor corresponding to each image frame based on the initial quantization parameters and initial bitrate factor corresponding to each initial encoded image. The initial bitrate factor is used to obtain the estimated quantization parameters of the initial encoded image. Based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image, the bit consumption deviation is obtained. The bit consumption deviation characterizes the difference between the actual bit consumption and the estimated bit consumption for encoding each initial encoded image. If the bit consumption deviation is not within a preset deviation range, the initial bitrate factor is updated, and the process of obtaining the estimated quantization parameters of the initial encoded image based on the initial quantization parameters and initial bitrate factor corresponding to each initial encoded image is repeated until the bit consumption deviation is within the preset deviation range. Then, the initial encoded image is encoded based on the estimated quantization parameters to obtain the bitstream. The bitstream obtained after video encoding is placed in a buffer for buffering and transmitted over the network. The video distribution server 20 can also push the encoded bitstream to the second terminal 30.
[0046] It should be understood that the above application scenarios are merely illustrative. The video encoding method of this invention can be applied to computer devices. The computer device can be an independent physical server or terminal, or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these.
[0047] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0048] Please see Figure 2 , Figure 2 The image shown is a video encoding method provided in an embodiment of this application. This method can be applied to electronic devices, such as... Figure 1 In the scenario shown, the server or terminal device, when executing the video encoding method, specifically performs steps S110 to S150:
[0049] Step S110: Obtain the initial encoded image obtained by encoding each image frame in the video to be encoded, and the initial encoding information corresponding to each initial encoded image.
[0050] The initial encoding information includes the initial complexity, initial quantization parameters, and initial code rate factor.
[0051] When encoding each image frame in the video to be encoded to obtain the initial encoded image, either a fixed bitrate encoding method or an average bitrate encoding method can be used, as long as the main information (initial encoding information) in the encoding process can be collected.
[0052] It should be understood that the above-described encoding of each image frame in the video to be encoded to obtain the initial encoded image is only illustrative. There are many other encoding methods, which will not be elaborated here.
[0053] The video to be encoded mentioned above can be any video that needs to be encoded or transmitted, such as live video, short video, etc.
[0054] Step S120: Perform scene switching detection on the video to be encoded to obtain the scene segments included in the video to be encoded and the scene type of each image frame included in each scene segment. Based on the initial complexity and scene type of each image frame in the scene segment, obtain the motion scene scaling factor corresponding to each image frame.
[0055] One method for detecting scene transitions in the video to be encoded and obtaining scene segments and the scene type of each image frame within those segments can be: 1) Utilizing the pre-analysis computational cost of X265 (an open-source, free software and function library for encoding videos conforming to the High Efficiency Video Coding (HEVC / H.265) standard) to determine scene transitions in the video, simultaneously calculating the motion amplitude of each frame within a single scene, and then determining the scene type of each image frame based on the calculated motion amplitude. 2) Alternatively, deep learning algorithms, such as TransNetV2, can be used for scene transition detection to obtain the scene segments included in the video to be encoded, and the motion vector of each image frame relative to the previous frame can be calculated, with the scene type of each image frame determined based on the calculated motion vector. 3) A third method can be based on a GOP (Group of Pictures) level video scene transition detection algorithm based on the H.264 / AVC compression domain, using related technologies to detect scene transitions, obtain the scene segments included in the video to be encoded, and calculate the motion vector of each image frame relative to the previous frame.
[0056] Please see Figure 3 In one possible implementation of this application, scene estimation can be performed on the video content. Utilizing the mature pre-analysis computation method of X265 (an open-source and free software and function library for encoding videos conforming to the High Efficiency Video Coding (HEVC / H.265) standard), motion analysis is performed on the group of pictures (GOP) to be encoded included in the pre-analysis window. By statistically analyzing the proportion of the maximum movement amplitude obtained from motion estimation of a fixed 16x16 block in each frame that exceeds a threshold, a decision is made as to whether the current group of pictures is in a smooth motion. Each frame of the group of pictures is then marked as to whether it is in a smooth motion scene. The pre-analysis window may also contain already encoded images.
[0057] It should be understood that the scene switching detection method shown above is only illustrative, and the determination of the scene type of each image frame is also only illustrative. There are many other processing methods that can be set according to actual needs.
[0058] The method for obtaining the motion scene scaling factor for each image frame based on the initial complexity and scene type within a scene segment can be as follows: First, the complexity corresponding to the scene type is obtained based on the initial complexity of each image frame belonging to the same scene type within a scene segment, and the motion scene scaling factor is then obtained based on the complexity corresponding to each scene type. Second, the complexity corresponding to the scene type is obtained based on the initial complexity of each image frame belonging to the same scene type within a scene segment, a target complexity is determined from the complexities corresponding to each scene type, and the motion scene scaling factor is obtained based on the target complexity. It should be understood that the above methods for obtaining the motion scene scaling factor for each image frame based on the initial complexity and scene type are merely illustrative; many other methods exist, which will not be elaborated upon here.
[0059] Please see Figure 4 In one possible implementation of this application, step S120 includes:
[0060] Step S122: Perform scene switching detection on the video to be encoded to obtain multiple scene segments included in the video to be encoded.
[0061] Step S124: Perform motion analysis on each image frame within each scene segment to obtain the scene type of each image frame. The scene type includes slow motion type or non-slow motion type.
[0062] One approach to motion analysis within each image frame of a scene segment is to perform motion analysis on each frame and determine whether the current image group is in a smooth motion by calculating the proportion of frames where the maximum MV (motion mean squared error) estimated from a fixed 16x16 block exceeds a threshold. Each frame in the image group is then labeled as belonging to a smooth motion scene. Alternatively, motion analysis can be performed on each image frame to determine whether the average movement of pixels exceeds a threshold, and each frame in the image group is then labeled as belonging to a smooth motion scene. It should be understood that the above motion analysis methods are merely illustrative, and many other analysis methods are possible.
[0063] Step S126: Obtain the first complexity based on the initial complexity of each image frame corresponding to the slow motion type within the same scene segment, and obtain the second complexity based on the initial complexity of the image frames corresponding to the non-slow motion type within the same scene segment.
[0064] Specifically, step S126 can be: averaging the initial complexities of image frames corresponding to slow motion types within the same scene segment to obtain a first complexity, and averaging the initial complexities of image frames corresponding to non-slow motion types within the same scene segment to obtain a second complexity. Alternatively, it can be using the median of the initial complexities of image frames corresponding to slow motion types within the same scene segment as the first complexity, and the median of the initial complexities of image frames corresponding to non-slow motion types within the same scene segment as the second complexity. Another option is to perform a weighted summation based on a Gaussian distribution on the initial complexities of image frames corresponding to slow motion types within the same scene to obtain the first complexity, and to perform a weighted summation based on a Gaussian distribution on the initial complexities of image frames corresponding to non-slow motion types within the same scene to obtain the second complexity. It should be understood that the calculation methods for the complexity of different scene types can be different, and the above-described methods for calculating the complexity corresponding to scene types are merely illustrative; there can be many other methods for calculating complexity.
[0065] Step S128: Based on the first complexity and the second complexity within the same scene segment, obtain the motion scene scaling factor corresponding to each image frame in the scene segment.
[0066] The method for obtaining the motion scene scaling factor corresponding to each image frame in a scene segment based on the first complexity and the second complexity within the same scene segment can be as follows: The scaling factor can be obtained by weighted summation of the first complexity and the second complexity within the same scene segment; alternatively, it can be obtained by the ratio of the first complexity to the second complexity within the same scene segment. It should be understood that the above methods for obtaining the motion scene scaling factor are merely illustrative, and there are many other possible methods.
[0067] In one possible implementation of this application, step S128 may specifically be: using the first complexity as the base and the second complexity as the exponent to perform a power exponent calculation to obtain the calculation result; if the calculation result is between the first preset value and the second preset value, then the calculation result is determined as the motion scene scaling factor; if the calculation result is greater than the first preset value, then the first preset value is determined as the motion scene scaling factor; if the calculation result is less than the second preset value, then the second preset value is determined as the motion scene scaling factor, and the first preset value is greater than the second preset value.
[0068] The first preset value can be 0.35, 0.4, 0.45 or 0.5, etc., and the second preset value can be 0.7, 0.75, 0.8 or 0.85, etc.
[0069] For example, the first preset value is 0.4, and the second preset value is 0.8. That is, in this exemplary embodiment, the average complexity value static_cplxAve corresponding to the slow motion scene within each segment can be obtained using the first complexity calculation formula. k : In the formula, static_cplxSum i N represents the initial complexity corresponding to the i-th image frame. k_static This represents the number of frames in the Kth scene with slow motion. The average complexity of non-slow motion scenes, `normal_cplxAve`, is calculated using the second complexity formula. k :
[0070]
[0071] In the formula, N k_normal This represents the number of frames in the non-slow motion scene within the Kth scene. Then, using the scaling factor calculation formula, the scaling factor for the slow motion scene within each segment is obtained by considering the average complexity of the slow motion scene and the average complexity of the non-slow motion scene within the same segment. The complexity calculation formula is as follows:
[0072] Step S130: Based on the initial quantization parameters and initial code rate factor corresponding to each initial coded image, obtain the estimated quantization parameters of the initial coded image.
[0073] One method for obtaining the estimated quantization parameters based on the initial quantization parameters and initial bitrate is as follows: For the target initial encoded image, the average or weighted sum of the initial quantization parameters corresponding to the target initial encoded image and the initial quantization parameters corresponding to a predetermined number of adjacent initial encoded images is calculated to obtain the target initial quantization parameters of the target initial encoded image. The estimated quantization parameters of the target initial encoded image are then obtained based on the target initial quantization parameters and the initial bitrate factor. Alternatively, for the target initial encoded image, the quantization parameter with the largest value is selected from the initial quantization parameters corresponding to the target initial encoded image and the initial quantization parameters corresponding to a predetermined number of adjacent initial encoded images as the target initial quantization parameters of the target initial encoded image. The estimated quantization parameters of the target initial encoded image are then obtained based on the target initial quantization parameters and the initial bitrate factor. It should be understood that the above methods for obtaining the estimated quantization parameters are merely illustrative, and there are many other ways to obtain them.
[0074] Please see Figure 5Specifically, in one possible implementation, the initial encoding information further includes the intra-frame estimated optimal block proportion and initial complexity of the initial encoded image, and the above step S130 may include:
[0075] Step S132: Based on the initial quantization parameters, initial complexity, initial bits, and the proportion of the optimal block in the intra-frame estimation of each initial encoded image in the same scene segment, obtain the estimated complexity of each initial encoded image in each scene segment.
[0076] Specifically, step S132 can be as follows: A target quantization parameter is obtained based on the initial quantization parameters corresponding to each initial encoded image. This can be achieved by averaging the initial quantization parameters of each initial encoded image or by selecting the initial quantization parameter whose value is the median from the initial quantization parameters of each initial encoded image. Then, based on the target quantization parameter, the initial complexity of each initial encoded image, the initial bits, and the proportion of the optimal block estimated within the frame, the estimated complexity of each initial encoded image is obtained.
[0077] In one possible implementation of this application, step S132 may specifically be: obtaining the average quantization parameter of each video to be encoded in the scene segment based on the initial quantization parameter corresponding to each initial encoded image in each scene segment; multiplying the average quantization parameter by the square of the difference between the preset parameter and the intra-frame estimated optimal block ratio corresponding to each initial encoded image and the initial bit corresponding to the initial encoded image to obtain the complexity calculation value corresponding to each initial encoded image; for the target initial encoded image, obtaining the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images by using the target initial encoded image as the Gaussian distribution center; and obtaining the estimated complexity of the target initial encoded image based on the complexity calculation value corresponding to each initial encoded image and the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images, wherein the target initial encoded image is any one of the multiple initial encoded images.
[0078] It should be understood that the aforementioned average quantization parameter is also the target quantization parameter. Considering that the initial bits generated when an image frame is encoded to obtain the initial encoded image typically include initial coefficient bits (coeffBits), initial motion information bits (mvbits), and other initial bits (miscbits) besides the two, in this application, when multiplying the average quantization parameter by the square of the difference between the preset parameter and the intra-frame estimated optimal block ratio corresponding to each initial encoded image, and the initial bits corresponding to that initial encoded image, to obtain the complexity calculation value corresponding to each initial encoded image, specifically, a target initial bit (Bits) can be obtained based on the initial quality parameter, the average quantization parameter, and the initial coefficient bits (coeffBits), initial motion information bits (mvbits), and other initial bits (miscbits) besides the two. Pred (i,qp ave ), where the target initial bits are Bits Pred (i,qp ave It can be calculated using bit-based formulas: In the formula, qp ave qp is the average quantization parameter. 1pass These are the initial quantization parameters.
[0079] After obtaining the target initial bits, the initial bits can be multiplied by the square of the difference between the preset parameters and the estimated optimal block proportion within the frame corresponding to each initial encoded image to obtain the complexity calculation value for each initial encoded image. Then, for the target initial encoded image, using it as the center of the Gaussian distribution, the Gaussian distribution probabilities of the target initial encoded image and its neighboring encoded images are obtained. Based on the complexity calculation values corresponding to each initial encoded image and the Gaussian distribution probabilities of the target initial encoded image and its neighboring encoded images, the estimated complexity of the target initial encoded image is obtained. The target initial encoded image is any one of multiple initial encoded images. In other words, the process of calculating the estimated complexity is as follows:
[0080]
[0081] Complexity i =cplxSum sum / Weigh t sum
[0082] In the formula, Gauss(j) is a Gaussian function used to perform a weighted average over a sliding window of length 40 (20 frames forward and 20 frames backward), IntraRatio represents the proportion of blocks with better intra-frame estimation in the current frame, and Complexity... i The estimated complexity of the initial encoded image for the i-th frame.
[0083] Step S134: Based on the estimated complexity and initial bit rate factor of each initial encoded image in the same scene segment, obtain the estimated quantization parameters of each initial encoded image in each scene segment.
[0084] Specifically, step S134 can be: processing the estimated complexity of the initial encoded image and dividing it by the initial code rate factor to obtain the estimated quantization parameter qp of the initial encoded image. i For example, the estimated quantization parameter qp can be calculated using the following formula. i ,in, In the formula, rateFactor is the initial bit rate factor, and qcompress is a numerical constant, the specific value of which is set according to actual needs.
[0085] It should be noted that if the current frame type is a P-frame or a GPB-frame, then qp i It refers to the result calculated by the model. If the current frame type is an I-frame or a B-frame, qp i Further calculations revealed the following:
[0086] In the above formula, ipoffset represents the QP interval between I-frames and GPB / P-frames, and pboffset represents the QP interval between B-frames and GPB / P-frames.
[0087] By employing the methods described in steps S132-S134 above to obtain the estimated quantization parameters, the fixed threshold of the base QP used in the complexity calculation during the second encoding is avoided. In 2-pass encoding, the QP of each frame is limited by the target bit rate and may all be greater than or less than the fixed threshold, leading to excessive overall deviation in the predicted QP and reduced accuracy of the code control prediction bits. This approach allows for adaptive adjustment of the base QP in the complexity estimation model using the average QP from one pass of encoding, optimizing complexity estimation and thus optimizing the estimated quantization parameters to improve their accuracy. Therefore, an optimization scheme is used to statistically analyze the average qscale_ave during the 1-pass encoding process to replace the fixed value in the estimated complexity calculation, narrowing the QP adjustment range and improving overall estimation accuracy.
[0088] Step S140: Obtain the bit consumption deviation based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image. The bit consumption deviation is used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each initial encoded image.
[0089] One method to obtain the bit consumption deviation based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image can be as follows: First, obtain an estimated bit consumption based on the estimated quantization parameters corresponding to each frame. Second, obtain the actual bit consumption based on the scene scaling factor corresponding to each image frame. The bit consumption deviation is then obtained based on the difference or ratio between the actual bit consumption and the estimated bit consumption. Alternatively, the estimated bit consumption for encoding each initial image frame within the same scene can be obtained based on the estimated quantization parameters corresponding to each frame. Then, the estimated bit consumption for each scene is accumulated to obtain the bit consumption of the video to be encoded. Finally, the actual bit consumption for encoding each initial image frame within the same scene can be obtained based on the motion scene scaling factor corresponding to each frame. Then, the actual bit consumption for each scene is accumulated to obtain the actual bit consumption of the video to be encoded.
[0090] Please see Figure 6 In one possible implementation of this application, step S140 may specifically include:
[0091] Step S142: Based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to each initial encoded image, obtain the estimated bit consumption of the video to be encoded.
[0092] For example, for each target initial encoded image, the estimated cost bits can be obtained based on the initial quality parameters, the estimated quantization parameters, the initial coefficient bits (coeffBits), the initial motion information bits (mvBits), and other initial bits besides the two (miscBits). Pred (i,qp i The estimated bit cost of the video to be encoded is obtained by summing the estimated bit cost of each initial encoded image. The estimated bit cost of each initial encoded image can be calculated using the following formula:
[0093]
[0094] Considering the potential for bit inaccuracies due to large QP deviations during bit estimation, to avoid such problems, in one possible implementation of this application, a bit estimate can be obtained based on the updated model coefficients of the prediction model after encoding is completed using 1pass, the actual bit output by the encoder, the estimated complexity, and the estimated quantization parameters corresponding to each initial encoded image. Then, a target estimated bit consumption can be determined based on the bit estimate corresponding to the initial encoded image and the estimated bit consumption, to participate in subsequent calculations.
[0095] Specifically, the bit estimate value corresponding to each initial coded image can be calculated using the bit estimate calculation formula. In the formula, satd i The estimated complexity of the i-th frame image is qp i For the estimated quantization parameters of the i-th frame image, coeff update ,offset update count update For 1-pass encoding, the model coefficients are updated based on the actual bits output by the encoder after encoding. When determining a target estimated bit consumption based on the bit estimate corresponding to the initial encoded image and the estimated bit consumption, the following estimation formula can be used to determine the target estimated bit consumption (Bits). clip ,
[0096] By adopting the above method, the current scheme can effectively solve the problem of qp estimation in 2-pass mode. cur With one actual coding qp 1pass When the deviation is large, the estimated bitrate is too small, leading to bitrate runaway problems in extreme cases. The calculation process for the estimated complexity is similar to that of the open-source platform X265, and will not be described in detail here.
[0097] Step S144: Based on the target bitrate, the video frame rate of the video to be encoded, the number of image frames in the video to be encoded, and the motion scene scaling factor corresponding to each image frame, obtain the actual bit consumption of the video to be encoded.
[0098] For example, for each scene segment, the target bitrate, the number of image frames in the video frame to be encoded within that scene segment, and the motion scene scaling factor corresponding to the image frames in that scene segment can be multiplied by the frame rate of the video to be encoded. The actual bits consumed within that scene segment are then summed to obtain the actual bit consumption of the video to be encoded. Specifically, the actual bit consumption (Bits) within the k-th scene segment can be calculated using a formula for calculating actual bit consumption. True-k ,in, In the formula R TThe target bitrate is represented by fps, which represents the video frame rate of the video to be encoded. SeqLen k,i The number of video frames within the k-th scene segment, static k,i is the motion scene scaling factor corresponding to the image frame in the k-th scene segment.
[0099] Step S146: Obtain the bit consumption deviation based on the actual bit consumption and the estimated bit consumption.
[0100] Specifically, the bit consumption deviation mentioned above can be the ratio or difference between the actual bit consumption and the estimated bit consumption, or it can be a parameter obtained based on the actual bit consumption and the estimated bit consumption to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each initial coded image. It can be set according to actual needs.
[0101] In one possible implementation, the bit consumption deviation stepMulti can be calculated using the following formula: In the formula, n is the number of scene segments, and SeqLen k,i Let static be the number of image frames in the k-th scene. k,i is the motion scene scaling factor corresponding to each initial image in the k-th scene.
[0102] If the bit consumption deviation is not within the preset deviation range, execute step S150: update the initial code rate factor and return to execute step S130. If the bit consumption deviation is within the preset deviation range, execute step S160: encode the initial encoded image based on the estimated quantization parameters of the initial encoded image.
[0103] One way to update the initial bitrate factor is to use a dynamic programming process to gradually select the rateFactor (bitrate factor) that meets the conditions from large to small. The rateFactor is incremented by the step size in the for loop. For example, each time a value can be selected from stepMult*10000 to stepMult*1E-7 and added to the initial bitrate factor as the updated initial bitrate factor.
[0104] It should be noted that the initial bitrate factor of each initial encoded image belonging to the same scene segment should be the same, and the updated initial bitrate factor of each initial encoded image belonging to the same scene segment should also be the same. This is to achieve scene segmentation based on scene switching and the motion amplitude of each frame within a single scene, adaptively determining the bitrate factor of each segment, thereby obtaining the estimated quantization parameters faster and more accurately.
[0105] By employing the video encoding method described above in this application, motion scene scaling factors can be set for each image frame in each scene included in the video to be encoded. Based on the motion scene scaling factors, more accurate estimated quantization parameters can be configured for each initial encoded image in the video to be encoded. This enables differentiated encoding when encoding based on the estimated quantization parameters of each initial encoded image, ensuring video quality. Furthermore, the estimated quantization parameters used in the encoding process are within a preset deviation range for bit consumption. Encoding the initial encoded image based on the estimated quantization parameters of the initial encoded image can effectively improve the accuracy of the overall bit rate and the real-time performance of the encoding.
[0106] It should also be noted that when obtaining the estimated quantization parameters for encoding, the number of bits consumed and the allocated bit data for each frame can be estimated again to ensure that the buffer (VBV) requirements are met and encoding is performed. Since the bit data has been pre-allocated for each frame, when there is a difference between the actual encoded bits and the allocated bits, the encoding model needs to be corrected to ensure the accuracy of the overall bit rate.
[0107] Please see Figure 7 Another embodiment of this application provides a video encoding method applicable to electronic devices, the method comprising:
[0108] Step S210: Obtain the initial encoded image obtained by encoding each image frame in the video to be encoded, and the initial encoding information corresponding to each initial encoded image.
[0109] The initial encoding information includes the initial complexity, initial quantization parameters, and initial code rate factor.
[0110] Step S220: Perform scene switching detection on the video to be encoded to obtain the scene segments included in the video to be encoded and the scene type of each image frame included in each scene segment. Based on the initial complexity and scene type of each image frame in the scene segment, obtain the motion scene scaling factor corresponding to each image frame.
[0111] For a detailed description of steps S210-S220 above, please refer to the detailed description of steps S110-S120 above. In this embodiment, they will not be described in detail.
[0112] The encoding method used to encode each image frame in the video to be encoded to obtain the initial encoded image is YUV. Each image frame in the video to be encoded has a color component, a luminance component, or a chrominance component. The initial encoding information also includes the complexity of each component and the initial consumption bits of each component.
[0113] YUV is a color encoding method commonly used in various video processing components. When encoding photos or videos, YUV takes into account human perception, allowing for reduced chroma bandwidth.
[0114] Step S230: Based on the complexity of each component in each image frame of the same scene segment, obtain the complexity ratio of each component in the scene segment.
[0115] Specifically, step S230 can be as follows: First, the initial complexities of the same component within the same scene segment are summed to obtain the calculated complexity value for each component. Then, the complexity percentage of each component within the scene segment is obtained based on the calculated complexity value. Alternatively, the complexity percentage of each component within each image frame can be obtained based on the initial complexity of each image frame and the complexity of each component within that image frame. Then, the complexity percentages of the same component in each image frame within the same scene segment are sorted by size, and the complexity percentage in the middle of the sorted list is selected as the complexity percentage of each component within the scene segment.
[0116] The complexity percentage for each component in each image frame can be calculated using the following formula. In the formula, Width refers to the result obtained by calculating the sum of squared differences between the corresponding image frame (YUV plane) in the j-th component of the i-th frame and the corresponding plane of the reconstructed image, and then opening the result. j This refers to the width of the j-th component in the image frame. The j-th component corresponds to the width of the image frame. The j-th component is identified by 0, 1, and 2, representing the three YUV components in the current frame, respectively.
[0117] For example, the complexity percentage of the j-th component in the K-th scene segment is Sence_Cplx. k,j It can be calculated based on the following formula: In the formula, N k The number of image frames included in the k-th scene segment.
[0118] Step S240: Based on the initial bit consumption of each component in the initial encoded image corresponding to the same scene segment, obtain the bit consumption ratio of each component in the scene segment.
[0119] Step S240 above can be: adding the initial consumption bits belonging to the same component in the same scene segment to obtain the consumption bit calculation value corresponding to each component; and obtaining the bit consumption ratio corresponding to each component in the scene segment based on the consumption bit calculation value corresponding to each component in the same scene segment. Alternatively, it can be based on the initial consumption bits corresponding to each frame and the initial consumption bits corresponding to each component in each frame to obtain the bit consumption ratio corresponding to each component in each frame; sorting the bit consumption ratios corresponding to the same component in each frame of the same scene segment according to their size; and selecting the bit consumption ratio in the middle of the sort as the bit consumption ratio of each component in the scene segment.
[0120] For example, the bit consumption ratio (Ratio) corresponding to the j-th component in the i-th image frame is calculated using the following formula. i,j , In the formula, Bits i,j Let Sj be the initial bit consumption of the j-th component in the i-th image frame. Then, let Sj be the bit consumption ratio of the j-th component in the K-th scene segment. k,j It can be calculated based on the following formula: In the formula, N k The number of image frames included in the k-th scene segment.
[0121] Step S250: Based on the complexity ratio and bit consumption ratio of each component within the same scene segment, obtain the quantization parameter offset of each initial encoded image in each component within the scene segment.
[0122] Specifically, step S250 can be as follows: for each target component, if the complexity ratio of the luminance component in the same scene segment is greater than a third preset value, and the bit consumption ratio of the target component in the scene segment is less than a fourth preset value, then the quantization parameter offset of the target component in the scene segment is obtained based on the bit consumption ratio of the target component in the scene segment, wherein the target component is any one of the color component, luminance component, or chrominance component in the initial encoded image; if the complexity ratio of the luminance component in the same scene segment is not greater than the third preset value, or the bit consumption ratio of the target component in the scene segment is not less than the fourth preset value, then a fifth preset value is obtained as the quantization parameter offset of each component in the scene segment.
[0123] It should be understood that the specific values of the third, fourth, and fifth preset values mentioned above can be set according to actual needs, and are not specifically limited in the embodiments of this application.
[0124] For example, taking the third preset value as 0.18, the fourth preset value as 0.95, and the fifth preset value as 0 as an example, if the proportion of bits consumed and the proportion of complexity in the current scene are less than a certain level, the quantization parameter offset (Scene_QPoffset) of the UV components in the current scene is calculated:
[0125] If the proportion of bits consumed and the proportion of complexity in the current scene are less than a certain level, calculate the quantization parameter offset (Scene_QPoffset) of the UV components in the current scene: Sence_Ratio k,j <thres1 and Sence_Clpx k,0 *thres2>Sence_Clpx k,j
[0126] In the above formula, Sense_Clpx k,0 This refers to the complexity ratio of the luminance component. α and β are thresholds that can be fitted from a large number of experiments. In this embodiment, α can be 1.3, β can be 4, Thres1 is 0.018, and Thres2 is 0.95. After calculating the QP offset Scene_QPoffset using the above formula, in the second encoding process, the quantization offset is superimposed on the calculated luminance quantization step size to encode the chrominance.
[0127] Step S260: Based on the initial quantization parameters and initial code rate factor corresponding to each initial coded image, obtain the estimated quantization parameters of the initial coded image.
[0128] Step S270: Obtain the bit consumption deviation based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image. The bit consumption deviation is used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each initial encoded image.
[0129] If the bit consumption deviation is not within the preset deviation range, execute step S280: update the initial bit rate factor, and return to execute step S260 until the bit consumption deviation is within the preset deviation range, then execute step S290: encode the initial encoded image based on the estimated quantization parameters of the initial encoded image and the quantization parameter offset of the initial encoded image in each component.
[0130] By adopting the video coding method of this application, the scene is segmented according to scene switching and the motion amplitude of each frame within a single scene. When the estimated bit consumption is inconsistent with the actual bit consumption, the bit rate factor is adjusted, thereby improving the 2-pass compression rate. At the same time, based on the segmented scene, the complexity of one-pass coding, and the bit information consumed by the YUV components, adaptive QP coding is adopted for luminance and chrominance, further improving the coding quality.
[0131] Specifically, such as Figure 8 The diagram shown is a flowchart of the video encoding process in this embodiment. During video encoding, the process first obtains the initial encoded images obtained by encoding each image frame in the video to be encoded, along with the initial encoding information corresponding to each initial encoded image. The initial encoding information includes the initial complexity, initial quantization parameters, and initial bitrate factor. Then, complexity optimization can be performed. During complexity optimization, bit correction is performed first. Specifically, the bit estimate value Bits_esti corresponding to each initial encoded image can be calculated using the bit estimation formula. 1pass , In the formula, satd i The estimated complexity of the i-th frame image is qp i For the estimated quantization parameters of the i-th frame image, coeff update ,offset update count update For 1-pass encoding, the model coefficients are updated based on the actual bits output by the encoder after encoding. When determining a target estimated bit consumption based on the bit estimate corresponding to the initial encoded image and the estimated bit consumption, the following estimation formula can be used to determine the target estimated bit consumption (Bits). clip ,
[0132] By adopting the above method, the current scheme can effectively solve the problem of qp estimation in 2-pass mode. cur With one actual coding qp 1pass When the deviation is large, the estimated bit rate is too small, which leads to bit rate runaway in extreme cases, thus completing bit correction.
[0133] After completing the bit correction, QP estimation optimization can be performed to obtain the predicted quantization parameters. During QP estimation optimization, complexity estimation optimization needs to be performed first. That is, the average QP from one pass of encoding (as in S132 and S134 in the aforementioned embodiment) is used to adaptively adjust the base QP in the complexity estimation model, optimizing the complexity estimation (obtaining the predicted complexity). This avoids using a fixed threshold for the base QP in existing 2-pass estimation. In 2-pass encoding, the QP of each frame is limited by the target bit rate and may all be greater than or less than the fixed threshold, leading to excessive overall deviation in the predicted QP (predicted quantization commitment) and reduced accuracy of the code control prediction bits. The optimization scheme uses the average qscale_ave from the 1-pass encoding process to replace the fixed value 1.0 in Formula 2, narrowing the QP adjustment range and improving overall estimation accuracy.
[0134] After completing the complexity estimation optimization to obtain the estimated complexity, the estimated quantization parameter qp can be calculated using the following formula. i ,in, In the formula, rateFactor is the initial bitrate factor, and qcompress is a numerical constant whose specific value is set according to actual needs. It should be noted that if the current frame type is a P-frame or a GPB-frame, then qp... i It refers to the result calculated by the model. If the current frame type is an I-frame or a B-frame, qp i Further calculations revealed the following:
[0135] In the above formula, ipoffset represents the QP interval between I-frames and GPB / P-frames, and pboffset represents the QP interval between B-frames and GPB / P-frames.
[0136] Rate factor optimization is possible. During this optimization process, it's necessary to determine the frame type for each image frame (frame type determination). If the current frame is a P-frame or a GPB-frame, then the qp... i It refers to the result calculated by the model. If the current frame type is an I-frame or a B-frame, qp i Further calculations revealed the following:
[0137] In the above formula, ipoffset represents the QP interval between I-frames and GPB / P-frames, and pboffset represents the QP interval between B-frames and GPB / P-frames.
[0138] Subsequently, the estimated quantization factor is determined based on the frame type, and the actual consumed bits and estimated consumed bits are obtained based on the estimated quantization factor. The bit estimation deviation is then obtained based on the actual consumed bits and estimated consumed bits. If the bit estimation deviation does not meet the conditions, the initial quantization parameters are adjusted, and then the estimated quantization parameters of each image frame are adjusted until the bit estimation deviation meets the conditions (less than the preset deviation value).
[0139] It should be noted that during the bitrate factor optimization process, scene segmentation and QP optimization under YUV encoding can be performed simultaneously. That is, under YUV encoding, the quantization offset parameters for luminance and chrominance are obtained.
[0140] Finally, when the bit prediction deviation meets the condition (less than the preset deviation value), the predicted quantization parameters corresponding to each image frame are output, and encoding is performed based on the predicted quantization parameters corresponding to each image frame and the quantization offset parameters in brightness and chrominance.
[0141] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.
[0142] Figure 9 This is a video encoding apparatus 300 according to one embodiment. The apparatus 300 includes: a data acquisition module 310, used to acquire initial encoded images obtained by encoding each image frame in the video to be encoded, and initial encoding information corresponding to each initial encoded image, the initial encoding information including initial complexity, initial quantization parameters, and initial bitrate factor; a scaling factor acquisition module 320, used to perform scene switching detection on the video to be encoded, obtain scene segments included in the video to be encoded and scene types of each image frame included in each scene segment, and obtain motion scene scaling factors corresponding to each image frame based on the initial complexity and scene type corresponding to each image frame in the scene segment; and a quantization parameter acquisition module 320. Module 330 is used to obtain the estimated quantization parameters of the initial encoded image based on the initial quantization parameters and initial bit rate factor corresponding to each initial encoded image; the deviation acquisition module 340 is used to obtain the bit consumption deviation based on the motion scene scaling factor corresponding to each image frame and the estimated quantization parameters of each initial encoded image, the bit consumption deviation being used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption for encoding each initial encoded image; the encoding module 350 is used to update the initial bit rate factor if the bit consumption deviation is not within the preset deviation range; if the bit consumption deviation is within the preset deviation range, the initial encoded image is encoded based on the estimated quantization parameters of the initial encoded image.
[0143] In one possible implementation, the scaling factor acquisition module 320 includes: a scene detection submodule, a motion analysis submodule, a complexity calculation submodule, and a scaling factor calculation submodule. The scene detection submodule is used to perform scene switching detection on the video to be encoded, obtaining multiple scene segments included in the video to be encoded; the motion analysis submodule is used to perform motion analysis on each image frame within each scene segment, obtaining the scene type of each image frame, where the scene type includes slow motion or non-slow motion; the complexity calculation submodule is used to obtain a first complexity based on the initial complexity corresponding to each image frame corresponding to the slow motion type within the same scene segment, and a second complexity based on the initial complexity corresponding to the image frames corresponding to the non-slow motion type within the same scene segment; the scaling factor calculation submodule is used to obtain the motion scene scaling factor corresponding to each image frame in the scene segment based on the first and second complexities within the same scene segment.
[0144] In one possible implementation, the scaling factor calculation submodule is further configured to perform exponential calculation using the first complexity as the base and the second complexity as the exponent to obtain the calculation result. If the calculation result is between the first preset value and the second preset value, the calculation result is determined as the motion scene scaling factor. If the calculation result is greater than the first preset value, the first preset value is determined as the motion scene scaling factor. If the calculation result is less than the second preset value, the second preset value is determined as the motion scene scaling factor, and the first preset value is greater than the second preset value.
[0145] In one possible implementation, the encoding method used to obtain the initial encoded image by encoding each image frame in the video to be encoded is YUV. Each image frame in the video to be encoded has a color component, a luminance component, or a chrominance component. The initial encoding information also includes the complexity corresponding to each component and the initial bit consumption corresponding to each component. The device further includes: a complexity ratio calculation module, a bit consumption ratio calculation submodule, and a parameter offset calculation submodule. The complexity ratio calculation module is used to obtain the complexity ratio of each component in the scene segment based on the complexity corresponding to each component in each image frame in the same scene segment. The bit consumption ratio calculation submodule is used to obtain the bit consumption ratio of each component in the scene segment based on the initial bit consumption corresponding to each component in the initial encoded image corresponding to the same scene segment. The parameter offset calculation submodule is used to obtain the quantization parameter offset of each initial encoded image in each component in the scene segment based on the complexity ratio and bit consumption ratio of each component in the same scene segment. The encoding module is also used to encode the initial encoded image based on the estimated quantization parameters of the initial encoded image and the quantization parameter offset of the initial encoded image in each component.
[0146] In one possible implementation, the parameter offset calculation submodule is further configured to, for each target component, if the complexity ratio of the luminance component within the same scene segment is greater than a third preset value, and the bit consumption ratio of the target component within the scene segment is less than a fourth preset value, then obtain the quantization parameter offset of the target component within the scene segment based on the bit consumption ratio of the target component within the scene segment, wherein the target component is any one of the color component, luminance component, or chrominance component in the initial encoded image; if the complexity ratio of the luminance component within the same scene segment is not greater than the third preset value, or the bit consumption ratio of the target component within the scene segment is not less than the fourth preset value, then obtain a fifth preset value as the quantization parameter offset of each component within the scene segment.
[0147] In one possible implementation, the initial encoding information further includes the initial bits of the initial encoded image and the intra-frame estimated optimal block ratio. The quantization parameter acquisition module 330 includes a complexity acquisition submodule and a quantization parameter acquisition submodule. The complexity acquisition submodule is used to obtain the estimated complexity of each initial encoded image in each scene segment based on the initial quantization parameters, initial complexity, initial bits, and intra-frame estimated optimal block ratio of each initial encoded image in the same scene segment. The quantization parameter acquisition submodule is used to obtain the estimated quantization parameters of each initial encoded image in each scene segment based on the estimated complexity and initial bit rate factor of each initial encoded image in the same scene segment.
[0148] In one possible implementation, the complexity calculation submodule is used to obtain the average quantization parameter of each video to be encoded in the scene segment based on the initial quantization parameter corresponding to each initial encoded image in each scene segment; multiply the average quantization parameter by the square of the difference between the preset parameter and the intra-frame estimated optimal block ratio corresponding to each initial encoded image, and the initial bit corresponding to the initial encoded image to obtain the complexity calculation value corresponding to each initial encoded image; for the target initial encoded image, using the target initial encoded image as the Gaussian distribution center, obtain the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images; and according to the complexity calculation value corresponding to each initial encoded image, and the Gaussian distribution probability of the target initial encoded image and its neighboring encoded images, obtain the estimated complexity of the target initial encoded image, where the target initial encoded image is any one of multiple initial encoded images.
[0149] In one possible implementation, the initial quantization parameters further include initial bits. The deviation acquisition module 340 includes a predicted bit acquisition submodule, an actual bit acquisition submodule, and a deviation acquisition submodule. The predicted bit acquisition submodule is used to obtain the estimated bit consumption of the video to be encoded based on the initial quantization parameters, the predicted quantization parameters, and the initial bits corresponding to each initial encoded image. The actual bit acquisition submodule is used to obtain the actual bit consumption of the video to be encoded based on the target bit rate, the video frame rate of the video to be encoded, the number of image frames in the video to be encoded, and the motion scene scaling factor corresponding to each image frame. The deviation acquisition submodule is used to obtain the bit consumption deviation based on the actual bit consumption and the predicted bit consumption.
[0150] In one possible implementation, the estimated bit acquisition submodule is further configured to obtain the predicted bits corresponding to the initial encoded image based on the initial quantization parameters, the estimated quantization parameters, and the initial bits corresponding to the initial encoded image; and to accumulate the predicted bits corresponding to each initial encoded image to obtain the estimated bit consumption of the video to be encoded.
[0151] In one possible implementation, the bit estimation submodule is further configured to obtain the first bit of the initial encoded image based on the initial quantization parameters and initial complexity of the initial encoded image; obtain the second bit of the initial encoded image based on the initial quantization parameters, the estimated quantization parameters, and the initial bit; and select the larger bit between the first bit and the second bit of the initial encoded image as the prediction bit of the initial encoded image.
[0152] The following will combine Figure 10 This application describes an electronic device.
[0153] Please see Figure 10 Based on the video encoding method provided in the above embodiments, this application also provides another electronic device 100 including a processor 102 capable of executing the aforementioned method. The electronic device 100 can be a server or a terminal device, and the terminal device can be a smartphone, tablet computer, computer, or portable computer, etc.
[0154] The electronic device 100 also includes a memory 104. The memory 104 stores a program that can execute the contents of the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104.
[0155] The processor 102 may include one or more cores for data processing and message matrix units. The processor 102 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and by calling data stored in the memory 104. Optionally, the processor 102 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 102 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 102 and may be implemented separately using a communication chip.
[0156] The memory 104 may include random access memory (RAM) or read-only memory (ROM). The memory 104 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data acquired by the electronic device 100 during use (e.g., initial encoded images and initial encoded information).
[0157] The electronic device 100 may also include a network module and a screen. The network module is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals, thereby enabling communication with communication networks or other devices, such as audio playback devices. The network module may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, SIM cards, memory, etc. The network module can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The screen can display interface content and facilitate data interaction.
[0158] In some embodiments, the electronic device 100 may further include a peripheral interface and at least one peripheral device. The processor 102, memory 104, and peripheral interface 106 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral interface via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency component 108, a positioning component 112, a camera 114, an audio component 116, a display screen 118, and a power supply 122.
[0159] Peripheral interface 106 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 102 and memory 104. In some embodiments, processor 102, memory 104 and peripheral interface 106 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 102, memory 104 and peripheral interface 106 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.
[0160] The radio frequency (RF) component 108 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF component 108 communicates with communication networks and other communication devices via electromagnetic signals. The RF component 108 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF component 108 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF component 108 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF component 108 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.
[0161] Positioning component 112 is used to locate the current geographic location of an electronic device to enable navigation or LBS (Location Based Service). Positioning component 112 can be a positioning component based on the US GPS (Global Positioning System), BeiDou system, or Galileo system.
[0162] Camera 114 is used to capture images or videos. Optionally, camera 114 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device 100, and the rear-facing camera is located on the back of the electronic device 100. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, camera 114 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.
[0163] Audio component 116 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to processor 102 for processing, or input to radio frequency component 108 for voice communication. For stereo acquisition or noise reduction purposes, there may be multiple microphones, each located at a different part of electronic device 100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from processor 102 or radio frequency component 108 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into sound waves that humans can hear, but also into sound waves that humans cannot hear for purposes such as ranging. In some embodiments, audio component 114 may also include a headphone jack.
[0164] Display screen 118 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 118 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 102 for processing. In this case, display screen 118 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 118, which serves as the front panel of electronic device 100; in other embodiments, there may be at least two display screens, respectively disposed on different surfaces of electronic device 100 or in a folded design; in still other embodiments, display screen 118 may be a flexible display screen, disposed on a curved or folded surface of electronic device 100. Furthermore, display screen 118 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 118 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).
[0165] Power supply 122 is used to supply power to various components in electronic device 100. Power supply 122 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 122 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.
[0166] This application also provides a structural block diagram of a computer-readable storage medium. The computer-readable medium stores program code, which can be called by a processor to execute the methods described in the above method embodiments.
[0167] Computer-readable storage media can be electronic storage devices such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, computer-readable storage media includes non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code can be compressed, for example, in a suitable form.
[0168] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the various optional implementations above.
[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method of video coding, the method comprising: The method includes: The initial encoded image and the initial encoding information corresponding to each initial encoded image are obtained by encoding each image frame in the video to be encoded. The initial encoding information includes the initial complexity, the initial quantization parameter and the initial bit rate factor. Scene switching detection is performed on the video to be encoded to obtain multiple scene segments included in the video to be encoded; Motion analysis is performed on each image frame within each scene segment to obtain the scene type of each image frame, wherein the scene type includes slow motion type or non-slow motion type; The first complexity is obtained based on the initial complexity of each image frame corresponding to the slow motion type within the same scene segment, and the second complexity is obtained based on the initial complexity of the image frames corresponding to the non-slow motion type within the same scene segment. For the same scene segment, a first complexity and a second complexity are used as the base and the second complexity as the exponent to perform a power calculation to obtain the result. If the calculation result is between a first preset value and a second preset value, the calculation result is determined as the motion scene scaling factor of the image frame in the scene segment. If the calculation result is greater than the first preset value, the first preset value is determined as the motion scene scaling factor. If the calculation result is less than the second preset value, the second preset value is determined as the motion scene scaling factor, and the first preset value is greater than the second preset value. Based on the initial quantization parameters and initial code rate factor corresponding to each initial coded image, the estimated quantization parameters of the initial coded image are obtained. The bit consumption deviation is obtained based on the target bit rate, the motion scene scaling factor corresponding to each of the image frames, and the estimated quantization parameters of each of the initial encoded images. The bit consumption deviation is used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption when encoding each of the initial encoded images. If the bit consumption deviation is not within the preset deviation range, the initial bit rate factor is updated, and the process returns to perform the execution of the estimated quantization parameters of the initial encoded image based on the initial quantization parameters and initial bit rate factor corresponding to each initial encoded image. This process continues until the bit consumption deviation is within the preset deviation range, at which point the initial encoded image is encoded based on the estimated quantization parameters of the initial encoded image.
2. The method of claim 1, wherein, The encoding method used to obtain the initial encoded image by encoding each image frame in the video to be encoded is YUV. Each image frame in the video to be encoded has a color component, a luminance component, or a chrominance component. The initial encoding information also includes the complexity of each component and the initial bit consumption of each component. The method further includes: Based on the complexity of each component in each image frame of the same scene segment, the complexity ratio of each component in the scene segment is obtained. Based on the initial bit consumption of each component in the initial encoded image corresponding to the same scene segment, the bit consumption ratio of each component in that scene segment is obtained. The quantization parameter offset of each component of the initial encoded image within the same scene segment is obtained based on the complexity ratio and bit consumption ratio of each component within the same scene segment. The encoding of the initial encoded image based on the estimated quantization parameters of the initial encoded image includes: The initial encoded image is encoded based on the estimated quantization parameters of the initial encoded image and the quantization parameter offset of the initial encoded image in each component.
3. The method of claim 2, wherein, The method of obtaining the quantization parameter offset of each component of each initial encoded image within the same scene segment based on the complexity ratio and bit consumption ratio of each component includes: For each target component, if the complexity ratio of the luminance component in the same scene segment is greater than the third preset value, and the bit consumption ratio of the target component in the scene segment is less than the fourth preset value, then the quantization parameter offset of the target component in the scene segment is obtained based on the bit consumption ratio of the target component in the scene segment. Here, the target component is any one of the color component, luminance component, or chrominance component in the initial encoded image. If the complexity ratio of the luminance component within the same scene segment is not greater than the third preset value, or the bit consumption ratio of the target component within the scene segment is not less than the fourth preset value, then the fifth preset value is obtained as the quantization parameter offset for each component within the scene segment.
4. The method of claim 1, wherein, The initial coding information also includes the initial bits and the intra-frame estimated optimal block ratio of the initial coded image. The step of obtaining the estimated quantization parameters of the initial coded image based on the initial quantization parameters and initial rate factor corresponding to each initial coded image includes: Based on the initial quantization parameters, initial complexity, initial bits, and intra-frame estimated optimal block ratio of each initial encoded image in the same scene segment, the estimated complexity of each initial encoded image in each scene segment is obtained. Based on the estimated complexity and initial bit rate factor of each initial encoded image in the same scene segment, the estimated quantization parameters of each initial encoded image in each scene segment are obtained.
5. The method of claim 4, wherein, The process of obtaining the estimated complexity of each initial encoded image in each scene segment based on the initial quantization parameters, initial complexity, initial bits, and the proportion of the optimal block estimated intra-frame for each initial encoded image in the same scene segment includes: Based on the initial quantization parameters corresponding to each initial encoded image in each scene segment, the average quantization parameters of each video to be encoded in that scene segment are obtained; Multiply the average quantization parameter by the square of the difference between the preset parameter and the intra-frame estimated optimal block ratio corresponding to each initial coded image, and the initial bit corresponding to the initial coded image, to obtain the complexity calculation value corresponding to each initial coded image. For a target initial encoded image, the Gaussian distribution probabilities of the target initial encoded image and its neighboring encoded images are obtained using the target initial encoded image as the center of the Gaussian distribution. Based on the complexity calculation value corresponding to each initial encoded image, the target initial encoded image and its neighboring encoded images, the estimated complexity of the target initial encoded image is obtained. The target initial encoded image is any one of multiple initial encoded images.
6. The method of claim 1, wherein, The initial quantization parameters also include initial bits. The step of obtaining the bit consumption deviation based on the target bit rate, the motion scene scaling factor corresponding to each of the image frames, and the estimated quantization parameters of each of the initial encoded images includes: Based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to each of the initial encoded images, the estimated bit consumption of the video to be encoded is obtained. Based on the target bitrate, the video frame rate of the video to be encoded, the number of image frames in the video to be encoded, and the motion scene scaling factor corresponding to each image frame, the actual bit consumption of the video to be encoded is obtained. The bit consumption deviation is obtained based on the actual bit consumption and the estimated bit consumption.
7. The method of claim 6, wherein, The step of obtaining the estimated bit consumption of the video to be encoded based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to each of the initial encoded images includes: Based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to the initial encoded image, the predicted bits corresponding to the initial encoded image are obtained; The predicted bits corresponding to each initial encoded image are accumulated to obtain the estimated bit consumption of the video to be encoded.
8. The method of claim 7, wherein, The step of obtaining the predicted bits corresponding to the initial encoded image based on the initial quantization parameters, estimated quantization parameters, and initial bits corresponding to the initial encoded image includes: The first bit of the initial encoded image is obtained based on the initial quantization parameters and initial complexity of the initial encoded image; The second bit of the initial encoded image is obtained based on the initial quantization parameters, the estimated quantization parameters, and the initial bits of the initial encoded image; The larger bit between the first and second bits of the initial encoded image is selected as the prediction bit of the initial encoded image.
9. A video encoding apparatus, comprising: The device includes: The data acquisition module is used to acquire the initial encoded image obtained by encoding each image frame in the video to be encoded, as well as the initial encoding information corresponding to each initial encoded image. The initial encoding information includes the initial complexity, initial quantization parameters, and initial bitrate factor. The scaling factor acquisition module is used to perform scene switching detection on the video to be encoded to obtain multiple scene segments included in the video to be encoded; perform motion analysis on each image frame in each scene segment to obtain the scene type of each image frame, wherein the scene type includes slow motion type or non-slow motion type; obtain a first complexity based on the initial complexity corresponding to each image frame corresponding to slow motion type in the same scene segment, and obtain a second complexity based on the initial complexity corresponding to the image frame corresponding to non-slow motion type in the same scene segment; for the first complexity and the second complexity of the same scene segment, use the first complexity as the base and the second complexity as the exponent to perform exponential calculation to obtain the calculation result; if the calculation result is between a first preset value and a second preset value, then the calculation result is determined as the motion scene scaling factor of the image frame in the scene segment; if the calculation result is greater than the first preset value, then the first preset value is determined as the motion scene scaling factor; if the calculation result is less than the second preset value, then the second preset value is determined as the motion scene scaling factor, wherein the first preset value is greater than the second preset value. The quantization parameter acquisition module is used to obtain the estimated quantization parameters of the initial encoded image based on the initial quantization parameters and initial code rate factor corresponding to each initial encoded image. The deviation acquisition module is used to obtain the bit consumption deviation based on the target bit rate, the motion scene scaling factor corresponding to each of the image frames and the estimated quantization parameters of each of the initial encoded images. The bit consumption deviation is used to characterize the bit consumption deviation between the actual bit consumption and the estimated bit consumption when encoding each of the initial encoded images. The encoding module is used to update the initial code rate factor if the bit consumption deviation is not within a preset deviation range; and to encode the initial encoded image based on the estimated quantization parameters of the initial encoded image if the bit consumption deviation is within the preset deviation range.
10. An electronic device, comprising: include: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1-8.
11. A computer readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-8.
12. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method described in any one of claims 1-8.
Citation Information
Patent Citations
Method to encode video with close multiple scene changes
US20170171545A1
