Video coding method and device

Abstract

The embodiment of the invention provides a video coding method and device. The method comprises the following steps: dividing a plurality of video frames of a video to be coded in a progressive length increasing manner to obtain a plurality of image groups; according to the technical scheme, the to-be-coded video is coded based on the image groups, the smoothness is preferentially guaranteed through the short image groups in the initial stage, and the compression efficiency is improved through the long image groups in the stable stage, so that the jamming rate of the video can be reduced under the condition that the BDaate loss is reduced as much as possible.

Description

Technical Field

[0001] This specification relates to the field of image processing technology, and in particular to a video encoding method and electronic device. Background Technology

[0002] Video coding technology is a technique that compresses redundant components in video images and represents video information using as little data as possible. In cloud transcoding, video coding technology is used to reduce bitrate, maintain image quality, and ensure playback compatibility on user devices.

[0003] In video coding technology, video frame sequences are typically encoded in units of Group of Pictures (GOPs) or Mini-GOPs. In live streaming and short video applications, it's crucial to minimize the bitrate while maintaining consistent image quality to reduce CDN (Content Delivery Network) bandwidth costs and stuttering rates. Current solutions employ a Constant Quantizer Parameter (CQP) allocation strategy to assign quantization parameters to each GOP, and then encode the video based on these parameters. However, this approach can cause stuttering when GOPs are long. Therefore, optimizing video stuttering rates has become a critical technical challenge.

[0004] The information in the background section is merely information known only to the inventor and does not imply that such information had entered the public domain before the date of this application, nor does it imply that it can be considered prior art in this disclosure. Summary of the Invention

[0005] This manual provides a video encoding method and electronic device that can optimize video stuttering rate with minimal loss of BDrate.

[0006] Firstly, embodiments of this specification provide a video encoding method, including:

[0007] Obtain the video to be encoded, which includes multiple video frames;

[0008] The multiple video frames are divided into multiple image groups using a progressively increasing length approach, wherein the length of the first image group is less than a predetermined length value; and

[0009] The target video is obtained by encoding the video to be encoded based on the image group.

[0010] In some example embodiments, based on the above scheme, the division of the multiple video frames into multiple image groups in a progressively increasing length manner includes:

[0011] The length of each image group is determined according to the value of a predetermined incremental sequence.

[0012] The plurality of video frames are divided based on the length of each of the image groups to obtain the plurality of image groups.

[0013] In some example embodiments, based on the above scheme, determining the length of each of the image groups according to the values ​​of a predetermined increasing sequence includes:

[0014] Determine the video type of the video to be encoded;

[0015] Based on the correspondence between the video type and the incremental sequence type, a predetermined incremental sequence corresponding to the video to be encoded is determined; and

[0016] The length of each image group is determined based on the value of a predetermined increasing sequence corresponding to the video to be encoded.

[0017] In some example embodiments, based on the above scheme, the general term formula of the predetermined increasing sequence is 2. n -1, where n is a positive integer greater than or equal to 1.

[0018] In some example embodiments, based on the above scheme, the division of the multiple video frames into multiple image groups in a progressively increasing length manner includes:

[0019] Based on the video type of the video to be encoded, determine the length of the first image group among the plurality of image groups, and use the first image group as the current image group;

[0020] The compression performance of the current image group is determined based on its length.

[0021] The length of the next image group is determined based on the compression performance of the current image group.

[0022] The multiple video frames are divided based on the length of the next image group to obtain the next image group.

[0023] In some example embodiments, based on the above scheme, encoding the video to be encoded based on the image group to obtain the target video includes:

[0024] Based on the length of the image group, determine the encoding parameters corresponding to the image group;

[0025] The target video is obtained by encoding each of the plurality of video frames based on the encoding parameters corresponding to each of the image groups.

[0026] In some example embodiments, based on the above scheme, the encoding parameters include quantization parameters, and determining the encoding parameters corresponding to the image group based on the length of the image group includes:

[0027] Based on the correspondence between the length of the image group and the quantization parameters, the quantization parameters corresponding to the image group are determined.

[0028] In some example embodiments, based on the above scheme, determining the encoding parameters corresponding to the image group based on the length of the image group includes:

[0029] If the length of the image group is less than a predetermined threshold, then the key regions of each image frame in the image group are determined;

[0030] Based on the length of the image group, the encoding parameters of the key regions of the image frame are determined.

[0031] In some example embodiments, based on the above scheme, the method further includes:

[0032] When a scene change is detected, the length of the image group is reset to the length value of the first image group;

[0033] Starting with the length of the first image group, the multiple video frames are re-divided in a progressively increasing manner.

[0034] In some example embodiments, based on the above scheme, the method further includes:

[0035] Determine the difference between the current frame and the previous frame;

[0036] If the difference between the current frame and the previous frame is greater than a predetermined threshold, then a scene switch is determined to have occurred.

[0037] In some example embodiments, based on the above scheme, the method further includes:

[0038] When switching from the previous image group to the current image group, at least one frame from the previous image group is used as the reference frame for the current image group.

[0039] Secondly, this specification also provides an electronic device, comprising: at least one storage medium storing at least one instruction set for video encoding processing; and at least one processor communicatively connected to the at least one storage medium, wherein, when the electronic device is running, the at least one processor reads the at least one instruction set and executes the video encoding method described in the first aspect of this specification according to the instructions of the at least one instruction set.

[0040] As can be seen from the above technical solutions, the video encoding method and electronic device provided in the embodiments of this specification divide multiple video frames of the video to be encoded in a progressively increasing manner to obtain multiple image groups; the video to be encoded is then encoded based on these image groups. On the one hand, the length of the first image group in the multiple image groups is less than a predetermined length value, which allows for initial encoding using a shorter image group (MiniGOP), reducing the decoding time between the first and second frames. On the other hand, dividing multiple video frames in a progressively increasing manner allows for gradually increasing the length of the generated image group (MiniGOP). By dynamically adjusting the length of the image group, smoothness is prioritized in the initial stage using a shorter MiniGOP, while compression efficiency is improved in the stable stage using a longer MiniGOP, thereby reducing the video stuttering rate while minimizing BDrate loss. Furthermore, the video to be encoded is then encoded according to the encoding parameters of the MiniGOP corresponding to its length to obtain the target video. The encoding parameters can be dynamically adjusted according to the length of the MiniGOP to further improve compression performance and reduce the video stuttering rate.

[0041] Other functions of the video encoding methods, devices, and apparatus provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the video encoding methods, devices, and apparatus provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A schematic diagram of the implementation environment of a video encoding method provided in an embodiment of this specification is shown;

[0044] Figure 2 A hardware structure diagram of an electronic device 200 provided according to an embodiment of this specification is shown;

[0045] Figure 3 A flowchart illustrating a video encoding method according to some embodiments of this specification is shown;

[0046] Figure 4 A schematic diagram of the structure of multiple image groups provided according to some embodiments of this specification is shown; and

[0047] Figure 5 A schematic diagram of a process for dividing multiple video frames is shown according to some embodiments of this specification. Detailed Implementation

[0048] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0049] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0050] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0051] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0052] It should be noted that the user data obtained in this manual is authorized by the user and does not involve user privacy.

[0053] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0054] QP (Quantization Parameter): In video encoding, it controls the step size of quantization. The larger the QP, the lower the bit rate, but the worse the image quality.

[0055] BD-Rate (Bjontegaard Delta Rate): This is an objective metric used in video compression to compare the rate-distortion performance or compression efficiency of two different video codecs or different settings of the same video codec within a range of bitrate or quality values. A higher BD-Rate indicates better encoder performance.

[0056] A GOP (Group of Pictures) is a collection of consecutive frames in video coding, typically starting with an I-frame (keyframe) followed by multiple P-frames (forward prediction frames) and B-frames (bidirectional prediction frames).

[0057] MiniGOP (Mini Group of Pictures): A substructure within a GOP used to describe the frame arrangement logic in temporal scalability coding.

[0058] In cloud transcoding, video encoding technology is used to reduce bitrate, maintain image quality, and ensure playback compatibility on user devices. Among related technical solutions, fixed-length MiniGOPs are used to encode the video. Taking an encoding scenario with a MiniGOP length of 16 as an example, Table 1 below shows the playback order, encoding / decoding order, and corresponding decoding speed of the 16 frames.

[0059] Table 1. Playback order, encoding / decoding order, and corresponding decoding speed of 16 frames.

[0060]

[0061]

[0062] Define the download time of the i-th frame. Decoding time The sum is T i :

[0063]

[0064] Referring to Table 1 above, the edit code sequence corresponding to the 2nd frame in the playback order is the 6th frame. The time interval between the decoding time of the 1st frame and the decoding time of the 2nd frame in the playback order is the longest, and the calculation method is as follows:

[0065]

[0066] Taking the statistical data in the table as an example, T max It can be calculated as follows:

[0067] T max =Download(874kbit+462kbit+129kbit+35kbit+15kbit+4kbit)+30ms

[0068] +15ms +7ms +5ms +3ms +2ms

[0069] T max =Download(1519kbit)+62ms

[0070] Where Download(x) represents the time required to download a file of size x, and T max This indicates the maximum time interval between playing the first frame and playing the second frame.

[0071] According to the above technical solution, when the MiniGOP (Minimum Group of Images) is long, the decoding time interval between the first and second frames is long, which can easily cause stuttering. Since increasing the MiniGOP can improve the BDrate on average, the MiniGOP length in short video-on-demand scenarios is often long in order to maximize the compression rate. However, a long MiniGOP can also affect the stuttering rate of the video. Therefore, how to optimize the stuttering rate of the video with minimal loss of BDrate has become a technical challenge to be solved.

[0072] Based on the above, embodiments of this specification provide a video encoding method and electronic device. The method divides multiple video frames of the video to be encoded into multiple image groups (MiniGOPs) using a progressively increasing length approach. Encoding is then performed based on these image groups. On one hand, the length of the first image group is less than a predetermined value, allowing for initial encoding using shorter MiniGOPs, reducing decoding time between the first and second frames. On the other hand, dividing the video frames using a progressively increasing length approach gradually increases the length of the generated MiniGOPs. By dynamically adjusting the length of the image groups, smoothness is prioritized in the initial stage using shorter MiniGOPs, while compression efficiency is improved in the stable stage using longer MiniGOPs. This reduces video stuttering while minimizing BDrate loss. Furthermore, the video to be encoded is then encoded according to the encoding parameters corresponding to the MiniGOP length to obtain the target video. Dynamically adjusting the encoding parameters based on the MiniGOP length further improves compression performance and reduces video stuttering.

[0073] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0074] Figure 1 A schematic diagram of the implementation environment of a video encoding method provided in an embodiment of this specification is shown.

[0075] See Figure 1 As shown, the implementation environment 100 may include a terminal 110, a server 130, and a database 140.

[0076] Terminal 110 is connected to server 130 via wireless network or wired network 120. Terminal 110 can be a mobile phone, tablet computer, laptop computer, or desktop computer, but is not limited to these.

[0077] Terminal 110 may store data or instructions for performing the video encoding method described in this specification. Terminal 110 may include hardware devices with data processing capabilities and the necessary programs to drive the hardware devices.

[0078] Server 130 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 130 provides background services for applications running on terminal 110.

[0079] Server 130 is equipped with an integrated development platform (IDE). An IDE, also known as an integrated development environment, is an application that provides a program development environment, typically including tools such as a code editor, compiler, debugger, and graphical user interface. Developers can write program code (i.e., program development) on the IDE. The IDE server can be a computing device specifically designed to implement video encoding methods. Server 130 can communicate with both terminal 110 and database 140.

[0080] Furthermore, server 130 may store data or instructions for executing the video encoding method described in this specification. Server 130 may include hardware devices with data processing capabilities and the necessary programs to drive the hardware devices. Of course, server 130 may also be merely a hardware device with data processing capabilities, or simply a program running on the hardware device. In some embodiments, server 130 may also be deployed as a plug-in on terminal 110, in which case server 130 stores data or instructions for executing the video encoding method corresponding to terminal 110 described in this specification.

[0081] Database 140 may store data and / or instructions. In some embodiments, database 140 may store task graphs and contexts corresponding to tasks. In some embodiments, database 140 may store data and / or instructions executed by server 130 or used to execute the video encoding methods described herein. Terminal 110 and server 130 have access to database 140, and terminal 110 and server 130 may access data or instructions stored in database 140 via a network. In some embodiments, database 140 may be directly connected to terminal 110 and server 130. In some embodiments, database 140 may be part of server 130. In some embodiments, database 140 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), or similar content, or any combination thereof. Exemplary mass storage may include non-transitory storage media such as disks, optical discs, and solid-state drives. Exemplary removable storage may include flash drives, floppy disks, optical discs, memory cards, zip disks, magnetic tapes, etc. Typical volatile read-write memory may include random access memory (RAM). Example RAMs may include dynamic RAM (DRAM), dual date rate synchronous dynamic RAM (DDRSDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero-capacitance RAM (Z-RAM), etc. Exemplary ROMs may include mask ROM (MROM), programmable ROM (PROM), virtual programmable ROM (PEROM), electronically programmable ROM (EEPROM), optical disc ROM (CD ROM), and digital multifunction disk ROM, etc.

[0082] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more, in which case other terminals may also be included in the above implementation environment. This specification does not limit the number of terminals or the type of devices in the embodiments.

[0083] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be described below in conjunction with the above implementation environment. In the following description, the terminal is also the terminal 110 in the above implementation environment, and the server is also the server 130 in the above implementation environment. The technical solutions provided by the embodiments of this specification can be applied to video encoding scenarios of multimedia platforms, such as short video platforms, live video platforms, or long video platforms.

[0084] Taking the technical solution provided in the embodiments of this specification as an example in the video encoding scenario of a short video platform, the terminal 110 acquires the video to be encoded, which includes multiple video frames; the multiple video frames are divided in a progressively increasing manner to obtain multiple image groups, wherein the length of the first image group in the multiple image groups is less than a predetermined length value, for example, the first image group is an I-frame with a length of 1; and the multiple video frames of the video to be encoded are encoded based on the encoding parameters corresponding to the image groups to obtain the target video.

[0085] It should be noted that the above description is based on the application of the technical solution provided in the embodiments of this specification to the video encoding scenario of a short video platform. The technical solution provided in the embodiments of this specification can also be applied to other appropriate video encoding scenarios, such as the video encoding scenarios of live streaming platforms or video conferencing platforms. The implementation process is the same as the above description and belongs to the same inventive concept, so it will not be repeated here.

[0086] It should be noted that the steps in the video encoding method in the example embodiments of this specification may be partially executed by the terminal, partially executed by the server, or entirely executed by the server or entirely by the client. This specification does not impose any special limitations on this.

[0087] based on Figure 1 The implementation environment shown below will be combined with... Figures 2-5 This specification provides a detailed description of the video encoding method and electronic device provided in the embodiments. It should be noted that the above-described implementation environment is shown only to facilitate understanding of the spirit and principles of this specification, and the embodiments of this specification are not limited in any way. Rather, the embodiments of this specification can be applied to any applicable scenario.

[0088] Figure 2 This is a schematic diagram of an electronic device 200 provided according to some embodiments of this specification. The electronic device 200 can execute the video encoding method described in this specification. The video encoding method is described in other parts of this specification. The electronic device 200 can be a general-purpose computer or a special-purpose computer. For example, the electronic device 200 can be a server, a personal computer, a portable computer (such as a laptop computer, tablet computer, etc.), or other electronic devices with computing capabilities.

[0089] The electronic device described in this specification may include one or more of the following components: processor 210, memory 220, input device 230, output device 240, and bus 250. The processor 210, memory 220, input device 230, and output device 240 may be connected to each other via bus 250.

[0090] Processor 210 may include one or more processing cores. Processor 210 connects to various parts within the electronic device using various interfaces and lines, and executes the video encoding method or video coding method described in this specification by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 210 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing displayed content, or training machine learning models; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 210, but may be implemented separately using a communication chip.

[0091] The memory 220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 220 may include non-transitory computer-readable storage medium. The memory 220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be an Android system, including systems deeply developed based on the Android system, an iOS system, including systems deeply developed based on the iOS system, or other systems.

[0092] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0093] The input device 230 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 240 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 230 and the output device 240 can be combined, and both the input device 230 and the output device 240 can be a touch display screen.

[0094] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0095] Figure 3 A flowchart of a video encoding method according to an embodiment of this specification is shown. As previously described, the electronic device 200 can execute the video encoding method of the embodiment of this specification. Specifically, the processor 210 can read the instruction set stored in its local storage medium and then execute the video encoding method of the embodiment of this specification according to the instructions of the instruction set. Hereinafter, steps S310 to S330 of the video encoding method will be described in detail with reference to the accompanying drawings.

[0096] like Figure 3 As shown, in step S310, the video to be encoded is obtained, which includes multiple video frames.

[0097] In the example embodiment, the video to be encoded refers to the original video file that needs to be compressed or transcoded, such as a video uploaded by a user to a video-on-demand platform. The video to be encoded serves as input data in the encoding process, where it is compressed by an encoder into a smaller file or adapted to a specific device / network transmission format. For example, an uncompressed YUV format video file (e.g., input.yuv) needs to be converted to an H.264 encoded MP4 file (e.g., output.mp4). A Group of Pictures (GOP) refers to a collection of frames in video encoding, typically starting with a keyframe (I-frame), followed by several prediction frames (B-frames) and reference frames (P-frames). Dividing the video to be encoded into multiple video frames optimizes video compression efficiency and supports fast localization and decoding.

[0098] In some example embodiments, the video to be encoded comprises multiple video frames. The electronic device 200 reads the video to be encoded (e.g., YUV, MP4, AVI), loads the video into memory or a buffer, and divides the multiple video frames of the video to be encoded into multiple Groups of Pictures (GOPs) at a predetermined length, such as 32 frames. Each GOP begins with an I-frame, followed by P-frames or B-frames. For example, the GOP structure can be IBBPBB. Alternatively, the electronic device 200 can automatically divide the video to be encoded into multiple GOPs at predetermined intervals using an encoder, or it can divide the video into multiple GOPs using a script.

[0099] In step S320, multiple video frames are divided in a progressively increasing length manner to obtain multiple image groups.

[0100] In the example implementation, the progressively increasing length of the MiniGOP is a dynamic grouping strategy. Its core idea is to gradually increase the length of the MiniGOP as the video progresses. Shorter MiniGOPs reduce latency but sacrifice compression ratio, while longer MiniGOPs increase compression ratio but increase latency. By progressively increasing the length, a dynamic balance can be found between these two, combining the low latency advantage of short MiniGOPs with the high compression efficiency advantage of long MiniGOPs. A MiniGOP is a substructure of a Group of Pictures (GOP). A GOP can be divided into multiple MiniGOPs. The length of a MiniGOP refers to the number of frames it contains in the temporal hierarchical structure. The core function of the MiniGOP length is to determine the frame reference relationships, hierarchical structure, and bitrate allocation strategy during encoding. The MiniGOP length determines the reference hierarchy and dependencies between frames. Lower-level frames (such as layer 0) are typically I-frames (keyframes) and do not depend on other frames. Higher-level frames (such as layer 1 and above) rely on lower-level frames for prediction (such as P-frames or B-frames). Electronic device 200 determines the length of the image group starting from a predetermined length and in a progressively increasing manner. Based on the determined length of the image group, multiple video frames of the video to be encoded are divided into multiple image groups, for example, the predetermined length is 1, 2 or 3, etc.

[0101] In an example embodiment, the electronic device 200 determines the length of each image group according to the value of a predetermined increasing sequence; based on the length of each image group, multiple video frames are divided to obtain multiple image groups. The predetermined increasing sequence can be a linear increasing sequence or an exponential increasing sequence, or other suitable increasing sequences.

[0102] For example, the general term formula for a predetermined increasing sequence is 2. n-1, where n is a positive integer greater than or equal to 1, and n represents the nth MiniGOP. For example, the length of the 1st MiniGOP is 1, the length of the 2nd MiniGOP is 3, the length of the 3rd MiniGOP is 7, the length of the 4th MiniGOP is 15, and so on.

[0103] In some example embodiments, the length of the first image group in a plurality of image groups is less than a predetermined length value. The electronic device 200 divides the multiple video frames of the video to be encoded into multiple image groups (GOPs), and divides the video frames of the current image group of the video to be encoded in a progressively increasing manner to obtain multiple mini-GOPs corresponding to the current image group GOP. For example, the electronic device 200 uses I-frames and P-frames as segmentation frames to divide the current GOP into multiple mini-GOPs.

[0104] In other example embodiments, the electronic device 200 directly divides multiple video frames of the video to be encoded in a progressively increasing manner, resulting in multiple image groups. For example... Figure 4 As shown, the I-frame in the current GOP can be used as the first MiniGOP1. After the first MiniGOP1, the second MiniGOP2 has a length of 4, the next MiniGOP3 has a length of 8, the next MiniGOP4 has a length of 16, and so on until the next I-frame appears. For example, the electronic device 200 determines the length of the image group in a progressively increasing manner. When dividing multiple video frames, an I-frame (keyframe) is inserted at the beginning of each MiniGOP to ensure decoding independence; P / B frames are allocated according to the MiniGOP length and the number of reference frames. For example: MiniGOP = 8 frames: IPBBPBBP; MiniGOP = 16 frames: IPBBPBBPBBPBBPBB.

[0105] Furthermore, in the example embodiment, the electronic device 200 determines the video type of the video to be encoded; determines a predetermined incremental sequence corresponding to the video to be encoded based on a preset correspondence between video types and incremental sequence types; and determines the length of each image group based on the value of the predetermined incremental sequence corresponding to the video to be encoded.

[0106] For example, electronic device 200 determines the video type by analyzing video content features such as motion intensity, scene change frequency, and texture complexity. Video types can include sports, animation, film, news, etc. A correspondence between video types and incremental sequence types is pre-established. The incremental sequence type defines a rule for increasing length, such as the initial length and the growth method. Table 2 below shows the correspondence between different video types and incremental sequence types.

[0107] Table 2. Correspondence between different video types and incremental sequence types

[0108] Video type initial length growth model Maximum length Sports Videos 2 Exponential growth, with an index base of 2 32 Animation 4 Linear growth, with a step size of 2 16 Film and television 4 Exponential growth, with an index base of 2 32

[0109] Electronic device 200 determines a predetermined incremental sequence corresponding to the video to be encoded based on a preset correspondence between video type and incremental sequence type; and determines the length of each image group based on the value of the predetermined incremental sequence corresponding to the video to be encoded. For example, if the video to be encoded is a sports video, the predetermined incremental sequence corresponding to the video to be encoded is determined to be 2 based on the correspondence between sports video type and incremental sequence type. n The length of each image group is determined based on the values ​​of a predetermined incremental sequence, namely 2, 4, 8, 16, and 32. The maximum length is a length threshold, which can be determined based on the coding performance of the image group. For example, if the rate-distortion cost improvement is less than 5% (or the improvement is less than a certain value), then the length is stopped from increasing. The length before stopping the increase is the length threshold.

[0110] By analyzing the video content of the video to be encoded and determining the incremental strategy that matches the video, we can select the appropriate image group length increment sequence for different video types, thereby further reducing the stuttering rate of the video.

[0111] In step S330, the video to be encoded is encoded based on the image group to obtain the target video.

[0112] In the example embodiment, MiniGOP-based coding is a hierarchical video compression technique. Its core idea is to divide the video into multiple MiniGOPs, perform efficient predictive coding within each MiniGOP, and maintain independence between groups to achieve parallel processing. The electronic device 200 determines the coding parameters of each frame within a MiniGOP based on the MiniGOP length, encodes each frame based on the coding parameters, obtains the coded frames corresponding to each MiniGOP, and thus obtains the target video.

[0113] The length of a Mini-GOP determines the distribution of frames (e.g., the number of I / P / B frames) and their reference relationships within the image group. Mini-GOPs of different lengths have different temporal prediction structures and compression potential, thus requiring customized coding parameters. In some example embodiments, the electronic device 200 determines the coding parameters corresponding to the image group based on its length; and encodes multiple video frames based on the coding parameters corresponding to each image group to obtain the target video.

[0114] In the example embodiment, the encoding parameters include quantization parameters. The electronic device 200 determines the quantization parameters corresponding to an image group based on the correspondence between the length of the image group and the quantization parameters. For example, a longer MiniGOP corresponds to a higher quantization parameter QP, and a shorter MiniGOP corresponds to a lower quantization parameter QP. The correspondence between the length of the image group and the quantization parameters is preset. The electronic device 200 determines the quantization parameter QP corresponding to the image group based on the length of the image group; and encodes multiple video frames based on the quantization parameters corresponding to each image group to obtain the target video.

[0115] For example, the electronic device 200 dynamically adjusts the QP value based on the Mini GOP length and scene complexity. For short Mini GOPs, the QP needs to be lowered to avoid a sudden drop in image quality, while for long Mini GOPs, a higher QP is allowed. For instance, assuming the Mini GOP length is 4 frames, such as IPBB, the quantization parameter QP for the I-frame is determined to be 25, and intra-frame prediction is performed for the I-frame; the quantization parameter QP for the P-frame is 28, referencing the previous I-frame; and the quantization parameter QP for the B-frame is 30, referencing the preceding and following frames.

[0116] Furthermore, since insufficient bitrate allocation in the initial short MiniGOP may cause block artifacts, in some example embodiments, if the length of the image group is less than a predetermined threshold, the electronic device 200 determines the key regions of each image frame in the image group; based on the length of the image group, it determines the encoding parameters of the key regions of the image frames. For example, the electronic device 200 uses a lightweight CNN model to identify the key regions of the image frames, i.e., the regions sensitive to human vision, and pre-sets the number of additional bits allocated to the key regions corresponding to image groups of different lengths. For example, for the key regions in the initial short GOP, i.e., the ROI (Region of Interest) regions, sub-partition encoding is used, allocating an additional 20% of bits.

[0117] According to the technical solution in the above example embodiment, intelligent bitrate reallocation significantly improves the subjective quality of the short MiniGOP stage while keeping the total bitrate unchanged, thus avoiding block effects.

[0118] Table 3 below shows Figure 4 The playback order, encoding / decoding order, and corresponding decoding speed of each frame in the second image group:

[0119] Table 3. Figure 4 The playback order, encoding / decoding order, and corresponding decoding speed of each frame in the second image group.

[0120] Playback order Encoding / decoding order Frame type Frame size (kbit) Decoding speed 1 1 I-slice 779 30ms 2 4 B-slice 8 2ms 3 3 B-slice 26 3ms 4 5 B-slice 1 2ms 5 2 B-slice 259 10ms

[0121] Referring to Table 3 above, the calculation method for the time interval between the decoding of the first frame and the decoding of the second frame in the playback order is as follows:

[0122]

[0123] T′ max =Download(779kbit+259kbit+26kbit+8kbit)+30ms+10ms+3ms

[0124] +2ms

[0125] T′ max =Download(1072kbit)+45ms

[0126] ΔT max =T max -T′ max =Download(447kbit)+17ms

[0127] Among them, T max T′ represents the maximum time interval between playing the first frame and playing the second frame when the MiniGOP length is 16. max ΔT represents the maximum time interval between playing the first frame and playing the second frame when the MiniGOP length is 16. max This represents the difference between the two.

[0128] Therefore, according to the technical solution in the above example embodiment, the method of progressively increasing the MiniGOP length saves 447kbit of download time and 17ms of decoding time; at the same time, the bdrate loss is 0.4%, thereby reducing the video stuttering rate while minimizing bdrate loss.

[0129] according to Figure 3In the example embodiment, the technical solution has several advantages. First, the length of the first image group in a plurality of image groups is less than a predetermined length value, allowing for initial encoding using a shorter image group (MiniGOP), thus reducing the decoding time between the first and second frames. Second, by dividing multiple video frames into groups with progressively increasing lengths, the length of the generated MiniGOPs can be gradually increased. By dynamically adjusting the length of the image groups, smoothness is prioritized in the initial stage using shorter MiniGOPs, while compression efficiency is improved in the stable stage using longer MiniGOPs. This reduces video stuttering while minimizing bitrate loss. Third, the video to be encoded is encoded according to the encoding parameters of the MiniGOP corresponding to its length, resulting in the target video. The encoding parameters can be dynamically adjusted according to the MiniGOP length, further improving compression performance and reducing video stuttering.

[0130] Furthermore, since increasing the length of the MiniGOP increases the computational load of the encoder, such as requiring bidirectional prediction of more B-frames, in the example embodiment, the electronic device 200 pre-allocates computational resources corresponding to the length of the MiniGOP, such as a reference frame buffer, according to the expected length of the MiniGOP. More computational resources are allocated for longer MiniGOPs, and less computational resources are allocated for shorter MiniGOPs.

[0131] By allocating corresponding computing resources according to the length of MiniGOP, resource utilization can be improved, stuttering caused by insufficient hardware resources can be avoided, and the stuttering rate of video can be further reduced.

[0132] Furthermore, in some example embodiments, when the electronic device 200 detects a scene change, it resets the length of the image group to the length value of the first image group; starting from the length value of the first image group, it re-divides multiple video frames in a progressively increasing manner. Taking a scene change in a movie video as an example, after detecting a scene change, the electronic device 200 resets the Mini GOP length to an initial value corresponding to the changed scene, such as Mini GOP = 4 frames, and progressively increases the subsequent video frames based on the initial value to adapt to scene changes and optimize compression performance. For example, the initial value is 2 for a high dynamic range scene and 8 for a low dynamic range scene.

[0133] For example, electronic device 200 determines the difference between the current frame and the previous frame; if the difference between the current frame and the previous frame is greater than a predetermined threshold, it determines that a scene switch has occurred. For example, electronic device 200 calculates the pixel difference between the current frame and the previous frame, such as pixel difference, SSIM value, or motion vector amplitude, etc. If the difference exceeds a predetermined threshold, such as SSIM being less than 0.5, it determines that a scene switch has occurred; when a scene switch occurs, the current GOP length is reset to an initial value (such as the default length of the first MiniGOP); starting from the reset Mini GOP length, the Mini GOP length is gradually increased.

[0134] Based on the technical solutions in the above example embodiments, when a scene change is detected, the length of the image group is reset to the initial value. This not only enables a quick response to the needs of the new scene, but also allows for the exploration of a better MiniGOP length through gradual increments, thereby achieving a balance between compression efficiency, image quality, and latency.

[0135] Furthermore, to ensure a smooth transition when switching MiniGOP lengths, in some example embodiments, when switching from a previous image group to the current image group, the electronic device 200 uses at least one frame from the previous image group as a reference frame for the current image group. At the MiniGOP switching point, the last frame of the previous MiniGOP is retained as an additional reference frame for the new MiniGOP. For example, at a MiniGOP switching point, such as switching from MiniGOP1 to MiniGOP2, the last frame of the previous MiniGOP (e.g., frame A) is directly used as an additional reference frame for the new MiniGOP. If the scene complexity is high (e.g., fast motion), the penultimate frame (e.g., frame B) can be additionally retained as an auxiliary reference frame to further improve prediction accuracy. Therefore, the reference frame may include: the first frame of the new MiniGOP, the last frame of the previous MiniGOP, and the penultimate frame of the previous MiniGOP.

[0136] In some example implementations, closer reference frames are given higher weights. The first frame (frame C) of a new MiniGOP is preferentially referenced to the last frame (frame A) of the previous MiniGOP, and then the second to last frame (frame B). The weight allocation is as follows: Weight(A) = w1, Weight(B) = w2, w1 > w2.

[0137] According to the above scheme, by sharing reference frames across MiniGOPs, the problem of reference frame fragmentation caused by changes in MiniGOP length is avoided. At the same time, the strategy of assigning higher weights to nearby reference frames improves prediction accuracy and coding efficiency.

[0138] Figure 5 A schematic diagram of a process for dividing multiple video frames is shown according to some embodiments of this specification.

[0139] Reference Figure 5 As shown, in step S510, the length of the first image group among multiple image groups is determined based on the video type of the video to be encoded.

[0140] In the example embodiment, the video type to be encoded may include high dynamic range (HMR) video, low dynamic range (LVR) video, and still video, etc. The length of the first MiniGOP is typically determined based on the video type and the initial scene; for example, the initial length is shorter for HMR video and longer for LVR video. Table 4 below shows the length of the first image group corresponding to different video types.

[0141] Table 4. Length of the first image group for different video types

[0142] Video type Length of the first MiniGOP High dynamic range video 2-4 Medium dynamic video 4-8 Low dynamic range video 8-16

[0143] Assuming the video type of a video sequence to be encoded is high dynamic range video, such as a sports event, the electronic device 200 determines the length of the first MiniGOP to be 3 frames based on the video type and the above correspondence; the first MiniGOP (3 frames) is encoded, and the corresponding compression performance, such as BDrate, is calculated.

[0144] In step S520, the compression performance corresponding to the current image group is determined based on the length of the current image group.

[0145] In the example embodiment, the first image group is used as the current image group. Compression performance refers to BD-Rate performance, such as the bitrate difference between the current encoder and the reference encoder at the same PSNR (Peak Signal-to-Noise Ratio). The lower the BD-Rate, the higher the compression efficiency while maintaining the same image quality. The electronic device 200 quantifies the compression performance of the current MiniGOP, such as BD-Rate and PSNR, to provide a basis for the next GOP length adjustment.

[0146] In step S530, the length of the next image group is determined based on the compression performance corresponding to the current image group.

[0147] In an example embodiment, after encoding a MiniGOP, the electronic device 200 evaluates the compression performance of the current image group at that length, and determines the length of the next MiniGOP in a progressively increasing manner based on the compression performance. For example, if the compression rate is improved and the image quality is not significantly reduced, the growth step of the MiniGOP is increased.

[0148] In some example implementations, the initial growth step size is set to 2. If the current MiniGOP's compression performance, such as BD-Rate, is relatively high and the PSNR is stable, the growth step size of the next Mini GOP is increased in a progressive manner, for example, to 4 (e.g., from 8 frames to 12 frames), to further compress the bitrate. If the current MiniGOP's BD-Rate is high or the PSNR drops significantly, the growth step size is reduced, for example, to 1 or 0, to maintain the compression ratio and protect the image quality.

[0149] In other example embodiments, assuming the video sequence to be encoded is a high dynamic range video, such as a sports event, the electronic device 200 determines the length of the first MiniGOP to be 3 frames based on the video type and the above correspondence; encodes the first MiniGOP (3 frames) and calculates the corresponding compression performance; and determines the length of the next MiniGOP based on the current content (e.g., motion intensity) and the compression performance of the previous MiniGOP. If the motion intensity corresponding to the video sequence increases, the length of the next MiniGOP is shortened (e.g., 2 frames); if the motion intensity decreases, the length of the next MiniGOP is increased (e.g., 4 frames).

[0150] According to the technical solution in the above example embodiment, the length of the next GOP is dynamically adjusted based on the compression performance of the current Mini GOP, maximizing bitrate savings while maintaining image quality.

[0151] In step S540, multiple video frames are divided based on the length of the next image group to obtain the next image group.

[0152] In an example embodiment, the electronic device 200 divides the remaining video frames into consecutive MinGOPs according to the determined next MiniGOP length and assigns frame types such as I / P / B frames. For example, the electronic device 200 inserts an I-frame (keyframe) at the beginning of each MiniGOP to ensure decoding independence; and assigns P / B frames according to the MiniGOP length and the number of reference frames. For example: GOP = 8 frames: IPBBPBBP; GOP = 16 frames: IPBBPBBPBBPBBPBB.

[0153] according to Figure 5 The technical solution in the example embodiment dynamically adjusts the length of the image group based on the video type and content characteristics, which can significantly improve compression efficiency while reducing initial latency.

[0154] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing video encoding. When the executable instructions are executed by a processor, they instruct the processor to perform the steps of the video encoding method described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on an electronic device 200, the program code causes the electronic device 200 to perform the steps of the video encoding method described in this specification. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the electronic device 200. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on electronic device 200, partially on electronic device 200, as a standalone software package, partially on electronic device 200 and partially on a remote computing device, or entirely on a remote computing device.

[0155] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multi-video coding and parallel processing are also possible or may be advantageous.

[0156] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0157] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0158] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0159] Each patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference, except for any identical content appearing in the related documents that may be inconsistent with or conflict with this document, or any identical document content that may have a limiting effect on the widest scope of the claims. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any included material and those related to this document, the terminology herein shall prevail.

[0160] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A video encoding method, comprising: Obtain the video to be encoded, which includes multiple video frames; The multiple video frames are divided into multiple image groups by progressively increasing length, wherein the length of the first image group in the multiple image groups is less than a predetermined length value; as well as The target video is obtained by encoding the video to be encoded based on the image group.

2. The method according to claim 1, wherein, The process of dividing the multiple video frames into multiple image groups using a progressively increasing length method includes: The length of each image group is determined according to the value of a predetermined incremental sequence. The plurality of video frames are divided based on the length of each of the image groups to obtain the plurality of image groups.

3. The method according to claim 2, wherein, Determining the length of each image group according to a predetermined increasing sequence of values ​​includes: Determine the video type of the video to be encoded; Based on the correspondence between the video type and the incremental sequence type, a predetermined incremental sequence corresponding to the video to be encoded is determined; and The length of each image group is determined based on the value of a predetermined increasing sequence corresponding to the video to be encoded.

4. The method according to claim 2, wherein, The general formula for the predetermined increasing sequence is 2. n -1, where n is a positive integer greater than or equal to 1.

5. The method according to claim 1, wherein, The process of dividing the multiple video frames into multiple image groups using a progressively increasing length method includes: Based on the video type of the video to be encoded, determine the length of the first image group among the plurality of image groups, and use the first image group as the current image group; The compression performance of the current image group is determined based on its length. The length of the next image group is determined based on the compression performance of the current image group. The multiple video frames are divided based on the length of the next image group to obtain the next image group.

6. The method according to claim 1, wherein, The process of encoding the video to be encoded based on the image group to obtain the target video includes: Based on the length of the image group, determine the encoding parameters corresponding to the image group; The target video is obtained by encoding each of the plurality of video frames based on the encoding parameters corresponding to each of the image groups.

7. The method according to claim 6, wherein, The encoding parameters include quantization parameters, and determining the encoding parameters corresponding to the image group based on the length of the image group includes: Based on the correspondence between the length of the image group and the quantization parameters, the quantization parameters corresponding to the image group are determined.

8. The method according to claim 6, wherein, The step of determining the encoding parameters corresponding to the image group based on the length of the image group includes: If the length of the image group is less than a predetermined threshold, then the key regions of each image frame in the image group are determined; Based on the length of the image group, the encoding parameters of the key regions of the image frame are determined.

9. The method according to claim 1, wherein, The method further includes: When a scene change is detected, the length of the image group is reset to the length value of the first image group; Starting with the length of the first image group, the multiple video frames are re-divided in a progressively increasing manner.

10. The method according to claim 9, wherein, The method further includes: Determine the difference between the current frame and the previous frame; If the difference between the current frame and the previous frame is greater than a predetermined threshold, then a scene switch is determined to have occurred.

11. The method according to claim 1, wherein, The method further includes: When switching from the previous image group to the current image group, at least one frame from the previous image group is used as the reference frame for the current image group.

12. An electronic device, comprising: At least one storage medium storing at least one instruction set for video encoding processing; as well as At least one processor is communicatively connected to the at least one storage medium. When the electronic device is running, the at least one processor reads the at least one instruction set and executes the video encoding method as described in any one of claims 1-11 according to the instructions of the at least one instruction set.

Images