Video encoding method, apparatus, device, medium, and program product

Abstract

The application discloses a video coding method, device, equipment, medium and program product, and applies to the field of encoders. The method comprises the following steps: acquiring a video frame sequence with a preset frame number in a to-be-coded video, and obtaining a current image group; determining a target end frame from the current image group based on the video features of the current image group; determining a starting frame to the target end frame in the current image group as a to-be-coded image group; and coding each video frame in the to-be-coded image group to obtain a coding code stream.

Description

Technical Field

[0001] This application relates to the field of encoders, and more particularly to a video encoding method, apparatus, device, medium, and program product. Background Technology

[0002] An encoder is a program or device used to compress video, reducing its size and bandwidth. A group of pictures (GOP) is a collection of video frames. An encoder divides video frames into GOPs and encodes the video frames in each GOP sequentially.

[0003] In related technologies, the encoder determines the optimal image group structure by minimizing the overall cost of video frames in the pre-analysis queue. Based on this optimal image group structure, the video frames are divided into image groups, and this optimal image group structure needs to be continuously updated.

[0004] However, the related techniques require the encoder to traverse every possible image group structure, resulting in high algorithm complexity and poor encoder performance. Summary of the Invention

[0005] This application provides a video encoding method, apparatus, device, medium, and program product. The technical solution is as follows:

[0006] On the one hand, a video encoding method is provided, the method comprising:

[0007] Obtain the video frame sequence with a preset number of frames from the video to be encoded to obtain the current image group;

[0008] Based on the video features of the current image group, determine the target last frame from the current image group;

[0009] The starting frame to the target ending frame in the current image group are determined as the image group to be encoded;

[0010] Each video frame in the group of images to be encoded is encoded to obtain an encoded bitstream.

[0011] In some embodiments, determining the specified analysis block of the maximum complexity frame compared to the starting frame includes:

[0012] Using the starting frame as a reference frame, determine the reference block in the starting frame for each pre-analysis block in the maximum complexity frame;

[0013] Based on the reference block corresponding to each pre-analysis block in the maximum complexity frame, the inter-frame mode complexity of each pre-analysis block in the maximum complexity frame is determined, and the motion vector of each pre-analysis block in the maximum complexity frame is determined.

[0014] If the inter-frame mode complexity of the current pre-analysis block of the frame with the highest complexity is less than the intra-frame mode complexity of the current pre-analysis block, it is determined that the current pre-analysis block adopts inter-frame mode; otherwise, it is determined that the current pre-analysis block adopts intra-frame mode.

[0015] The pre-analysis block in the frame with the highest complexity that adopts the inter-frame mode and whose motion vector is greater than the vector threshold is determined as the designated pre-analysis block.

[0016] In some embodiments, the method further includes:

[0017] If the starting frame meets the segmentation conditions, the step of determining the target ending frame from the maximum ending frame and the at least one preset frame based on the frame features of the maximum ending frame and at least one preset frame is executed; otherwise, the maximum ending frame is determined as the target ending frame.

[0018] In some embodiments, the method further includes:

[0019] Determine the full intra-frame mode complexity of the starting frame;

[0020] If the total complexity of the starting frame is less than or equal to the complexity threshold, the starting frame satisfies the partitioning condition.

[0021] The complexity threshold is related to the number of pre-analysis blocks in the starting frame.

[0022] In some embodiments, obtaining a video frame sequence with a preset number of frames in the video to be encoded to obtain the current image group includes:

[0023] Read the video frame sequence of the first frame number in the video to be encoded;

[0024] The video frame sequence of the first frame number is stored in the pre-analysis queue;

[0025] The video frame sequence with the preset number of frames in the pre-analysis queue is determined as the current image group;

[0026] The maximum value of the first frame number is the pre-configured maximum number of read frames.

[0027] In some embodiments, the method further includes:

[0028] Each video frame in the image group to be encoded is removed from the pre-analysis queue to obtain the removed pre-analysis queue, which stores the video frame sequence of the second frame number.

[0029] After encoding each video frame in the image group to be encoded to obtain the encoded bitstream, the pre-analysis queue after being pushed out is supplemented with a video frame sequence of the first number of frames, and the step of determining the video frames of the preset number of frames in the pre-analysis queue as the current image group is repeated.

[0030] On the other hand, a video encoding apparatus is provided, the apparatus comprising:

[0031] The acquisition module is used to acquire a video frame sequence with a preset number of frames from the video to be encoded, and obtain the current image group;

[0032] The processing module is used to determine the target last frame from the current image group based on the video features of the current image group;

[0033] The determining module is used to determine the starting frame to the target ending frame in the current image group as the image group to be encoded;

[0034] The encoding module is used to encode each video frame in the image group to be encoded to obtain an encoded bitstream.

[0035] On the other hand, an encoder is provided, the encoder comprising: an encoding unit storing a computer program, the computer program being loaded and executed by the encoding unit to implement the video encoding method as described above.

[0036] On the other hand, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the video encoding method as described above.

[0037] On the other hand, a computer-readable storage medium is provided that stores a computer program, which is loaded and executed by a processor to implement the video encoding method described above.

[0038] On the other hand, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein a processor retrieves the computer instructions from the computer-readable storage medium, causing the processor to load and execute them to implement the video encoding method as described above.

[0039] The beneficial effects of the technical solutions provided in this application include at least the following:

[0040] This paper presents a novel adaptive method for partitioning image groups to be encoded. After determining the current image group from the video to be encoded, the target last frame can be determined from the current image group based on its video features. This allows the image group from the start frame to the target last frame to be defined as the image group to be encoded, thus enabling the encoding of each video frame within the image group. On one hand, compared to related techniques, this method eliminates the need to traverse every possible image group structure; the image group to be encoded can be determined simply by identifying the target last frame, significantly reducing algorithm complexity. On the other hand, the video features of the current image group can reflect various information such as scene and resolution of each video frame. This information is related to the encoding quality and performance of the video. Since the target last frame is determined based on the video features of the current image group, it better reflects the overall video features of the current image group, conforming to the encoding characteristics of the video to be encoded, improving the accuracy of the target last frame, and thus improving compression efficiency and encoding performance. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a structural block diagram of a computer system provided in an exemplary embodiment of this application;

[0043] Figure 2 This is a schematic diagram of a video encoding method provided in an exemplary embodiment of this application;

[0044] Figure 3 This is a flowchart of a video encoding method provided in an exemplary embodiment of this application;

[0045] Figure 4 This is a flowchart of a video encoding method provided in an exemplary embodiment of this application;

[0046] Figure 5 This is a flowchart of a video encoding method provided in an exemplary embodiment of this application;

[0047] Figure 6 This is a schematic diagram of a video encoding method provided in an exemplary embodiment of this application;

[0048] Figure 7 This is a block diagram of a video encoding apparatus provided in an exemplary embodiment of this application;

[0049] Figure 8This is a structural block diagram of an encoder provided in an exemplary embodiment of this application;

[0050] Figure 9 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0054] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0055] It should be noted that this application may display a prompt interface, pop-up window, or output voice prompts before and during the collection of user-related data (e.g., videos, videos to be encoded, video frames). These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps related to collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise, if no confirmation is received from the user, the steps to collect user-related data end, i.e., no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0056] First, let me briefly introduce the terms used in the embodiments of this application:

[0057] An encoder is a program or device used to compress video, reducing its size and bandwidth. An encoder divides video frames into several image groups and encodes each video frame in each image group sequentially, resulting in a encoded bitstream.

[0058] A Group of Pictures (GOP) is a collection of video frames, representing a sequence of video frames. The size of a GOP is equal to the number of frames in that GOP. For example, a GOP with a size of 16 means that it contains 16 frames.

[0059] In this embodiment, the image group includes: a current image group and an image group to be encoded. The current image group is an image group that includes a video frame sequence with a preset number of frames. For example, if the preset number of frames is 32, then the current image group is the first 32 consecutive frames in the video to be encoded. The image group to be encoded is the final image group that needs to be encoded, determined by further subdividing the current image group. For example, if the number of frames in the image group to be encoded is 16, then the image group to be encoded is the first 16 consecutive frames in the current image group.

[0060] The maximum end frame (max_index) indicates the maximum number of frames in the image group to be encoded, or the maximum number of frames supported by the image group. Specifically, the index of the maximum end frame within the current image group indicates the maximum number of frames in the image group to be encoded. For example, if the current image group has 32 frames and the maximum end frame is the 28th frame in the current image group, then the maximum number of frames supported by the image group to be encoded is 28, meaning the maximum size of the image group to be encoded is 28. In this embodiment, the maximum end frame is one of the following video frames in the current image group: a video frame that meets the resolution requirement, or the end frame of a scene before a scene change.

[0061] Video frames that meet the resolution criteria are considered high-quality frames rich in detail and texture information. Whether a frame meets the resolution criteria is determined based on its intra-frame mode complexity. A high intra-frame mode complexity indicates that the video frame contains more detail and texture information, making it a high-quality frame. When high-quality frames are encoded with high quality, they can provide a better reference for other blurry frames, improving the overall encoding quality of the video being encoded.

[0062] When the current image group involves two or more scenes, the last frame of the scene before the scene switch is the last frame of the first scene. Specifically, it is the last frame of the video frame corresponding to scene 1 before switching from scene 1 to scene 2. For example, if the current image group includes 32 frames, frames 1 to 16 involve scene 1, and frames 17 to 32 involve scene 2, then the last frame of the scene before the scene switch is frame 16.

[0063] Target end frame: Used to indicate the optimized frame number of the image group to be encoded, which is less than or equal to the maximum frame number supported by the image group. Specifically, the index of the target end frame within the current image group is used to indicate the optimized frame number of the image group to be encoded. The target end frame is determined from the maximum end frame and at least one preset frame.

[0064] At least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the largest last frame. The image group structure can be characterized by the size (number of frames) of the image group to be encoded, and the image group structure conditions refer to the conditions that the size of the image group to be encoded must satisfy. In this embodiment, the size of the image group to be encoded is set to a first size, a second size, or a third size. Then, at least one preset frame is a preset frame in the current image group that is located before the largest last frame and makes the size of the image group to be encoded the first size, the second size, or the third size. In one example, the size of the image group to be encoded is set to 16, 8, or 4. At least one preset frame includes at least one of the following: a first preset frame, a second preset frame, and a third preset frame. In one example, the first preset frame is the 16th frame, the second preset frame is the 8th frame, and the third preset frame is the 4th frame.

[0065] For example, if the current image group has 32 frames, and the largest last frame is the 28th frame in the current image group, then the maximum number of frames supported by the image group to be encoded is 28. The target last frame can be the largest last frame, i.e., the 28th frame, then the image group to be encoded consists of frames 1 to 28 of the current image group, and the size of the image group to be encoded is 28. Alternatively, the target last frame can be the first preset frame, i.e., the 16th frame, then the image group to be encoded consists of frames 1 to 16 of the current image group, and the size of the image group to be encoded is 16. Alternatively, the target last frame can be the second preset frame, i.e., the 8th frame, then the image group to be encoded consists of frames 1 to 8 of the current image group, and the size of the image group to be encoded is 8. Alternatively, the target last frame can be the third preset frame, i.e., the 4th frame, then the image group to be encoded consists of frames 1 to 4 of the current image group, and the size of the image group to be encoded is 4.

[0066] Prediction complexity: In the field of video coding, the prediction complexity of a current video frame measures the ease with which the pixel value at each position of that current video frame is predicted. In some embodiments, prediction complexity reflects the spatial and temporal variation characteristics of the current video frame and its correlation with other video frames (e.g., adjacent video frames). For the pixel value at each position of the current video frame, prediction can be performed using at least one of intra-frame mode and inter-frame mode. Accordingly, the prediction complexity in this embodiment can include the prediction complexity corresponding to intra-frame mode and the prediction complexity corresponding to inter-frame mode.

[0067] Intra-frame mode: Also known as intra-prediction mode or intra-coding mode, it uses the pixel values ​​in the current video frame and / or the relationships between pre-analysis blocks for encoding. Specifically, intra-frame mode predicts the pixel value at each position in each pre-analysis block using only information from the current video frame itself, without relying on other video frames. The prediction complexity of intra-frame mode is called intra-frame mode complexity. It should also be noted that full intra-frame mode complexity refers to the prediction complexity when using intra-frame mode to predict the pixel value at each position in each pre-analysis block; that is, when every pre-analysis block of the video frame uses intra-frame mode.

[0068] Inter-frame mode, also known as inter-frame prediction mode or inter-frame coding mode, utilizes the temporal correlation of video frames for encoding. Specifically, inter-frame mode uses a video frame as a reference frame and, through a motion search algorithm, determines the best-matching reference block in the reference frame for each pre-analyzed block in the current video frame. It then predicts the pixel value at each position of each pre-analyzed block and calculates the motion vector (MV) for each pre-analyzed block, which indicates the positional changes of the reference block in the reference frame. In some examples, the motion vector is also called the displacement vector. The prediction complexity of inter-frame mode is called inter-frame mode complexity.

[0069] Pre-analysis blocks are pre-divided pixel blocks within a video frame; that is, a video frame includes several pre-analysis blocks. Each pre-analysis block can use either intra-frame mode or inter-frame mode. When using inter-frame mode, each pre-analysis block has a corresponding motion vector.

[0070] A preanalysis block is an N×N block. N can take at least one of the following values: 16, 8, or 4. The value of N is related to at least one of the following factors: prediction accuracy, computational load, signal-to-noise ratio (SNR), video coding standard, and computer equipment performance. To achieve higher prediction accuracy, smaller preanalysis blocks can capture finer motions. However, smaller preanalysis blocks may increase computational load; therefore, a balance needs to be struck between prediction accuracy and computational load to determine the preanalysis block size. When the SNR is higher than a threshold, smaller preanalysis blocks can be used to capture details in the video frame; when the SNR is lower than a threshold, larger preanalysis blocks can be used to smooth noise. Video coding standards can specify the size of the preanalysis block and its partitioning method. Higher computer equipment performance allows for the processing of smaller preanalysis blocks. Therefore, the size of the preanalysis block can be set according to actual technical needs.

[0071] In this embodiment, we take an 8×8 block as an example for each pre-analysis block. The number of 8×8 blocks in a video frame is calculated as follows: scale the video frame to its original preset size, for example, 1 / 4 (scale the width and height to 1 / 2 of their original size), and divide the scaled video frame by 64 to obtain the number of 8×8 blocks. Scaling can be achieved through downsampling. 64 is the pixel size of an 8×8 block. For example, if the video frame size is 1920×1080, divide by 4, then divide by 64 to obtain the number of 8×8 blocks.

[0072] How to calculate the complexity of the intra-frame mode for frame A:

[0073] The A-frame is divided into several pre-analysis blocks.

[0074] Traverse several pre-analysis blocks. For the current pre-analysis block, refer to the 35 intra prediction modes in the High Efficiency Video Coding (HEVC) coding standard to predict the pixel value at each position in the current pre-analysis block, and obtain the predicted pixel value.

[0075] Subtract the predicted pixel value from the original pixel value at each position of the current pre-analysis block to obtain the residual value at each position. Perform a Hadamard Transform (HT) on the residual values ​​and sum them (satd) to obtain the intra-mode complexity of the current pre-analysis block. The Hadamard Transform and summation (stad) involves summing the absolute values ​​of the residual values ​​after the Hadamard Transform. Repeat this process until the intra-mode complexity of each pre-analysis block for several pre-analysis blocks is obtained. Accumulate the intra-mode complexity of each pre-analysis block for several pre-analysis blocks to obtain the total intra-mode complexity of frame A using the intra mode.

[0076] In some embodiments, dividing an A-frame into several pre-analysis blocks can be based on the size of the pre-analysis blocks, dividing the A-frame into several pre-analysis blocks of the same size in a left-to-right, top-to-bottom order. Optionally, dividing an A-frame into several pre-analysis blocks can be done by first scaling the A-frame to its original preset size, for example, 1 / 4 (scaling the width and height to 1 / 2 of their original sizes), and then dividing the scaled A-frame into several pre-analysis blocks.

[0077] Frame A is any frame in the current image group. The algorithm in this embodiment is applicable to calculating the intra-frame mode complexity of any frame in the current image group, such as the intra-frame mode complexity of the largest last frame, the intra-frame mode complexity of the first preset frame, the intra-frame mode complexity of the second preset frame, and the intra-frame mode complexity of each video frame in the current image group.

[0078] How to calculate the prediction complexity of frame A compared to frame B:

[0079] The A-frame is divided into several pre-analysis blocks.

[0080] Using B-frame as reference frame, traverse several pre-analysis blocks. For the current pre-analysis block in the several pre-analysis blocks, determine the corresponding reference block in B-frame, determine the inter-frame mode complexity of the current pre-analysis block, and record the motion vector (mv) of the current pre-analysis block.

[0081] If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, then the current pre-analysis block is determined to use inter-frame mode, and its prediction complexity is equal to the inter-frame mode complexity. Otherwise, the current pre-analysis block is determined to use intra-frame mode, and its prediction complexity is equal to the intra-frame mode complexity. This process is repeated until the prediction complexity of each of several pre-analysis blocks is determined. In this embodiment, the prediction complexity of each pre-analysis block can be understood as the smaller of the intra-frame mode complexity and the inter-frame mode complexity. The prediction complexities of each of several pre-analysis blocks are summed to obtain the prediction complexity of frame A compared to frame B.

[0082] The calculation method for determining the inter-frame mode complexity of the current pre-analysis block is as follows: Based on the reference block corresponding to the current pre-analysis block in frame A, the pixel value at each position of the current pre-analysis block is predicted using inter-frame mode. The predicted pixel value at each position is then subtracted from the original pixel value of the reference block to obtain the residual value at each position. A Hadamard transform is performed on the residual values ​​at each position, and the summation (stad) yields the inter-frame mode complexity of the current pre-analysis block. The Hadamard transform and summation (stad) involves summing the absolute values ​​of the residuals after the Hadamard transform. In some possible implementations, the motion vector (mv) of the current pre-analysis block can be multiplied by a preset coefficient and added to the inter-frame mode complexity to obtain the final inter-frame mode complexity.

[0083] In some embodiments, dividing the A-frame into several pre-analysis blocks can be based on the size of the pre-analysis blocks, dividing the A-frame into several pre-analysis blocks of the same size in a left-to-right, top-to-bottom order. Optionally, dividing the A-frame into several pre-analysis blocks can be done by first scaling the A-frame to its original preset size, for example, 1 / 4 (scaling the width and height to 1 / 2 of their original sizes), scaling the B-frame to its original preset size as well, for example, 1 / 4 (scaling the width and height to 1 / 2 of their original sizes), and then dividing the scaled A-frame into several pre-analysis blocks.

[0084] Frame A and Frame B are any two frames in the current image group. It should also be noted that the algorithm in this embodiment is applicable to calculating the prediction complexity of any frame A compared to any frame B in the current image group. For example, the following are related to: the prediction complexity of the largest last frame compared to the starting frame, the prediction complexity of the largest last frame compared to the previous frame, the prediction complexity of the first preset frame compared to the starting frame, and the prediction complexity of the second preset frame compared to the starting frame.

[0085] How to calculate the prediction complexity of frame A compared to frames B and C:

[0086] The A-frame is divided into several pre-analysis blocks.

[0087] Using only B-frames as reference frames, several pre-analysis blocks are traversed. For the current pre-analysis block, the corresponding reference block in the B-frame is determined, the inter-frame mode complexity of the current pre-analysis block is determined, and the motion vector (mv) of the current pre-analysis block is recorded. If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, it is determined that the current pre-analysis block adopts inter-frame mode and the prediction complexity of the current pre-analysis block is inter-frame mode complexity; otherwise, it is determined that the current pre-analysis block adopts intra-frame mode and the prediction complexity of the current pre-analysis block is intra-frame mode complexity. This process is repeated until the first prediction complexity of each pre-analysis block is determined.

[0088] Using only frame C as the reference frame, several pre-analysis blocks are traversed. For the current pre-analysis block, the corresponding reference block in frame C is determined, the inter-frame mode complexity of the current pre-analysis block is determined, and the motion vector (mv) of the current pre-analysis block is recorded. If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, it is determined that the current pre-analysis block adopts inter-frame mode and the prediction complexity of the current pre-analysis block is inter-frame mode complexity; otherwise, it is determined that the current pre-analysis block adopts intra-frame mode and the prediction complexity of the current pre-analysis block is intra-frame mode complexity. This process is repeated until the second prediction complexity of each pre-analysis block is determined.

[0089] Simultaneously, using B-frames and C-frames as reference frames, several pre-analysis blocks are traversed. For the current pre-analysis block among the several pre-analysis blocks, the corresponding reference block in the B-frame is determined, and the same steps as in the aforementioned embodiment using B-frames as reference frames are performed to obtain the first sub-prediction complexity of each of the several pre-analysis blocks; and, the corresponding reference block in the C-frame is determined, and the same steps as in the aforementioned embodiment using C-frames as reference frames are performed to obtain the second sub-prediction complexity of each of the several pre-analysis blocks; the first sub-prediction complexity and the second sub-prediction complexity of each of the several pre-analysis blocks are weighted and summed to obtain the third prediction complexity of each of the several pre-analysis blocks.

[0090] In this embodiment, the reference block corresponding to the current pre-analysis block in frame B is the same as or different from the reference block when frame B is the only reference frame. The reference block corresponding to the current pre-analysis block in frame C is the same as or different from the reference block when frame C is the only reference frame. When both frame B and frame C are used as reference frames, each pre-analysis block has two reference blocks.

[0091] The minimum prediction complexity among the first, second, and third prediction complexities of each of the several pre-analysis blocks is determined as the final prediction complexity for each pre-analysis block. Simultaneously, the reference block (which can be one or two) corresponding to the minimum complexity is determined as the final reference block, and the motion vector corresponding to the minimum complexity is determined as the final motion vector. The final prediction complexities of each of the several pre-analysis blocks are summed to obtain the prediction complexity of frame A compared to frames B and C.

[0092] In some embodiments, dividing the A-frame into several pre-analysis blocks can be based on the size of the pre-analysis blocks, dividing the A-frame into several pre-analysis blocks of the same size in a left-to-right, top-to-bottom order. Optionally, dividing the A-frame into several pre-analysis blocks can be done by first scaling the A-frame to its original preset size, for example, 1 / 4 (scaling the width and height to 1 / 2 of their original sizes), scaling the B-frame and C-frame to their original preset sizes, 1 / 4 (scaling the width and height to 1 / 2 of their original sizes), and then dividing the scaled A-frame into several pre-analysis blocks.

[0093] Frames A, B, and C are any three frames in the current image group. It should also be noted that the algorithm in this embodiment is applicable to calculating the prediction complexity of any frame A in the current image group compared to any frame B and any frame C, for example, as discussed below: the prediction complexity of the first preset frame compared to the starting frame and the maximum ending frame.

[0094] Figure 1 This is a structural block diagram of a computer system provided in an exemplary embodiment of this application. The computer system 100 can be implemented as a system architecture for a video encoding method. The computer system 100 includes: a terminal 120, an encoder 130, and a server 140.

[0095] Terminal 120 can be at least one of the following: mobile phone, tablet computer, vehicle terminal (vehicle system), wearable device, PC (Personal Computer), unmanned reservation terminal, smart home appliance, smart voice interaction device, or unmanned vending terminal. A client application for the target application can be installed and run on terminal 120. This target application can be a video encoding / decoding application or an application that provides video encoding / decoding functions; this application embodiment does not limit the specific form of the target application. Furthermore, this application embodiment does not limit the form of the target application, including but not limited to Apps (Applications), mini-programs, etc., installed on terminal 120, and can also be in web page form.

[0096] The encoder 130 can be an electronic device dedicated to video encoding / decoding, and can be a standalone device or built into a professional video processing device. The video processing device includes, but is not limited to, at least one of the following: video editing workstation, high-definition camera / webcam, professional server, broadcast television transmission platform, and video transcoding platform. The encoder 130 includes an encoding unit for implementing video encoding. The encoding unit can be implemented as at least one of the following: chip, processor, hardware circuit, and logic circuit; this application embodiment does not limit this.

[0097] Server 140 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. Server 140 can be the backend server for the target application of terminal 120 or the backend server for encoder 130, used to provide backend services. Optionally, server 140 can also be implemented as a node in a blockchain system.

[0098] Terminal 120, encoder 130 and server 140 communicate with each other via wired or wireless network.

[0099] The video encoding method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Figure 1 Taking the implementation environment of the scheme shown as an example, the video encoding method can be executed by the terminal 120. For example, the video encoding method can be executed by the client of the target application installed and running in the terminal 120, the video encoding method can be executed by the encoder 130, the video encoding method can be executed by the server 140, or the terminal 120 and the server 140 can interact and cooperate to execute the method, or the terminal 120 and the encoder 130 can interact and cooperate to execute the method, or the encoder 130 and the server 140 can interact and cooperate to execute the method, or the terminal 120, the encoder 130 and the server 140 can interact and cooperate to execute the method. The embodiments of this application do not limit the method.

[0100] Those skilled in the art will understand that the number of terminals 120, encoders 130, and servers 140 can be more or less. For example, there may be only one terminal 120, encoder 130, and server 140, or dozens, hundreds, or more terminals 120, encoders 130, and servers 140. This application does not limit the number or type of terminals 120, encoders 130, and servers 140.

[0101] In related technologies, video coding and image grouping methods include the following:

[0102] Method 1: Divide the input video into several image groups according to a fixed number of frames, and encode each video frame in each image group sequentially. The fixed number of frames is a pre-configured maximum number of frames (gop_size). This method cannot adaptively divide image groups based on the video content.

[0103] Method 2: Use the Viterbi algorithm to find the optimal picture group structure by minimizing the total cost of video frames in the pre-analysis queue. The optimal picture group structure is defined as the one that minimizes the cumulative cost of all video frames in the pre-analysis queue. Minimizing the cumulative cost means that temporal correlation is fully utilized, which minimizes the residual value in the formal encoding process, thereby reducing the bit consumption of encoding.

[0104] Assuming there are a total of n frames in the pre-analysis queue, the Viterbi algorithm is used to calculate the cumulative minimum cost, which involves splitting the overall minimum cost into three parts: 1. The minimum cost of the first i frames; 2. The cost of inserting B frames from frame i to frame (n-1); 3. The cost of using P frames in frame n. The optimal image group structure is found through dynamic programming. The image group structure is represented by a list of frame types, with each image group consisting of the next B frame from the current P frame to the next P frame. The steps are briefly described below:

[0105] Suppose we need to determine the optimal structure for the first i frames, where 2 <= i <= n.

[0106] 1. The cost of initializing the optimal frame type list is 1LL<<62.

[0107] 2. Traverse j within the range of 2 to i-1 to find the list of frame types with the lowest cost.

[0108] 2.1) Construct a list of candidate frame types.

[0109] 2.1.1) Copy the optimal frame type of the first j frames to the candidate frame type list;

[0110] 2.1.2) Set the frame type from j+1 to i-1 to B-frame;

[0111] 2.1.3) Set the frame type of the i-th frame to P frame.

[0112] 2.2) The cost of calculating the list of candidate frame types.

[0113] 2.2.1) Calculate the cost of the next P-frame with the current P-frame as a reference, according to the order of the pre-analysis queue from front to back, and add the cost to the candidate frame type list;

[0114] 2.2.2) Calculate the cost of B-frames between two P-frames and add it to the cost of the candidate frame type list:

[0115] 2.2.2.1) If there is only one B-frame, calculate the cost of that B-frame with reference to the current P-frame and the next P-frame;

[0116] 2.2.2.2) If there are multiple B-frames, then:

[0117] a) Calculate the cost of the intermediate frame between two P-frames, with reference to the current P-frame and the next P-frame. The index of the intermediate frame in the pre-analysis queue is middle = cur_p + (next_p - cur_p) / 2; where cur_p is the index of the current P-frame and next_p is the index of the next P-frame;

[0118] b) For a B-frame between the current P-frame and the intermediate frame, calculate the cost of this B-frame with reference to the current P-frame and the intermediate frame.

[0119] c) For a B-frame between the next P-frame and the intermediate frame, calculate the cost of this B-frame with reference to the next P-frame and the intermediate frame.

[0120] 2.3) If the cost of the current candidate frame type list is less than the optimal cost, then update the optimal cost to the current cost and record the current candidate frame type list.

[0121] Method 3: Divide the image into groups based on the similarity of video frames. The steps are briefly described below:

[0122] 1. Iterate through frames 0 to n-2 in the pre-analysis queue and set the frame type for each frame.

[0123] Assume the current analysis is of frame i:

[0124] 1.1) Calculate the cost of the (i+2)-th frame relative to the i-th frame, which is recorded as cost2p1; if the proportion of pre-analysis blocks that adopt the intra (intra) mode for the (i+2)-th frame compared with the i-th frame exceeds the threshold, set the (i+1)-th frame and the i-th frame as P-frames, end the current loop, then return to step 1, and continue determining the frame type starting from the (i+2)-th frame; otherwise, enter step 1.2);

[0125] 1.2) Calculate the cost when the (i+1)-th frame takes the i-th frame and the (i+2)-th frame as references, which is recorded as cost1b1. Calculate the cost of the (i+1)-th frame compared with the i-th frame, which is recorded as cost1p0; calculate the cost of the (i+2)-th frame compared with the (i+1)-th frame, which is recorded as cost2p0. If cost1p0+cost2p0 < cost1b1+cost2p1, set the (i+1)-th frame as a P-frame, end the current loop, then return to step 1, and continue determining the frame type starting from the (i+1)-th frame; otherwise, enter step 1.3);

[0126] 1.3) Set the (i+1)-th frame as a B-frame, traverse from the (i+2)-th frame to the end of the pre-analysis queue, and determine the frame type of each frame. Record the current frame as j;

[0127] 1.3.1) Calculate the cost of the (j+1)-th frame compared with the i-th frame; if the cost exceeds the threshold, or the proportion of pre-analysis blocks adopting intra (intra) mode exceeds the threshold, set the j-th frame as a P-frame, end the current loop, then return to step 1, and continue determining the frame type starting from the j-th frame;

[0128] 1.3.2) Otherwise, set the j-th frame as a B-frame, and repeat step 1.3).

[0129] For Method 2 and Method 3, it is necessary to traverse every possible group of pictures structure, which has high algorithm complexity and limited overall performance. Table 1 shows the results obtained on the msu2022 test set when the maximum group of pictures size is set to 16, with x265 disabling adaptive group of pictures, the algorithm of Method 3 taken as Test 1 (test1), and the algorithm of Method 2 taken as Test 2 (test2).

[0130] The first row of Table 1 shows the results of Test 1, and the second row shows the results of Test 2. A negative speed value in Table 1 indicates a speed reduction. For the video quality assessment metrics in Table 1—Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Visual Multimethod Assessment Fusion (VMAF)—negative values ​​indicate gain, while positive values ​​indicate loss. Table 1 shows that the algorithm of Method 3 experiences an overall speed reduction of 4.95%, and a significant loss in all video quality assessment metrics; the algorithm of Method 2 experiences an overall speed reduction of 6.84%, with some loss in video quality assessment metrics, but the overall gain is not significant.

[0131] Table 1

[0132]

[0133]

[0134] This application provides a video encoding method that can adaptively divide a group of images to be encoded based on video features, so as to encode each video frame in the group of images to be encoded. Figure 2 This is a schematic diagram of a video encoding method provided in an exemplary embodiment of this application. The method is executed by a computer device, which includes... Figure 1 The terminal 120 and / or encoder 130 and / or server 140. In this embodiment, taking the execution by encoder 130 as an example, the steps are briefly described as follows:

[0135] Obtain a video frame sequence with a preset number of frames from the video to be encoded 10 to obtain the current image group 20. The preset number of frames is the number of frames pre-configured in the current image group 20. For example, if the preset number of frames is 32, then the current image group is the first 32 frames of the video to be encoded 10. Based on the video features of the current image group 20, determine the target end frame from the current image group 20. For example, the target end frame is the 4th frame in the current image group 20. Determine the image group 30 to be encoded from the start frame to the target end frame in the current image group 20. Encode each video frame in the image group 30 to be encoded to obtain the encoded bitstream 40. And output the encoded bitstream 40.

[0136] It should also be noted that the steps in this embodiment can be executed iteratively. For example, each video frame in the image group 30 to be encoded is derived from the current image group 20, resulting in the derived current image group. After encoding each video frame in the image group 30 to obtain the encoded bitstream 40, video frames in the video 10 to be encoded are acquired to supplement the derived current image group into a new current image group 20 with a preset number of frames. The step of determining the target last frame from the current image group 20 based on the video features of the current image group 20 is then re-executed, thereby continuing iterative encoding until the entire video 10 to be encoded has been encoded. Please refer to the following for detailed examples.

[0137] Figure 3 This is a flowchart illustrating a video encoding method provided in an exemplary embodiment of this application. The method is executed by a computer device, which includes... Figure 1 The terminal 120 and / or encoder 130 and / or server 140. Taking the execution by encoder 130 as an example, the method includes at least some of the steps 220, 240, 260, and 280:

[0138] Step 220: Obtain the video frame sequence of the preset number of frames in the video to be encoded to obtain the current image group.

[0139] The video to be encoded is the video that needs to be encoded. The video to be encoded consists of several video frames, which together form a video frame sequence. That is, the video frames in the video to be encoded are arranged in order, for example, in chronological order.

[0140] The current image group is a sequence of video frames with a preset number of frames. These preset frame sequences are consecutive and arranged chronologically in the video to be encoded. The preset frame number is a pre-configured number of frames for the current image group, and its maximum value is the maximum pre-configured number of frames for the current image group, which can be set according to actual technical needs. For example, the maximum preset frame number can be set to 32.

[0141] It should also be noted that when determining the current image group, if the number of remaining uncoded frames in the video to be encoded is greater than or equal to this maximum value, then the preset frame number corresponding to the current image group is this maximum value. At the end of the video to be encoded, if the number of remaining uncoded frames in the video to be encoded is less than this maximum value, then the preset frame number corresponding to the current image group is the number of remaining uncoded frames.

[0142] The current image group can be determined directly based on the video to be encoded. For example, the video to be encoded is acquired, and its video frames are read frame by frame to obtain a video frame sequence with a preset number of frames, thus obtaining the current image group. Alternatively, the video to be encoded can be pre-divided into several video frames, and several video frames can be acquired. A video frame sequence with a preset number of frames can then be extracted from these video frames to obtain the current image group.

[0143] The current image group can also be obtained based on a pre-analysis queue. The pre-analysis queue is a fixed-size queue used to store the video frames to be analyzed frame by frame. The size of the pre-analysis queue can be set according to actual technical needs. For example, the size of the pre-analysis queue can be set to at least one of the following: 48, 64, or 80, that is, the pre-analysis queue is used to store 48 video frames, 64 video frames, or 80 video frames. In some embodiments, the size of the pre-analysis queue is consistent with the maximum number of frames pre-configured by the encoder.

[0144] For example, the process involves reading the first number of video frames from the video to be encoded; storing this first number of video frames in a pre-analysis queue; and determining the current image group from the video frames in the pre-analysis queue with a preset number of frames. The maximum value of the first number of frames is the pre-configured maximum number of frames to read. For instance, if the size of the pre-analysis queue is set to 48 and the preset number of frames is set to 32, then the first 48 video frames from the video to be encoded are read and stored in the pre-analysis queue, and the first 32 video frames in the pre-analysis queue are determined as the current image group.

[0145] Step 240: Based on the video features of the current image group, determine the target last frame from the current image group.

[0146] Video features refer to the features related to each video frame in the current image group.

[0147] In some embodiments, video features can be sequence features of a sequence of video frames formed between consecutive or non-consecutive video frames in the current image group, or frame features of a single video frame in the current image group. Sequence features can be features related to sharpness or features related to the scene.

[0148] In some embodiments, for the sequence features in the video features of the current image group, the sequence features are used to characterize at least one of the following information: motion information, scene change information, and temporal correlation information. Motion information includes at least one of the following: the trajectory, speed, and direction of objects in the video frame sequence of the current image group. Scene change information is used to characterize scene transitions. Temporal correlation information is used to characterize the temporal order and dependencies between the video frame sequences of the current image group, which is beneficial for capturing the development and logic of the plot content. For the frame features in the video features of the current image group, the frame features are used to characterize at least one of the following information: texture information, edge information, and brightness and color change information. Regions with rich texture information have more complex pixel value changes, and texture information can characterize the complexity of video frames. Edge regions are areas in video frames where the degree of pixel value change is greater than a threshold; edge information is used to indicate the structure of objects in video frames. Brightness and color change information is used to indicate the visual dynamics and visual richness of video frames.

[0149] In one possible implementation, the video frame sequence corresponding to the current image group is input into a feature extraction model to obtain the video features of the current image group. The video features include the sequence features of the current image group and the frame features of each video frame. The feature extraction model is a pre-trained neural network model, which may include at least one of the following: Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM) networks. CNNs are used to extract information such as the shape, texture, and color of objects in the video frames, as well as information such as the category and scene type of the objects. Recurrent Neural Networks and LSTM networks are used to capture the temporal order and dependencies between video frames and to extract motion-related information of objects.

[0150] The target last frame is a video frame determined from the current image group that can serve as the last frame of the image group to be encoded. The image group to be encoded is the final group of images to be encoded, determined by further subdividing the current image group. The method for determining the target last frame will be described in detail in the following embodiments.

[0151] In some embodiments, the size of the image group to be encoded is less than or equal to the size of the current image group, that is, the number of frames in the image group to be encoded is less than or equal to the number of frames in the current image group. For example, the current image group is the first 32 consecutive frames in the video to be encoded, and the image group to be encoded is the first 16 consecutive frames in the current image group.

[0152] Step 260: Determine the image group to be encoded from the starting frame to the target ending frame in the current image group.

[0153] The image group to be encoded is the final group of images that needs to be encoded, determined by further subdividing the current image group. The starting frame is the first video frame of the current image group, and can also be understood as the first video frame of the image group to be encoded. After determining the target ending frame from the current image group, the image group from the starting frame to the target ending frame is defined as the image group to be encoded.

[0154] Step 280: Encode each video frame in the image group to be encoded to obtain the encoded bitstream.

[0155] For example, each video frame in the image group to be encoded is encoded sequentially to obtain the encoded bitstream. The encoding method can be at least one of the following: entropy coding and lossy coding. Among them, entropy coding is a lossless compression method. Lossy coding can be implemented through intra-frame and inter-frame compression, motion compensation, etc. The encoding in this embodiment also conforms to the encoding and decoding standards, such as the HEVC standard, specifically the H.26x series and MPEG series standards.

[0156] In summary, the video encoding method provided in this application obtains a video frame sequence with a preset number of frames from the video to be encoded, thus obtaining the current image group. The preset number of frames is a pre-configured number of frames for the current image group. Based on the video features of the current image group, the target end frame is determined from the current image group. The starting frame to the target end frame in the current image group is determined as the image group to be encoded. Each video frame in the image group to be encoded is encoded to obtain the encoded bitstream. Accordingly, a novel adaptive method for dividing the image group to be encoded is provided. After determining the current image group from the video to be encoded, the target end frame can be determined from the current image group based on its video features, and then the starting frame to the target end frame of the current image group can be determined as the image group to be encoded, thereby achieving the encoding of each video frame in the image group to be encoded. On the one hand, compared with the methods of related technologies, it is not necessary to traverse every possible image group structure. The image group to be encoded can be determined by determining the target end frame, which greatly reduces the algorithm complexity. On the other hand, the target end frame is determined based on the video features of the current image group, which conforms to the encoding characteristics of the video to be encoded, improving the accuracy of the target end frame, thereby improving compression efficiency and encoding performance.

[0157] It should also be noted that the following embodiments may use terms such as first, second, third, etc. to describe various information, but these terms should not be limited to these terms and do not indicate the order of these information or the sequence of calculation steps. They are only used to distinguish information of the same type from each other.

[0158] In some embodiments, at least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the largest last frame. The image group structure can be characterized by the size (number of frames) of the image group to be encoded, and the image group structure conditions refer to the conditions that the size of the image group to be encoded must satisfy. In this embodiment, if the size of the image group to be encoded is set to a first size, a second size, or a third size, then at least one preset frame is a preset frame in the current image group that is located before the largest last frame and makes the size of the image group to be encoded the first size, the second size, or the third size. In one example, the size of the image group to be encoded is set to 16, 8, or 4.

[0159] At least one preset frame includes at least one of the following: a first preset frame, a second preset frame, and a third preset frame. In one example, the first preset frame is frame 16, the second preset frame is frame 8, and the third preset frame is frame 4. When frame 16 is the target end frame, the size of the image group to be encoded is 16; when frame 8 is the target end frame, the size of the image group to be encoded is 8; and when frame 4 is the target end frame, the size of the image group to be encoded is 4.

[0160] To reduce algorithm complexity, the process of adaptively dividing the images to be encoded into groups is split into two parts.

[0161] Part 1: The size of the image group to be encoded is divided into a numerical value (max_index). This value can be a non-power of two (except for 32, 16, 8, 4, 2) or a power of two. This value is used to indicate the maximum number of frames supported by the image group to be encoded.

[0162] Part Two: When the value (max_index) obtained in Part One is greater than or equal to 16, determine whether the size of the image group to be encoded needs to be further divided into 16, 8 or 4. The value of 16, 8 or 4 here is used to indicate the optimal number of frames for the image group to be encoded.

[0163] The first part of the process is also referred to as: determining whether it is necessary to divide the images to be encoded into groups according to sharpness / scene. The second part of the process is also referred to as: determining whether it is necessary to divide the size of the image groups to be encoded into 16, 8, or 4. It should also be noted that, according to actual needs and / or experimental verification, 16, 8, or 4 in this embodiment can be set to other values, which are not limited here.

[0164] In some other possible implementations, when the value (max_index) obtained in the first part is greater than or equal to 8 and less than 16, execution can directly begin by determining whether the size of the image group to be encoded needs to be further divided into 8 or 4. Alternatively, when the value (max_index) obtained in the first part is greater than or equal to 4 and less than 8, execution can directly begin by determining whether the size of the image group to be encoded needs to be further divided into 4. That is, based on the sequence number of at least one preset frame, the closest preset frame that is less than the value (max_index) is determined, and execution begins by determining whether the size of the image group to be encoded needs to be divided into the sequence number corresponding to the preset frame, thereby determining the image group to be encoded. Alternatively, in some other possible implementations, when the value (max_index) obtained in the first part is less than 16, subsequent judgments can be skipped, and the largest last frame can be directly determined as the target last frame, thereby determining the image group to be encoded.

[0165] For example, step 240 specifically includes steps 300 and 400:

[0166] Step 300: Based on the sequence features of the current image group, determine the largest last frame from the current image group;

[0167] Step 400: Based on the frame features of the maximum last frame and at least one preset frame, determine the target last frame from the maximum last frame and at least one preset frame; wherein, the sequence number of the maximum last frame in the current image group is used to indicate the maximum number of frames in the image group to be encoded, the sequence number of the target last frame in the current image group is used to indicate the optimized number of frames in the image group to be encoded, the optimized number of frames is less than or equal to the maximum number of frames, and at least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the maximum last frame.

[0168] Video features include sequence features and frame features. Sequence features refer to the features of several consecutive or non-consecutive video frames in the current image group; this embodiment uses the features of consecutive video frames as an example. Frame features refer to the features of a single video frame in the current image group. As an example, sequence features can be features related to sharpness or features related to the scene. For example, sharpness can be related to at least one of resolution, bitrate, and number of pixels; the scene can be related to at least one of objects, scenery, people, lighting, color, composition, and shooting method.

[0169] The last frame refers to the final frame. The maximum last frame is the last frame corresponding to the maximum number of frames in the image group to be encoded. For example, if the maximum number of frames in the image group to be encoded is 16, then the maximum last frame is the 16th frame. The target last frame is the last frame corresponding to the optimized number of frames in the image group to be encoded. For example, after optimization, if the optimized number of frames in the image group to be encoded is 8, then the target last frame is the 8th frame. The sequence number of the maximum last frame within the current image group indicates the maximum number of frames in the image group to be encoded. The sequence number of the target last frame within the current image group indicates the optimized number of frames in the image group to be encoded, where the optimized number of frames is less than or equal to the maximum number of frames.

[0170] At least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the largest last frame. The image group structure can be characterized by the size (number of frames) of the image group to be encoded, and the image group structure conditions refer to the conditions that the size of the image group to be encoded must satisfy. In this embodiment, the size of the image group to be encoded is set to a first size, a second size, or a third size. Then, at least one preset frame is a preset frame in the current image group that is located before the largest last frame and makes the size of the image group to be encoded the first size, the second size, or the third size. At least one preset frame includes at least one of the following: a first preset frame, a second preset frame, and a third preset frame. In one example, the size of the image group to be encoded is set to 16, 8, or 4, the first preset frame is the 16th frame, the second preset frame is the 8th frame, and the third preset frame is the 4th frame.

[0171] For example, based on the sequence characteristics of the current image group, the maximum end frame (max_index) is determined from the current image group. Based on the frame characteristics of the maximum end frame and at least one preset frame, the target end frame is determined from the maximum end frame and at least one preset frame. Here, the index of the maximum end frame in the current image group indicates the maximum number of frames in the image group to be encoded, and the index of the target end frame in the current image group indicates the optimized number of frames in the image group to be encoded, where the optimized number of frames is less than or equal to the maximum number of frames. That is, the size of the image group to be encoded is less than or equal to the size of the current image group.

[0172] This embodiment describes a method for adaptively dividing the image group to be encoded. By determining the maximum last frame, the maximum number of frames supported by the image group to be encoded can be determined. By determining the target last frame, the maximum number of frames in the image group to be encoded is further optimized, thereby improving the accuracy of the image group to be encoded, which is beneficial for subsequent encoding of the image group to be encoded.

[0173] Specifically, step 400 is implemented as steps 420 and 440:

[0174] Step 420: Statistical features of the frame features of the largest last frame and at least one preset frame;

[0175] Step 440: Based on the relationship between statistical features and thresholds, determine the target end frame from the largest end frame and at least one preset frame; wherein the threshold is related to statistical features and at least one preset frame.

[0176] Statistical features are obtained by performing operations on the frame features of the largest last frame and at least one preset frame. In some embodiments, statistical features include features related to prediction complexity, features related to motion vectors, and features related to noise. For example, features related to prediction complexity may be prediction complexity or the ratio between different prediction complexities; features related to noise may be noise level; and features related to motion vectors may be motion vectors or the ratio between different motion vectors.

[0177] The threshold is pre-configured and is related to the statistical feature and at least one preset frame. Specifically, the type, value, and range of the threshold are related to the type of the statistical feature and to each of the at least one preset frame. For example, if the statistical feature is noise-related, the threshold is a pre-configured noise threshold. If the statistical feature is related to a first preset frame, the value and range of the threshold are related to the first preset frame.

[0178] In this embodiment, by statistically analyzing the frame features of the maximum last frame and at least one preset frame, the target last frame can be determined from the maximum last frame and at least one preset frame based on the relationship between these statistical features and the threshold. This determines the final size of the image group to be encoded, simplifies the algorithm steps, reduces the algorithm complexity, and improves the accuracy of the image group to be encoded, which is beneficial for subsequent encoding of the image group.

[0179] • Determine whether the size of the image group to be encoded needs to be divided into 16.

[0180] In some embodiments, at least one preset frame includes a first preset frame, and the statistical features include prediction complexity-related features of the maximum last frame and the first preset frame, respectively, and noise-related features of the maximum last frame; wherein, the prediction complexity of the maximum last frame is used to measure the difficulty of predicting the pixel value at each position of the maximum last frame. The prediction complexity of the first preset frame is used to measure the difficulty of predicting the pixel value at each position of the first preset frame. Step 420 is implemented as steps 4211, 4212, 4213, 4214, 4215, and 4216:

[0181] Step 4211: Determine the prediction complexity of the maximum end frame compared to the start frame; determine the prediction complexity of the maximum end frame compared to the previous frame; determine the prediction complexity of the first preset frame compared to the start frame; determine the prediction complexity of the first preset frame compared to the start frame and the maximum end frame.

[0182] Step 4212: Calculate the first proportion of the specified pre-analysis block in the largest last frame;

[0183] Step 4213: Calculate the second ratio of the prediction complexity of the largest last frame compared to the starting frame, and the full intra-frame mode complexity compared to the largest last frame.

[0184] Step 4214: Calculate the third ratio of the prediction complexity of the largest last frame compared to the previous frame and the intra-frame mode complexity compared to the largest last frame.

[0185] Step 4215: Calculate the prediction complexity of the first preset frame compared to the starting frame and the maximum last frame, and the fourth ratio of the prediction complexity of the first preset frame compared to the starting frame.

[0186] Step 4216: Calculate the noise level of the largest last frame; where the full intra-frame mode complexity is the prediction complexity when predicting the pixel value at each position of each pre-analysis block using intra-frame mode; the specified pre-analysis block is a pre-analysis block using inter-frame mode and whose motion vector is greater than the vector threshold, the vector threshold is pre-configured, and the pre-analysis block is a pre-divided pixel block.

[0187] The start frame is the first video frame of the current picture group, also understood as the first video frame of the picture group to be encoded. The maximum end frame indicates the maximum number of frames supported by the picture groups to be encoded from the current picture group; it is also understood as the last frame when the picture group to be encoded is of maximum size. The preceding frame is a video frame that precedes and is adjacent to the maximum end frame. For example, the maximum end frame is frame max_indexx, and the preceding frame is frame max_index-1. Or, the maximum end frame is frame n, and the preceding frame is frame n-1.

[0188] The first preset frame is the preset frame in the current image group that precedes the largest last frame. The designated pre-analysis block is a pre-analysis block that uses inter-frame mode and whose motion vector is greater than a vector threshold, which is pre-configured. The pre-analysis block is a pre-divided pixel block; in this embodiment, the pre-analysis block is set to 8×8 blocks. In this embodiment, since the prediction complexity is determined by the largest last frame compared to the starting frame, the designated pre-analysis block is the designated pre-analysis block in the largest last frame when the starting frame is used as the reference frame.

[0189] The first ratio (large_mv_frac) is the proportion of the specified pre-analysis blocks in the largest last frame when the start frame is used as the reference frame. The second ratio (cost_ratio) is the ratio of the prediction complexity of the largest last frame compared to the start frame (when the start frame is used as the reference frame), and the ratio of the intra-frame mode complexity of the largest last frame. This can also be understood as the ratio of the prediction complexity of the largest last frame compared to the start frame to the intra-frame mode complexity of the largest last frame. The third ratio (inter_with_prev) is the ratio of the prediction complexity of the largest last frame compared to the previous frame (when the previous frame is used as the reference frame), and the ratio of the intra-frame mode complexity of the largest last frame. This can also be understood as the ratio of the prediction complexity of the largest last frame compared to the previous frame to the intra-frame mode complexity of the largest last frame. The fourth ratio (poc16_inter) is the ratio of the prediction complexity of the first preset frame compared to the start frame and the maximum end frame (when both the start frame and the maximum end frame are used as reference frames), to the prediction complexity of the first preset frame compared to the start frame (when the start frame is used as a reference frame). It can also be understood as the ratio of the prediction complexity of the first preset frame compared to the start frame and the maximum end frame to the prediction complexity of the first preset frame compared to the start frame. The noise level (noise_level) is calculated based on the gradient.

[0190] In some embodiments, the threshold includes a noise threshold, a first threshold, a second threshold, a third threshold, and a fourth threshold; step 440 is implemented as step 441:

[0191] Step 441: If the noise level of the maximum end frame is less than the noise threshold, and the first proportion is greater than the first threshold, and the second proportion is greater than the second threshold, and the third proportion is less than the third threshold, and the fourth proportion is greater than the fourth threshold, then the first preset frame is determined as the target end frame; otherwise, the maximum end frame is determined as the target end frame; wherein, the noise threshold, the first threshold, the second threshold, the third threshold, and the fourth threshold are pre-configured and used to indicate the characteristics of the maximum end frame and the first preset frame.

[0192] The noise level (noise_level), the first ratio (large_mv_frac), the second ratio (cost_ratio), the third ratio (inter_with_prev), and the fourth ratio (poc16_inter) each have corresponding pre-configured thresholds. The value of these thresholds is used to indicate the characteristics of the maximum end frame and the first preset frame; that is, the value of the threshold can be understood as being related to the maximum end frame and the first preset frame. In some embodiments, the value of the threshold is specifically related to the statistical characteristics of the maximum end frame and the first preset frame; that is, the value of the threshold is used to indicate the statistical characteristics of the maximum end frame and the first preset frame.

[0193] For example, if the noise level of the largest last frame is less than the noise threshold, and the first ratio (large_mv_frac) is greater than the first threshold, the second ratio (cost_ratio) is greater than the second threshold, the third ratio (inter_with_prev) is less than the third threshold, and the fourth ratio (poc16_inter) is greater than the fourth threshold, then it means that the features of the largest last frame and the first preset frame meet the pre-configured conditions, and the size of the image group to be encoded can be divided into 16, and the first preset frame can be determined as the target last frame, that is, the 16th frame is determined as the target last frame, and the size of the image group to be encoded is 16; otherwise, the largest last frame is determined as the target last frame, that is, the largest last frame (max_index) is determined as the target last frame, that is, the size of the image group to be encoded is the maximum supported size.

[0194] This embodiment provides a method for determining whether the size of the image group to be encoded needs to be divided into 16. The first preset frame is predetermined as the 16th frame, which simplifies the algorithm steps and reduces the algorithm complexity.

[0195] To determine the prediction complexity of the maximum end frame compared to the start frame, the calculation method for the prediction complexity of frame A compared to frame B can be referenced, where frame A is the maximum end frame and frame B is the start frame. Step 4211 is implemented as steps 42111, 42112, 42113, 42114, and 42115:

[0196] Step 42111: Divide the largest last frame into several pre-analysis blocks;

[0197] Step 42112: Using the starting frame as the reference frame, traverse several pre-analysis blocks. For the current pre-analysis block in the several pre-analysis blocks, determine the corresponding reference block in the starting frame.

[0198] Step 42113: Based on the reference block corresponding to the current pre-analysis block in the starting frame, determine the inter-frame mode complexity of the current pre-analysis block, and determine the motion vector of the current pre-analysis block;

[0199] Step 42114: If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, determine that the current pre-analysis block adopts inter-frame mode and the prediction complexity of the current pre-analysis block is the inter-frame mode complexity; otherwise, determine that the current pre-analysis block adopts intra-frame mode and the prediction complexity of the current pre-analysis block is the intra-frame mode complexity, and repeat the process until the prediction complexity of each pre-analysis block of several pre-analysis blocks is determined.

[0200] Step 42115: Accumulate the prediction complexity of each pre-analysis block of several pre-analysis blocks to obtain the prediction complexity of the maximum last frame compared to the starting frame.

[0201] For example, the largest end frame is divided into several pre-analysis blocks. In one example, the pre-analysis blocks are 8×8 blocks. Using the start frame as the reference frame, the several pre-analysis blocks are traversed. For the current pre-analysis block, the corresponding reference block in the start frame is determined. Based on the reference block corresponding to the current pre-analysis block in the start frame, the inter-frame mode complexity of the current pre-analysis block is determined, and the motion vector of the current pre-analysis block is determined.

[0202] If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, then the current pre-analysis block is determined to use inter-frame mode, and its prediction complexity is equal to the inter-frame mode complexity. Otherwise, the current pre-analysis block is determined to use intra-frame mode, and its prediction complexity is equal to the intra-frame mode complexity. This process is repeated until the prediction complexity of each of several pre-analysis blocks is determined. The prediction complexities of each of several pre-analysis blocks are accumulated to obtain the maximum prediction complexity of the last frame compared to the starting frame.

[0203] In some embodiments, dividing the maximum end frame into several pre-analysis blocks can be based on the size of the pre-analysis blocks, dividing the maximum end frame into several pre-analysis blocks of the same size in a left-to-right, top-to-bottom order. Optionally, dividing the maximum end frame into several pre-analysis blocks can be done by first scaling the maximum end frame to 1 / 4 of its original size (scaling the width and height to 1 / 2 of their original sizes), scaling the start frame to 1 / 4 of its original size (scaling the width and height to 1 / 2 of their original sizes), and then dividing the scaled maximum end frame into several pre-analysis blocks.

[0204] This embodiment provides a method for calculating the prediction complexity of the maximum end frame compared to the start frame. It should also be noted that the maximum end frame and start frame in this embodiment can be replaced with any two other video frames, thereby obtaining the prediction complexity of one video frame compared to another video frame, which improves data processing efficiency.

[0205] Regarding the calculation method of the inter-frame mode complexity of the current pre-analysis block, step 42113 is implemented as steps 42117, 42118, and 42119:

[0206] Step 42117: Based on the reference block corresponding to the current pre-analysis block in the starting frame, use the inter-frame mode to predict the pixel value at each position of the current pre-analysis block to obtain the predicted pixel value;

[0207] Step 42118: Subtract the original pixel value of the reference block corresponding to the current analysis block in the starting frame from the predicted pixel value of each position of the current pre-analysis block to obtain the residual value of each position;

[0208] Step 42119: Perform Hadamard transform on the residual values ​​at each position and sum them to obtain the inter-frame mode complexity of the current pre-analysis block.

[0209] For example, based on the reference block corresponding to the current pre-analysis block in the starting frame, an inter-frame mode is used to predict the pixel value at each position of the current pre-analysis block, resulting in a predicted pixel value. The predicted pixel value at each position of the current pre-analysis block is then subtracted from the original pixel value of the reference block corresponding to the current analysis block in the starting frame, yielding a residual value at each position. Finally, a Hadamard transform is performed on the residual values ​​at each position, and the summation (stad) is performed to obtain the inter-frame mode complexity of the current pre-analysis block.

[0210] This embodiment provides a method for calculating the inter-frame mode complexity of the current pre-analysis block, which can improve the accuracy of data calculation and improve data processing efficiency.

[0211] • Determine whether the size of the image group to be encoded needs to be divided into 8 parts.

[0212] In some embodiments, at least one preset frame includes a first preset frame, and the statistical features include features of the first preset frame related to prediction complexity; step 420 is implemented as steps 4221, 4222, 4223, 4224, and 4225:

[0213] Step 4221: Determine the prediction complexity of the first preset frame compared to the starting frame;

[0214] Step 4222: Calculate the fifth proportion of the specified pre-analysis block in the first preset frame;

[0215] Step 4223: Calculate the sixth proportion of the pre-analysis blocks using intra-frame mode in the first preset frame;

[0216] Step 4224: Calculate the seventh ratio of the prediction complexity of the first preset frame compared to the starting frame and the full intra-frame mode complexity compared to the first preset frame.

[0217] Step 4225: Calculate the intra-frame mode complexity of the starting frame and its eighth ratio compared to the intra-frame mode complexity of the first preset frame.

[0218] The start frame is the first video frame of the current image group, also understood as the first video frame of the image group to be encoded. The first preset frame is the preset frame in the current image group that is located before the largest last frame.

[0219] The designated pre-analysis block is a pre-analysis block that uses inter-frame mode and whose motion vector is greater than a vector threshold, which is pre-configured. In this embodiment, since the prediction complexity of the first preset frame compared to the starting frame is determined, the designated pre-analysis block is the designated pre-analysis block in the first preset frame when the starting frame is used as the reference frame.

[0220] The fifth ratio (large_mv_frac) is the proportion of specified pre-analysis blocks in the first preset frame when the starting frame is used as the reference frame. The sixth ratio (intra_frac) is the proportion of pre-analysis blocks using intra-frame mode in the first preset frame when the starting frame is used as the reference frame. The seventh ratio (cost_ratio16) is the ratio of the prediction complexity of the first preset frame compared to the starting frame (when the starting frame is used as the reference frame), and the ratio of the total intra-frame mode complexity of the first preset frame. This can also be understood as the ratio of the prediction complexity of the first preset frame compared to the starting frame to the total intra-frame mode complexity of the first preset frame. The eighth ratio (similarity) is the ratio of the total intra-frame mode complexity of the starting frame compared to the total intra-frame mode complexity of the first preset frame. This can also be understood as the ratio of the total intra-frame mode complexity of the starting frame to the total intra-frame mode complexity of the first preset frame. The closer the eighth ratio (similarity) is to 1, the more similar the starting frame and the first preset frame are.

[0221] In some embodiments, at least one preset frame further includes a second preset frame, which is a preset frame in the current image group that precedes the first preset frame; the threshold includes a fifth threshold, a sixth threshold, a first interval range, and a second interval range; step 440 is implemented as step 442:

[0222] Step 442: If the fifth ratio is less than the fifth threshold, the sixth ratio is less than the sixth threshold, the seventh ratio is within the first interval range, and the eighth ratio is within the second interval range, the second preset frame is determined as the target end frame; otherwise, the first preset frame is kept as the target end frame; wherein, the fifth threshold, the sixth threshold, the first interval range, and the second interval range are pre-configured and used to indicate the characteristics of the first preset frame.

[0223] The fifth ratio (large_mv_frac), the sixth ratio (intra_frac), the seventh ratio (cost_ratio16), and the eighth ratio (similarity) each have corresponding pre-configured thresholds and / or ranges. The value of the threshold and the size of the range are used to indicate the characteristics of the first preset frame; that is, the value of the threshold and the size of the range are related to the first preset frame. In some embodiments, the value of the threshold and the size of the range are specifically related to the statistical characteristics of the first preset frame; that is, the value of the threshold is used to indicate the statistical characteristics of the first preset frame.

[0224] For example, if the fifth ratio (large_mv_frac) is less than the fifth threshold, the sixth ratio (intra_frac) is less than the sixth threshold, the seventh ratio (cost_ratio16) is within the first interval range, and the eighth ratio (similarity) is within the second interval range, then it means that the features of the first preset frame meet the pre-configured conditions, the size of the image group to be encoded can be divided into 8, and the second preset frame can be determined as the target end frame, that is, the 8th frame is determined as the target end frame, and the size of the image group to be encoded is 8; otherwise, the first preset frame remains unchanged as the target end frame, that is, the size of the image group to be encoded remains unchanged at 16.

[0225] This embodiment provides a method for determining whether the size of the image group to be encoded needs to be divided into 8. The second preset frame is predetermined as the 8th frame, which simplifies the algorithm steps and reduces the algorithm complexity.

[0226] • Determine whether the size of the image group to be encoded needs to be divided into 4.

[0227] In some embodiments, at least one preset frame includes a second preset frame, and the statistical features include features of the second preset frame related to prediction complexity; step 420 is implemented as steps 4231, 4232, and 4233:

[0228] Step 4231: Determine the prediction complexity of the second preset frame compared to the starting frame;

[0229] Step 4232: Calculate the ninth proportion of the specified pre-analysis block in the second preset frame;

[0230] Step 4233: Calculate the prediction complexity of the second preset frame compared to the starting frame, and the tenth ratio of the intra-frame mode complexity of the second preset frame compared to the starting frame.

[0231] The start frame is the first video frame of the current image group, also understood as the first video frame of the image group to be encoded. The second preset frame is the preset frame in the current image group that precedes the first preset frame.

[0232] The designated pre-analysis block is a pre-analysis block that uses inter-frame mode and whose motion vector is greater than a vector threshold, which is pre-configured. In this embodiment, since the prediction complexity of the second preset frame compared to the starting frame is determined, the designated pre-analysis block is the designated pre-analysis block in the second preset frame when the starting frame is used as the reference frame.

[0233] The ninth ratio (large_mv_frac) is the proportion of the specified pre-analysis blocks in the second preset frame when the starting frame is used as the reference frame. The tenth ratio (cost_ratio) is the ratio of the prediction complexity of the second preset frame compared to the starting frame (when the starting frame is used as the reference frame), and compared to the intra-frame mode complexity of the second preset frame. It can also be understood as the ratio of the prediction complexity of the second preset frame compared to the starting frame to the intra-frame mode complexity of the second preset frame.

[0234] In some embodiments, at least one preset frame further includes a third preset frame, which is a preset frame in the current image frame that precedes the second preset frame; the threshold includes a seventh threshold and a third interval range; step 440 is implemented as step 443:

[0235] Step 443: If the ninth proportion is greater than the seventh threshold and the tenth proportion is within the third interval range, the third preset frame is determined as the target end frame; otherwise, the second preset frame is kept as the target end frame. The seventh threshold and the third interval range are pre-configured and used to indicate the characteristics of the second preset frame.

[0236] The ninth ratio (large_mv_frac) and the tenth ratio (cost_ratio) each have corresponding pre-configured thresholds and / or ranges. The value of the threshold and the size of the range are used to indicate the characteristics of the second preset frame; that is, the value of the threshold and the size of the range can also be understood as being related to the second preset frame. In some embodiments, the value of the threshold and the size of the range are specifically related to the statistical characteristics of the second preset frame; that is, the value of the threshold is used to indicate the statistical characteristics of the second preset frame.

[0237] For example, if the ninth ratio (large_mv_frac) is greater than the seventh threshold and the tenth ratio (cost_ratio) is within the third interval range, it means that the features of the second preset frame meet the pre-configured conditions. The size of the image group to be encoded can be divided into 4, and the third preset frame can be determined as the target end frame. That is, the fourth frame is determined as the target end frame, and the size of the image group to be encoded is 4. Otherwise, the second preset frame is kept as the target end frame, that is, the size of the image group to be encoded is kept as 8.

[0238] This embodiment provides a method for determining whether the size of the image group to be encoded needs to be divided into 4. The third preset frame is predetermined as the fourth frame, which simplifies the algorithm steps and reduces the algorithm complexity.

[0239] • Determine whether it is necessary to divide the images to be encoded into groups based on sharpness / scene.

[0240] In some embodiments, step 300 is implemented as steps 310, 320, 330, and 340:

[0241] Step 310: Determine the full intra-frame mode complexity of each video frame in the current image group;

[0242] Step 320: Determine the maximum complexity in the intra-frame mode complexity of each video frame, and determine the video frame with the maximum complexity as the maximum complexity frame of the current image group.

[0243] Step 330: Determine the proportion of the maximum complexity frame to the specified pre-analysis block of the starting frame;

[0244] Step 340: If the maximum complexity is less than the eighth threshold, the sequence number of the maximum complexity frame in the current image group is greater than the ninth threshold, and the proportion of the specified pre-analysis block is greater than the tenth threshold, the maximum complexity frame is determined as the maximum end frame; where the eighth threshold is calculated based on the image resolution, and the ninth and tenth thresholds are pre-configured.

[0245] For the current image group, determine the intra-mode complexity of each video frame in the current image group. From the intra-mode complexity of each video frame, determine the maximum complexity (max_intra), and designate the video frame with the maximum complexity as the maximum complexity frame of the current image group. This maximum complexity frame has a corresponding index (max_index) in the current image group. Using the starting frame as a reference frame, determine the ratio (large_mv_ratio) of the maximum complexity frame to a specified pre-analysis block of the starting frame. The specified pre-analysis block is a pre-analysis block using inter-frame mode and whose motion vectors are greater than a vector threshold.

[0246] If the maximum complexity (max_intra) is less than the eighth threshold, and the index of the maximum complexity frame in the current image group (max_index) is greater than the ninth threshold, and the proportion of the specified pre-analysis block (large_mv_ratio) is greater than the tenth threshold, the maximum complexity frame is determined as the maximum end frame. That is, the maximum number of frames supported by the image group to be encoded can be represented by this maximum end frame.

[0247] The eighth threshold is calculated based on the image resolution, while the ninth and tenth thresholds are pre-configured. In some embodiments, the formula for calculating the eighth threshold is: Eighth threshold = Preset coefficient × Video frame width × Video frame height. Since the video frames in the current image group are obtained from the same video to be encoded, the eighth threshold is the same for each video frame. In one example, the preset coefficient is 0.65.

[0248] In some embodiments, when two or more scenes are involved in the current image group, scene switching is also taken into account, and video frames before the scene switching are assigned to the same current image group and the same image group to be encoded as much as possible. Step 300 is also implemented as steps 350 and 360:

[0249] Step 350: Determine the last frame of the first scene in the current image group;

[0250] Step 360: Determine the frame that appears first in the sequence between the last frame of the scene and the frame with the highest complexity as the maximum last frame.

[0251] The first scene is the scene that appears first in the current image group, or it can be understood as the scene before the scene switch. The last frame of the scene is the last frame of the first scene before the scene switch. For example, if the current image group consists of 32 frames, with frames 1 to 16 involving scene 1 and frames 17 to 32 involving scene 2, then the last frame of the first scene is frame 16.

[0252] A scene can be related to at least one of objects, scenery, people, lighting, color, composition, and shooting method. In some embodiments, different scenes in the current image group are pre-labeled, or determined by transition video frames in each video frame of the current image group, or obtained by scene recognition model performing scene recognition on each video frame in the current image group. For example, the scene recognition model is a pre-trained neural network model used to recognize the scene in each video frame of the current image group. The scene recognition model includes at least one of the following: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and time series models.

[0253] For example, the last frame of the first scene in the current image group is determined; the frame that appears first between the last frame and the frame with the highest complexity is determined as the maximum last frame. For example, if the current image group includes 32 frames, frames 1 to 16 relate to scene 1, frames 17 to 32 relate to scene 2, the frame with the highest complexity is frame 18, and the last frame of scene 1 is frame 16, then frame 16 is determined as the maximum last frame.

[0254] This embodiment provides a method for determining whether to divide the images to be encoded into groups based on sharpness / scene. Specifically, using the frame with the highest complexity as the largest last frame ensures the identification of high-quality frames containing more detail and texture information. When high-quality frames are encoded with high quality, they provide a better reference for other blurry frames, improving the overall encoding quality of the video, especially suitable for videos with significant motion and blurriness. Using the last frame of the scene as the largest last frame ensures that video frames before scene transitions are grouped into the same current image group and the same image group to be encoded, improving encoding efficiency.

[0255] To calculate the intra-frame mode complexity of each video frame in the current image group, refer to the method for calculating the intra-frame mode complexity of frame A, where frame A is each video frame in the current image group. For the current video frame, step 310 is implemented as steps 311, 312, 313, 314, and 315:

[0256] Step 311: Divide the current video frame into several pre-analysis blocks;

[0257] Step 312: Traverse several pre-analysis blocks. For the current pre-analysis block in the several pre-analysis blocks, use intra-frame mode to predict the pixel value at each position of the current pre-analysis block and obtain the predicted pixel value.

[0258] Step 313: Subtract the predicted pixel value from the original pixel value at each position of the current pre-analysis block to obtain the residual value at each position;

[0259] Step 314: Perform Hadamard transform on the residual value at each position and sum them to obtain the intra-mode complexity of the current pre-analysis block. Repeat this process until the intra-mode complexity of each pre-analysis block of several pre-analysis blocks is obtained.

[0260] Step 315: Accumulate the intra-frame mode complexity of each pre-analysis block to obtain the full intra-frame mode complexity of the current video frame.

[0261] For example, the current video frame is divided into several pre-analysis blocks; the several pre-analysis blocks are traversed, and for the current pre-analysis block, the intra mode is used to predict the pixel value at each position of the current pre-analysis block to obtain the predicted pixel value.

[0262] Subtract the predicted pixel value from the original pixel value at each position of the current pre-analysis block to obtain the residual value at each position. Perform a Hadamard transform on the residual values ​​at each position and sum them (stad) to obtain the intra-frame mode complexity of the current pre-analysis block. Repeat this process until the intra-frame mode complexity of each pre-analysis block for several pre-analysis blocks is obtained. Accumulate the intra-frame mode complexity of each pre-analysis block for several pre-analysis blocks to obtain the total intra-frame mode complexity of the current video frame.

[0263] This embodiment provides a method for calculating the intra-frame complexity of the current video frame, which can improve the accuracy of data calculation and increase data processing efficiency.

[0264] In some embodiments, step 330 is implemented as steps 331 and 332:

[0265] Step 331: Determine the specified analysis block of the maximum complexity frame compared to the starting frame;

[0266] Step 332: Determine the proportion of the specified pre-analysis blocks in the maximum complexity frame based on the number of specified pre-analysis blocks and the number of pre-analysis blocks in the maximum complexity frame.

[0267] For example, using the starting frame as a reference frame, the specified pre-analysis of the maximum complexity (max_intra) frame compared to the starting frame is determined. Based on the number of specified pre-analysis blocks in the maximum complexity (max_intra) frame and the total number of pre-analysis blocks, the ratio (large_mv_ratio) of the specified pre-analysis blocks in the maximum complexity (max_intra) frame is determined by dividing the number of specified pre-analysis blocks by the total number of pre-analysis blocks. Here, the number of pre-analysis blocks refers to the total number of pre-analysis blocks in the maximum complexity frame.

[0268] Specifically, step 331 includes steps 3311, 3312, 3313, and 3314:

[0269] Step 3311: Using the starting frame as the reference frame, determine the reference block in the starting frame for each pre-analysis block in the maximum complexity frame.

[0270] Step 3312: Based on the reference block corresponding to each pre-analysis block in the starting frame, determine the inter-frame mode complexity of each pre-analysis block in the maximum complexity frame, and determine the motion vector of each pre-analysis block in the maximum complexity frame.

[0271] Step 3313: If the inter-frame mode complexity of the current pre-analysis block of the maximum complexity frame is less than the intra-frame mode complexity of the current pre-analysis block, determine that the current pre-analysis block adopts inter-frame mode; otherwise, determine that the current pre-analysis block adopts intra-frame mode.

[0272] Step 3314: The pre-analysis block in the frame with the highest complexity that adopts inter-frame mode and whose motion vector is greater than the vector threshold is identified as the specified pre-analysis block.

[0273] For example, taking the starting frame as the reference frame, the reference block corresponding to each pre-analysis block in the maximum complexity (max_intra) frame is determined in the starting frame; based on the reference block corresponding to each pre-analysis block in the maximum complexity frame, the inter-frame (inter) mode complexity of each pre-analysis block in the maximum complexity frame is determined, and the motion vector (mv) of each pre-analysis block in the maximum complexity frame is determined.

[0274] If the inter-frame complexity of the current pre-analysis block in the frame with the highest complexity is less than the intra-frame complexity, the current pre-analysis block is determined to use inter-frame mode; otherwise, the current pre-analysis block is determined to use intra-frame mode. Pre-analysis blocks in the frame with the highest complexity that use inter-frame mode and whose motion vectors are greater than a vector threshold are identified as designated pre-analysis blocks.

[0275] This embodiment provides a method for calculating the specified analysis block of the maximum complexity frame compared to the starting frame. It should also be noted that the maximum complexity frame and the starting frame in this embodiment can be replaced with any two other video frames, thereby obtaining the specified pre-analysis block of one video frame compared to another video frame, which improves data processing efficiency.

[0276] In some embodiments, the method further includes step 500:

[0277] Step 500: If the starting frame meets the segmentation conditions, perform the step of determining the target end frame from the maximum end frame and at least one preset frame based on the frame features of the maximum end frame and at least one preset frame; otherwise, determine the maximum end frame as the target end frame.

[0278] The starting frame is the first video frame of the current image group, also understood as the first video frame of the image group to be encoded. The partitioning condition is the condition that the current image group needs to be further divided into 16, 8, or 4 sizes. When the starting frame meets the partitioning condition, the step of determining the target ending frame from the frame features of the maximum ending frame and at least one preset frame is performed; otherwise, the maximum ending frame is directly determined as the target ending frame, thereby determining the image group to be encoded.

[0279] In some embodiments, the method for determining whether the starting frame meets the segmentation conditions is further implemented as steps 510 and 520:

[0280] Step 510: Determine the full intra-frame mode complexity of the starting frame;

[0281] Step 520: If the complexity of the entire frame of the starting frame is less than or equal to the complexity threshold, the starting frame satisfies the partitioning condition; wherein, the complexity threshold is related to the number of pre-analysis blocks in the starting frame.

[0282] For example, the intra-frame mode complexity of the starting frame is determined. If the intra-frame mode complexity of the starting frame is less than or equal to a complexity threshold, the starting frame meets the partitioning condition, and the steps to partition the current image group into 16, 8, or 4 frames need to be performed. Otherwise, if the intra-frame mode complexity of the starting frame is greater than the complexity threshold, no further judgment is performed, and the largest last frame is directly determined as the target last frame, thus determining the image group to be encoded. The complexity threshold is related to the number of pre-analyzed blocks in the starting frame. In one example, the complexity threshold is expressed as F(N), where F is a pre-configured function and N is the number of pre-analyzed blocks in the starting frame.

[0283] In this embodiment, if the starting frame meets the partitioning condition, the subsequent judgment is performed; otherwise, the subsequent judgment is not performed, and the largest last frame is directly determined as the target last frame, which can improve data processing efficiency.

[0284] In some embodiments, the current image group is obtained based on a pre-analysis queue. The pre-analysis queue is a fixed-size queue used to store video frames to be analyzed frame by frame. Specifically, step 220 is implemented as steps 221, 222, and 223:

[0285] Step 221: Read the first frame of the video to be encoded;

[0286] Step 222: Save the first number of video frames in the pre-analysis queue;

[0287] Step 223: Determine the video frames of the preset number of frames in the pre-analysis queue as the current image group; wherein, the maximum value of the first frame number is the pre-configured maximum number of read frames.

[0288] For example, video frames of the first frame number in the video to be encoded are read sequentially according to time. The maximum value of the first frame number is a pre-configured maximum number of frames to read. The video frames of the first frame number are stored in a pre-analysis queue, and the video frames of a preset frame number in the pre-analysis queue are determined as the current image group. Optionally, the preset frame number is less than or equal to the first frame number.

[0289] In some embodiments, the size of the current image group is represented as: gop_size = min(cfg_gop_size, num_frames). Here, cfg_gop_size is the pre-configured maximum image group size, and num_frames is the number of the first frame in the pre-analysis queue. The maximum value of the first frame is pre-configured, representing the maximum number of frames that can be read into the pre-analysis queue at one time. In one example, at the end of the video to be encoded, the number of the first frame in the pre-analysis queue is the number of remaining unencoded frames.

[0290] This embodiment introduces a pre-analysis queue, which can determine the current image group and improve the accuracy of the current image group.

[0291] In some embodiments, the current image group is obtained based on a pre-analysis queue. After determining the image group to be encoded from the current image group, each video frame of the image group to be encoded is dequeued from the pre-analysis queue and encoded. This embodiment supports iterative execution of encoding. For example, step 292 is included after step 260, and step 294 is included after step 280:

[0292] Step 292: Remove each video frame from the pre-analysis queue in the image group to be encoded to obtain the pre-analysis queue after removal. The pre-analysis queue after removal contains the video frames of the second frame number.

[0293] Step 294: After encoding each video frame in the image group to be encoded and obtaining the encoded bitstream, the pre-analysis queue is replenished with the first number of video frames, and the step of determining the preset number of video frames in the pre-analysis queue as the current image group is repeated.

[0294] After determining the image group to be encoded from the current image group, each video frame in the image group to be encoded is removed from the pre-analysis queue, resulting in a pre-analysis queue containing a second number of video frames. This second number of video frames represents the remaining video frames after removing all video frames from the image group to be encoded from the pre-analysis queue.

[0295] After encoding each video frame in the image to be encoded to obtain the encoded bitstream, the pre-analysis queue is replenished with the first number of video frames, forming a new pre-analysis queue. The newly replenished video frames are placed after the second number of video frames in chronological order. After forming the new pre-analysis queue, the step of determining the preset number of video frames in the pre-analysis queue as the current image group is repeated, thus continuing to iterate and determine the new current image group and the new image group to be encoded, and performing encoding, until the entire video to be encoded has been encoded.

[0296] This embodiment provides an iterative execution method for the video encoding method. By iteratively executing the method, it is possible to ensure that the entire video to be encoded has been encoded, thereby improving the accuracy of encoding the video.

[0297] The following is a general description of the video encoding method provided in this application, executed by an encoder, using flowcharts and diagrams as examples. In this embodiment, the first preset frame in the aforementioned embodiments is the 16th frame, the second preset frame is the 8th frame, and the third preset frame is the 4th frame.

[0298] Application Scenarios

[0299] The video encoding method provided in this application can be integrated into various types of encoders for adaptively dividing image groups. It is suitable for scenarios insensitive to latency and buffering, such as offline compression. In other embodiments, it is also suitable for videos with significant motion and blurriness to be encoded, as well as videos with scene transitions. Specifically, based on video characteristics, the size of the image group to be encoded is divided into a numerical value. This value supports a non-power-two value (values ​​other than 32, 16, 8, 4, and 2), and also supports a power-two value. Furthermore, the size of the image group to be encoded is adaptively divided into 16, 8, or 4 to improve the encoder's compression efficiency and encoding performance.

[0300] • Technical Implementation

[0301] refer to Figure 4 The overall steps of the video encoding method performed by the encoder are as follows:

[0302] 1. Input the video to be encoded into the encoder;

[0303] 2. Read the video frame sequence and store it in the pre-analysis queue;

[0304] 3. Calculate video features to determine the size of the image group to be encoded;

[0305] 4. Remove each video frame from the pre-analysis queue and encode it;

[0306] 5. Output encoded bitstream;

[0307] 6. Determine the pre-analysis queue after the exit, and repeat steps 2-4.

[0308] For example, if the pre-analysis queue stores 48 video frames and the current image group size is 32, then the first 32 video frames in the pre-analysis queue are determined as the current image group. Based on video features, the size of the image to be encoded is determined to be 16, so the image group to be encoded consists of the first 16 video frames in the current image group. These first 16 video frames are then deduced from the pre-analysis queue, encoded, and the resulting encoded bitstream is obtained. In the next iteration, video frames are read, and the pre-analysis queue is replenished to 48 video frames, forming a new pre-analysis queue. The steps related to determining the size of the image group to be encoded based on video features are then executed again.

[0309] refer to Figure 5 The steps for adaptively dividing the image into groups are as follows:

[0310] 1. Calculate the maximum number of frames supported by the group of images to be encoded;

[0311] And; 2. Determine whether it is necessary to divide the images to be encoded into groups according to sharpness / scene;

[0312] 3. Determine whether the size of the image group to be encoded needs to be divided into 16;

[0313] If yes, divide the size of the image group to be encoded into 16 and execute 4; otherwise, execute 6 directly.

[0314] 4. Determine whether the size of the image group to be encoded needs to be divided into 8 parts;

[0315] If yes, divide the size of the image group to be encoded into 8 and execute 5; otherwise, execute 6 directly.

[0316] 5. Determine whether the size of the image group to be encoded needs to be divided into 4;

[0317] If yes, the size of the image group to be encoded is divided into 4 and 6 is executed; otherwise, 6 is executed directly.

[0318] 6. Determine the group of images to be encoded.

[0319] • For 1. Calculating the maximum number of frames supported by the image group to be encoded; and 2. Determining whether it is necessary to divide the image group to be encoded according to sharpness / scene; the implementation steps are as follows:

[0320] 1. The size of the current image group is represented as: gop_size = min(cfg_gop_size, num_frames). Where cfg_gop_size is the pre-configured maximum image group size, and num_frames is the number of the first frame in the pre-analysis queue. The maximum value of the first frame is pre-configured, representing the maximum number of frames read into the pre-analysis queue at once. For example, the first frame number of the pre-analysis queue can be set to at least one of the following: 48, 64, or 80. The size of the current image group is a preset number of frames, for example, the preset number of frames could be 32. At the end of the video, the first frame number in the pre-analysis queue is the number of remaining unencoded frames.

[0321] 2. For video frames in the pre-analysis queue ranging from the starting frame to `gop_size`, i.e., for the current image group, determine the intra-frame mode complexity of each video frame in the current image group. Calculate the maximum intra-frame mode complexity (`max_intra`) for each video frame. Determine the index (`max_index`) of the frame with the maximum complexity in the pre-analysis queue.

[0322] 3. Determine the ratio (large_mv_ratio) of the specified pre-analysis block in the max_index frame (maximum complexity frame) compared to the starting frame. The specified pre-analysis block is a pre-analysis block that uses inter-frame mode and whose motion vector (mv) exceeds a pre-configured vector threshold.

[0323] 4. If the maximum complexity (max_intra) is less than the threshold calculated based on the image resolution, and the index (max_index) is greater than the pre-configured threshold, and the proportion of the specified pre-analysis block (large_mv_ratio) is greater than the pre-configured threshold, the maximum number of frames supported by the image group to be encoded is determined as the index (max_index) of the maximum complexity frame. That is, the maximum complexity frame can be used as the maximum end frame of the image group to be encoded, and the maximum size supported by the image group to be encoded is the index (max_index).

[0324] Here, the index (max_index) can be a non-power-of-two value or a power-of-two value. The threshold calculated based on image resolution is: preset coefficient × width of video frame × height of video frame. In one example, the preset coefficient is 0.65.

[0325] In some embodiments, when the current image group involves two or more scenes, the last frame of the first scene in the current image group is determined; the frame that appears first among the last frame of the scene and the frame with the highest complexity is determined as the maximum last frame. That is, the maximum size supported by the image group to be encoded is determined as the sequence number corresponding to the last frame of the scene.

[0326] • For step 3, determine whether the size of the image group to be encoded needs to be divided into 16; the implementation steps are as follows:

[0327] If the maximum supported size of the image group to be encoded is greater than or equal to 16, the optimal number of frames for the image group to be encoded is determined, thereby determining the optimal size of the image group to be encoded.

[0328] 1. If the intra-complexity of the starting frame in the current image group is greater than the complexity threshold, then no further judgment is performed, and the image group to be encoded is directly removed from the pre-analysis queue and encoding is performed. If the intra-complexity of the starting frame in the current image group is less than or equal to the complexity threshold, then further judgment is performed to determine the number of optimized frames for the image group to be encoded.

[0329] The complexity threshold is related to the number of pre-analyzed blocks in the starting frame. In one example, the complexity threshold is denoted as F(N), where F is a pre-configured function and N is the number of pre-analyzed blocks in the starting frame.

[0330] 2. Based on the maximum size supported by the image group to be encoded, determine the maximum last frame of the image group to be encoded. Calculate the prediction complexity of the maximum last frame compared to the starting frame (frame 1); calculate the prediction complexity of frame 16 compared to the starting frame; calculate the prediction complexity of frame 16 compared to both the starting frame and the maximum last frame.

[0331] 3. Analyze the features related to prediction complexity of the largest last frame:

[0332] The first proportion (large_mv_frac) of the specified pre-analysis block in the largest last frame is calculated, where the specified pre-analysis block refers to the pre-analysis block that uses inter-frame mode and whose motion vector (mv) is greater than the vector threshold;

[0333] The prediction complexity of the largest last frame with the starting frame as the reference frame is calculated as the second ratio (cost_ratio) to the complexity of its own intra-frame mode.

[0334] The prediction complexity of the largest last frame is calculated as the previous frame as the reference frame, and then compared to the third proportion of the complexity of the intra-frame mode when using the intra mode (inter_with_prev); where the largest last frame is the nth frame, and the previous frame is the (n-1)th frame.

[0335] The prediction complexity of frame 16 with the start frame and the largest end frame as reference frames is calculated as the fourth proportion of the prediction complexity when frame 16 is only with the start frame as reference frame (poc16_inter).

[0336] 4. Statistically analyze the noise-related features of the largest last frame: noise level.

[0337] 5. If the noise level of the largest last frame is less than the pre-configured threshold, the third ratio (inter_with_prev) is less than the pre-configured threshold, the first ratio (large_mv_frac) is greater than the pre-configured threshold, the second ratio (cost_ratio) is greater than the pre-configured threshold, and the fourth ratio (poc16_inter) is greater than the pre-configured threshold, then the size of the image group to be encoded is set to 16, that is, the 16th frame is determined as the target last frame of the image group to be encoded.

[0338] • For step 4, determine whether the size of the image group to be encoded needs to be divided into 8 parts; the implementation steps are as follows:

[0339] 1. Calculate the prediction complexity of the 16th frame in the current image group compared to the starting frame (frame 1).

[0340] 2. Analyze the features related to prediction complexity in frame 16:

[0341] The fifth proportion (large_mv_frac) of the specified pre-analysis block in frame 16 is calculated; where the specified pre-analysis block refers to the pre-analysis block that uses inter-frame mode and whose motion vector (mv) is greater than the vector threshold;

[0342] The sixth proportion (intra_frac) of the pre-analysis block using intra mode in frame 16 is calculated.

[0343] The seventh ratio (cost_ratio16) is calculated when the prediction complexity of frame 16 is taken as the reference frame and compared with the complexity of frame 16 when it is in intra mode.

[0344] The intra-frame complexity of the starting frame is compared to the eighth similarity of the intra-frame complexity of the 16th frame.

[0345] 3. If the sixth ratio (intra_frac) is less than the pre-configured threshold, the fifth ratio (large_mv_frac) is less than the pre-configured threshold, the seventh ratio (cost_ratio16) is within the pre-configured range, and the eighth ratio (similarity) is within the pre-configured range, then the size of the image group to be encoded is divided into 8, that is, the 8th frame is determined as the target end frame of the image group to be encoded; otherwise, the size of the image group to be encoded remains unchanged at 16.

[0346] • For step 5, determine whether the size of the image group to be encoded needs to be divided into 4; the implementation steps are as follows:

[0347] 1. Calculate the prediction complexity of the 8th frame in the current image group compared to the starting frame (the 1st frame).

[0348] 2. Analyze the features related to prediction complexity in frame 8:

[0349] The ninth proportion (large_mv_frac) of the specified pre-analysis block in the 8th frame is calculated; where the specified pre-analysis block refers to the pre-analysis block that uses inter-frame mode and whose motion vector (mv) is greater than the vector threshold;

[0350] The prediction complexity of frame 8, with the starting frame as the reference frame, is calculated as the tenth ratio of the complexity of frame 8 when using the intra mode.

[0351] 3. If the ninth ratio (large_mv_frac) is greater than the pre-configured threshold and the tenth ratio (cost_ratio) is within the pre-configured range, the size of the image group to be encoded is divided into 4, that is, the fourth frame is determined as the target end frame of the image group to be encoded; otherwise, the size of the image group to be encoded remains unchanged at 8.

[0352] The steps for calculating the intra-frame mode complexity of a video frame in the aforementioned embodiments are as follows:

[0353] The current video frame is scaled to its original preset size, for example, 1 / 4 (width and height are scaled to 1 / 2 of their original size). The scaled current video frame is divided into several pre-analysis blocks, each an 8×8 block. These pre-analysis blocks are traversed. For the current pre-analysis block, referring to the 35 intra-prediction modes in the HEVC coding standard, the pixel value at each position of the current pre-analysis block is predicted, yielding the predicted pixel value. The original pixel value at each position of the current pre-analysis block is subtracted from the predicted pixel value, yielding the residual value at each position. A Hadamard transform is performed on the residual values, and the summation (satd) is calculated to obtain the intra-prediction mode complexity of the current pre-analysis block. This process is repeated until the intra-prediction mode complexity of each of the several pre-analysis blocks is obtained. The intra-prediction mode complexity of each of the several pre-analysis blocks is accumulated to obtain the full intra-prediction mode complexity of the current video frame using full intra-prediction mode.

[0354] The steps for calculating the prediction complexity of frame A compared to frame B in the aforementioned embodiments are as follows:

[0355] Scale both A-frame and B-frame to their original preset size, for example, 1 / 4 (width and height scaled to 1 / 2 of their original size). Divide the scaled A-frame into several pre-analysis blocks, each an 8×8 block. Iterate through these pre-analysis blocks. For the current pre-analysis block, determine the best-matching reference block in the scaled B-frame through motion search, and record the motion vector (mv) and inter-frame mode complexity at this point. The inter-frame mode complexity is calculated as follows: subtract the original pixel value of the reference block from the original pixel value of each position in the current pre-analysis block in the scaled A-frame to obtain the residual value at each position. Perform a Hadamard transform on the residual values ​​at each position and sum them (satd) to obtain the inter-frame mode complexity of the current pre-analysis block.

[0356] If the inter-frame (inter) mode complexity of the current pre-analysis block is less than the intra-frame (intra) mode complexity, then the current pre-analysis block is considered to be using inter-frame (inter) mode; otherwise, the current pre-analysis block is considered to be using intra-frame (intra) mode. The prediction complexity of each pre-analysis block is accumulated to obtain the prediction complexity of frame A compared to frame B.

[0357] For example, taking the current image group size as 32 as an example, refer to... Figure 6As shown in (1), the current image group is determined. Assuming the maximum number of frames supported by the image group to be encoded is 28, if the intra-frame complexity of the starting frame is greater than the complexity threshold, no further judgment is made, and the size of the image group to be encoded is 28. If the intra-frame complexity of the starting frame is less than or equal to the complexity threshold, it is further determined whether the size of the image group to be encoded needs to be divided into 16. If it is necessary to divide the size of the image group to be encoded into 16, then refer to... Figure 6 As shown in (2) above, otherwise keep it unchanged at 28. Continue to determine whether it is necessary to divide the size of the image group to be encoded into 8. If it is necessary to divide the size of the image group to be encoded into 8, then refer to Figure 6 As shown in (3) above, otherwise keep it unchanged at 16. Continue to determine whether it is necessary to divide the size of the image group to be encoded into 4. If it is necessary to divide the size of the image group to be encoded into 4, refer to Figure 6 As shown in (4) above, otherwise keep it unchanged at 8. Encode each video frame of the image group to be encoded to obtain the encoded bitstream.

[0358] Beneficial effects

[0359] Dividing the images to be encoded into groups based on sharpness allows for the selection of sharper, more textured video frames from videos with significant motion and blurriness. Referring to Table 2, the method described in this embodiment achieves a performance gain of over 20% on the MSU2022 test set. Further dividing the image groups into sizes of 16, 8, or 4, as shown in Table 3, yields an average performance gain of 0.5% on the MSU2022 test set, with a speeddown of 2.11%.

[0360] Table 2

[0361] frames psnr psnr_mse ssim ssim_msu y-vmaf y-vmaf-neg msu2022 1000 -20.54％ -19.02％ -22.94％ -22.55％ -30.02％ -28.74％

[0362] Table 3

[0363] speed psnr psnr_mse ssim ssim_msu y-vmaf y-vmaf_neg msu2022 -2.11％ -0.81％ -0.66％ -0.43％ -0.48％ -0.44％ -0.25％ average -2.11％ -0.81％ -0.66％ -0.43％ -0.48％ -0.44％ -0.25％

[0364] Figure 7 This is a block diagram of a video encoding apparatus provided in an exemplary embodiment of this application. The video encoding apparatus 800 includes at least some of the following modules: an acquisition module 820, a processing module 840, a determination module 860, and an encoding module 880.

[0365] The acquisition module 820 is used to acquire a preset number of video frames in the video to be encoded, and obtain the current image group;

[0366] Processing module 840 is used to determine the target last frame from the current image group based on the video features of the current image group;

[0367] The determining module 860 is used to determine the starting frame to the target ending frame in the current image group as the image group to be encoded.

[0368] The encoding module 880 is used to encode each video frame in the image group to be encoded to obtain an encoded bitstream.

[0369] In some embodiments, the processing module 840 is configured to:

[0370] Based on the sequence features of the current image group, determine the largest last frame from the current image group;

[0371] Based on the frame features of the maximum last frame and at least one preset frame, the target last frame is determined from the maximum last frame and the at least one preset frame;

[0372] Wherein, the sequence number of the maximum last frame in the current image group is used to indicate the maximum number of frames in the image group to be encoded, the sequence number of the target last frame in the current image group is used to indicate the optimized number of frames in the image group to be encoded, the optimized number of frames is less than or equal to the maximum number of frames, and the at least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the maximum last frame.

[0373] In some embodiments, the processing module 840 is configured to:

[0374] Statistical features are analyzed in the frame features of the largest last frame and the at least one preset frame;

[0375] Based on the relationship between the statistical features and the threshold, the target end frame is determined from the maximum end frame and the at least one preset frame;

[0376] The threshold is related to the statistical features and the at least one preset frame.

[0377] In some embodiments, the at least one preset frame includes a first preset frame, which is a preset frame in the current image group located before the maximum last frame; the statistical features include features related to prediction complexity of the maximum last frame and the first preset frame, respectively, and noise-related features of the maximum last frame; the processing module 840 is configured to:

[0378] Determine the prediction complexity of the maximum end frame compared to the start frame; determine the prediction complexity of the maximum end frame compared to the previous frame; determine the prediction complexity of the first preset frame compared to the start frame; determine the prediction complexity of the first preset frame compared to the start frame and the maximum end frame.

[0379] Calculate the first proportion of the specified pre-analysis block in the largest last frame;

[0380] The prediction complexity of the maximum end frame compared to the starting frame as the reference frame is calculated as a second ratio of the intra-frame mode complexity of the maximum end frame.

[0381] The third ratio is calculated as follows: the prediction complexity of the maximum last frame compared to the previous frame, and the intra-frame mode complexity compared to the maximum last frame.

[0382] The prediction complexity of the first preset frame compared to the starting frame and the maximum ending frame is calculated as a fourth ratio to the prediction complexity of the first preset frame compared to the starting frame.

[0383] The noise level of the largest last frame is statistically analyzed;

[0384] The intra-frame mode complexity is the prediction complexity when predicting the pixel value at each position of each pre-analysis block using intra-frame mode; the specified pre-analysis block is a pre-analysis block using inter-frame mode and whose motion vector is greater than a vector threshold, wherein the vector threshold is pre-configured and the pre-analysis block is a pre-divided pixel block.

[0385] In some embodiments, the threshold includes a noise threshold, a first threshold, a second threshold, a third threshold, and a fourth threshold; the processing module 840 is configured to:

[0386] If the noise level of the maximum last frame is less than the noise threshold, and the first proportion is greater than the first threshold, and the second proportion is greater than the second threshold, and the third proportion is less than the third threshold, and the fourth proportion is greater than the fourth threshold, then the first preset frame is determined as the target last frame; otherwise, the maximum last frame is determined as the target last frame.

[0387] The noise threshold, the first threshold, the second threshold, the third threshold, and the fourth threshold are all pre-configured and used to indicate the characteristics of the maximum end frame and the first preset frame.

[0388] In some embodiments, the at least one preset frame includes a first preset frame, and the statistical features include features of the first preset frame related to prediction complexity; the processing module 840 is configured to:

[0389] Determine the prediction complexity of the first preset frame compared to the starting frame;

[0390] The fifth proportion of the specified pre-analysis block in the first preset frame is calculated.

[0391] The sixth proportion of the pre-analysis blocks using intra-frame mode in the first preset frame is calculated.

[0392] The prediction complexity of the first preset frame with the starting frame as the reference frame is calculated as a seventh ratio to the full intra-frame mode complexity of the first preset frame.

[0393] The total intra-frame mode complexity of the starting frame is calculated as an eighth ratio to the total intra-frame mode complexity of the first preset frame.

[0394] In some embodiments, the at least one preset frame further includes a second preset frame, the second preset frame being a preset frame in the current image group preceding the first preset frame; the threshold includes a fifth threshold, a sixth threshold, a first interval range, and a second interval range; the processing module 840 is configured to:

[0395] If the fifth ratio is less than the fifth threshold, and the sixth ratio is less than the sixth threshold, and the seventh ratio is within the first interval range, and the eighth ratio is within the second interval range, then the second preset frame is determined as the target end frame; otherwise, the first preset frame remains unchanged as the target end frame.

[0396] The fifth threshold, the sixth threshold, the first interval range, and the second interval range are pre-configured and used to indicate the characteristics of the first preset frame.

[0397] In some embodiments, the at least one preset frame includes a second preset frame, and the statistical features include features of the second preset frame related to prediction complexity; the processing module 840 is configured to:

[0398] Determine the prediction complexity of the second preset frame compared to the starting frame;

[0399] Determine the ninth proportion of the specified pre-analysis block in the second preset frame;

[0400] Determine the prediction complexity of the second preset frame when the starting frame is used as the reference frame, and the tenth ratio of the intra-frame mode complexity of the second preset frame.

[0401] In some embodiments, the at least one preset frame further includes a third preset frame, the third preset frame being a preset frame in the current image frame preceding the second preset frame; the threshold includes a seventh threshold and a third interval range; the processing module 840 is configured to:

[0402] If the ninth ratio is greater than the seventh threshold and the tenth ratio is within the third interval range, the third preset frame is determined as the target end frame; otherwise, the second preset frame is kept as the target end frame.

[0403] The seventh threshold and the third interval range are pre-configured and used to indicate the characteristics of the second preset frame.

[0404] In some embodiments, the processing module 840 is configured to:

[0405] The maximum last frame is divided into several pre-analysis blocks;

[0406] Using the starting frame as a reference frame, the plurality of pre-analysis blocks are traversed, and for the current pre-analysis block among the plurality of pre-analysis blocks, the corresponding reference block in the starting frame is determined;

[0407] Based on the reference block corresponding to the current pre-analysis block in the starting frame, determine the inter-frame mode complexity of the current pre-analysis block, and determine the motion vector of the current pre-analysis block;

[0408] If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, then the current pre-analysis block is determined to use inter-frame mode and the prediction complexity of the current pre-analysis block is the inter-frame mode complexity; otherwise, the current pre-analysis block is determined to use intra-frame mode and the prediction complexity of the current pre-analysis block is the intra-frame mode complexity, and this process is repeated until the prediction complexity of each of the plurality of pre-analysis blocks is determined.

[0409] The prediction complexity of each of the several pre-analysis blocks is summed to obtain the prediction complexity of the maximum last frame compared to the starting frame.

[0410] In some embodiments, the processing module 840 is configured to:

[0411] Based on the reference block corresponding to the current pre-analysis block in the starting frame, the pixel value at each position of the current pre-analysis block is predicted using an inter-frame mode to obtain the predicted pixel value.

[0412] The predicted pixel value at each position of the current pre-analysis block is subtracted from the original pixel value of the reference block corresponding to the current analysis block in the starting frame to obtain the residual value at each position.

[0413] Perform a Hadamard transform on the residual value at each position and sum them to obtain the inter-frame mode complexity of the current pre-analysis block.

[0414] In some embodiments, the processing module 840 is configured to:

[0415] Determine the full intra-frame mode complexity of each video frame in the current image group;

[0416] Determine the maximum complexity among the intra-frame mode complexities of each video frame, and determine the video frame with the maximum complexity as the maximum complexity frame of the current image group.

[0417] Determine the ratio of the maximum complexity frame to a specified pre-analysis block of the starting frame;

[0418] If the maximum complexity is less than the eighth threshold, the sequence number of the maximum complexity frame in the current image group is greater than the ninth threshold, and the proportion of the specified pre-analysis block is greater than the tenth threshold, the maximum complexity frame is determined as the maximum end frame.

[0419] The eighth threshold is calculated based on the image resolution, while the ninth and tenth thresholds are pre-configured.

[0420] In some embodiments, the processing module 840 is further configured to:

[0421] Determine the last frame of the first scene in the current image group;

[0422] The frame that appears first in the order of the last frame of the scene and the frame with the highest complexity is determined as the maximum last frame.

[0423] In some embodiments, the processing module 840 is configured to:

[0424] The current video frame is divided into several pre-analysis blocks;

[0425] Traverse the plurality of pre-analysis blocks. For the current pre-analysis block among the plurality of pre-analysis blocks, use intra-frame mode to predict the pixel value at each position of the current pre-analysis block to obtain the predicted pixel value.

[0426] Subtract the predicted pixel value from the original pixel value at each position of the current pre-analysis block to obtain the residual value at each position;

[0427] Perform a Hadamard transform on the residual value at each position and sum them to obtain the intra-mode complexity of the current pre-analysis block. Repeat this process until the intra-mode complexity of each of the several pre-analysis blocks is obtained.

[0428] The intra-frame mode complexity of the current video frame is obtained by summing the intra-frame mode complexity of each of the several pre-analysis blocks.

[0429] In some embodiments, the processing module 840 is configured to:

[0430] Determine the specified analysis block of the maximum complexity frame compared to the starting frame;

[0431] The proportion of the specified pre-analysis blocks in the maximum complexity frame is determined based on the number of specified pre-analysis blocks and the number of pre-analysis blocks in the maximum complexity frame.

[0432] In some embodiments, the processing module 840 is configured to:

[0433] Using the starting frame as a reference frame, determine the reference block in the starting frame for each pre-analysis block in the maximum complexity frame;

[0434] Based on the reference block corresponding to each pre-analysis block in the maximum complexity frame, the inter-frame mode complexity of each pre-analysis block in the maximum complexity frame is determined, and the motion vector of each pre-analysis block in the maximum complexity frame is determined.

[0435] If the inter-frame mode complexity of the current pre-analysis block of the frame with the highest complexity is less than the intra-frame mode complexity of the current pre-analysis block, it is determined that the current pre-analysis block adopts the inter-frame mode; otherwise, it is determined that the current pre-analysis block adopts the intra-frame mode.

[0436] The pre-analysis block in the frame with the highest complexity that adopts the inter-frame mode and whose motion vector is greater than the vector threshold is determined as the designated pre-analysis block.

[0437] In some embodiments, the processing module 840 is further configured to:

[0438] If the starting frame meets the segmentation conditions, the step of determining the target ending frame from the maximum ending frame and the at least one preset frame based on the frame features of the maximum ending frame and at least one preset frame is executed; otherwise, the maximum ending frame is determined as the target ending frame.

[0439] In some embodiments, the processing module 840 is configured to:

[0440] Determine the full intra-frame mode complexity of the starting frame;

[0441] If the total complexity of the starting frame is less than or equal to the complexity threshold, the starting frame satisfies the partitioning condition.

[0442] The complexity threshold is related to the number of pre-analysis blocks in the starting frame.

[0443] In some embodiments, the acquisition module 820 is used for:

[0444] Read the video frame sequence of the first frame number in the video to be encoded;

[0445] The video frame sequence of the first frame number is stored in the pre-analysis queue;

[0446] The video frame sequence with the preset number of frames in the pre-analysis queue is determined as the current image group;

[0447] The maximum value of the first frame number is the pre-configured maximum number of read frames.

[0448] In some embodiments, the acquisition module 820 is further configured to:

[0449] Each video frame in the image group to be encoded is removed from the pre-analysis queue to obtain the removed pre-analysis queue, which stores the video frame sequence of the second frame number.

[0450] After encoding each video frame in the image group to be encoded to obtain the encoded bitstream, the pre-analysis queue after being pushed out is supplemented with a video frame sequence of the first number of frames, and the step of determining the video frames of the preset number of frames in the pre-analysis queue as the current image group is repeated.

[0451] It should be noted that the specific limitations of the one or more video encoding devices 800 provided above can be found in the limitations of the video encoding method above, and will not be repeated here. Each module of the above device can be implemented entirely or partially by software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0452] This application also provides an encoder, which includes an encoding unit storing a computer program. The computer program is loaded and executed by the encoding unit to implement the video encoding methods provided in the above-described method embodiments.

[0453] For example, Figure 8 This is a structural block diagram of an encoder provided in an exemplary embodiment of this application. The encoder 900 includes: an encoding unit 901, a storage unit 902, an input interface 903, and an output interface 904.

[0454] Input interface 903 is the input section of encoder 900, used to receive raw video to be encoded. The source of the video to be encoded includes at least one of the following: various sensors, audio and video devices, and other data sources.

[0455] The encoding unit 901 is the core component of the encoder 900, used to encode the video to be encoded. During the encoding process, the encoding unit 901 converts the video to be encoded into a specific format according to a preset encoding standard, such as the HEVC standard, specifically the H.26x series or MPEG series standards, to obtain the encoded bitstream. After encoding, the video to be encoded can not only compress the data size, but also improve its anti-interference ability to a certain extent, and improve the stability and reliability during transmission or storage.

[0456] In some embodiments, the encoding unit 901 may be implemented in at least one hardware form selected from chips, hardware circuits, logic circuits, processors, digital signal processing (DSP), field-programmable gate arrays (FPGA), and programmable logic arrays (PLA).

[0457] Storage unit 902 is used to temporarily store the encoded bitstream, which facilitates the rapid reading of the encoded bitstream for subsequent transmission or storage. The capacity and read / write speed of storage unit 902 can be set according to actual technical needs.

[0458] In some embodiments, storage unit 902 may include one or more computer-readable storage media, which may be non-transitory. Storage unit 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices.

[0459] Output interface 904 is the output section of encoder 900, used to output the encoded bitstream to external devices or networks. Output interface 904 needs to be compatible with various standards and protocols to facilitate the output of the encoded bitstream to external devices or networks.

[0460] In some embodiments, the encoding unit 901, storage unit 902, input interface 903, and output interface 904 can be connected via a bus or signal lines. Various external devices can be connected to the input interface 903 and output interface 904 via a bus, signal lines, or circuit board. The input interface 903 and output interface 904 can be used to connect at least one input / output (I / O) related external device to the encoding unit 901 and storage unit 902. In some examples, the encoding unit 901, storage unit 902, input interface 903, and output interface 904 are integrated on the same chip or circuit board; in other examples, any one or two of the encoding unit 901, storage unit 902, input interface 903, and output interface 904 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0461] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the encoder and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0462] This application also provides a computer device, which includes a processor and a memory, wherein the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the video encoding method provided in the above-described method embodiments.

[0463] For example, Figure 9 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Optionally, the computer device is a server 1000.

[0464] Typically, server 1000 includes a processor 1001 and memory 1002.

[0465] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a central processing unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1001 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0466] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the video encoding method provided in the above-described method embodiments.

[0467] In some embodiments, the server 1000 may optionally include an input interface 1003 and an output interface 1004. The processor 1001, memory 1002, and input interfaces 1003 and 1004 can be connected via a bus or signal lines. Various external devices can be connected to the input interfaces 1003 and 1004 via a bus, signal lines, or a circuit board. The input interfaces 1003 and 1004 can be used to connect at least one input / output (I / O) related external device to the processor 1001 and memory 1002. In some embodiments, the processor 1001, memory 1002, and input interfaces 1003 and 1004 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, memory 1002, and input interfaces 1003 and 1004 can be implemented on separate chips or circuit boards, and this application does not limit this.

[0468] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0469] In an exemplary embodiment, this application also provides a chip, which includes programmable logic circuits and / or computer instructions, and is used to implement the video encoding methods provided in the above-described method embodiments when the chip is running on a computer device.

[0470] This application also provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the video encoding methods provided in the above-described method embodiments.

[0471] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the processor of the computer device to load and execute the video encoding method provided in the above-described method embodiments.

[0472] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0473] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0474] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0475] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video encoding method, characterized in that, The method includes: Obtain the video frame sequence with a preset number of frames from the video to be encoded to obtain the current image group; Based on the video features of the current image group, determine the target last frame from the current image group; The starting frame to the target ending frame in the current image group are determined as the image group to be encoded; Each video frame in the group of images to be encoded is encoded to obtain an encoded bitstream.

2. The method according to claim 1, characterized in that, The step of determining the target last frame from the current image group based on the video features of the current image group includes: Based on the sequence features of the current image group, determine the largest last frame from the current image group; Based on the frame features of the maximum last frame and at least one preset frame, the target last frame is determined from the maximum last frame and the at least one preset frame; Wherein, the sequence number of the maximum last frame in the current image group is used to indicate the maximum number of frames in the image group to be encoded, the sequence number of the target last frame in the current image group is used to indicate the optimized number of frames in the image group to be encoded, the optimized number of frames is less than or equal to the maximum number of frames, and the at least one preset frame is a preset frame in the current image group that satisfies the image group structure conditions and is located before the maximum last frame.

3. The method according to claim 2, characterized in that, The step of determining the target end frame from the maximum end frame and the at least one preset frame based on frame features of the maximum end frame and at least one preset frame includes: Statistical features are analyzed in the frame features of the largest last frame and the at least one preset frame; Based on the relationship between the statistical features and the threshold, the target end frame is determined from the maximum end frame and the at least one preset frame; The threshold is related to the statistical features and the at least one preset frame.

4. The method according to claim 3, characterized in that, The at least one preset frame includes a first preset frame, which is a preset frame in the current image group that is located before the maximum end frame; the statistical features include features related to prediction complexity of the maximum end frame and the first preset frame, respectively, and noise-related features of the maximum end frame; The statistical features in the frame features of the largest last frame and the at least one preset frame include: Determine the prediction complexity of the maximum last frame compared to the starting frame; Determine the prediction complexity of the maximum last frame compared to the previous frame; Determine the prediction complexity of the first preset frame compared to the starting frame; Determine the prediction complexity of the first preset frame compared to the starting frame and the maximum ending frame; Calculate the first proportion of the specified pre-analysis block in the largest last frame; The prediction complexity of the maximum last frame compared to the starting frame is calculated as a second ratio to the intra-frame mode complexity of the maximum last frame. The third ratio is calculated as follows: the prediction complexity of the maximum last frame compared to the previous frame, and the intra-frame mode complexity compared to the maximum last frame. The prediction complexity of the first preset frame compared to the starting frame and the maximum ending frame is calculated as a fourth ratio to the prediction complexity of the first preset frame compared to the starting frame. The noise level of the largest last frame is statistically analyzed; The intra-frame mode complexity is the prediction complexity when predicting the pixel value at each position of each pre-analysis block using intra-frame mode; the specified pre-analysis block is a pre-analysis block using inter-frame mode and whose motion vector is greater than a vector threshold, wherein the vector threshold is pre-configured and the pre-analysis block is a pre-divided pixel block.

5. The method according to claim 4, characterized in that, The thresholds include a noise threshold, a first threshold, a second threshold, a third threshold, and a fourth threshold; Determining the target last frame from the largest last frame and the at least one preset frame based on the relationship between the statistical features and the threshold includes: If the noise level of the maximum last frame is less than the noise threshold, and the first proportion is greater than the first threshold, and the second proportion is greater than the second threshold, and the third proportion is less than the third threshold, and the fourth proportion is greater than the fourth threshold, then the first preset frame is determined as the target last frame; otherwise, the maximum last frame is determined as the target last frame. The noise threshold, the first threshold, the second threshold, the third threshold, and the fourth threshold are all pre-configured and used to indicate the characteristics of the maximum end frame and the first preset frame.

6. The method according to any one of claims 3 to 5, characterized in that, The at least one preset frame includes a first preset frame, and the statistical features include features of the first preset frame related to prediction complexity. The statistical features in the frame features of the largest last frame and the at least one preset frame further include: Determine the prediction complexity of the first preset frame compared to the starting frame; The fifth proportion of the specified pre-analysis block in the first preset frame is calculated. The sixth proportion of the pre-analysis blocks using intra-frame mode in the first preset frame is calculated. The prediction complexity of the first preset frame compared to the starting frame is calculated as a seventh ratio of the intra-frame mode complexity compared to the first preset frame. The total intra-frame mode complexity of the starting frame is calculated as an eighth ratio to the total intra-frame mode complexity of the first preset frame.

7. The method according to claim 6, characterized in that, The at least one preset frame further includes a second preset frame, which is a preset frame in the current image group that precedes the first preset frame; the threshold includes a fifth threshold, a sixth threshold, a first interval range, and a second interval range; Determining the target last frame from the largest last frame and the at least one preset frame based on the relationship between the statistical features and the threshold includes: If the fifth ratio is less than the fifth threshold, and the sixth ratio is less than the sixth threshold, and the seventh ratio is within the first interval range, and the eighth ratio is within the second interval range, then the second preset frame is determined as the target end frame; otherwise, the first preset frame remains unchanged as the target end frame. The fifth threshold, the sixth threshold, the first interval range, and the second interval range are pre-configured and used to indicate the characteristics of the first preset frame.

8. The method according to any one of claims 3 to 7, characterized in that, The at least one preset frame includes a second preset frame, and the statistical features include features of the second preset frame related to prediction complexity; The statistical features in the frame features of the largest last frame and the at least one preset frame further include: Determine the prediction complexity of the second preset frame compared to the starting frame; The ninth proportion of the specified pre-analysis block in the second preset frame is calculated. The prediction complexity of the second preset frame compared to the starting frame is calculated as a tenth ratio of the intra-frame mode complexity of the second preset frame.

9. The method according to claim 8, characterized in that, The at least one preset frame further includes a third preset frame, which is a preset frame in the current image frame that precedes the second preset frame; the threshold includes a seventh threshold and a third interval range; Determining the target last frame from the largest last frame and the at least one preset frame based on the relationship between the statistical features and the threshold includes: If the ninth ratio is greater than the seventh threshold and the tenth ratio is within the third interval range, the third preset frame is determined as the target end frame; otherwise, the second preset frame is kept as the target end frame. The seventh threshold and the third interval range are pre-configured and used to indicate the characteristics of the second preset frame.

10. The method according to claim 4, characterized in that, Determining the prediction complexity of the maximum last frame compared to the starting frame includes: The maximum last frame is divided into several pre-analysis blocks; Using the starting frame as a reference frame, the plurality of pre-analysis blocks are traversed, and for the current pre-analysis block among the plurality of pre-analysis blocks, the corresponding reference block in the starting frame is determined; Based on the reference block corresponding to the current pre-analysis block in the starting frame, determine the inter-frame mode complexity of the current pre-analysis block, and determine the motion vector of the current pre-analysis block; If the inter-frame mode complexity of the current pre-analysis block is less than the intra-frame mode complexity of the current pre-analysis block, then the current pre-analysis block is determined to use inter-frame mode and the prediction complexity of the current pre-analysis block is the inter-frame mode complexity; otherwise, the current pre-analysis block is determined to use intra-frame mode and the prediction complexity of the current pre-analysis block is the intra-frame mode complexity, and this process is repeated until the prediction complexity of each of the plurality of pre-analysis blocks is determined. The prediction complexity of each of the several pre-analysis blocks is summed to obtain the prediction complexity of the maximum last frame compared to the starting frame.

11. The method according to claim 10, characterized in that, The step of determining the inter-frame mode complexity of the current pre-analysis block based on the reference block corresponding to the current pre-analysis block in the starting frame includes: Based on the reference block corresponding to the current pre-analysis block in the starting frame, the pixel value at each position of the current pre-analysis block is predicted using an inter-frame mode to obtain the predicted pixel value. The predicted pixel value at each position of the current pre-analysis block is subtracted from the original pixel value of the reference block corresponding to the current analysis block in the starting frame to obtain the residual value at each position. Perform a Hadamard transform on the residual value at each position and sum them to obtain the inter-frame mode complexity of the current pre-analysis block.

12. The method according to any one of claims 2 to 11, characterized in that, The step of determining the largest last frame from the current image group based on the sequence features of the current image group includes: Determine the full intra-frame mode complexity of each video frame in the current image group; Determine the maximum complexity among the intra-frame mode complexities of each video frame, and determine the video frame with the maximum complexity as the maximum complexity frame of the current image group. Determine the ratio of the maximum complexity frame to a specified pre-analysis block of the starting frame; If the maximum complexity is less than the eighth threshold, the sequence number of the maximum complexity frame in the current image group is greater than the ninth threshold, and the proportion of the specified pre-analysis block is greater than the tenth threshold, the maximum complexity frame is determined as the maximum end frame. The eighth threshold is calculated based on the image resolution, while the ninth and tenth thresholds are pre-configured.

13. The method according to claim 12, characterized in that, The method further includes: Determine the last frame of the first scene in the current image group; The frame that appears first in the order of the last frame of the scene and the frame with the highest complexity is determined as the maximum last frame.

14. The method according to claim 12, characterized in that, Determining the intra-frame mode complexity of each video frame in the current image group includes: Divide the current video frame into several pre-analysis blocks; Traverse the plurality of pre-analysis blocks. For the current pre-analysis block among the plurality of pre-analysis blocks, use intra-frame mode to predict the pixel value at each position of the current pre-analysis block to obtain the predicted pixel value. Subtract the predicted pixel value from the original pixel value at each position of the current pre-analysis block to obtain the residual value at each position; Perform a Hadamard transform on the residual value at each position and sum them to obtain the intra-mode complexity of the current pre-analysis block. Repeat this process until the intra-mode complexity of each of the several pre-analysis blocks is obtained. The intra-frame mode complexity of the current video frame is obtained by summing the intra-frame mode complexity of each of the several pre-analysis blocks.

15. The method according to claim 12, characterized in that, Determining the ratio of the maximum complexity frame to the specified pre-analysis block of the starting frame includes: Determine the specified analysis block of the maximum complexity frame compared to the starting frame; The proportion of the specified pre-analysis blocks in the maximum complexity frame is determined based on the number of specified pre-analysis blocks and the number of pre-analysis blocks in the maximum complexity frame.

16. A video encoding device, characterized in that, The device includes: The acquisition module is used to acquire a video frame sequence with a preset number of frames from the video to be encoded, and obtain the current image group; The processing module is used to determine the target last frame from the current image group based on the video features of the current image group; The determining module is used to determine the starting frame to the target ending frame in the current image group as the image group to be encoded; The encoding module is used to encode each video frame in the image group to be encoded to obtain an encoded bitstream.

17. An encoder, characterized in that, The encoder includes an encoding unit that stores a computer program, which is loaded and executed by the encoding unit to implement the video encoding method as described in any one of claims 1 to 15.

18. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the video encoding method as described in any one of claims 1 to 15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the video encoding method as described in any one of claims 1 to 15.

20. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, from which a processor retrieves the computer instructions, causing the processor to load and execute them to implement the video encoding method as described in any one of claims 1 to 15.

Images