Video coding method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a video coding method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining a first coefficient level corresponding to a to-be-coded frame and second coefficient levels corresponding to a preset number of preorder coded frames, counting the second coefficient level to determine a first proportion value of a non-complex frame in a preset number of pre-coded frames, and determining a target coefficient level corresponding to the to-be-coded frame according to the first coefficient level and the first proportion value; obtaining non-square division intensity corresponding to the target coefficient level, and non-square division information and rate-distortion cost information of each coded frame, and adjusting the non-square division intensity according to the non-square division information and the rate-distortion cost information to obtain target non-square division intensity; and coding the to-be-coded frame based on the target coefficient grade and the target non-square division intensity. According to the scheme, the related coding parameters of the to-be-coded frame can be optimized, the coding efficiency is improved, and the coding requirement of a low-delay scene is met.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video coding technology, and in particular to a video coding method, apparatus, device, storage medium, and program product. Background Art

[0002] With the rapid development of video coding technology, new-generation international coding standards such as VVC, AV1, and AVS3 have made significant breakthroughs in compression efficiency. However, in order to improve the compression rate, these standards generally adopt high-computational complexity implementation methods, which leads to a significant increase in encoding time, making practical applications challenging. The specific core challenge lies in the high diversity of video content, from intense motion to static scenes, from clear and complex textures to blurry and noisy pictures. Videos with different characteristics require differentiated compression tools to achieve the best encoding effect. For example, complex block division is suitable for processing pictures with rich textures, 128 The large 128-bit Coding Tree Unit (CTU) is suitable for simple, low-bitrate scenarios, while affine motion compensation effectively handles videos with complex motions like scaling and rotation. Therefore, encoder optimization requires a comprehensive consideration of various scenarios, striking the optimal balance between computational complexity and video quality.

[0003] In related technologies, encoders use a lookahead pre-processing mechanism, which pre-analyzes subsequent frame sequences to guide decisions such as bitrate allocation and QP (Quantization Parameter) adjustments. However, for low-latency scenarios like video conferencing and live streaming, this severely limits the number of future frames that can be analyzed, resulting in insufficient information acquisition. Furthermore, due to computational complexity considerations, the lookahead pre-processing mechanism can only process images at reduced resolutions, such as half or quarter resolution, which reduces information reliability. Furthermore, encoders in related technologies also use frame-level coefficient ranking to classify each frame and perform adaptive processing. However, this lacks temporal continuity analysis, resulting in less accurate coding parameter decisions. Summary of the Invention

[0004] The embodiments of the present application provide a video encoding method, apparatus, device, storage medium and program product to solve the problems of low encoding efficiency in related technologies, which cannot meet the encoding requirements of low-latency scenarios, and low accuracy of encoding parameter decisions. The method can combine the relevant information of the encoded frames to perform time domain correlation analysis, optimize and adjust the relevant encoding parameters of the frames to be encoded, improve the accuracy of encoding parameter decisions, and improve encoding efficiency to meet the encoding requirements of low-latency scenarios.

[0005] In a first aspect, an embodiment of the present application provides a video encoding method, the method comprising: obtaining a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames, performing statistics on the second coefficient levels to determine a first proportion of non-complex frames in the preset number of encoded frames, and determining a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion; obtaining a non-square partition strength corresponding to the target coefficient level, as well as non-square partition information and rate-distortion cost information of each coded frame in the preceding preset number of coded frames, and adjusting the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; The to-be-encoded frame is encoded based on the target coefficient level and the target non-square partitioning strength.

[0006] In a second aspect, an embodiment of the present application further provides a video encoding device, including: a coefficient level determination module configured to obtain a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames in a preceding order, perform statistics on the second coefficient levels to determine a first proportion of non-complex frames in the preset number of encoded frames in the preceding order, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion; a partition strength determination module configured to obtain a non-square partition strength corresponding to the target coefficient level, and non-square partition information and rate-distortion cost information of each coded frame in a preceding preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; The encoding module is configured to encode the to-be-encoded frame based on the target coefficient level and the target non-square partitioning strength.

[0007] In a third aspect, an embodiment of the present application further provides a video encoding device, the device comprising: one or more processors; a storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding method described in the embodiment of the present application.

[0008] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are configured to execute the video encoding method described in the embodiment of the present application.

[0009] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device executes the video encoding method described in the embodiment of the present application.

[0010] In an embodiment of the present application, the second coefficient level is statistically determined to determine the first proportion value of non-complex frames in a preset number of coded frames, and the target coefficient level corresponding to the frame to be coded is determined based on the first coefficient level and the first proportion value. The coefficient level information of multiple historical coded frames can be combined to adjust the coefficient level of the frame to be coded, thereby improving the accuracy of judging the frame type of the frame to be coded; the non-square division strength is adjusted according to the non-square division information and the rate-distortion cost information to obtain the target non-square division strength, and the benefit of the frame to be coded can be estimated based on the relevant decision information of the aforementioned multiple coded frames, fully considering the time domain correlation and optimizing the adjustment of the non-square division strength. The above scheme can perform time domain correlation analysis based on the relevant information of the coded frames, optimize and adjust the relevant coding parameters of the frame to be coded, improve the accuracy of the coding parameter decision, and improve the coding efficiency to meet the coding requirements of low-latency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of a video encoding method provided in an embodiment of the present application; Figure 2 A flowchart of a video encoding method including a process of determining a target non-square partition strength provided in an embodiment of the present application; Figure 3 A flowchart of a video encoding method including a process of determining a target coefficient level provided in an embodiment of the present application; Figure 4 A flowchart of a video encoding method including a process of determining a first scale value provided in an embodiment of the present application; Figure 5 A flowchart of a video encoding method including a process of determining a target non-square partition strength provided in an embodiment of the present application; Figure 6 A flowchart of a video encoding method including a process of obtaining level information of an encoded frame by referring to a delayed frame number provided in an embodiment of the present application; Figure 7 A structural block diagram of a video encoding device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the embodiments of the present application, rather than all structures.

[0013] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0014] The video encoding method provided in the embodiments of the present application can adjust the encoding parameters of the frames to be encoded in combination with the relevant information of the encoded frames, thereby improving the accuracy of the encoding parameter decisions and enhancing the encoding efficiency. Related application scenarios may include: video conferencing, video calls, indoor and outdoor live broadcasts, and short videos. The several application scenarios listed above are only exemplary and explanatory. In actual applications, the video encoding method can also be used in other scenarios, and the embodiments of the present application do not limit this.

[0015] In the video encoding method provided in the embodiment of the present application, the execution entity of each step can be a computer device, which refers to any electronic device with data calculation, processing and storage capabilities, such as a server, etc., and the embodiment of the present application does not limit this.

[0016] In related technologies, new-generation video coding standards such as AV1 (Alliance for Open Media Video 1) and VVC (Versatile Video Coding) have introduced non-square partitioning techniques, rather than being limited to the traditional square block partitioning. This allows for more flexible adaptation to the structural characteristics of video content. Key partitioning methods include horizontal partitioning, vertical partitioning, a combination of quadtree and binary tree partitioning, and extended quadtree partitioning with 1:4 or 4:1 ratios. These more complex partitioning techniques enable more precise segmentation along object edges, reducing cross-edge blocking artifacts and improving prediction and transform efficiency. For areas with directional textures, non-square partitioning can reduce prediction residuals, better align motion boundaries during motion compensation, and reduce unnecessary subdivision levels. In specific applications, coding complexity can be controlled by limiting partition depth and configuring minimum or maximum partition sizes. Non-square partitioning can help save bitrate, especially for scenes with distinct geometric structures. Taking the SVT-AV1 encoder of the AV1 standard as an example, the coefficient level is used as the content complexity indicator, and the decision process of non-square partitioning and square partitioning is dynamically adjusted to achieve a balance between coding efficiency and computational complexity. By calculating the noise level of the input frame, calculating the content complexity based on motion estimation distortion and QP (quantization parameter) value, adaptively adjusting the thresholds of different resolution levels, and further fine-tuning the thresholds according to the noise level, each frame is finally divided into a corresponding coefficient level, which is one of the very low complexity level, low complexity level, normal complexity level and high complexity level. Specifically, the coefficient level is determined as follows: calculate the frame-level noise factor; count the 8-based noise factors calculated by the frame to be encoded in the open-loop motion estimation stage; The motion estimation distortion of the 8-block image is calculated. Multiple thresholds are fine-tuned based on QP, sequence resolution, and frame-level noise factors to obtain final thresholds for classifying the image into very low complexity, low complexity, normal complexity, and high complexity levels. The calculated motion estimation distortion is compared with these thresholds to determine the coefficient level to which the frame belongs. For frames of very low or low complexity levels, the number of candidate non-square partitioning modes can be reduced, the rate-distortion cost threshold for non-square partitioning can be increased to make it less likely that a non-square partition will be selected, or the evaluation of certain complex non-square partition types can be skipped. Furthermore, larger square blocks can be used to reduce the quadtree partition depth and simplify transform type selection. For frames of high complexity levels, the number of candidate non-square partitioning modes can be increased, the rate-distortion threshold for non-square partitioning can be lowered, allowing for deeper recursive partitioning, while also allowing for smaller square block partitions, increasing the quadtree partition depth, or using more precise transform type selection. Although the aforementioned method can classify each frame by calculating the coefficient level at the frame level and perform adaptive processing, it lacks time domain continuity analysis and only considers the characteristics of a single frame, which limits the accuracy of the decision. In addition, the Lookahead pre-processing mechanism is used to guide decisions such as bit rate allocation and QP adjustment by pre-analyzing subsequent frame sequences. However, it strictly limits the number of future frames that can be analyzed for low-latency scenarios such as video conferencing and live broadcasts, resulting in insufficient information acquisition. Moreover, due to computational complexity considerations, the Lookahead pre-processing mechanism can only process reduced-resolution images such as half-resolution or quarter-resolution, which reduces information reliability. Based on this, the embodiments of the present application aim to provide a video encoding method, apparatus, device, storage medium, and program product to solve the problem that the encoding efficiency of the related technology is low, the encoding requirements of low-latency scenarios cannot be met, and the accuracy of encoding parameter decisions is low.

[0017] Figure 1 A flowchart of a video encoding method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the video encoding method specifically includes the following steps: Step S101: Obtain a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames in the preceding sequence, perform statistics on the second coefficient level to determine a first proportion value of non-complex frames in the preset number of encoded frames in the preceding sequence, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion value.

[0018] The first coefficient level corresponding to the frame to be coded may be originally determined by the encoder and may represent the content complexity of the frame to be coded. The preceding preset number of coded frames may be a plurality of previously coded video frames based on the frame to be coded. The second coefficient level may represent the content complexity of the coded frame. For example, the coefficient levels may include an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level. If the second coefficient level corresponding to a coded frame is an extremely low complexity level or a low complexity level, the coded frame may be considered a non-complex frame. By statistically analyzing the second coefficient levels corresponding to each of the preceding preset number of coded frames, a first ratio of non-complex frames may be determined. This first ratio represents the proportion of non-complex frames in the preceding preset number of coded frames relative to the total number of frames in the preceding preset number of coded frames, and may, to a certain extent, reflect the possible trend of the complexity of the frame to be coded. Therefore, by combining the originally determined first coefficient level corresponding to the frame to be coded and the first ratio, and fully considering the coefficient level distribution of the coded frames, a target coefficient level for the frame to be coded can be more accurately determined.

[0019] In one embodiment, the process of specifically determining the target coefficient level can be to first adjust the first proportional value according to the specific level type of the first coefficient level to obtain a second proportional value. For example, if the frame to be encoded is a non-complex frame, the first proportional value can be adjusted upward by a preset value to obtain a second proportional value; if the frame to be encoded is a normal frame or a complex frame, the first proportional value can be adjusted downward by a preset value to obtain a second proportional value. The second proportional value is then compared with multiple set numerical ranges. Different numerical ranges correspond to different coefficient levels. For example, four numerical ranges are set, corresponding to an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level, respectively. By determining in which numerical range the second proportional value falls, the frame to be encoded can be determined to be the target coefficient level corresponding to that numerical range.

[0020] In one embodiment, an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level are arranged in order of complexity from low to high. The specific process of determining the target coefficient level may be to compare the first ratio value with a set threshold value. If the first ratio value is greater than the threshold value, the first coefficient level may be used as a benchmark, and a coefficient level with a complexity reduction of one level may be selected as the target coefficient level of the frame to be encoded. For example, if the first coefficient level is a normal complexity level, the low complexity level may be selected as the target coefficient level. If the first ratio value is less than or equal to the threshold value, the first coefficient level may be used as a benchmark, and a coefficient level with a complexity increase of one level may be selected as the target coefficient level of the frame to be encoded. For example, if the first coefficient level is an extremely low complexity level, the low complexity level may be selected as the target coefficient level.

[0021] Step S102: Obtain a non-square partition strength corresponding to a target coefficient level, as well as non-square partition information and rate-distortion cost information of each coded frame in a preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength.

[0022] Among them, the encoders corresponding to different coefficient levels are originally set with corresponding non-square division strengths. The non-square division strength can represent the complexity of the non-square division strategy adopted by the encoding. If the non-square division strength is greater, the complexity of the non-square division strategy adopted by the encoding is greater; if the non-square division strength is smaller, the complexity of the non-square division strategy adopted by the encoding is smaller. The non-square division information of each encoded frame in the pre-set number of encoded frames can be the selection result of whether each encoding block in all the encoding blocks in the encoded frame that attempt to perform non-square division performs non-square division, which can specifically include counting the number of encoding blocks that select non-square division and the number of encoding blocks that do not select non-square division. The rate-distortion cost information can include the rate-distortion cost corresponding to whether each encoding block in all the encoding blocks in the encoded frame that attempt to perform non-square division selects non-square division.

[0023] In one embodiment, the process of determining the target non-square partitioning strength can be to first calculate the block-level yield ratio based on the first rate-distortion cost corresponding to the selection of non-square partitioning and the second rate-distortion cost of not selecting non-square partitioning for each coding block in all coding blocks that attempt to perform non-square partitioning in the coded frame, and add up the multiple block-level yield ratios obtained to obtain the frame-level yield ratio. Then calculate the first sum of the number of coding blocks that select non-square partitioning and the number of coding blocks that do not select non-square partitioning. Divide the frame-level yield ratio by the first sum result to obtain the frame-level yield. The frame-level yield calculated based on multiple coded frames can be used to calculate the average yield. The average yield is compared with multiple set numerical ranges, for example, a first numerical range and a second numerical range are set, wherein the lower limit value of the first numerical range is greater than the upper limit value of the second numerical range. Since the average yield is larger, a more complex non-square partitioning strategy can be selected. Therefore, if the average yield is within the first numerical range, the non-square division strength can be increased by a preset step size to obtain the target non-square division strength. If the average yield is within the second numerical range, the non-square division strength can be decreased by a preset step size to obtain the target non-square division strength.

[0024] In one embodiment, the process of determining the target non-square partitioning strength can be to obtain the average yield according to the aforementioned calculation process, and then compare the average yield with a plurality of set numerical ranges, for example, a first numerical range and a second numerical range are set, wherein the first numerical range is larger than the second numerical range. As the average yield is larger, a more complex non-square partitioning strategy can be selected. Therefore, if the average yield is within the first numerical range, the non-square partitioning strength can be multiplied by the preset upward adjustment ratio to obtain the target non-square partitioning strength. If the average yield is within the second numerical range, the non-square partitioning strength can be multiplied by the preset downward adjustment ratio to obtain the target non-square partitioning strength.

[0025] Step S103: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0026] Among them, the first coefficient level and non-square division strength are the coding parameters selected for optimization and adjustment in this embodiment. After calculating the target coefficient level and target non-square division strength, the frame to be coded can be encoded in combination with other existing coding parameters.

[0027] In the above, the second coefficient level is statistically determined to determine the first proportion value of the non-complex frames in the preceding preset number of coded frames, and the target coefficient level corresponding to the frame to be coded is determined based on the first coefficient level and the first proportion value. The coefficient level information of multiple historical coded frames can be combined to adjust the coefficient level of the frame to be coded, thereby improving the accuracy of judging the frame type of the frame to be coded; the non-square division strength is adjusted according to the non-square division information and the rate-distortion cost information to obtain the target non-square division strength, and the benefit of the frame to be coded can be estimated based on the relevant decision information of the aforementioned multiple coded frames, and the time domain correlation can be fully considered to optimize and adjust the non-square division strength. The above scheme can combine the relevant information of the coded frames to perform time domain correlation analysis, optimize and adjust the relevant coding parameters of the frame to be coded, improve the accuracy of the coding parameter decision, and improve the coding efficiency to meet the coding requirements of low-latency scenarios.

[0028] Figure 2 A flowchart of a video encoding method including a process of determining a target non-square partition strength is provided in an embodiment of the present application, wherein the non-square partition information includes a first number of coding blocks that select non-square partition and a second number of coding blocks that do not select non-square partition, and the rate-distortion cost information includes a first rate-distortion cost corresponding to the selection of non-square partition for each coding block and a second rate-distortion cost corresponding to the selection of non-square partition. Figure 2 As shown, the video encoding method specifically includes the following steps: Step S201: Obtain a first coefficient level corresponding to the frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames in the preceding sequence, perform statistics on the second coefficient level to determine a first proportion value of non-complex frames in the preset number of encoded frames in the preceding sequence, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion value.

[0029] Step S202: Obtain a non-square partitioning strength corresponding to a target coefficient level, as well as non-square partitioning information and rate-distortion cost information for each of a predetermined number of encoded frames, wherein the non-square partitioning information includes a first number of coding blocks for which non-square partitioning is selected and a second number of coding blocks for which non-square partitioning is not selected, and the rate-distortion cost information includes a first rate-distortion cost corresponding to selecting non-square partitioning for each coding block and a second rate-distortion cost for not selecting non-square partitioning.

[0030] Step S203, add the first number and the second number to obtain a first addition result; divide the second rate-distortion cost corresponding to each coding block by the first rate-distortion cost to obtain a block-level benefit ratio, and add the block-level benefit ratios corresponding to each coding block to obtain a frame-level benefit ratio corresponding to the encoded frame; divide the frame-level benefit ratio by the first addition result to obtain a frame-level yield, and calculate an average yield based on the frame-level yield corresponding to each encoded frame; compare the average yield with a second preset threshold, and adjust the non-square partition strength based on the comparison result to obtain a target non-square partition strength.

[0031] The non-square partitioning information and rate-distortion cost information are the results of the encoder's non-square partitioning decision and can serve as reference information for adjusting the strength of the non-square partitioning. The second rate-distortion cost corresponds to the distortion of not selecting non-square partitioning for the coding block, while the first rate-distortion cost corresponds to the distortion of selecting non-square partitioning for the coding block. Dividing the second rate-distortion cost corresponding to each coding block by the first rate-distortion cost yields a block-level benefit ratio, which is used to quantify the benefit of selecting non-square partitioning for each coding block. Therefore, the sum of the block-level benefit ratios corresponding to each coding block in an encoded frame can be used as the frame-level benefit ratio of the encoded frame. This frame-level benefit ratio quantifies the benefit of selecting non-square partitioning for each encoded frame. Furthermore, after obtaining the frame-level benefit ratio corresponding to each encoded frame, the frame-level benefit ratio can be divided by the first summation result to obtain a frame-level yield ratio, which quantifies the unit benefit of selecting non-square partitioning for the coding block in the encoded frame. By averaging the frame-level yield ratios corresponding to each encoded frame, an average yield ratio can be obtained. This average yield ratio represents the average benefit of selecting non-square partitioning for a preset number of encoded frames.

[0032] It is understandable that the higher the average yield, the more complex the non-square partitioning strategy can be selected, ensuring less speed loss while improving the peak signal-to-noise ratio benefit; and the smaller the average yield, the simpler the non-square partitioning strategy can be selected to reduce computing power waste and improve coding efficiency. Therefore, the average yield can be compared with the second preset threshold, and the non-square partitioning strength can be adjusted based on the comparison result to obtain the target non-square partitioning strength, wherein the larger the average yield, the larger the non-square partitioning strength should be, and the smaller the average yield, the smaller the non-square partitioning strength should be.

[0033] In one embodiment, the average yield is compared with a second preset threshold. If the average yield is greater than the second preset threshold, the non-square division strength can be increased according to a preset step size to obtain the target non-square division strength; if the average yield is less than or equal to the second preset threshold, the non-square division strength can be reduced according to a preset step size to obtain the target non-square division strength.

[0034] In one embodiment, the second preset threshold may include a first threshold and a second threshold, wherein the first threshold is greater than the second threshold. The average rate of return is compared with the second preset threshold. If the average rate of return is greater than the first threshold, the non-square partitioning strength may be increased by a preset step size to obtain a target non-square partitioning strength; if the average rate of return is less than the second threshold, the non-square partitioning strength may be decreased by a preset step size to obtain the target non-square partitioning strength.

[0035] Step S204: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0036] As described above, by calculating the frame-level yield corresponding to each coded frame, the unit yield of the non-square division selection of the coding block in the coded frame can be accurately quantified; and by averaging the frame-level yields corresponding to multiple coded frames to obtain the average yield, the division yields of multiple coded frames can be smoothed, providing reliable information for adjusting the non-square division strength; by adjusting the non-square division strength based on the comparison result to obtain the target non-square division strength, the complexity of the non-square division of the frame to be coded can be adaptively adjusted, thereby optimizing the coding efficiency and ensuring the coding quality.

[0037] Figure 3 The flowchart of a video encoding method including a process of determining a target coefficient level is provided in an embodiment of the present application. Figure 3 As shown, the video encoding method specifically includes the following steps: Step S301: Obtain a first coefficient level corresponding to a frame to be encoded and second coefficient levels corresponding to a preset number of encoded frames, and perform statistics on the second coefficient levels to determine a first proportion of non-complex frames in the preset number of encoded frames.

[0038] Step S302: When the first coefficient level is a preset complexity level, increase the first proportional value by a preset value corresponding to the preset complexity level to obtain a second proportional value; compare the second proportional value with the first preset threshold value, and determine the target coefficient level corresponding to the frame to be encoded based on the comparison result.

[0039] The coefficient level can be subdivided into multiple different levels to represent the content complexity of the coded frame. The preset complexity level can be a related level indicating that the frame to be coded is a non-complex frame. For example, the coefficient level can include an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level. Correspondingly, the preset complexity level can include an extremely low complexity level and a low complexity level.

[0040] In one embodiment, if the first coefficient level is an extremely low complexity level, the first proportional value may be increased by a first preset value, the first preset value ranging from 0.5 to 1, for example, the first preset value is 0.6. In one embodiment, if the first coefficient level is a low complexity level, the first proportional value may be increased by a second preset value, the second preset value ranging from 0 to 0.5, for example, the second preset value is 0.3. Thus, the first proportional value may be offset-adjusted based on the first coefficient level originally determined for the frame to be encoded to obtain a second proportional value, so that the judgment of whether the frame to be encoded is a non-complex frame is more accurate, which is beneficial to improving encoding efficiency. The larger the second proportional value, the less complex the content of the frame to be encoded.

[0041] In one embodiment, the coefficient level may include an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level. Correspondingly, the first preset threshold may include multiple thresholds, for example, the multiple thresholds may be 1, 0.6, and 0.3. If the second ratio value is greater than 1, it can be determined that the target coefficient level corresponding to the frame to be encoded is an extremely low complexity level; if the second ratio value is less than or equal to 1 and greater than 0.6, it can be determined that the target coefficient level corresponding to the frame to be encoded is a low complexity level; if the second ratio value is less than or equal to 0.6 and greater than 0.3, it can be determined that the target coefficient level corresponding to the frame to be encoded is a normal complexity level; if the second ratio value is less than or equal to 0.3, it can be determined that the target coefficient level corresponding to the frame to be encoded is a high complexity level. Of course, the foregoing is only an exemplary description. The specific settings of the first preset threshold and coefficient level can be adaptively adjusted by the developer according to the encoding requirements and empirical data of the actual application scenario, and this application does not limit this.

[0042] Step S303: Obtain the non-square partition strength corresponding to the target coefficient level, as well as the non-square partition information and rate-distortion cost information of each coded frame in a preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain the target non-square partition strength.

[0043] Step S304: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0044] As described above, by performing an offset adjustment on the first proportional value based on the specific first coefficient level of the frame to be encoded, the original level information of the frame to be encoded and the distribution information of the non-complex frames in the encoded frames can be comprehensively referred to, thereby improving the accuracy of determining the target coefficient level corresponding to the frame to be encoded, which is beneficial to optimizing the encoding efficiency.

[0045] Figure 4 The flowchart of a video encoding method including a process of determining a first ratio value is provided in an embodiment of the present application. Figure 4 As shown, the video encoding method specifically includes the following steps: Step S401: Obtain a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to a predetermined number of preceding encoded frames.

[0046] Step S402: Count a third number of encoded frames whose second coefficient level is a preset complexity level among the preceding preset number of encoded frames, where the preset complexity level indicates that the corresponding encoded frame is a non-complex frame; and divide the third number by the number of frames of the preceding preset number of encoded frames to obtain a first ratio value.

[0047] Among them, the preset complexity level indicates that the corresponding encoded frame is a non-complex frame. For example, the second coefficient level may include an extremely low complexity level, a low complexity level, a normal complexity level, and a high complexity level, and the preset complexity level may be an extremely low complexity level and a low complexity level. Counting the third number of encoded frames whose second coefficient level is the preset complexity level in the preceding preset number of encoded frames is equivalent to counting the number of non-complex frames in the multiple encoded frames. The third number is divided by the number of frames of the preceding preset number of encoded frames to obtain a first ratio value, and the proportion of non-complex frames relative to the multiple encoded frames can be determined.

[0048] Step S403: Determine a target coefficient level corresponding to the frame to be encoded according to the first coefficient level and the first scale value.

[0049] Step S404: Obtain the non-square partition strength corresponding to the target coefficient level, as well as the non-square partition information and rate-distortion cost information of each coded frame in a preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength.

[0050] Step S405: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0051] As described above, by dividing the third number of encoded frames corresponding to the preset complexity level by the number of frames of the previous preset number of encoded frames to obtain the first ratio value, the proportion of non-complex frames relative to multiple encoded frames can be accurately calculated, providing reliable reference information for the subsequent determination of the target coefficient level.

[0052] Figure 5 The flowchart of a video encoding method including a process of determining a target non-square partition strength is provided in an embodiment of the present application. Figure 5 As shown, the video encoding method specifically includes the following steps: Step S501: Obtain a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames in the preceding sequence, perform statistics on the second coefficient level to determine a first proportion value of non-complex frames in the preset number of encoded frames in the preceding sequence, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion value.

[0053] Step S502: Obtain a non-square partitioning strength corresponding to a target coefficient level, as well as non-square partitioning information and rate-distortion cost information for each of a predetermined number of encoded frames, wherein the non-square partitioning information includes a first number of coding blocks for which non-square partitioning is selected and a second number of coding blocks for which non-square partitioning is not selected, and the rate-distortion cost information includes a first rate-distortion cost corresponding to selecting non-square partitioning for each coding block and a second rate-distortion cost for not selecting non-square partitioning.

[0054] Step S503: add the first number and the second number to obtain a first addition result; divide the second rate-distortion cost corresponding to each coding block by the first rate-distortion cost to obtain a block-level revenue ratio, and add the block-level revenue ratios corresponding to each coding block to obtain a frame-level revenue ratio corresponding to the encoded frame; divide the frame-level revenue ratio by the first addition result to obtain a frame-level yield, and calculate the average yield based on the frame-level yields corresponding to each encoded frame.

[0055] Step S504: compare the average rate of return with a second preset threshold, wherein the second preset threshold includes a first threshold and a second threshold, and the first threshold is greater than the second threshold.

[0056] Step S505: When the average rate of return is greater than the first threshold, the non-square partitioning strength is increased by a preset step size to obtain a target non-square partitioning strength.

[0057] If the average rate of return is greater than the first threshold, it can be considered that the expected return of selecting non-square partitioning for the to-be-encoded frame is high, and the complexity of the non-square partitioning can be increased accordingly to improve encoding quality. For example, the first threshold can be 1.6. Specific other values ​​can be adaptively adjusted by developers based on the encoding requirements of actual application scenarios and empirical data, and are not limited in this application.

[0058] Step S506: When the average rate of return is less than the second threshold, the non-square partitioning strength is reduced by a preset step size to obtain a target non-square partitioning strength.

[0059] If the average rate of return is less than the second threshold, it can be considered that the expected return of selecting non-square partitioning for the encoded frame is low. The complexity of the non-square partitioning can be reduced accordingly to improve encoding efficiency. For example, the first threshold can be 0.8. Other specific values ​​can be adaptively adjusted by developers based on the encoding requirements of actual application scenarios and empirical data, and are not limited in this application.

[0060] Step S507: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0061] As described above, by comparing the first threshold and the second threshold set for the average yield, and adjusting the non-square division strength based on the comparison result, the non-square division strength can be adjusted to adapt to the different yield expectations of selecting non-square division for the frame to be encoded, effectively balancing the encoding efficiency and encoding quality, and optimizing the encoding performance.

[0062] Figure 6 The flowchart of a video encoding method provided by an embodiment of the present application includes a process of obtaining level information of an encoded frame by referring to the number of delayed frames. Figure 6 As shown, the video encoding method specifically includes the following steps: Step S601: Obtain encoding parallelism and a minimum group of pictures size, multiply the encoding parallelism by the minimum group of pictures size to obtain the number of delayed frames; determine a target encoding frame whose frame number is the number of delayed frames away from the frame to be encoded, obtain second coefficient levels corresponding to a preset number of encoded frames preceding the target encoding frame, and perform statistics on the second coefficient levels to determine a first proportion value of non-complex frames in the preset number of encoded frames preceding the target encoding frame.

[0063] Since the encoder can use parallel coding, for example, when performing parallel coding based on a minimum group of pictures (GOP), if the encoding progress of the preset number of coded frames preceding the frame to be encoded is uncertain, some blocks or frames may be coded while the remaining frames are still waiting to be encoded. Therefore, each time the same sequence is encoded, the encoding result of the preceding frame is uncertain at a certain moment, leading to inconsistent encoding. Therefore, it is necessary to consider the parallel encoding situation and set a separate delay frame number to ensure that the encoding result of the encoded frame is certain and accurate, thereby ensuring consistency of the encoding results across multiple encodings. The encoding parallelism can be the maximum number of GOPs that the encoder allows for parallel encoding. For example, if a maximum of x additional GOPs are allowed for parallel encoding, the encoding parallelism is x+1. If the GOP size is y, the delay frame number is (x+1)*y. Assuming the frame number to be encoded is M, the preset number is K, and the delay frame number is D, then before considering the delay frame number D, the second coefficient levels corresponding to frames MK through M-1 are obtained. After considering the delayed frame number D, the target coding frame whose frame number is the delayed frame number from the frame to be coded is used as the benchmark, that is, the MDth frame is used as the benchmark, and the second coefficient levels corresponding to the MDKth frame to the MD-1th frame are obtained.

[0064] Step S602: Obtain a first coefficient level corresponding to the frame to be encoded.

[0065] Step S603: Determine a target coefficient level corresponding to the frame to be encoded according to the first coefficient level and the first scale value.

[0066] Step S604: Obtain the non-square partition strength corresponding to the target coefficient level, as well as the non-square partition information and rate-distortion cost information of each coded frame in a preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength.

[0067] Step S605: Encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0068] In the above, the encoding parallelism and the minimum picture group size are obtained, and the encoding parallelism and the minimum picture group size are multiplied to obtain the number of delayed frames. The number of delayed frames can be accurately set considering the parallel encoding situation of the encoder, and the target encoding frame whose frame number is the number of delayed frames away from the frame to be encoded is determined, and the second coefficient level corresponding to each of the preset number of encoded frames preceding the target encoding frame is obtained, which can ensure the encoding consistency in the scenario of encoding the same content multiple times.

[0069] Figure 7This is a structural block diagram of a video encoding device provided in an embodiment of the present application. The device is configured to execute the video encoding method provided in the above embodiment and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the device specifically includes: The coefficient level determination module 701 is configured to obtain a first coefficient level corresponding to the frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames, perform statistics on the second coefficient levels to determine a first ratio value of non-complex frames in the preset number of encoded frames, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first ratio value. a partition strength determination module 702 configured to obtain a non-square partition strength corresponding to a target coefficient level, and non-square partition information and rate-distortion cost information of each coded frame in a predetermined number of preceding coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; The encoding module 703 is configured to encode the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0070] In the above, the second coefficient level is statistically determined to determine the first proportion value of the non-complex frames in the preceding preset number of coded frames, and the target coefficient level corresponding to the frame to be coded is determined based on the first coefficient level and the first proportion value. The coefficient level information of multiple historical coded frames can be combined to adjust the coefficient level of the frame to be coded, thereby improving the accuracy of judging the frame type of the frame to be coded; the non-square division strength is adjusted according to the non-square division information and the rate-distortion cost information to obtain the target non-square division strength, and the benefit of the frame to be coded can be estimated based on the relevant decision information of the aforementioned multiple coded frames, and the time domain correlation can be fully considered to optimize and adjust the non-square division strength. The above scheme can combine the relevant information of the coded frames to perform time domain correlation analysis, optimize and adjust the relevant coding parameters of the frame to be coded, improve the accuracy of the coding parameter decision, and improve the coding efficiency to meet the coding requirements of low-latency scenarios.

[0071] In one possible embodiment, the non-square partitioning information includes a first number of coding blocks that select the non-square partitioning and a second number of coding blocks that do not select the non-square partitioning; the rate-distortion cost information includes a first rate-distortion cost corresponding to selecting the non-square partitioning for each coding block and a second rate-distortion cost corresponding to not selecting the non-square partitioning; the partitioning strength determining module 702 is further configured to: Adding the first quantity to the second quantity to obtain a first addition result; Dividing the second rate-distortion cost corresponding to each coding block by the first rate-distortion cost to obtain a block-level benefit ratio, and adding the block-level benefit ratios corresponding to each coding block to obtain a frame-level benefit ratio corresponding to the coded frame; Dividing the frame-level revenue ratio by the first addition result to obtain a frame-level yield, and calculating an average yield based on the frame-level yields corresponding to each encoded frame; The average rate of return is compared with a second preset threshold, and the non-square partitioning strength is adjusted based on the comparison result to obtain a target non-square partitioning strength.

[0072] In one possible embodiment, the coefficient level determination module 701 is further configured to: In a case where the first coefficient level is a preset complexity level, increasing the first proportional value by a preset value corresponding to the preset complexity level to obtain a second proportional value; The second ratio value is compared with the first preset threshold value, and a target coefficient level corresponding to the frame to be encoded is determined based on the comparison result.

[0073] In one possible embodiment, the coefficient level determination module 701 is further configured to: counting a third number of coded frames having a second coefficient level of a preset complexity level among a preset number of coded frames in a preceding sequence, wherein the preset complexity level indicates that the corresponding coded frames are non-complex frames; The first ratio value is obtained by dividing the third number by the number of frames of the preceding preset number of encoded frames.

[0074] In a possible embodiment, the second preset threshold includes a first threshold and a second threshold, the first threshold is greater than the second threshold, and the division strength determination module 702 is further configured to: When the average rate of return is greater than the first threshold, the non-square partitioning strength is increased by a preset step size to obtain a target non-square partitioning strength.

[0075] In a possible embodiment, the partition strength determination module 702 is further configured to: When the average rate of return is less than the second threshold, the non-square partitioning strength is reduced by a preset step size to obtain a target non-square partitioning strength.

[0076] In a possible embodiment, the invention further includes a delay frame number determination module configured to: Obtain encoding parallelism and minimum GOP size, and multiply the encoding parallelism by the minimum GOP size to obtain the number of delayed frames; The coefficient level determination module 701 is further configured to: A target coding frame whose frame sequence number is the delay frame number from the frame to be coded is determined, and second coefficient levels corresponding to a preset number of coded frames preceding the target coding frame are obtained.

[0077] Figure 8 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application is shown in FIG. Figure 8As shown, the device includes a processor 801, a memory 802, an input device 803 and an output device 804; the number of processors 801 in the device can be one or more. Figure 8 In the embodiment, a processor 801 is used as an example; the processor 801, the memory 802, the input device 803 and the output device 804 in the device can be connected by a bus or other means. Figure 8 The example of the connection via bus is taken. The memory 802, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the video encoding method in the embodiment of the present application. The processor 801 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 802, that is, implements the above-mentioned video encoding method. The input device 803 can be configured to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 804 may include a display device such as a display screen.

[0078] An embodiment of the present application also provides a non-volatile storage medium containing computer-executable instructions, which, when executed by a computer processor, is configured to perform a video encoding method described in the above embodiment, which includes: obtaining a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames, performing statistics on the second coefficient level to determine a first proportion value of non-complex frames in the preset number of encoded frames, and determining a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion value; obtaining a non-square partition strength corresponding to the target coefficient level, and non-square partition information and rate-distortion cost information of each encoded frame in the preset number of encoded frames, adjusting the non-square partition strength based on the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; and encoding the frame to be encoded based on the target coefficient level and the target non-square partition strength.

[0079] It is worth noting that in the embodiment of the above-mentioned video encoding device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not configured to limit the scope of protection of the embodiments of this application.

[0080] In some possible implementations, various aspects of the methods provided herein may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is configured to cause the computer device to execute the steps of the methods according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the video encoding method described in the embodiments of the present application. The program product may be implemented using a combination of one or more readable media.

Claims

1. A video encoding method, characterized in that: include: obtaining a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames, performing statistics on the second coefficient levels to determine a first proportion of non-complex frames in the preset number of encoded frames, and determining a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion; obtaining a non-square partition strength corresponding to the target coefficient level, as well as non-square partition information and rate-distortion cost information of each coded frame in the preceding preset number of coded frames, and adjusting the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; The to-be-encoded frame is encoded based on the target coefficient level and the target non-square partitioning strength.

2. The video encoding method according to claim 1, wherein: The non-square partition information includes a first number of coding blocks that select the non-square partition and a second number of coding blocks that do not select the non-square partition, and the rate-distortion cost information includes a first rate-distortion cost corresponding to selecting the non-square partition for each coding block and a second rate-distortion cost corresponding to not selecting the non-square partition; The adjusting the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength includes: adding the first amount to the second amount to obtain a first addition result; Dividing the second rate-distortion cost corresponding to each coding block by the first rate-distortion cost to obtain a block-level benefit ratio, and adding the block-level benefit ratios corresponding to the coding blocks to obtain a frame-level benefit ratio corresponding to the coded frame; Dividing the frame-level revenue ratio by the first addition result to obtain a frame-level yield, and calculating an average yield based on the frame-level yields corresponding to each of the encoded frames; The average rate of return is compared with a second preset threshold, and based on the comparison result, the non-square partitioning strength is adjusted to obtain a target non-square partitioning strength.

3. The video encoding method according to claim 1, wherein: The determining, according to the first coefficient level and the first scale value, a target coefficient level corresponding to the to-be-encoded frame includes: In a case where the first coefficient level is a preset complexity level, increasing the first proportional value by a preset value corresponding to the preset complexity level to obtain a second proportional value; The second ratio value is compared with a first preset threshold value, and a target coefficient level corresponding to the frame to be encoded is determined based on the comparison result.

4. The video encoding method according to claim 1, wherein: The performing statistical analysis on the second coefficient level to determine a first proportion value of non-complex frames in the preceding preset number of encoded frames includes: Counting a third number of coded frames having a second coefficient level of a preset complexity level among the preceding preset number of coded frames, wherein the preset complexity level indicates that the corresponding coded frame is a non-complex frame; The first ratio value is obtained by dividing the third number by the number of frames of the preceding preset number of encoded frames.

5. The video encoding method according to claim 2, wherein: The second preset threshold includes a first threshold and a second threshold, the first threshold is greater than the second threshold, and the adjusting the non-square partition strength based on the comparison result to obtain the target non-square partition strength includes: When the average rate of return is greater than a first threshold, the non-square partitioning strength is increased by a preset step size to obtain a target non-square partitioning strength.

6. The video encoding method according to claim 5, wherein: The adjusting the non-square partition strength based on the comparison result to obtain a target non-square partition strength further includes: When the average rate of return is less than a second threshold, the non-square partitioning strength is reduced by a preset step size to obtain a target non-square partitioning strength.

7. The video encoding method according to any one of claims 1 to 6, characterized in that: Before obtaining the second coefficient levels corresponding to the first preset number of encoded frames, the method further includes: Obtaining encoding parallelism and a minimum group of pictures size, and multiplying the encoding parallelism by the minimum group of pictures size to obtain a number of delayed frames; The step of obtaining the second coefficient levels corresponding to the first predetermined number of encoded frames comprises: A target coding frame whose frame sequence number is the number of delayed frames from the frame to be coded is determined, and second coefficient levels corresponding to respective preset numbers of coded frames preceding the target coding frame are obtained.

8. A video encoding device, characterized in that: include: a coefficient level determination module configured to obtain a first coefficient level corresponding to a frame to be encoded and a second coefficient level corresponding to each of a preset number of encoded frames in a preceding order, perform statistics on the second coefficient levels to determine a first proportion of non-complex frames in the preset number of encoded frames in the preceding order, and determine a target coefficient level corresponding to the frame to be encoded based on the first coefficient level and the first proportion; a partition strength determination module configured to obtain a non-square partition strength corresponding to the target coefficient level, and non-square partition information and rate-distortion cost information of each coded frame in the preceding preset number of coded frames, and adjust the non-square partition strength according to the non-square partition information and the rate-distortion cost information to obtain a target non-square partition strength; The encoding module is configured to encode the to-be-encoded frame based on the target coefficient level and the target non-square partitioning strength.

9. A video encoding device, characterized in that The device includes: one or more processors; and a storage device configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the video encoding method according to any one of claims 1 to 7.

10. A non-volatile storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, the computer executable instructions are configured to perform the video encoding method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that The computer program is stored in a computer-readable storage medium, and at least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device performs the video encoding method according to any one of claims 1 to 7.

Images