Video encoding method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a video coding method, device, electronic equipment and storage medium. The video coding method comprises: obtaining feature information of a target coding unit in a video frame to be coded, feature information of a parent coding unit of the target coding unit, and feature information of a neighboring coding unit; determining a prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the neighboring coding unit; and coding the target coding unit based on the determined prediction mode. The video coding method, device, electronic equipment and storage medium according to the present disclosure can solve the problem that traversing the prediction mode occupies a large amount of resources and is extremely time-consuming, and can determine the prediction mode of the coding unit to be coded, thereby saving the calculation complexity in the mode selection aspect of the inter-coded frame and improving the coding efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of video coding technology, and in particular to a video coding method, apparatus, electronic device and storage medium. Background Technology

[0002] In video coding, each video frame is typically encoded. There are three types of coded frames: I-frames, P-frames, and B-frames. I-frames are intra-coded frames, P-frames are forward reference frames, and B-frames are bi-directional reference frames. An I-frame is an independent frame containing all its own information and can be decoded independently without referencing other frames; it uses intra-frame prediction for encoding. P-frames and B-frames are inter-coded frames, and they can be encoded using either intra-frame prediction or inter-frame prediction.

[0003] In existing methods for encoding inter-frame coded frames, it is necessary to traverse all preset prediction modes, including intra-frame prediction modes and inter-frame prediction modes, and then select the prediction mode with the lowest rate-distortion cost as the final prediction mode for encoding.

[0004] However, with the development of coding technology, in order to improve the coding performance of encoders, prediction modes are constantly being added and optimized. This results in the process of traversing prediction modes in existing coding methods consuming a lot of computing resources and being extremely time-consuming, which is not conducive to improving coding efficiency. Summary of the Invention

[0005] This disclosure provides a video encoding method, apparatus, electronic device, and storage medium to at least solve the problem of excessive resource consumption and extremely time-consuming traversal prediction modes in related technologies. The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a video coding method is provided, the video coding method comprising: acquiring feature information of a target coding unit, feature information of a parent coding unit of the target coding unit, and feature information of adjacent coding units in a video frame to be encoded; determining a prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units; and encoding the target coding unit based on the determined prediction mode, wherein the target coding unit is partitioned from the parent coding unit, the adjacent coding units are coding units in the video frame to be encoded that are adjacent to the target coding unit, and the prediction mode is intra-frame prediction or inter-frame prediction.

[0007] Optionally, the step of determining the prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units includes: inputting the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units into a preset neural network model to obtain estimated prediction mode-related information of the target coding unit; and determining the prediction mode of the target coding unit based on the estimated prediction mode-related information of the target coding unit.

[0008] Optionally, the prediction mode related information is a first probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and / or a second probability that the best candidate prediction mode of the target coding unit is inter-frame prediction, wherein the best candidate prediction mode refers to the prediction mode with the lowest rate-distortion cost among the predetermined candidate prediction modes.

[0009] Optionally, the step of determining the prediction mode of the target coding unit based on the estimated prediction mode information of the target coding unit includes: comparing one of the first probability and the second probability with a preset probability threshold; when the comparison result indicates that the first probability is greater than the probability threshold, determining the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the probability threshold, determining the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; and when the comparison result indicates that the first probability is equal to the probability threshold, calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

[0010] Optionally, the step of determining the prediction mode of the target coding unit based on the estimated prediction mode information of the target coding unit includes: comparing one of the first probability and the second probability with a preset first probability threshold and a second probability threshold, wherein the first probability threshold is less than the second probability threshold; when the comparison result indicates that the first probability is greater than the second probability threshold, determining the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the first probability threshold, determining the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, determining the prediction mode of the target coding unit by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

[0011] Optionally, the feature information of the target coding unit includes at least one of the following: a first feature related to the division of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

[0012] Optionally, the first feature includes the size of the target coding unit and the partitioning depth of the target coding unit; the second feature includes the statistical value and quantization parameter of the prediction residual of the target coding unit in the first candidate prediction mode; the third feature includes the texture information of the target coding unit; the fourth feature includes the coding level of the video frame; and the fifth feature includes the rate-distortion cost of the target coding unit in the second candidate prediction mode.

[0013] Optionally, the feature information of the parent coding unit includes at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in a third candidate prediction mode. The third candidate prediction mode is all candidate prediction modes under the prediction mode corresponding to the best candidate prediction mode of the parent coding unit's previous level coding unit. The parent coding unit is partitioned from the previous level coding unit. The feature information of the adjacent coding unit includes: features related to the partitioning of the adjacent coding unit, features related to the best candidate prediction mode of the adjacent coding unit, and inter-frame features of the video frame related to the adjacent coding unit. The best candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among multiple predetermined candidate prediction modes.

[0014] Optionally, the features related to the partitioning of the parent coding unit include the size of the parent coding unit; the features related to the best candidate prediction mode of the parent coding unit include the prediction mode of the best candidate prediction mode of the parent coding unit; the inter-frame features of the video frame related to the parent coding unit include the motion vector of the parent coding unit; the features related to the partitioning of the adjacent coding units include the size of the adjacent coding units; the features related to the best candidate prediction mode of the adjacent coding units include the prediction mode of the best candidate prediction mode of the adjacent coding units; and the inter-frame features of the video frame related to the adjacent coding units include the motion vector of the adjacent coding units.

[0015] According to a second aspect of the present disclosure, a video encoding apparatus is provided, the video encoding apparatus comprising: an acquisition unit configured to acquire feature information of a target coding unit, feature information of a parent coding unit of the target coding unit, and feature information of adjacent coding units in a video frame to be encoded; a determination unit configured to determine a prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units; and an encoding unit configured to encode the target coding unit based on the determined prediction mode, wherein the target coding unit is partitioned from the parent coding unit, the adjacent coding units are coding units in the video frame to be encoded that are adjacent to the target coding unit, and the prediction mode is intra-frame prediction or inter-frame prediction.

[0016] Optionally, the determining unit is further configured to: input the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units into a preset neural network model to obtain estimated prediction mode related information of the target coding unit; and determine the prediction mode of the target coding unit based on the estimated prediction mode related information of the target coding unit.

[0017] Optionally, the prediction mode related information is a first probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and / or a second probability that the best candidate prediction mode of the target coding unit is inter-frame prediction, wherein the best candidate prediction mode refers to the prediction mode with the lowest rate-distortion cost among the predetermined candidate prediction modes.

[0018] Optionally, the determining unit is further configured to: compare one of the first probability and the second probability with a preset probability threshold; when the comparison result indicates that the first probability is greater than the probability threshold, determine the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the probability threshold, determine the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is equal to the probability threshold, determine the prediction mode of the target coding unit by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

[0019] Optionally, the determining unit is further configured to: compare one of the first probability and the second probability with a preset first probability threshold and a second probability threshold, wherein the first probability threshold is less than the second probability threshold; when the comparison result indicates that the first probability is greater than the second probability threshold, determine the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the first probability threshold, determine the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, determine the prediction mode of the target coding unit by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

[0020] Optionally, the feature information of the target coding unit includes at least one of the following: a first feature related to the division of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

[0021] Optionally, the first feature includes the size of the target coding unit and the partitioning depth of the target coding unit; the second feature includes the statistical value and quantization parameter of the prediction residual of the target coding unit in the first candidate prediction mode; the third feature includes the texture information of the target coding unit; the fourth feature includes the coding level of the video frame; and the fifth feature includes the rate-distortion cost of the target coding unit in the second candidate prediction mode.

[0022] Optionally, the feature information of the parent coding unit includes at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in a third candidate prediction mode. The third candidate prediction mode is all candidate prediction modes under the prediction mode corresponding to the best candidate prediction mode of the parent coding unit's previous level coding unit. The parent coding unit is partitioned from the previous level coding unit. The feature information of the adjacent coding unit includes: features related to the partitioning of the adjacent coding unit, features related to the best candidate prediction mode of the adjacent coding unit, and inter-frame features of the video frame related to the adjacent coding unit. The best candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among multiple predetermined candidate prediction modes.

[0023] Optionally, the features related to the partitioning of the parent coding unit include the size of the parent coding unit; the features related to the best candidate prediction mode of the parent coding unit include the prediction mode of the best candidate prediction mode of the parent coding unit; the inter-frame features of the video frame related to the parent coding unit include the motion vector of the parent coding unit; the features related to the partitioning of the adjacent coding units include the size of the adjacent coding units; the features related to the best candidate prediction mode of the adjacent coding units include the prediction mode of the best candidate prediction mode of the adjacent coding units; and the inter-frame features of the video frame related to the adjacent coding units include the motion vector of the adjacent coding units.

[0024] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform the video encoding method described herein.

[0025] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the video encoding method according to the present disclosure.

[0026] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer instructions that, when executed by a processor, implement the video encoding method according to the present disclosure.

[0027] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0028] The prediction mode of the unit to be coded can be determined, thereby allowing all candidate prediction modes in either inter-frame prediction or intra-frame prediction to be skipped, saving computational complexity in mode selection for inter-frame coded frames and improving coding efficiency.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0031] Figure 1 This is a flowchart illustrating a video encoding method according to an exemplary embodiment.

[0032] Figure 2 This is a schematic diagram illustrating the division of coding units in a video coding method according to an exemplary embodiment.

[0033] Figure 3 This is a schematic diagram illustrating the encoding sequence in a video encoding method according to an exemplary embodiment.

[0034] Figure 4 This is a schematic diagram of a neural network for a video coding method according to an exemplary embodiment.

[0035] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment.

[0036] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0038] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0039] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0040] It should also be noted that the video encoding method and apparatus according to the exemplary embodiments of this disclosure can be applied to devices used for computing services, including but not limited to personal computers, tablet computers, smartphones, etc.

[0041] In video coding, a hybrid coding method based on coding blocks can be adopted. Specifically, each frame of the video can be divided into multiple coding blocks, and then intra-frame or inter-frame prediction can be performed on the basis of coding blocks. Then, the prediction residuals are transformed and quantized. Finally, the prediction mode information and the quantized prediction residuals are entropy encoded to obtain the coded bitstream.

[0042] Generally, to adapt to diverse video content and features, most video codecs support using intra-prediction modes for the coding block in intra-coded frames. However, for inter-coded frames, it is necessary to iterate through prediction modes for the coding block. Rate-Distortion Optimization (RDO) can then be used to calculate the prediction mode with the lowest rate-distortion cost from a set of preset prediction modes, which can then be used as the coding mode for that coding block.

[0043] In this approach, the traversal process consumes a large amount of computational resources and is extremely time-consuming. In some cases, the computation time spent on prediction mode selection in inter-frame coded frames can even exceed 20% of the total coding time of the current frame. Such a traversal process makes it difficult to apply complex and diverse prediction modes in practice.

[0044] To address the resource-intensive and time-consuming nature of the aforementioned method of traversing prediction modes, one approach is to calculate the motion and detail features of the block to be encoded and select inter-frame prediction modes that meet pre-defined conditions for RDO (Restricted Decision Optimization). This effectively skips unnecessary candidate modes from a variety of inter-frame prediction modes, reducing the encoder's computational complexity and making it suitable for practical applications. However, this method only considers inter-frame prediction modes and does not account for the existence of intra-frame prediction modes in the blocks to be encoded within the inter-frame coded frames. Furthermore, traditional methods for calculating motion and detail features do not accurately fit the characteristics of different blocks to be encoded, making it difficult to accurately remove candidate modes that do not meet the conditions from various inter-frame prediction modes.

[0045] On the other hand, multi-layer convolutional neural networks can be used to obtain the prediction mode of the current block to be encoded, thereby further reducing the traversal of candidate intra-frame prediction modes and lowering the computational complexity of intra-frame prediction. However, this method only considers intra-frame prediction modes and does not take into account inter-frame prediction scenarios. In addition, the more complex convolutional neural network will also introduce greater computational complexity.

[0046] In view of the above problems, a video encoding method, video encoding apparatus, electronic device, computer-readable storage medium, and computer program product according to exemplary embodiments of the present disclosure will be provided below with reference to the accompanying drawings.

[0047] Figure 1 This is a flowchart illustrating a video encoding method according to an exemplary embodiment, such as... Figure 1 As shown, the video encoding method may include the following steps:

[0048] In step S10, the feature information of the target coding unit, the feature information of the parent coding unit of the target coding unit, and the feature information of the adjacent coding units in the video frame to be encoded can be obtained.

[0049] To facilitate understanding of this step, the process of dividing the coding unit is described below. Specifically, in the video encoding process, video frame images can first be captured from the video to be encoded, and then the video frames to be encoded are divided into multiple coding units. Figure 2 An example schematic diagram of coding unit partitioning is shown, such as Figure 2 As shown, firstly, the video frames to be encoded can be initially divided into segments of a predetermined size. This initial division can be equal, meaning each encoded unit after division has the same shape and size. For each encoded unit obtained after the initial division, depth division can be performed. This depth division can be based on predetermined rules or models to further divide the encoded units into secondary, tertiary, or other subdivisions. For example, the size of each encoded unit after the initial division can be 64*64 pixels. Based on the pixel characteristics of the encoded unit, the division depth can be determined according to predetermined rules or models to further divide the encoded unit. For instance, with a division depth of 2, the encoded units obtained from the initial division can be further divided into four 32*32 pixel encoded units. With a division depth of 3, the encoded units obtained from the initial division can be further divided into secondary and tertiary divisions. That is, the encoded units obtained from the initial division are divided into four 32*32 pixel encoded units, and then each of the secondary divisions is further divided into four 16*16 pixel encoded units.

[0050] Here, a suitable segmentation depth can be determined based on the features of pixels in the initial coding unit. The segmentation depth can be a balance between the accuracy of the coding unit prediction and the computational cost. That is, too fine a segmentation can improve accuracy, but it will increase the computational cost; too coarse a segmentation can reduce the computational cost, but it will reduce the accuracy. The determined segmentation depth can be a value obtained by balancing the two. Here, the predetermined rule or model used to determine the segmentation depth can be arbitrary, such as existing methods for segmenting video coding blocks. This disclosure does not impose any particular restrictions on it.

[0051] The above describes the process of dividing coding units. After determining the division depth, the final divided coding units can be obtained. It should be noted that the shape of the final divided coding units can be arbitrary, such as squares, rectangles, etc. Different coding units can have different division depths, therefore, different coding units can have different sizes. For example, ... Figure 2 As shown, adjacent coding units can have different sizes. Figure 2In this context, the number on each coding unit can represent the partitioning depth of that coding unit.

[0052] In step S10, the target coding unit can be a coding unit obtained from the final division of the video frame, for example... Figure 2 The coding units A, B, C, and D in the code can have a defined partition depth. The target coding unit can be partitioned from the parent coding unit. That is, the partition depth of the parent coding unit can be one level less than the partition depth of the target coding unit. Taking the example above, if the target coding unit is 16*16 pixels in size and its partition depth is 3, then its parent coding unit can be 32*32 pixels in size and its partition depth is 2. The parent coding unit can be partitioned from the initial coding unit with a pixel size of 64*64.

[0053] by Figure 2 For example, the initial division of the video frame into coding unit A1 (with a pixel size of 64*64) is further divided into four coding units (with pixel sizes of 32*32), including coding units B2 and A2. Coding unit B2 is then divided into four coding units (with pixel sizes of 16*16), including coding units B and C. Therefore, the parent coding unit of coding units B and C is coding unit B2 obtained from the second division. Coding unit A2, however, is not divided into three parts. In the final grid, coding unit A is coding unit A2 obtained from the second division, with a pixel size of 32*32, and its parent coding unit remains coding unit A1.

[0054] Adjacent coding units are coding units in a video frame that are adjacent to the target coding unit. Here, since the final partitioning depth and size of the resulting coding units can be different, the size of the target coding unit and its adjacent coding units can also be different. Therefore, on any one of the four edges of the target coding unit, there can be one or more coding units adjacent to it, such as... Figure 2 As shown, taking target coding unit A as an example, the coding units adjacent to the left of coding unit A are two coding units B and C, and the coding unit adjacent to the top of coding unit A is only one coding unit D.

[0055] In step S20, the prediction mode of the target coding unit can be determined based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units.

[0056] In this step, since the target coding unit is divided from the parent coding unit and is related to the pixel features of its neighboring coding units, the prediction mode of the target coding unit can be determined based on the feature information of the target coding unit, its parent coding unit, and its neighboring coding units, so as to encode the target coding unit using the specific mode under the determined prediction mode.

[0057] Here, the prediction mode can be intra-frame prediction or inter-frame prediction. Each of the intra-frame and inter-frame prediction modes can include multiple candidate prediction modes, which can be any coded prediction mode, such as the existing Merge mode used for inter-frame prediction.

[0058] As an example, in step S20, the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units can be input into a preset neural network model to obtain the estimated prediction mode information of the target coding unit; based on the estimated prediction mode information of the target coding unit, the prediction mode of the target coding unit is determined.

[0059] Specifically, the prediction mode information of the target coding unit estimated by the neural network can indicate the first probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and / or the second probability that the best candidate prediction mode of the target coding unit is inter-frame prediction. Here, the best candidate prediction mode refers to the prediction mode with the minimum rate-distortion cost among the predetermined candidate prediction modes. The predetermined candidate prediction modes may include inter-frame prediction modes and intra-frame prediction modes.

[0060] Here, the neural network model can be a pre-trained neural network used to predict the prediction mode of the target coding unit. For example, training samples can be obtained, which may include sample coding units and corresponding coding labels. The coding labels indicate whether the best candidate prediction mode for the sample coding unit is an inter-frame prediction mode or an intra-frame prediction mode. The sample coding units can be input into the initial neural network model to output a probability value for each sample coding unit as an inter-frame prediction mode and / or an intra-frame prediction mode. The probability values ​​output by the model can be compared with the pre-labeled coding labels corresponding to the sample coding units, thereby training the neural network model. In this way, through the training of the neural network, the probability values ​​of both intra-frame and inter-frame prediction modes can be better fitted and predicted, and the mode selection can be accelerated based on these probability values.

[0061] According to exemplary embodiments of this disclosure, a lightweight neural network can be used as the aforementioned neural network model. For example, such as Figure 4As shown, a fully connected lightweight neural network with only 3 layers can be used to predict whether the target coding unit needs to skip the traversal of candidate prediction modes in the entire class of intra-frame prediction or inter-frame prediction.

[0062] Here, given the probabilities of intra-frame prediction and / or inter-frame prediction, the prediction mode can be determined in the following way:

[0063] When only the probability of intra-frame prediction or inter-frame prediction is calculated, for example, when the output node of the above neural network model is only one, the prediction mode can be determined by presetting a probability threshold.

[0064] In one example, the step of determining the prediction mode of the target coding unit based on the estimated prediction mode information of the target coding unit may include: comparing one of a first probability and a second probability with a preset probability threshold; when the comparison result indicates that the first probability is greater than the probability threshold, determining the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the probability threshold, determining the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; and when the comparison result indicates that the first probability is equal to the probability threshold, determining the prediction mode of the target coding unit by calculating the rate-distortion cost by traversing all candidate prediction modes for intra-frame prediction and all candidate prediction modes for inter-frame prediction for the target coding unit.

[0065] Specifically, a probability threshold T can be set, for example, 0.5. When the probability of intra-frame prediction (or inter-frame prediction) is greater than the probability threshold, intra-frame prediction (or inter-frame prediction) can be selected as the prediction mode, and the traversal of all inter-frame prediction (or intra-frame prediction) modes is skipped. When the probability of intra-frame prediction (or inter-frame prediction) is less than the probability threshold, inter-frame prediction (or intra-frame prediction) can be selected as the prediction mode, and the traversal of all intra-frame prediction (or inter-frame prediction) modes is skipped. When the probability of intra-frame prediction (or inter-frame prediction) is equal to the probability threshold, no selection needs to be made, and the process of traversing all candidate prediction modes is still performed.

[0066] In another example, the step of determining the prediction mode of the target coding unit based on the prediction mode-related information of the estimated target coding unit may include: comparing one of the first probability and the second probability with a preset first probability threshold and a second probability threshold, where the first probability threshold is less than the second probability threshold; when the comparison result indicates that the one is greater than the second probability threshold, determining the prediction mode corresponding to the one as the prediction mode of the target coding unit; when the comparison result indicates that the one is less than the first probability threshold, determining the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the one is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, determining the prediction mode of the target coding unit by calculating the rate-distortion cost for all candidate prediction modes of intra-frame prediction and all candidate prediction modes of inter-frame prediction for the target coding unit.

[0067] Specifically, a first probability threshold T1 and a second probability threshold T2 (T1 < T2) can be set to determine whether to skip the traversal of prediction modes in the entire large category of intra-frame or inter-frame. For example, T1 can be 0.2 and T2 can be 0.8. When the probability of intra-frame prediction (or inter-frame prediction) is greater than the second probability threshold T2, the traversal of all inter-frame prediction (or intra-frame prediction) modes is skipped; when the probability of intra-frame prediction (or inter-frame prediction) is less than the first probability threshold T1, the traversal of all intra-frame prediction (or inter-frame prediction) modes is skipped; when the probability of intra-frame prediction (or inter-frame prediction) is less than or equal to the second probability threshold T2 and greater than or equal to the first probability threshold T1, no selection is made and the process of traversing all candidate prediction modes is still executed. In this way, a confidence interval can be set for the probability of mode prediction to ensure the accuracy of prediction mode selection.

[0068] In the case of calculating the probabilities of both intra-frame prediction and inter-frame prediction, for example, the output nodes of the above neural network model are two. In one example, the probability thresholds can be set for both probabilities by the above-mentioned method of presetting probability thresholds to determine the prediction mode; in another example, the sizes of the probabilities of intra-frame prediction and inter-frame prediction can be compared, and the mode with a larger probability can be selected while skipping the traversal of the mode with a smaller probability.

[0069] Thus, in the video coding method according to this disclosure, the prediction mode can be selected based on a lightweight neural network, thereby allowing skipping of candidate prediction modes for an entire class. For example, the traversal of all candidate prediction modes under intra-frame prediction or inter-frame prediction can be skipped. Compared with the existing method that does not distinguish prediction modes and traverses all candidate prediction modes of all types, the traversal of all candidate prediction modes under a model can be eliminated simply by adding the prediction process of a lightweight neural network, which greatly reduces the workload of the traversal process, simplifies the computational complexity, and improves coding efficiency.

[0070] However, it should be noted that in the video coding method according to this disclosure, the way to determine the prediction mode based on the feature information of the target coding unit, the parent coding unit and the adjacent coding units is not limited to using a neural network, but can also use existing statistical methods to calculate the probability.

[0071] In step S30, the target coding unit can be encoded based on the determined prediction pattern.

[0072] According to an exemplary embodiment of this disclosure, since the information of the target coding unit, the parent coding unit, and the adjacent coding units are taken into account, the prediction mode of the target coding unit can be determined in step S20. Therefore, in step S30, prediction modes other than the determined prediction mode can be skipped. For example, if the prediction mode of the target coding unit is determined to be intra-frame prediction, then during the process of traversing the candidate prediction modes, all inter-frame prediction candidate prediction modes can be skipped, and only the intra-frame prediction candidate prediction modes can be traversed. For example, the rate-distortion cost of each intra-frame prediction candidate prediction mode can be compared.

[0073] It should be noted that this paper aims to identify prediction models, and does not impose any special restrictions on the specific candidate prediction models included in each prediction model.

[0074] As mentioned above, by considering the feature information of the target coding unit, the parent coding unit, and the adjacent coding units, the prediction mode of the target coding unit can be determined. Here, the feature information of the target coding unit, the parent coding unit, and the adjacent coding units refers to information that can characterize the characteristics of the corresponding coding units. By statistically analyzing the characteristics of the three, a suitable prediction mode for the target coding unit can be determined. In this way, the mode selection of the unit to be coded (e.g., the target coding unit) in the inter-frame coded frame can be predicted and accelerated. The probability of the unit to be coded selecting the intra-frame prediction mode and the inter-frame prediction mode can be directly predicted, thus skipping the traversal of candidate prediction modes in the entire intra-frame or inter-frame mode, saving the computational complexity of mode selection in the inter-frame coded frame.

[0075] The following will provide specific examples of the feature information of the target coding unit, the parent coding unit, and the adjacent coding units.

[0076] The feature information of the target coding unit may include at least one of the following: a first feature related to the division of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

[0077] Specifically, the first feature related to the partitioning of the target coding unit can be obtained based on partitioning information from dividing the video frames. For example, the first feature may include the size of the target coding unit and the partitioning depth of the target coding unit. Figure 2 Taking coding unit B as the target coding unit as an example, its size can be 16*16 pixels and its division depth can be 3.

[0078] The second feature associated with the prediction result of the target coding unit under the predetermined first candidate prediction mode may include the statistical value and quantization parameter of the prediction residual of the target coding unit under the predetermined first candidate prediction mode.

[0079] Here, the predetermined first candidate prediction mode can be any prediction mode, which can be an intra-frame prediction mode or an inter-frame prediction mode. The target coding unit can be encoded and predicted according to the predetermined first candidate prediction mode based on the existing encoder to obtain the encoding prediction result. For example, the quantization parameter (QP) and prediction residual can be obtained, and relevant statistical values, such as the mean of the prediction residual, can be extracted from the prediction residual.

[0080] Here, quantization parameters reflect the spatial detail compression of coding units. Specifically, quantization parameters are inversely proportional to bitrate and bit rate. Smaller quantization parameters preserve more detail in the coding units, resulting in a higher bitrate; larger quantization parameters preserve less detail, resulting in a lower bitrate, increased image distortion, and decreased image quality. Quantization parameters are determined by video resolution, input frame rate, and bitrate, and reflect the quality performance of the encoder.

[0081] Furthermore, the first candidate prediction mode described herein can be one or more. This disclosure does not impose specific restrictions on the specific type, form, and number of the first candidate prediction modes, nor does it impose specific restrictions on the method of obtaining the prediction result of the target coding unit under the first candidate prediction mode.

[0082] A third feature related to the pixels within the target coding unit may include, for example, the texture information of the target coding unit, which may include the mean and variance of the pixel values ​​of the target coding unit and the gradient values ​​of the pixel values ​​of the target coding unit in the horizontal / vertical direction (i.e., the width / height direction of the target coding unit).

[0083] A fourth feature associated with a video frame may include, for example, the coding level of the video frame, which can be determined when capturing video frame images from the video to be encoded. Specifically, all I-frames and P-frames have a coding level of 0, while the coding level of B-frames can be determined based on the coding order determined when capturing video frame images from the video to be encoded.

[0084] Figure 3 This illustrates an example of the encoding order determined when capturing video frame images from a video to be encoded. For example... Figure 3 As shown, the arrows point in the direction of the reference frames. For example, frame 8 is a B-frame, referencing the preceding frames 0 and 16. Frame 0 is an I-frame, and frame 16 is a P-frame. Therefore, frame 8 is a level 0 frame within the B-frame family. Figure 3 As can be seen, frame 4 is also a B-frame, which references the preceding frame 0 and frame 8. Frame 0 is an I-frame and frame 8 is a B-frame. Since it references the preceding frame which is a level 0 B-frame, frame 4 is a level 1 B-frame. Similarly, frame 2 references frame 4 which is a level 1 B-frame, so frame 2 is a level 2 B-frame.

[0085] It should be noted that the process of determining the encoding order when capturing video frame images from the video to be encoded can be implemented by any existing encoding method. This disclosure does not impose any particular limitation on this, as long as the encoding level of the video frame in which the target encoding unit is located can be determined based on the known encoding order.

[0086] The fifth feature associated with the prediction result of the target coding unit in the predetermined second candidate prediction mode may include, for example, the rate-distortion cost of the target coding unit in the predetermined second candidate prediction mode.

[0087] Here, the predetermined second candidate prediction mode can be any prediction mode, which can be an intra-frame prediction mode or an inter-frame prediction mode. The second candidate prediction mode can be the same as or different from the first candidate prediction mode described above. The target coding unit can be encoded and predicted according to the predetermined second candidate prediction mode based on the existing encoder to obtain the rate-distortion cost.

[0088] As an example, rate-distortion cost can be determined by the following expression:

[0089] Rate distortion cost = D + lambda × R

[0090] Where D represents encoding distortion, R represents the encoded bitrate, and lambda is a preset coefficient.

[0091] Furthermore, the second candidate prediction mode described herein can be one or more. This disclosure does not impose specific restrictions on the specific type, form, and number of the second candidate prediction modes, nor does it impose specific restrictions on the method of obtaining the rate-distortion cost of the target coding unit under the second candidate prediction mode.

[0092] The feature information of the parent coding unit may include at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in the third candidate prediction mode.

[0093] Specifically, the features associated with the division of the parent coding unit can be obtained based on the division information of the video frames, such as the size of the parent coding unit.

[0094] Features associated with the best candidate prediction mode of the parent coding unit may include the prediction mode of the best candidate prediction mode of the parent coding unit. Here, the best candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among a plurality of predetermined candidate prediction modes. The plurality of predetermined candidate prediction modes mentioned herein may include any prediction modes, which may include intra-frame prediction modes and / or inter-frame prediction modes.

[0095] For the parent coding unit, coding prediction can be performed based on the existing encoder according to multiple predetermined candidate prediction modes to obtain rate-distortion costs, thereby determining the best candidate prediction mode. In this way, it can be determined whether the prediction mode of the best candidate prediction mode is intra-frame prediction or inter-frame prediction.

[0096] Inter-frame features associated with a parent coding unit in a video frame include the parent coding unit's motion vector (MV) value. The parent coding unit's motion vector value refers to the motion vector between the parent coding unit and its matching coding unit in a reference frame of the video frame. Specifically, there is a certain correspondence between the coding units in the video frame containing the parent coding unit and the reference frame. Existing inter-frame coding methods can be used to find the best-matching coding unit between two frames. For example, the closest matching coding unit to the parent coding unit can be found in the reference frame of the video frame containing the parent coding unit. Therefore, the motion vector between the parent coding unit and the matching coding unit can be calculated based on their positions in the video frame. Furthermore, the calculation of the motion vector can be implemented using any existing coding method, and this disclosure does not impose any particular limitations on this.

[0097] The aforementioned third candidate prediction mode can be any of the candidate prediction modes corresponding to the best candidate prediction mode in the next higher level coding unit of the parent coding unit. Here, the parent coding unit is derived from that next higher level coding unit. Figure 2 For example, the target coding unit is coding unit B, the parent coding unit is coding unit B2, and the coding unit above the parent coding unit is coding unit A1.

[0098] Specifically, the optimal candidate prediction mode of the parent coding unit can be determined using the existing encoder. This allows us to determine the prediction mode corresponding to the optimal candidate prediction mode, i.e., whether it is intra-frame prediction or inter-frame prediction. The parent coding unit can then be traversed and encoded according to all possible prediction modes within the optimal candidate prediction mode of the parent coding unit to obtain the rate-distortion cost across all possible prediction modes. For example, if the optimal candidate prediction mode for coding unit A1 is determined to be intra-frame prediction, the rate-distortion cost of the parent coding unit across all intra-frame candidate prediction modes under the intra-frame prediction type can be calculated as the feature information of the parent coding unit.

[0099] The feature information of adjacent coding units may include: features related to the division of adjacent coding units, features related to the best candidate prediction mode of adjacent coding units, and inter-frame features of video frames related to adjacent coding units.

[0100] Specifically, the features associated with the partitioning of adjacent coding units can be obtained based on the partitioning information of the video frames, which may include, for example, the size of the adjacent coding units.

[0101] Features associated with the best candidate prediction mode of neighboring coding units may include the prediction mode of the best candidate prediction mode of neighboring coding units. Here, the best candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among a plurality of predetermined candidate prediction modes. The plurality of predetermined candidate prediction modes mentioned herein may include any prediction modes, which may include intra-frame prediction modes and / or inter-frame prediction modes.

[0102] For adjacent coding units, coding prediction can be performed based on the existing encoder according to multiple predetermined candidate prediction modes to obtain rate-distortion costs, thereby determining the best candidate prediction mode. In this way, it can be determined whether the prediction mode of the best candidate prediction mode is intra-frame prediction or inter-frame prediction.

[0103] Inter-frame features related to adjacent coding units in a video frame can include the motion vector values ​​of adjacent coding units. Similar to the meaning of the motion vector value of a parent coding unit, the motion vector value of an adjacent coding unit refers to the motion vector between the adjacent coding unit and its matching coding unit in the reference frame of the video frame. Specifically, there is a certain correspondence between the coding units in the video frame containing the adjacent coding unit and the reference frame. Existing inter-frame coding methods can be used to find the best-matching coding unit between two frames. For example, the closest matching coding unit can be found in the reference frame of the video frame containing the adjacent coding unit. Therefore, the motion vector between the adjacent coding unit and the matching coding unit can be calculated based on their positions in the video frame. Furthermore, the calculation of motion vectors can be implemented using any existing coding method, and this disclosure does not impose any particular limitations on this.

[0104] The feature information of the target coding unit, the parent coding unit, and the adjacent coding unit has been described above by example. However, their respective feature information is not limited to the specific items described above.

[0105] It should be noted that when using the aforementioned lightweight neural network to determine the prediction mode, in one case, the feature information of the target coding unit includes the size of the target coding unit, the partitioning depth of the target coding unit, the statistical value and quantization parameters of the prediction residual of the target coding unit in the predetermined first candidate prediction mode, the texture information of the target coding unit, and the coding level of the video frame. The feature information of the parent coding unit includes the size of the parent coding unit, the prediction mode of the best candidate prediction mode of the parent coding unit, the motion vector of the parent coding unit, and the rate-distortion cost of the parent coding unit in the third candidate prediction mode. Furthermore, the feature information of adjacent coding units includes the size of the adjacent coding unit, the prediction mode of the best candidate prediction mode of the adjacent coding unit, and the motion vector of the adjacent coding unit. In this case, the accuracy of the neural network prediction results can be improved while ensuring the computational speed of the neural network, thereby determining the prediction mode more efficiently.

[0106] In another case, based on the above, the feature information of the target coding unit can also include the rate-distortion cost of the target coding unit in a predetermined second candidate prediction mode. In this way, the computational accuracy of the neural network model can be improved, and the coding loss under the same coding time can be further reduced in the entire video coding process.

[0107] In video coding, to reduce the computational complexity of the encoder and meet the real-time requirements of practical coding applications, rapid selection of prediction modes is one of the main ways to reduce coding time. As described above, the video coding method according to the exemplary embodiments of this disclosure, for the mode selection problem of intra-frame prediction and inter-frame prediction of the target coding unit in inter-frame coded frames, can select the intra-frame or inter-frame prediction mode of the target coding unit to be encoded, pre-removing some candidate prediction modes that are unnecessary to traverse. Thus, by simply utilizing CPU computing resources, the computational complexity of mode selection in inter-frame coded frames can be greatly reduced with minimal introduced coding loss, thereby improving coding speed and efficiency.

[0108] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment. (Refer to...) Figure 5 The video encoding device includes an acquisition unit 100, a determination unit 200, and an encoding unit 300.

[0109] The acquisition unit 100 can be configured to acquire feature information of the target coding unit, the feature information of the parent coding unit of the target coding unit, and the feature information of the adjacent coding units in the video frame to be encoded.

[0110] The determining unit 200 can be configured to determine the prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units.

[0111] The coding unit 300 can be configured to encode the target coding unit based on a determined prediction pattern.

[0112] Here, the target coding unit is divided from the parent coding unit, and the adjacent coding unit is the coding unit in the video frame to be encoded that is adjacent to the target coding unit. The prediction mode is intra-frame prediction or inter-frame prediction.

[0113] As an example, the determining unit 200 can also be configured to: input the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units into a preset neural network model to obtain the estimated prediction mode information of the target coding unit; and determine the prediction mode of the target coding unit based on the estimated prediction mode information of the target coding unit.

[0114] As an example, prediction mode related information can be the probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and / or the probability that the best candidate prediction mode of the target coding unit is inter-frame prediction. The best candidate prediction mode can refer to the prediction mode with the lowest rate-distortion cost among the predetermined candidate prediction modes.

[0115] As an example, the determining unit 200 can also be configured to: compare one of the first probability and the second probability with a preset probability threshold; when the comparison result indicates that the first probability is greater than the probability threshold, determine the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the probability threshold, determine the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is equal to the probability threshold, determine the prediction mode of the target coding unit by calculating the rate-distortion cost by traversing all candidate prediction modes of intra-frame prediction and all candidate prediction modes of inter-frame prediction for the target coding unit.

[0116] As an example, the determining unit 200 can also be configured to: compare one of a first probability and a second probability with a preset first probability threshold and a second probability threshold, wherein the first probability threshold is less than the second probability threshold; when the comparison result indicates that the first probability is greater than the second probability threshold, determine the prediction mode corresponding to the first probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than the first probability threshold, determine the prediction mode corresponding to the other of the first probability and the second probability as the prediction mode of the target coding unit; when the comparison result indicates that the first probability is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, determine the prediction mode of the target coding unit by calculating the rate-distortion cost for traversing all candidate prediction modes of intra-frame prediction and all candidate prediction modes of inter-frame prediction for the target coding unit.

[0117] As an example, the feature information of the target coding unit may include at least one of the following: a first feature related to the partitioning of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

[0118] In this example, the first feature may include the size and partitioning depth of the target coding unit; the second feature may include the statistics and quantization parameters of the prediction residual of the target coding unit in the first candidate prediction mode; the third feature may include the texture information of the target coding unit; the fourth feature may include the coding level of the video frame; and the fifth feature may include the rate-distortion cost of the target coding unit in the second candidate prediction mode.

[0119] As an example, the feature information of the parent coding unit may include at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in the third candidate prediction mode, wherein the third candidate prediction mode is all candidate prediction modes in the prediction mode corresponding to the best candidate prediction mode of the parent coding unit, and the parent coding unit is partitioned from the parent coding unit.

[0120] The feature information of adjacent coding units may include: features related to the division of adjacent coding units, features related to the best candidate prediction mode of adjacent coding units, and inter-frame features of video frames related to adjacent coding units. The best candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among multiple predetermined candidate prediction modes.

[0121] In this example, features related to the partitioning of the parent coding unit may include the size of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit may include the prediction mode of the best candidate prediction mode of the parent coding unit, and inter-frame features of the video frame related to the parent coding unit may include the motion vector of the parent coding unit.

[0122] Features associated with the partitioning of adjacent coding units may include the size of the adjacent coding units, features associated with the best candidate prediction modes of the adjacent coding units may include the prediction modes of the best candidate prediction modes of the adjacent coding units, and inter-frame features of video frames associated with adjacent coding units may include the motion vectors of the adjacent coding units.

[0123] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0124] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 6 As shown, the electronic device 10 includes a processor 101 and a memory 102 for storing processor-executable instructions. Here, when executed by the processor, the processor executes the video encoding method as described in the exemplary embodiments above.

[0125] As an example, electronic device 10 is not necessarily a single device, but can be a collection of any means or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 10 can also be part of an integrated control system or system manager, or can be configured to interface with electronic devices locally or remotely (e.g., via wireless transmission).

[0126] In electronic device 10, processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0127] The processor 101 can execute instructions or code stored in the memory 102, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transmission protocol.

[0128] The memory 102 may be integrated with the processor 101, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 102 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 102 and the processor 101 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 101 to read files stored in the memory 102.

[0129] In addition, the electronic device 10 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 10 may be interconnected via a bus and / or network.

[0130] In an exemplary embodiment, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the video encoding method as described in the exemplary embodiment above. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), flash memory, non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, or a BD-R... LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0131] In an exemplary embodiment, a computer program product may also be provided, which includes computer instructions that, when executed by a processor, implement the video encoding method as described in the exemplary embodiment above.

[0132] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0133] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video encoding method, characterized in that, The video encoding method includes: Obtain the feature information of the target coding unit, the feature information of the parent coding unit of the target coding unit, and the feature information of the adjacent coding units in the video frame to be encoded; Based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units, the prediction mode of the target coding unit is determined; The target coding unit is encoded based on a determined prediction pattern. The target coding unit is derived from the parent coding unit, the adjacent coding unit is the coding unit in the video frame to be encoded that is adjacent to the target coding unit, and the prediction mode is intra-frame prediction or inter-frame prediction. The step of determining the prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units includes: inputting the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units into a preset neural network model to obtain estimated prediction mode-related information of the target coding unit; and determining the prediction mode of the target coding unit based on the estimated prediction mode-related information of the target coding unit. The prediction mode related information includes the first probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and the second probability that the best candidate prediction mode of the target coding unit is inter-frame prediction. The best candidate prediction mode refers to the prediction mode with the lowest rate-distortion cost among the predetermined candidate prediction modes.

2. The video encoding method according to claim 1, characterized in that, The step of determining the prediction mode of the target coding unit based on the estimated prediction mode-related information of the target coding unit includes: Compare one of the first probability and the second probability with a preset probability threshold; When the comparison result indicates that one of them is greater than the probability threshold, the prediction mode corresponding to the one is determined as the prediction mode of the target coding unit; when the comparison result indicates that one is less than the probability threshold, the prediction mode corresponding to the other of the first probability and the second probability is determined as the prediction mode of the target coding unit; when the comparison result indicates that one is equal to the probability threshold, the prediction mode of the target coding unit is determined by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

3. The video encoding method according to claim 1, characterized in that, The step of determining the prediction mode of the target coding unit based on the estimated prediction mode-related information of the target coding unit includes: One of the first probability and the second probability is compared with a preset first probability threshold and a preset second probability threshold, wherein the first probability threshold is less than the second probability threshold; When the comparison result indicates that one of them is greater than the second probability threshold, the prediction mode corresponding to the first one is determined as the prediction mode of the target coding unit; when the comparison result indicates that one of them is less than the first probability threshold, the prediction mode corresponding to the other of the first probability and the second probability is determined as the prediction mode of the target coding unit; when the comparison result indicates that one of them is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, the prediction mode of the target coding unit is determined by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

4. The video encoding method according to claim 1, characterized in that, The feature information of the target coding unit includes at least one of the following: a first feature related to the division of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

5. The video encoding method according to claim 4, characterized in that, The first feature includes the size of the target coding unit and the partitioning depth of the target coding unit; the second feature includes the statistical value and quantization parameter of the prediction residual of the target coding unit in the first candidate prediction mode; the third feature includes the texture information of the target coding unit; the fourth feature includes the coding level of the video frame; and the fifth feature includes the rate-distortion cost of the target coding unit in the second candidate prediction mode.

6. The video encoding method according to claim 1, characterized in that, The feature information of the parent coding unit includes at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in the third candidate prediction mode. The third candidate prediction mode is all candidate prediction modes under the prediction mode corresponding to the best candidate prediction mode of the parent coding unit, where the parent coding unit is derived from the parent coding unit. The feature information of the adjacent coding units includes: features related to the division of the adjacent coding units, features related to the best candidate prediction mode of the adjacent coding units, and inter-frame features of the video frames related to the adjacent coding units. The optimal candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among multiple predetermined candidate prediction modes.

7. The video encoding method according to claim 6, characterized in that, The features related to the partitioning of the parent coding unit include the size of the parent coding unit; the features related to the best candidate prediction mode of the parent coding unit include the prediction mode of the best candidate prediction mode of the parent coding unit; and the inter-frame features of the video frame related to the parent coding unit include the motion vector of the parent coding unit. The features associated with the partitioning of the adjacent coding units include the size of the adjacent coding units, the features associated with the best candidate prediction mode of the adjacent coding units include the prediction mode of the best candidate prediction mode of the adjacent coding units, and the inter-frame features of the video frame associated with the adjacent coding units include the motion vectors of the adjacent coding units.

8. A video encoding device, characterized in that, The video encoding device includes: The acquisition unit is configured to acquire feature information of a target coding unit, feature information of the parent coding unit of the target coding unit, and feature information of adjacent coding units in the video frame to be encoded. The determining unit is configured to determine the prediction mode of the target coding unit based on the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units; The encoding unit is configured to encode the target encoding unit based on a determined prediction pattern. The target coding unit is derived from the parent coding unit, the adjacent coding unit is the coding unit in the video frame to be encoded that is adjacent to the target coding unit, and the prediction mode is intra-frame prediction or inter-frame prediction. The determining unit is further configured to: input the feature information of the target coding unit, the feature information of the parent coding unit, and the feature information of the adjacent coding units into a preset neural network model to obtain estimated prediction mode-related information of the target coding unit; and determine the prediction mode of the target coding unit based on the estimated prediction mode-related information of the target coding unit. The prediction mode related information includes a first probability that the best candidate prediction mode of the target coding unit is intra-frame prediction and a second probability that the best candidate prediction mode of the target coding unit is inter-frame prediction. The best candidate prediction mode refers to the prediction mode with the lowest rate-distortion cost among the predetermined candidate prediction modes.

9. The video encoding apparatus according to claim 8, characterized in that, The determining unit is further configured to: Compare one of the first probability and the second probability with a preset probability threshold; When the comparison result indicates that one of them is greater than the probability threshold, the prediction mode corresponding to the one is determined as the prediction mode of the target coding unit; when the comparison result indicates that one is less than the probability threshold, the prediction mode corresponding to the other of the first probability and the second probability is determined as the prediction mode of the target coding unit; when the comparison result indicates that one is equal to the probability threshold, the prediction mode of the target coding unit is determined by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

10. The video encoding apparatus according to claim 8, characterized in that, The determining unit is further configured to: One of the first probability and the second probability is compared with a preset first probability threshold and a preset second probability threshold, wherein the first probability threshold is less than the second probability threshold; When the comparison result indicates that one of them is greater than the second probability threshold, the prediction mode corresponding to the first one is determined as the prediction mode of the target coding unit; when the comparison result indicates that one of them is less than the first probability threshold, the prediction mode corresponding to the other of the first probability and the second probability is determined as the prediction mode of the target coding unit; when the comparison result indicates that one of them is less than or equal to the second probability threshold and greater than or equal to the first probability threshold, the prediction mode of the target coding unit is determined by calculating the rate-distortion cost for the target coding unit by traversing all candidate prediction modes of the intra-frame prediction and all candidate prediction modes of the inter-frame prediction.

11. The video encoding apparatus according to claim 8, characterized in that, The feature information of the target coding unit includes at least one of the following: a first feature related to the division of the target coding unit, a second feature related to the prediction result of the target coding unit in a predetermined first candidate prediction mode, a third feature related to the pixels within the target coding unit, a fourth feature related to the video frame, and a fifth feature related to the prediction result of the target coding unit in a predetermined second candidate prediction mode.

12. The video encoding apparatus according to claim 11, characterized in that, The first feature includes the size of the target coding unit and the partitioning depth of the target coding unit; the second feature includes the statistical value and quantization parameter of the prediction residual of the target coding unit in the first candidate prediction mode; the third feature includes the texture information of the target coding unit; the fourth feature includes the coding level of the video frame; and the fifth feature includes the rate-distortion cost of the target coding unit in the second candidate prediction mode.

13. The video encoding apparatus according to claim 8, characterized in that, The feature information of the parent coding unit includes at least one of the following: features related to the partitioning of the parent coding unit, features related to the best candidate prediction mode of the parent coding unit, inter-frame features of the video frame related to the parent coding unit, and rate-distortion cost of the parent coding unit in the third candidate prediction mode. The third candidate prediction mode is all candidate prediction modes under the prediction mode corresponding to the best candidate prediction mode of the parent coding unit, where the parent coding unit is derived from the parent coding unit. The feature information of the adjacent coding units includes: features related to the division of the adjacent coding units, features related to the best candidate prediction mode of the adjacent coding units, and inter-frame features of the video frames related to the adjacent coding units. The optimal candidate prediction mode refers to the candidate prediction mode with the lowest rate-distortion cost among multiple predetermined candidate prediction modes.

14. The video encoding apparatus according to claim 13, characterized in that, The features related to the partitioning of the parent coding unit include the size of the parent coding unit; the features related to the best candidate prediction mode of the parent coding unit include the prediction mode of the best candidate prediction mode of the parent coding unit; and the inter-frame features of the video frame related to the parent coding unit include the motion vector of the parent coding unit. The features associated with the partitioning of the adjacent coding units include the size of the adjacent coding units, the features associated with the best candidate prediction mode of the adjacent coding units include the prediction mode of the best candidate prediction mode of the adjacent coding units, and the inter-frame features of the video frame associated with the adjacent coding units include the motion vectors of the adjacent coding units.

15. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions. Wherein, when the processor executes the processor, it causes the processor to perform the video encoding method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the video encoding method according to any one of claims 1 to 7.

17. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the video encoding method according to any one of claims 1 to 7.

Images