Joint rate distortion optimization method, apparatus and device for VVC intra-frame coding

Abstract

The invention relates to the technical field of VVC coding, and provides a joint rate distortion optimization method, device and equipment for VVC intra-frame coding, and the method comprises the steps: obtaining a target attribute and a coding decision value of a current coding unit, and an optimal coding decision path group of a previous coding unit; determining a beam value of the current coding unit according to the target attribute; determining a coding decision path candidate set of the current coding unit according to the coding decision value and the optimal coding decision path group of the previous coding unit; according to a beam search algorithm, selecting coding decision paths with the number of beam values from the coding decision path candidate set to serve as an optimal coding decision path group of the current coding unit; and determining a final coding decision path according to the optimal coding decision path group of the last coding unit. According to the embodiment of the invention, the scalability adjustment of the calculation overhead of VVC coding can be realized, so that the method is suitable for different VVC intra-frame coding application scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of Versatile Video Coding (VVC), and in particular to a joint rate-distortion optimization method, device and apparatus for VVC intra-frame coding. Background Art

[0002] VVC is a new video coding standard jointly developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC). Compared with the previous generation High Efficiency Video Coding (HEVC), the VVC codec can reduce the size of video data by about 50% while ensuring the same video quality.

[0003] Rate-distortion optimization (RDO), that is, RDO is a coding mode decision, an optimization method to improve video compression performance. Its principle is to optimize the average peak signal-to-noise ratio (picture quality) and bit rate (the amount of data required for encoding) of the video at the same time in order to achieve an optimal balance point (that is, the balance between video bit rate and video quality).

[0004] However, existing rate-distortion optimization methods ignore the dependencies between coding units and can only achieve suboptimal coding compression performance. How to efficiently consider the dependencies between coding units while having good scalability of computational overhead and adapting to VVC intra-frame coding application scenarios under different computing resources, joint rate-distortion optimization methods in this regard still need to be developed. Summary of the invention

[0005] The purpose of the embodiments of this specification is to provide a joint rate-distortion optimization method, device and equipment for VVC intra-frame coding, so as to efficiently jointly optimize the decisions between different coding units and adapt to VVC intra-frame coding application scenarios under different computing resources at the same time.

[0006] To achieve the above objectives, on the one hand, an embodiment of this specification provides a joint rate-distortion optimization method for VVC intra-frame coding, including:

[0007] Obtain target attributes, coding decision values, and the optimal coding decision path group of the previous coding unit of the current coding unit;

[0008] Determining a beam value of the current coding unit according to the target attribute;

[0009] Determine a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit;

[0010] Selecting, from the coding decision path candidate set according to a beam search algorithm, coding decision paths having the same number of beam values ​​as an optimal coding decision path group for the current coding unit;

[0011] The final coding decision path is determined according to the optimal coding decision path group of the last coding unit.

[0012] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of this specification, the target attribute includes a size attribute of the coding unit.

[0013] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of this specification, determining the beam value of the current coding unit according to the target characteristic includes:

[0014] According to the formula Determining a beam value of the current coding unit;

[0015] Among them, b i is the beam value of the i-th coding unit, b is the default global beam value, W i is the width of the i-th coding unit, H i is the height of the i-th coding unit, b' i is the adjusted beam value, and b' i <b,S d is the size attribute hyperparameter value of the encoding unit.

[0016] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of the present specification, determining the coding decision path candidate set of the current coding unit according to the coding decision value and the optimal coding decision path group of the previous coding unit includes:

[0017] When the level of the previous coding unit is a coding tree unit level, a coding decision path candidate set of the current coding unit is determined according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

[0018] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of the present specification, determining the coding decision path candidate set of the current coding unit according to the coding decision value and the optimal coding decision path group of the previous coding unit includes:

[0019] When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a coding unit of the same level, the coding decision path candidate set of the current coding unit is determined according to the coding decision value and all optimal coding decision paths in the optimal coding decision path group of the previous coding unit.

[0020] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of the present specification, determining the coding decision path candidate set of the current coding unit according to the coding decision value and the optimal coding decision path group of the previous coding unit includes:

[0021] When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a sub-coding unit of a coding unit at the same level, a coding decision path candidate set of a first sub-coding unit of the current coding unit is determined according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

[0022] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of the present specification, selecting the coding decision paths of the number of beam values ​​from the coding decision path candidate set according to the beam search algorithm as the optimal coding decision path group for the current coding unit includes:

[0023] Selecting, from a set of coding decision path candidates corresponding to when the current coding unit adopts a non-division decision, a group of non-division optimal coding decision paths from the previous coding unit to the current coding unit with the number of beam values ​​by using a beam search algorithm;

[0024] Selecting, from the candidate set of coding decision paths corresponding to the current coding unit when the current coding unit adopts the partition decision, a group of optimal coding decision paths for partitioning from the previous coding unit to the current coding unit with the number of beam values ​​by using a beam search algorithm;

[0025] Determine the cumulative Lagrangian cost of each optimal coding decision path in the non-partitioned optimal coding decision path group and the partitioned optimal coding decision path group;

[0026] And according to the size of the accumulated Lagrangian cost, the optimal coding decision paths of the number of beam values ​​are selected as the optimal coding decision path group of the current coding unit.

[0027] In the joint rate-distortion optimization method for VVC intra-frame coding in the embodiment of this specification, determining the final coding decision path according to the optimal coding decision path group of the last coding unit includes:

[0028] The optimal coding decision path group of the last coding unit with the smallest cumulative Lagrangian cost is determined as the final coding decision path.

[0029] On the other hand, the embodiment of this specification also provides a joint rate-distortion optimization device for VVC intra-frame coding, including:

[0030] A parameter acquisition module is used to obtain the target attribute of the current coding unit, the coding decision value, and the optimal coding decision path group of the previous coding unit;

[0031] A beam value determination module, used to determine the beam value of the current coding unit according to the target attribute;

[0032] A candidate set determination module, used to determine a candidate set of coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit;

[0033] A path group selection module, configured to select, from the coding decision path candidate set according to a beam search algorithm, coding decision paths of the number of beam values ​​as the optimal coding decision path group for the current coding unit;

[0034] The decision path determination module is used to determine a final coding decision path according to the optimal coding decision path group of the last coding unit.

[0035] On the other hand, an embodiment of the present specification further provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0036] On the other hand, an embodiment of the present specification further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0037] On the other hand, an embodiment of the present specification further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor of a computer device, the computer program executes instructions of the above method.

[0038] It can be seen from the technical solutions provided in the above embodiments of this specification that, in the embodiments of this specification, before encoding each coding unit, the RDO of the current coding unit decision (division, prediction and transformation) can refer to the decision path of the number of beam values ​​of the previous coding unit, and can dynamically determine the beam value of the current coding unit according to the target attribute of the current coding unit. The dependency between the coding units will be introduced into the RDO in a low-complexity manner, thereby realizing the joint optimization of VVC intra-frame coding, improving the coding performance, and at the same time having good computational scalability, which is conducive to better adapting to VVC intra-frame coding scenarios under different computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0040] Figure 1 A schematic diagram of an application environment for joint rate-distortion optimization for VVC intra-frame coding in some embodiments of this specification is shown;

[0041] Figure 2 A flowchart of a joint rate-distortion optimization method for VVC intra-frame coding in some embodiments of this specification is shown;

[0042] Figure 3 A schematic diagram of coding unit division in an exemplary embodiment of this specification is shown;

[0043] Figure 4 Shows Figure 2 A processing flow chart of selecting an optimal coding decision path group according to a beam search algorithm in the method shown;

[0044] Figure 5 A schematic diagram of a joint rate-distortion optimization process for VVC intra-frame coding in an exemplary embodiment of this specification is shown;

[0045] Figure 6 A structural block diagram of a joint rate-distortion optimization device for VVC intra-frame coding in some embodiments of this specification is shown;

[0046] Figure 7 The structure block diagram of the computer device in some embodiments of the present specification is shown.

[0047] [Description of Reference Numerals]

[0048] 10. Client;

[0049] 20. Server;

[0050] 61. Parameter acquisition module;

[0051] 62. Beam value determination module;

[0052] 63. Candidate set determination module;

[0053] 64. Path group selection module;

[0054] 65. Decision path determination module;

[0055] 702. Computer equipment;

[0056] 704, processor;

[0057] 706. Memory;

[0058] 708, driving mechanism;

[0059] 710, input / output interface;

[0060] 712. Input devices;

[0061] 714. Output devices;

[0062] 716. Presentation equipment;

[0063] 718. Graphical user interface;

[0064] 720, network interface;

[0065] 722. Communication link;

[0066] 724. Communication bus. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0068] Joint RDO refers to the joint optimization of coding decisions (or decisions) between different coding units (Coding Unit, CU) to improve the performance of VVC intra-frame coding (Intra Coding). Among them, the coding decisions include various combinations of division (i.e. CU division), prediction and transformation allowed by VVC intra-frame coding. The existing joint RDO technology for previous standards cannot be applied to the huge parameter space of VVC intra-frame coding due to complexity; and the complexity cannot be dynamically scaled, making it difficult to adapt to different application scenarios (mainly referring to different computing power resource scenarios). The embodiments of this specification provide a new joint rate-distortion optimization solution for VVC intra-frame coding.

[0069] Figure 1 A schematic diagram of an application environment for joint rate-distortion optimization for VVC intra-frame coding in some embodiments of the present specification is shown in the figure; the application environment includes a client 10 and a server 20; the client 10 can provide an image frame to be encoded to the server 20; the image frame can be composed of multiple coding units; the server 20 can obtain the target attribute, coding decision value and optimal coding decision path group of the previous coding unit of the current coding unit; determine the beam value of the current coding unit according to the target attribute; determine the coding decision path candidate set of the current coding unit according to the coding decision value and the optimal coding decision path group of the previous coding unit; select the coding decision paths with the beam value number from the coding decision path candidate set according to the beam search algorithm as the optimal coding decision path group of the current coding unit; determine the final coding decision path according to the optimal coding decision path group of the last coding unit.

[0070] In some embodiments of the present specification, the client 10 may be a mobile terminal (i.e., a smart phone), a display, a desktop computer, a tablet computer, a laptop computer, a digital assistant, or a smart wearable device, etc. Among them, the smart wearable device may include a smart bracelet, a smart watch, smart glasses, or a smart helmet, etc. Of course, the client 10 is not limited to the above-mentioned electronic devices with a certain entity, and it may also be software running in the above-mentioned electronic devices.

[0071] In some embodiments of the present specification, the server 20 may be an electronic device with computing and network interaction functions; or may be software running in the electronic device and providing business logic for data processing and network interaction.

[0072] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by this specification. In actual application, there may be multiple clients 10 and multiple servers 20, and this specification does not impose any limitation.

[0073] The embodiment of this specification provides a joint rate-distortion optimization method for VVC intra-frame coding, which can be applied to the above-mentioned server side, refer to Figure 2 As shown, in some embodiments of the present specification, the joint rate-distortion optimization method for VVC intra-frame coding may include the following steps:

[0074] Step 201: Obtain target attributes, coding decision values, and an optimal coding decision path group of a previous coding unit of a current coding unit.

[0075] Step 202: Determine a beam value of the current coding unit according to the target attribute.

[0076] Step 203: Determine a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit.

[0077] Step 204: Select coding decision paths of the number of beam values ​​from the coding decision path candidate set according to a beam search algorithm as the optimal coding decision path group for the current coding unit.

[0078] Step 205: Determine a final coding decision path according to the optimal coding decision path group of the last coding unit.

[0079] In an embodiment of the present specification, before encoding each coding unit, the RDO of the current coding unit mode (division, prediction and transformation) can refer to the decision path of the number of beam values ​​of the previous coding unit, and can determine the beam value of the current coding unit according to the target attribute of the current coding unit. Therefore, the dependency between the coding units in an image frame can be introduced into the RDO in a low-complexity manner, and at the same time have good computational scalability, which is conducive to better adapting to VVC intra-frame coding scenarios under different computing resources and improving the compression performance of VVC intra-frame coding.

[0080] In some embodiments of the present specification, the target attribute may include a size attribute (such as height and width) of the coding unit; thus, by adjusting the beam value through the size attribute of the coding unit, the computational scalability of VVC intra-frame coding can be achieved. The height and width can be calculated in pixels. In some other embodiments of the present specification, the target attribute may also adopt other attributes of the coding unit that can characterize its characteristics.

[0081] In the embodiments of the present specification, the coding decision value refers to the value range in each stage of the coding decision. For example, in some embodiments of the present specification, the coding decision includes three stages: division, prediction and transformation. Division may include: horizontal binary tree (HBT) division, vertical binary tree (VBT) division, horizontal ternary tree (HTT) division, vertical ternary tree (VTT) division and quadtree (QT) division and other optional division decisions; for each sub-coding unit after division, according to the VVC standard, VVC intra-frame prediction may include sixty-seven conventional angle modes and multi-row pixel references, matrix intra-frame prediction and other optional prediction decisions; according to the VVC standard, VVC intra-frame transform may include DCT2, DST7 and DCT8 and other optional transformation decisions. It can be seen that the optional coding decision space is actually a Product space, where {D i} is the possible partition space of the current i-th coding unit, {P i,j} is the prediction space of the jth sub-coding unit of the i-th coding unit. Similarly, {T i,j} is the transformation space of the corresponding sub-coding unit.

[0082] It should be understood that, in the embodiments of the present specification, the current coding unit, the previous coding unit, the next coding unit, and all coding units are coding units in the same image frame.

[0083] For example, in Figure 3 In the schematic diagram of coding unit division shown in FIG. 1 , the image frame is divided into 8 coding units: CU1 to CU8. When dividing at the coding tree unit (CTU) level, the image frame is divided into four CTU-level coding units (such as Figure 3 As shown by the cross-line in the middle, CU5 is one of the CTU-level coding units. When dividing at the non-CTU level, a vertical binary tree is used to further divide the CTU-level coding unit on the upper left into two non-CTU-level coding units, CU1 and CU2 (as shown in FIG. Figure 3 For the CTU-level coding unit in the upper right corner, a horizontal binary tree is used to further divide it into two non-CTU-level coding units, CU3 and CU4 (as shown in the vertical dotted line in the upper left corner); for ... Figure 3 For the CTU-level coding unit at the lower left (i.e., CU5), no non-CTU-level division is performed; for the CTU-level coding unit at the lower right, a horizontal binary tree is first used to further divide it into two non-CTU-level coding units (i.e., Figure 3As shown by the horizontal dotted line in the lower right corner, CU6 is one of them), and then the non-CTU-level coding units below CU6 are further divided into two non-CTU-level coding units CU7 and CU8 using a vertical binary tree (as shown in FIG. Figure 3 vertical dotted line at the lower right).

[0084] like Figure 3 As shown, if the current coding unit is CU2, the previous coding unit is CU1, and the next coding unit is CU3; all coding units are CU1-CU8.

[0085] Generally, each coding unit in the same image frame determines its coding decision path according to the order of the coding units; before encoding the current coding unit (i.e., when determining the optimal coding decision path group for encoding the current coding unit), the decision of the optimal coding decision path group for the previous coding unit has been completed. Existing algorithms assume that the decisions between coding units are independent of each other and only retain one optimal coding decision path in the optimal coding decision path group. In view of the spatial correlation between adjacent pixels in the same image frame, the decisions between adjacent coding units in the same image frame are actually correlated; therefore, when determining the optimal coding decision path group for the current coding unit, the correlation can be introduced by retaining multiple optimal coding decision paths. Among them, the number of optimal coding decision paths contained in the optimal coding decision path group of different coding units can be different.

[0086] The beam search algorithm selects multiple alternatives for the input sequence at each time step based on conditional probability. The number of multiple alternatives depends on a parameter called beam width, which is the beam value. In the embodiment of this specification, the beam value of the current coding unit is used to characterize: the number of optimal coding decision paths retained after the current coding unit performs RDO (that is, the number of coding decision paths in the optimal coding decision path group of the current coding unit).

[0087] In some embodiments of the present specification, taking the target attribute as the height and width of the coding unit as an example, determining the beam value of the current coding unit according to the target characteristic may include:

[0088] According to the formula Determine the beam value of the current coding unit; wherein b i is the beam value of the i-th coding unit, b is the default global beam value, W i is the width of the i-th coding unit, H i is the height of the i-th coding unit, b' i is the adjusted beam value (customizable), and b' i <b,Sd is the size attribute hyperparameter value of the encoding unit. Since b' i It can be customized, that is, b' can be adjusted according to its own computing power resources. i , which can not only achieve the computational scalability of VVC intra-frame coding to adapt to different application scenarios, but also improve the utilization of computing resources in different application scenarios.

[0089] In some embodiments of the present specification, in order to effectively avoid the pressure of cache and data access during grid coding (i.e., coding unit coding) and further reduce the coding complexity, the dependency between adjacent CUs of different partition depths can be appropriately truncated. Therefore, according to the coding decision value and the optimal coding decision path group of the previous coding unit, determining the coding decision path candidate set of the current coding unit may include:

[0090] 1) When the level of the previous coding unit is the coding tree unit level, the coding decision path candidate set of the current coding unit is determined according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

[0091] That is, when the level of the previous coding unit is the CTU level, no matter whether the current coding unit will be further divided, only the best coding decision path in the best coding decision path group of the previous coding unit can be used as the RDO reference of the current coding unit, and the other coding decision paths in the best coding decision path group of the previous coding unit are equivalent to being truncated (that is, these coding decision paths are not considered).

[0092] For example, in Figure 3 In the exemplary embodiment shown, if the current coding unit is CU6, the previous coding unit CU5 is a CTU-level coding unit, and the optimal coding decision path group of CU5 contains two (the optimal coding decision path of CU5, and the second optimal coding decision path of CU5); since CU5 is at the CTU level, only the optimal coding decision path of CU5 can be used as the RDO reference of CU6.

[0093] 2) When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a coding unit of the same level, the coding decision path candidate set of the current coding unit is determined according to the coding decision value and all optimal coding decision paths in the optimal coding decision path group of the previous coding unit.

[0094] When the level of the previous coding unit is a non-coding tree unit level, and the current coding unit is a coding unit of the same level (that is, the current coding unit and the previous coding unit are coding units of the same level or brother coding units), at this time, all coding decision paths in the optimal coding decision path group of the previous coding unit can be used as RDO references of the previous coding unit, that is, no truncation is performed in this case.

[0095] For example, in Figure 3 In the exemplary embodiment shown, if the current coding unit is CU2, and the previous coding unit CU1 is a non-CTU-level coding unit, the optimal coding decision path group of CU1 includes two (the optimal coding decision path of CU1, and the second optimal coding decision path of CU1); since CU1 is a non-CTU-level and CU2 is a coding unit of the same level as CU1, the optimal coding decision path and the second optimal coding decision path of CU1 can both be used as RDO references for CU2.

[0096] 3) When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a sub-coding unit of a coding unit at the same level, determine the coding decision path candidate set of the first sub-coding unit of the current coding unit according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

[0097] When the level of the previous coding unit is a non-coding tree unit level, and the current coding unit is a sub-coding unit of a coding unit at the same level as the previous coding unit, only the best coding decision path in the best coding decision path group of the previous coding unit can be used as the RDO reference of the current coding unit, while other coding decision paths in the best coding decision path group of the previous coding unit are equivalent to being truncated.

[0098] For example, in Figure 3 In the exemplary embodiment shown, if the current coding unit is CU7, the previous coding unit CU6 is a non-CTU-level coding unit, and the optimal coding decision path group of CU6 contains two (the optimal coding decision path of CU6, and the second optimal coding decision path of CU6); since CU6 is a non-CTU level, and CU7 is a sub-coding unit of the same-level coding unit adjacent to CU6 (the same-level coding unit adjacent to CU6 is further divided into CU7 and CU8), only the optimal coding decision path of CU6 can be used as the RDO reference of CU7, and the second optimal coding decision path of CU6 is equivalent to being truncated.

[0099] refer to Figure 4As shown, in some embodiments of the present specification, selecting the coding decision paths of the number of beam values ​​from the coding decision path candidate set according to the beam search algorithm as the optimal coding decision path group for the current coding unit may include the following steps:

[0100] Step 401: From the candidate set of coding decision paths corresponding to when the current coding unit adopts a non-split decision, use a beam search algorithm to select a group of non-split optimal coding decision paths with the number of beam values ​​from the previous coding unit to the current coding unit.

[0101] The current coding unit adopts a non-division decision, which means that the current coding unit is not divided. Correspondingly, the current coding unit adopts a division decision, which means that the current coding unit is divided.

[0102] Step 402: From the candidate set of coding decision paths corresponding to the partition decision adopted by the current coding unit, a beam search algorithm is used to select a group of optimal coding decision paths for partitioning from the previous coding unit to the current coding unit, the number of which is the beam value.

[0103] Step 403: Determine the cumulative Lagrangian cost of each optimal coding decision path in the non-partitioned optimal coding decision path group and the partitioned optimal coding decision path group.

[0104] In the embodiments of the present specification, the cumulative Lagrangian cost may be used as an evaluation index of the RDO decision, and is used to screen out the number of RDO paths with the beam value.

[0105] Assume that the i-th encoding stage contains b i The space of optimal coding decision paths is For one of the paths, m i is the coding decision (partitioning, prediction and transformation) of the i-th CU in the path.

[0106] For the i+1th encoding stage, b i+1 The optimal coding decision path will be updated with the cumulative Lagrangian cost The smallest b i+1 Path in,

[0107] Obviously, in the above formula, hb i Sub-Lagrangian cost Calculate, where h represents m i+1 The number of possible decisions. When updating to the last CU (i.e. X n ), according to the cost By tracing back the corresponding paths, we can get the final overall coding decision. If b1=b2=b3=b4=b, the coding complexity is O(nbh).

[0108] Step 404: According to the size of the accumulated Lagrangian cost, select the optimal coding decision paths of the number of beam values ​​as the optimal coding decision path group of the current coding unit.

[0109] In some embodiments of the present specification, determining the final coding decision path according to the optimal coding decision path group of the last coding unit means: determining the one with the smallest cumulative Lagrangian cost in the optimal coding decision path group of the last coding unit as the final coding decision path.

[0110] refer to Figure 5 As shown, in an exemplary embodiment of the present specification, X0 is the previous coding unit of size 16×16. For the same-level CU of size 16×16 on the right side of X0, in the non-partitioned RDO decision, there are 2 reference paths at X0; after trying binary tree (BT) and ternary tree (TT) partitions, QT partitioning is performed. Suppose the four sub-CUs of size 8×8 are X1, X2, X3 and X4 respectively. Due to the truncation in the scheme, only the first optimal (minimum cumulative Lagrangian cost) path of X0 will be referenced by X1. In the figure, joint RDO optimization based on beam search is used for X1, X2, X3 and X4. Suppose b1=b2=b3=b4=2, m i The possible decision values ​​are α, β, χ. From the X1 stage, the 2-way optimal coding decision path is For the X2 stage, the possible coding decision paths are {α-α, α-β, α-χ, χ-α, χ-β, χ-χ}, and finally {α-α, χ-α} is selected. Repeat this step until X4 completes the decision, and the final two optimal coding decision paths under QT are {χ-α-β-α} and {χ-α-β-β}. After completing the coding from X1 to X4, collect the two paths from X1 to X4 under QT partitioning, compare them with the completed non-partitioning, BT and TT decision screening paths, and update the final optimal coding decision path group of the current 16×16 coding unit. This process is also called the generalized Breiman, Friedman, Olshen and Stone (i.e. G-BFOS) algorithm. If there is no next coding unit at this time, the first optimal coding decision path in the optimal coding decision path group is the coding decision path of the current image; otherwise, the optimal coding decision path group still retains two coding decision paths for reference for subsequent coding units.

[0111] Although the process flows described above include multiple operations that occur in a particular order, it should be clearly understood that the processes may include more or fewer operations, which may be performed sequentially or in parallel (eg, using parallel processors or a multi-threaded environment).

[0112] Corresponding to the above-mentioned joint rate-distortion optimization method for VVC intra-frame coding, the embodiment of this specification also provides a joint rate-distortion optimization device for VVC intra-frame coding, which can be configured on the above-mentioned server, referring to Figure 6 As shown, in some embodiments of the present specification, a joint rate-distortion optimization device for VVC intra-frame coding may include:

[0113] A parameter acquisition module 61 is used to acquire the target attribute of the current coding unit, the coding decision value, and the optimal coding decision path group of the previous coding unit;

[0114] A beam value determination module 62, configured to determine a beam value of the current coding unit according to the target attribute;

[0115] A candidate set determination module 63, configured to determine a candidate set of coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit;

[0116] A path group selection module 64 is used to select the coding decision paths of the number of beam values ​​from the coding decision path candidate set according to a beam search algorithm as the optimal coding decision path group of the current coding unit;

[0117] The decision path determination module 65 is used to determine a final coding decision path according to the optimal coding decision path group of the last coding unit.

[0118] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0119] It should be noted that in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and fully authorized by all parties.

[0120] The embodiments of this specification also provide a computer device. Figure 7As shown, in some embodiments of the present specification, the computer device 702 may include one or more processors 704, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 702 may also include any memory 706, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 706 and can be run on the processor 704, when the computer program is run by the processor 704, can execute the instructions of the joint rate-distortion optimization method for VVC intra-frame coding described in any of the above embodiments. Non-limiting, for example, the memory 706 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 702. In one embodiment, when the processor 704 executes the associated instructions stored in any memory or combination of memories, the computer device 702 can perform any operation of the associated instructions. The computer device 702 also includes one or more drive mechanisms 708 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0121] The computer device 702 may also include an input / output interface 710 (I / O) for receiving various inputs (via input device 712) and for providing various outputs (via output device 714). A specific output mechanism may include a presentation device 716 and an associated graphical user interface 718 (GUI). In other embodiments, the input / output interface 710 (I / O), input device 712, and output device 714 may not be included, and the computer device 702 may be used as a computer device in a network. The computer device 702 may also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the components described above together.

[0122] The communication link 722 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 722 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0128] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0129] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0130] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0131] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0132] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0133] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0134] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A joint rate-distortion optimization method for VVC intra-frame coding, characterized in that: include: Obtain target attributes, coding decision values, and the optimal coding decision path group of the previous coding unit of the current coding unit; Determining a beam value of the current coding unit according to the target attribute; Determine a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit; Selecting, from the coding decision path candidate set according to a beam search algorithm, coding decision paths having the same number of beam values ​​as an optimal coding decision path group for the current coding unit; The final coding decision path is determined according to the optimal coding decision path group of the last coding unit.

2. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The target attributes include a size attribute of a coding unit.

3. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 2, characterized in that: The determining the beam value of the current coding unit according to the target characteristic includes: According to the formula Determining a beam value of the current coding unit; Among them, b i is the beam value of the i-th coding unit, b is the default global beam value, W i is the width of the i-th coding unit, H i is the height of the i-th coding unit, b i ' is the adjusted beam value, and b i '<b,S d is the size attribute hyperparameter value of the encoding unit.

4. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The step of determining a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit includes: When the level of the previous coding unit is a coding tree unit level, a coding decision path candidate set of the current coding unit is determined according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

5. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The step of determining a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit includes: When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a coding unit of the same level, the coding decision path candidate set of the current coding unit is determined according to the coding decision value and all optimal coding decision paths in the optimal coding decision path group of the previous coding unit.

6. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The step of determining a set of candidate coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit includes: When the level of the previous coding unit is a non-coding tree unit level and the current coding unit is a sub-coding unit of a coding unit at the same level, a coding decision path candidate set of a first sub-coding unit of the current coding unit is determined according to the coding decision value and the best one in the optimal coding decision path group of the previous coding unit.

7. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The step of selecting, from the coding decision path candidate set according to the beam search algorithm, coding decision paths having the number of beam values ​​as the optimal coding decision path group for the current coding unit comprises: Selecting, from a set of coding decision path candidates corresponding to when the current coding unit adopts a non-division decision, a group of non-division optimal coding decision paths from the previous coding unit to the current coding unit with the number of beam values ​​by using a beam search algorithm; Selecting, from the candidate set of coding decision paths corresponding to the current coding unit when the current coding unit adopts the partition decision, a group of optimal coding decision paths for partitioning from the previous coding unit to the current coding unit with the number of beam values ​​by using a beam search algorithm; Determine the cumulative Lagrangian cost of each optimal coding decision path in the non-partitioned optimal coding decision path group and the partitioned optimal coding decision path group; And according to the size of the accumulated Lagrangian cost, the optimal coding decision paths of the number of beam values ​​are selected as the optimal coding decision path group of the current coding unit.

8. The joint rate-distortion optimization method for VVC intra-frame coding according to claim 1, characterized in that: The determining a final coding decision path according to the optimal coding decision path group of the last coding unit includes: The optimal coding decision path group of the last coding unit with the smallest cumulative Lagrangian cost is determined as the final coding decision path.

9. A joint rate-distortion optimization device for VVC intra-frame coding, characterized in that: include: A parameter acquisition module is used to obtain the target attribute of the current coding unit, the coding decision value, and the optimal coding decision path group of the previous coding unit; A beam value determination module, used to determine the beam value of the current coding unit according to the target attribute; A candidate set determination module, used to determine a candidate set of coding decision paths for the current coding unit according to the coding decision value and the optimal coding decision path group for the previous coding unit; A path group selection module, configured to select, from the coding decision path candidate set according to a beam search algorithm, coding decision paths of the number of beam values ​​as the optimal coding decision path group for the current coding unit; The decision path determination module is used to determine a final coding decision path according to the optimal coding decision path group of the last coding unit.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 8.

11. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor of a computer device, the computer program executes instructions of the method according to any one of claims 1 to 8.

Images