Complexity control method for panoramic video coding and coder thereof

Abstract

The invention discloses a complexity control method for panoramic video coding and an encoder thereof, and the control method comprises the steps: firstly setting the complexity of each coding frame, and on this basis, according to the characteristic that the coding rate distortion cost of the panoramic video coding on the high-efficiency video coding under the equidistant cylindrical projection changes in dimension, determining the complexity of each coding frame according to the characteristic; constructing a complexity control model based on the game theory and coding rate distortion cost change; under the condition that the complexity of the coding frame is limited, the complexity of the coding frame is obtained by solving the constructed complexity control model, obtaining the optimal solution and distributing the complexity of each dimension in the frame, so that the complexity control is more accurately realized, the problem of distribution of the panoramic video coding complexity is effectively solved, and the coding complexity of the panoramic video is effectively improved. The video complexity optimization is realized, the panoramic video coding quality is ensured, and the coding quality is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding, and in particular to a complexity control method for panoramic video coding. Background Art

[0002] Panoramic videos usually have higher resolution and larger field of view than traditional videos, such as 360-degree viewing angle. Such high resolution makes each frame more detailed, but it also greatly increases the complexity of encoding. In order to process panoramic videos, the technology of mapping panoramic videos to planes (such as cube mapping, spherical mapping, etc.) is usually adopted. This mapping converts panoramic videos into flat videos suitable for processing by traditional encoders, but the mapped flat videos still face the encoding complexity challenge brought by high resolution. Compared with the commonly used 1080p videos, the high resolution of panoramic videos (such as 2K and 4K) requires the encoder to process more pixels and more complex details, which not only increases the computational burden, but may also affect the encoding efficiency and real-time processing capabilities.

[0003] In traditional video coding, GOP-level complexity optimization: by optimizing the complexity allocation of key frames and predicted frames to improve coding efficiency, but for the complexity control of panoramic videos, this method needs to be further adjusted to adapt to the characteristics of panoramic videos; some encoders adopt a fixed complexity allocation strategy, that is, using the same coding complexity for all frames or regions, but it cannot effectively handle high-resolution and complex areas in panoramic videos; the dynamic complexity adjustment strategy optimizes complexity allocation by real-time analysis of video content and coding requirements. It can be adjusted according to the actual complexity of the video frame, but a more complex model is required to adapt to the high-resolution characteristics of panoramic videos. Therefore, it is necessary to build a new model to control the complexity allocation of each strip in the frame to achieve appropriate complexity allocation, thereby achieving a balance between complexity optimization and coding performance. Summary of the invention

[0004] The purpose of the present invention is to provide a complexity control method for panoramic video coding to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a complexity control method for panoramic video coding, the method comprising:

[0006] Set the complexity of each coding frame, and build a complexity control model based on game theory and coding rate distortion cost variation through the characteristics of coding rate distortion cost variation in dimensions;

[0007] After solving the intermediate variables through the Newton iteration method using the complexity control model, the complexity resources of the CU encoded in the dimension are obtained, and the complexity of each dimension in the frame is allocated to achieve complexity control.

[0008] Furthermore, the complexity of each coded frame is set, and a complexity control model based on game theory and coding rate distortion cost variation is constructed through the coding rate distortion cost variation characteristics in the dimension, including:

[0009] Set the complexity of the current encoding frame;

[0010] Map the panoramic video into a planar video, divide the dimension according to the size of the maximum coding CU of the efficient video coding, and set the complexity allocation function;

[0011] Set the effect function in game theory and define the change of coding complexity;

[0012] The objective function is constructed by combining the Nash equilibrium theory in game theory to obtain a complexity control model to ensure a balance between the minimum coding quality and coding complexity.

[0013] Furthermore, the expression of the complexity allocation function of the panoramic video is:

[0014]

[0015] Among them, N represents the number of stripes after dimension division, n represents the index, and C n Indicates the complexity of the nth stripe allocation, Indicates the normalized value of the number of CUs encoded in the nth slice, C t Indicates the complexity of the set coded frame.

[0016] Furthermore, the expression of the effect function in the game theory is:

[0017] Among them, ΔJ n Indicates the change in the coding rate distortion cost of the nth strip. Similarly, define the minimum effect function Used to ensure minimum encoding quality.

[0018] Furthermore, the objective function is expressed as:

[0019] in, Indicates the encoding complexity corresponding to the lowest encoding quality.

[0020] Furthermore, after solving the intermediate variables by Newton iteration method using the complexity control model, the complexity resources of the CU encoded in the dimension are obtained, and the complexity of each dimension in the frame is allocated to realize the complexity control, including:

[0021] The objective function is used to convert the product into the sum and difference through the ln(·) function, and the Lagrange multiplier method is introduced to convert it to obtain the objective conversion function;

[0022] Solve the maximum value of the target transfer function under KKT conditions;

[0023] Based on the dependencies of coding performance in different dimensions, the simplification of error propagation between strips, and the target conversion function, the Newton iteration method is used to obtain the optimal solution and calculate the complexity of the coded frame.

[0024] The complexity of each dimension in the frame is allocated according to the complexity of the obtained coded frame to achieve complexity control.

[0025] Furthermore, the expression of the target conversion function is:

[0026]

[0027] Among them, λ and θ n is the Lagrange coefficient.

[0028] Furthermore, the expression of the KKT condition is:

[0029]

[0030] because and The condition, θ n The parameter value is 0.

[0031] Furthermore, based on the dependencies of coding performance in different dimensions, the simplification of error propagation between stripes, and the target conversion function, the Newton iteration method is used to obtain the optimal solution. The complexity of the coded frame is calculated as follows:

[0032] Since there is a dependency between encoding performance in different dimensions, It is expressed as:

[0033]

[0034] Among them, τ n，f It is expressed as:

[0035] Due to the error propagation between strips, τ n，f It is expressed as:

[0036] Among them, ε is the error transfer parameter, τ f It is expressed as:

[0037] Then It is expressed as:

[0038] Simplify expression (10) to:

[0039] Based on expression (11), the complexity of the coded frame is solved

[0040] Due to the need to meet the conditions Then C f The expression is:

[0041] Substituting expression (13) into The expression is:

[0042]

[0043] make

[0044] Obtain the optimal λ through Newton iteration method * , then C f It is expressed as:

[0045] in, δ is the empirical value of 0.8;

[0046] C f The further expression of is:

[0047] An encoder implements the steps of the complexity control method for panoramic video encoding as described above during encoding.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention first sets the complexity of each coding frame, and on this basis, according to the characteristic that the coding rate distortion cost varies in the existing dimension of panoramic video coding under equidistant cylindrical projection on high-efficiency video coding, based on this characteristic, a complexity control model based on game theory and coding rate distortion cost variation is constructed; under the condition that the complexity of the coding frame is limited, the constructed complexity control model is solved to obtain the optimal solution, the complexity of the coding frame is obtained, and the complexity of each dimension in the frame is distributed to ensure more accurate complexity control, effectively solve the problem of panoramic video coding complexity distribution, achieve video complexity optimization while ensuring the quality of panoramic video coding, and effectively improve the coding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of the complexity control method for panoramic video coding of the present invention;

[0050] Figure 2 This is a flow chart of step S1 in the complexity control method for panoramic video encoding of the present invention;

[0051] Figure 3This is a flow chart of step S2 in the complexity control method for panoramic video encoding of the present invention;

[0052] Figure 4 This is a complexity control flow chart for panoramic video encoding according to the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] like Figure 1 to Figure 3 As shown, this embodiment provides a complexity control method for panoramic video coding, and the method includes the following steps:

[0055] S1: Set the complexity of each coded frame, and build a complexity control model based on game theory and rate-distortion cost variation through the dimensional rate-distortion cost variation characteristics;

[0056] In this embodiment, the specific steps of S1 include:

[0057] S11: Set the complexity of the current encoding frame;

[0058] S12: Map the panoramic video into a planar video, divide the dimension according to the size of the maximum coding CU of the efficient video coding, and set the complexity allocation function;

[0059] S13: Set the effect function in game theory and define the change of coding complexity;

[0060] S14: The objective function is constructed by combining the Nash equilibrium theory in game theory to obtain a complexity control model to ensure a balance between the minimum coding quality and coding complexity.

[0061] S2: After solving the intermediate variables using the Newton iteration method using the complexity control model, the complexity resources of the CU encoded in the dimension are obtained, and the complexity of each dimension in the frame is allocated to achieve complexity control.

[0062] In this embodiment, the specific steps of S2 include:

[0063] S21: Using the objective function to realize product-to-difference conversion through ln(·) function, and introducing Lagrange multiplier method for conversion, the objective conversion function is obtained;

[0064] S22: Solve the maximum value of the target conversion function under KKT conditions;

[0065] S23: Based on the dependencies between coding performances in different dimensions, the simplification of error propagation between slices, and the target conversion function, the Newton iteration method is used to obtain the optimal solution and the complexity of the coded frame is calculated;

[0066] S24: Allocate the complexity of each dimension in the frame according to the obtained complexity of the coded frame to achieve complexity control.

[0067] Specifically, the complexity of the current coded frame can be set according to the characteristics of the panoramic video. After the panoramic video is mapped into a planar video, the complexity of different strips or regions may be uneven. Therefore, it is necessary to set the corresponding complexity according to the characteristics of each strip and set the complexity C of each coded frame. t , the panoramic video is mapped into a planar video. According to the characteristic that the rate-distortion cost of the panoramic video coding in the equirectangular projection varies in the dimension, the dimension is divided according to the size of the maximum coding CU of the efficient video coding, and the complexity allocation function is set. The expression is:

[0068] Among them, N represents the number of stripes after dimension division, n represents the index, and C n Indicates the complexity of the nth stripe allocation, Indicates the normalized value of the number of CUs encoded in the nth slice;

[0069] Set the effect function in game theory, the expression is: Define the change in coding complexity, where ΔJ n Indicates the change in the coding rate distortion cost of the nth strip. Similarly, the minimum effect function can be defined Used to ensure minimum encoding quality;

[0070] Combining the Nash equilibrium theory in game theory, the allocation function is constructed as the objective function, and its expression is:

[0071]

[0072] in, Indicates the coding complexity corresponding to the minimum coding quality; obtains the complexity control model to ensure the balance between the minimum coding quality and the coding complexity;

[0073] Using the constructed objective function, the product is converted to sum and difference through the ln(·) function, and the Lagrange multiplier method is introduced for conversion to obtain the objective conversion function, which is expressed as:

[0074]

[0075] Among them, λ and θn is the Lagrange coefficient;

[0076] The target conversion function is solved for the maximum value of the target conversion function under the KKT (Karush-Kuhn-Tucker) condition. The expression of the KKT condition is:

[0077]

[0078] because and The condition, θ n The parameter value of is 0;

[0079] Since there is a dependency between encoding performance in different dimensions, It is expressed as:

[0080]

[0081] Among them, τ n，f It is expressed as:

[0082] Due to the error propagation between strips, τ n，f It is expressed as:

[0083] Among them, ε is the error transfer parameter, τ f It is expressed as:

[0084] Then It is expressed as:

[0085] Simplify expression (10) to:

[0086] Based on expression (11), the complexity of the coded frame is solved

[0087] Due to the need to meet the conditions Then C f The expression is:

[0088] Substituting expression (13) into The expression is:

[0089]

[0090] make

[0091] Obtain the optimal λ through Newton iteration method * , then C f It is expressed as:

[0092] in, δ is the empirical value of 0.8;

[0093] C f The further expression of is:

[0094] The complexity of each dimension in the frame is allocated according to the complexity of the obtained coded frame to achieve complexity control.

[0095] This embodiment further provides an encoder, which implements the steps of the complexity control method for panoramic video encoding when encoding, such as Figure 4 As shown, prepare the encoder and input video frame; before starting encoding, perform complexity analysis on the current frame to obtain frame encoding complexity. The complexity may include texture details of the image, changes in motion vectors, static or dynamic characteristics of the region, etc. During analysis, the complexity index can be evaluated and internal encoding processing can be performed on the frame, such as intra-frame prediction, transformation, quantization and other operations; divide the current frame into N strips according to latitude, and these strips can be rectangular areas or areas of other shapes, which can be adjusted according to actual needs; according to the constructed complexity control model, obtain the encoding complexity of each strip; during the encoding process, according to the complexity predicted by the complexity control model, adjust the complexity allocation scheme of each strip, according to the predicted complexity allocation result, adjust the complexity allocation of the encoder, realize the optimization of encoding complexity, perform corresponding encoding operations on each strip, end the encoding process of the current frame, and output the encoding result.

[0096] The present invention first sets the complexity of each coding frame, and on this basis, according to the characteristic that the coding rate distortion cost changes in the existing dimension of panoramic video coding under equidistant cylindrical projection on high-efficiency video coding, based on this characteristic, constructs a complexity control model based on game theory and coding rate distortion cost change; under the condition that the complexity of the coding frame is limited, the constructed complexity control model is solved to obtain the optimal solution, the complexity of the coding frame is obtained, and the complexity of each dimension in the frame is distributed to ensure more accurate complexity control, effectively solve the problem of panoramic video coding complexity distribution, achieve video complexity optimization while ensuring the quality of panoramic video coding, and effectively improve the coding quality.

[0097] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A complexity control method for panoramic video coding, characterized in that: The method comprises: Set the complexity of each coding frame, and build a complexity control model based on game theory and coding rate distortion cost variation through the characteristics of coding rate distortion cost variation in dimensions; After solving the intermediate variables through the Newton iteration method using the complexity control model, the complexity resources of the CU encoded in the dimension are obtained, and the complexity of each dimension in the frame is allocated to achieve complexity control.

2. The complexity control method for panoramic video coding according to claim 1, characterized in that: The complexity of setting each coding frame and constructing a complexity control model based on game theory and coding rate distortion cost variation through the coding rate distortion cost variation characteristics in the dimension include: Set the complexity of the current encoding frame; Map the panoramic video into a planar video, divide the dimension according to the size of the maximum coding CU of the efficient video coding, and set the complexity allocation function; Set the effect function in game theory and define the change of coding complexity; The objective function is constructed by combining the Nash equilibrium theory in game theory to obtain a complexity control model to ensure a balance between the minimum coding quality and coding complexity.

3. The complexity control method for panoramic video coding according to claim 2, characterized in that: The expression of the complexity allocation function of the panoramic video is: Among them, N represents the number of stripes after dimension division, n represents the index, and C n Indicates the complexity of the nth stripe allocation, Indicates the normalized value of the number of CUs encoded in the nth slice, C t Indicates the complexity of the set coded frame.

4. The complexity control method for panoramic video coding according to claim 3, characterized in that: The expression of the effect function in the game theory is: Among them, ΔJ n Indicates the change in the coding rate distortion cost of the nth strip. Similarly, define the minimum effect function Used to ensure minimum encoding quality.

5. The complexity control method for panoramic video coding according to claim 4, characterized in that: The expression of the objective function is: in, Indicates the encoding complexity corresponding to the lowest encoding quality.

6. The complexity control method for panoramic video coding according to claim 5, characterized in that: After solving the intermediate variables by Newton iteration method using the complexity control model, the complexity resources of the CU encoded in the dimension are obtained, and the complexity of each dimension in the frame is allocated to realize the complexity control, which includes: The objective function is used to convert the product into the sum and difference through the ln(·) function, and the Lagrange multiplier method is introduced to convert it to obtain the objective conversion function; Solve the maximum value of the target transfer function under KKT conditions; Based on the dependencies of coding performance in different dimensions, the simplification of error propagation between strips, and the target conversion function, the Newton iteration method is used to obtain the optimal solution and calculate the complexity of the coded frame. The complexity of each dimension in the frame is allocated according to the complexity of the obtained coded frame to achieve complexity control.

7. The complexity control method for panoramic video coding according to claim 6, characterized in that: The expression of the target conversion function is: Among them, λ and θ n is the Lagrange coefficient.

8. The complexity control method for panoramic video coding according to claim 7, characterized in that: The expression of the KKT condition is: because and The condition, θ n The parameter value is 0.

9. The complexity control method for panoramic video coding according to claim 8, characterized in that: Based on the dependencies between coding performance in different dimensions, the simplification of error propagation between stripes, and the target conversion function, the Newton iteration method is used to obtain the optimal solution. The complexity of the coded frame is calculated as follows: Since there is a dependency between encoding performance in different dimensions, It is expressed as: Among them, τ n，f It is expressed as: Due to the error propagation between strips, τ n，f It is expressed as: Among them, ε is the error transfer parameter, τ f It is expressed as: Then It is expressed as: Simplify expression (10) to: Based on expression (11), the complexity of the coded frame is solved Due to the need to meet the conditions Then C f The expression is: Substituting expression (13) into The expression is: make Obtain the optimal λ through Newton iteration method * , then C f It is expressed as: in, δ is the empirical value of 0.8; C f The further expression of is:

10. An encoder, characterized in that: The encoder implements the steps of the complexity control method for panoramic video coding as described in any one of claims 1 to 9 during encoding.

Images