A method for inter-frame image bit rate control based on Lyapunov stability
Through the inter-frame image code rate control method based on Lyapunov stability, the code rate of video encoding is dynamically adjusted, and the problem of difficult to stabilize the code rate control during the video encoding process is solved, and the stability and adaptability of video quality are achieved, improving the quality of video User experience and coding efficiency.
Patent Information
- Application Number
- CN202410980948.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-22
AI Technical Summary
During the video encoding process, it is difficult for the prior art to find the best balance point between video quality, network conditions, real-time and computing resources, especially when video quality and network conditions change rapidly, making it difficult to effectively control the bit rate Stable, affecting the video playback experience.
The inter-frame image bit rate control method based on Lyapunov stability is adopted. The Lyapunov function is selected through predefined state variables to obtain the stability judgment amount, and the bit rate is controlled based on real-time feedback, and the bit rate is dynamically adjusted to adapt to the video. Changes in content and network conditions.
Effectively control the fluctuations in video frame quality, ensure the quality stability during video playback, optimize the video encoding process, reduce unnecessary data transmission and storage, adapt to various bandwidth and storage needs, and improve user experience.
Smart Images

Figure CN118524217B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an inter-frame image bit rate control method based on Lyapunov stability. Background Art
[0002] Rate Control is a key component in video encoding, and its main goal is to optimize video quality and compression efficiency within limited network bandwidth or storage space. During the video encoding process, the original video data is converted into a bit stream, and this process involves quantizing the video signal to reduce the amount of data. The role of rate control is to control the size of the output bit stream while ensuring video quality to ensure that it meets specific transmission or storage requirements.
[0003] As video resolution continues to increase and network conditions become more diverse, bitrate control faces multiple challenges. The most important one is to find the best balance between video quality, network conditions, real-time performance, and computing resources, while also taking into account the compatibility of different video standards. Summary of the invention
[0004] In order to improve the stability of bit rate control and ensure the quality of encoding, the present invention proposes an inter-frame image bit rate control method based on Lyapunov stability, comprising the steps of:
[0005] S1: predefine state variables based on factors that characterize video frame stability and quality deviation;
[0006] S2: Lyapunov function selection based on the goal of minimizing the deviation of the system target state quantity;
[0007] S3: acquiring the stability judgment amount of the current video encoding process according to the selected Lyapunov function and the current state variable;
[0008] S4: bit rate control is performed under preset threshold regulation according to the stability judgment amount fed back in real time.
[0009] Furthermore, in the step S1, the predefined state variable is the deviation between the video frame and the target video quality.
[0010] Furthermore, the deviation between the video frame and the target video quality is obtained by the following formula:
[0011]
[0012] In the formula, is the n-1th frame image, is the nth frame image, is the texture quality of the n-1th frame image, is the texture quality of the nth frame image, is the similarity between the n-1th frame image and the nth frame image at time t under the Euclidean distance, is the target video quality, is the actual video quality of the current frame at time t, is the deviation between the current video frame and the target video quality.
[0013] Furthermore, in the step S2, the selected Lyapunov function is expressed as the following formula:
[0014]
[0015] In the formula, is the Lyapunov function, is the state variable at time t.
[0016] Furthermore, in the step S3, the stability judgment quantity is the derivative of the Lyapunov function, which is obtained by the following formula:
[0017]
[0018] In the formula, is the derivative of the Lyapunov function, is the state variable at time t-1, For the interval time.
[0019] Furthermore, in the step S4, when the stability judgment amount is greater than a preset threshold, the bit rate is increased; when the stability judgment amount is less than the preset threshold, the bit rate is reduced.
[0020] Furthermore, in the step S4, the relationship between the bit rate and the stability judgment amount can be indirectly expressed by the state variable, which is expressed as the following formula:
[0021]
[0022] In the formula, is the bit rate at time t after adjustment, is the reference bit rate, k is the control gain parameter used to adjust the bit rate sensitivity, is an approximation of the quality difference between two consecutive frames.
[0023] Furthermore, the gain parameter k is obtained by the following formula:
[0024]
[0025] Where S is the frame size, M is the motion complexity, and is the weight coefficient.
[0026] Compared with the prior art, the present invention has at least the following beneficial effects:
[0027] (1) The inter-frame image bit rate control method based on Lyapunov stability described in the present invention ensures that the derivative of the Lyapunov function is non-positive based on a preset threshold comparison, thereby effectively controlling the fluctuation of the video frame quality, thereby ensuring the quality stability during video playback, which is crucial to improving the user experience, especially when processing dynamic scenes and high-resolution video streams;
[0028] (2) Through the real-time feedback mechanism, the video encoding bit rate can be dynamically adjusted based on changes in video content characteristics and network conditions, thereby automatically responding to real-time changes in video quality and network conditions to optimize the video encoding process;
[0029] (3) By continuously adjusting the bit rate, the entire video encoding process is optimized, which helps reduce unnecessary data transmission and storage while maintaining video quality and adapting it to various bandwidth and storage requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A step diagram of an inter-frame image bit rate control method based on Lyapunov stability. DETAILED DESCRIPTION
[0031] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.
[0032] When transmitting video over the Internet or wireless networks, bandwidth is usually limited. Rate control ensures that the bit rate of the video stream matches the available bandwidth, avoiding data overflow and packet loss, thereby maintaining a smooth playback experience. When the complexity of the video content varies greatly, rate control can avoid drastic fluctuations in the bit rate and maintain a stable output bit rate, which is particularly important for real-time streaming media transmission. To achieve these goals, the rate control algorithm usually dynamically adjusts the quantization parameter (QP) according to the characteristics of the video content, and continuously optimizes this process through a feedback mechanism to achieve an output that is closest to the target bit rate while maintaining good video quality. In the present invention, Figure 1 As shown, a method for controlling the bit rate of an inter-frame image based on Lyapunov stability is proposed, comprising the steps of:
[0033] S1: predefine state variables based on factors that characterize video frame stability and quality deviation;
[0034] S2: Lyapunov function selection based on the goal of minimizing the deviation of the system target state quantity;
[0035] S3: acquiring the stability judgment amount of the current video encoding process according to the selected Lyapunov function and the current state variable;
[0036] S4: bit rate control is performed under preset threshold regulation according to the stability judgment amount fed back in real time.
[0037] Lyapunov stability theory is an important concept in the theory of dynamical systems. In this concept, if the state of a dynamical system starts from a neighborhood of an equilibrium point, and as time goes by, the state of the system not only remains in this neighborhood but also tends to the equilibrium point itself, then we call this equilibrium point asymptotically stable. This means that after being disturbed, the system can not only return to the neighborhood of the equilibrium state, but also eventually converge to the equilibrium state. Based on the asymptotic stability of Lyapunov stability, we apply it to the bit rate control in video transmission to ensure that the bit rate can adapt to changes in network conditions and change asymptotically.
[0038] In order to apply Lyapunov stability to bit rate control, the system model we construct must be able to reflect factors such as network delay, packet loss rate, bandwidth availability, and data transmission rate. Therefore, when choosing the system state variable When the video frame quality is stable and the quality deviation is large, we need to select influencing factors that can characterize the stability and quality deviation of the video frame. Since changes in these factors will have a direct or indirect impact on the video quality, in this embodiment, the deviation between the video frame and the target video quality, that is, the quality error of the video frame, is used as the required state variable, which is obtained by the following formula:
[0039]
[0040] in, is the target video quality, is the actual video quality of the current frame at time t, It is obtained by calculating the image similarity based on content. First, select appropriate texture features to describe the image content, and then represent each image as a vector form of the selected features. We choose to use texture as a feature, represented as ,in represents the texture quality of the i-th frame image, and finally uses the Euclidean distance as the indicator of similarity measurement, then the previous and next two frames of images and The similarity between them can be expressed as:
[0041]
[0042] In the formula, is the n-1th frame image, is the nth frame image, is the similarity between the n-1th frame image and the nth frame image at time t under the Euclidean distance, represents the norm of a vector (such as the L2 norm), Used to normalize the Euclidean distance to ensure that the similarity is in the range [0, 1].
[0043] At the same time, we also need to choose a suitable Lyapunov function To evaluate the stability of a system, an energy or cost function is usually chosen, which represents the "energy" or "cost" of the system deviating from the target state. Here, the function we choose is expressed as follows:
[0044]
[0045] This choice reflects the energy of the system deviating from the target state. In order to minimize this energy and ensure that the system evolves to a stable state, thereby ensuring the stability of video quality, we calculate The time derivative of , which is the required stability judgment amount, to determine whether the system state is stable (less than or equal to 0 is considered stable):
[0046]
[0047] here, The time interval between two consecutive frames can be ) is approximately obtained by the mass difference:
[0048]
[0049] This can be converted to:
[0050]
[0051] At this point, you can use the real-time state variables , to calculate the current stability judgment, when (i.e. the system deviates from stability), then increase the bit rate To reduce the state variables The rate of increase of (i.e. the system deviates from stability), then maintain or reduce the bit rate Steady state variables .
[0052] Among them, the bit rate and stability judgment The change relationship between them can be achieved through the state variables Indirect expression, expressed as the following formula:
[0053]
[0054] In the formula, is the base bit rate, and k is the control gain parameter used to adjust the bit rate sensitivity. The gain parameter k can be expressed as a linear combination of the frame size S and the motion complexity M:
[0055]
[0056] The frame size S directly affects the coding rate. Generally, the larger the frame size, the higher the required coding rate. The motion complexity M reflects the amount of motion in the image. The higher the motion complexity, the higher the coding complexity, and more bits are required for coding. and It is used to balance the influence of various features on the gain parameter k. Here, the gain parameter k and the reference bit rate It is dynamically adjusted according to system performance and network conditions.
[0057] Here, we adjust the bit rate in real time , the goal is to make the state variable The change of is as small as possible, so that the stability judgment quantity Keep it non-positive.
[0058] Using real-time feedback mechanism, based on observed state variables and stability judgment , adjust the bit rate of each frame This approach optimizes the entire video encoding process by continuously adjusting to respond to real-time changes in video quality and network conditions.
[0059] In summary, the inter-frame image bit rate control method based on Lyapunov stability described in the present invention ensures that the derivative of the Lyapunov function is non-positive based on a preset threshold comparison, thereby effectively controlling the fluctuation of the video frame quality, which ensures the quality stability during video playback, which is crucial to improving user experience, especially when processing dynamic scenes and high-resolution video streams.
[0060] Through the real-time feedback mechanism, the video encoding bit rate can be dynamically adjusted based on changes in video content characteristics and network conditions, thereby automatically responding to real-time changes in video quality and network conditions to optimize the video encoding process.
[0061] By continuously adjusting the bit rate, the entire video encoding process is optimized, which helps reduce unnecessary data transmission and storage while maintaining video quality and adapting it to various bandwidth and storage requirements.
[0062] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0063] In addition, in the present invention, descriptions such as "first", "second", "one", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0064] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0065] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
Claims
1. A method for controlling the bit rate of an inter-frame image based on Lyapunov stability, characterized in that: Includes steps: S1: predefine state variables based on factors that characterize video frame stability and quality deviation; S2: Lyapunov function selection based on the goal of minimizing the deviation of the system target state quantity; S3: acquiring the stability judgment amount of the current video encoding process according to the selected Lyapunov function and the current state variable; S4: bit rate control under preset threshold regulation is performed according to the stability judgment amount fed back in real time; In the step S1, the predefined state variable is the deviation between the video frame and the target video quality; The deviation between the video frame and the target video quality is obtained by the following formula: In the formula, is the n-1th frame image, is the nth frame image, is the texture quality of the n-1th frame image, is the texture quality of the nth frame image, is the similarity between the n-1th frame image and the nth frame image at time t under the Euclidean distance, is the target video quality, is the actual video quality of the current frame at time t, is the deviation between the current video frame and the target video quality; In the step S2, the selected Lyapunov function is expressed as the following formula: In the formula, is the Lyapunov function, is the state variable at time t; In the step S3, the stability judgment quantity is the derivative of the Lyapunov function, which is obtained by the following formula: In the formula, is the derivative of the Lyapunov function, is the state variable at time t-1, is the interval time; In the step S4, when the stability judgment amount is greater than a preset threshold, the bit rate is increased; when the stability judgment amount is less than the preset threshold, the bit rate is reduced; In the step S4, the relationship between the bit rate and the stability judgment amount can be indirectly expressed by the state variable, which is expressed as the following formula: In the formula, is the bit rate at time t after adjustment, is the reference bit rate, k is the control gain parameter used to adjust the bit rate sensitivity, is an approximation of the quality difference between two consecutive frames.
2. The method for controlling the bit rate of an inter-frame image based on Lyapunov stability according to claim 1, characterized in that: The gain parameter k is obtained by the following formula: Where S is the frame size, M is the motion complexity, and is the weight coefficient.
