Inter-frame Image Bitrate Control Method, Device and Medium Based on Stochastic Optimal Control

Abstract

The present invention discloses an inter-frame image bitrate control method, device and medium based on stochastic optimal control, which relates to the technical field of image processing, and includes the steps of: determining system parameters with the goal of minimizing the total cost of video transmission, where the system parameters include state variables, control variables and random variables; based on the system parameters and the optimization goal, constructing a Bellman equation that expresses the immediate cost at the current moment and the expected cost at future moments in a recursive manner according to the theory of stochastic optimal control; based on the constructed Bellman equation, through backward iteration, calculating the expected costs of each control variable under the optimal expected cost according to the current state variables; dynamically adjusting the bitrate under image resolution regulation according to the control variable corresponding to the optimal total cost of the current state variables. The present invention can automatically adapt to network changes, effectively adjust the bitrate of inter-frame images, thereby improving the overall efficiency of video coding and video quality, and optimizing the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an inter-frame image bitrate control method, device, and medium based on stochastic optimal control. Background Art

[0002] Video compression technology reduces the size of video files by optimizing encoding algorithms. With the increasing application of AI technology in video encoding, the encoder parameters are optimized through machine learning and deep learning, further improving the video compression effect. Video compression technology is widely used in the medical field, supporting remote diagnosis, medical image storage, and surgical live broadcast. It improves the efficiency and quality of medical services and promotes the progress of medical education and research.

[0003] Among them, the bitrate control methods for video compression include constant bitrate (CBR), variable bitrate (VBR), and dynamic bitrate control. CBR maintains a stable bitrate and is suitable for scenarios with strict network transmission requirements, but may have low efficiency when the picture complexity changes. VBR dynamically adjusts the bitrate according to the picture content, improving the encoding efficiency and quality performance, and is suitable for applications with large demand variations. Dynamic bitrate control combines the advantages of CBR and VBR, and can flexibly control the overall bitrate while ensuring quality, but the implementation complexity is high and more computing resources are required. The selection of the appropriate method should be weighed according to specific application requirements and quality standards. Summary of the Invention

[0004] In order to better handle the video compression and transmission situation under complex working conditions, the present invention proposes an inter-frame image bitrate control method based on stochastic optimal control, including the steps of:

[0005] S1: Determine the system parameters with the optimization goal of minimizing the total cost of video transmission, where the system parameters include state variables, control variables, and random variables;

[0006] S2: Based on the system parameters and the optimization goal, construct a Bellman equation that recursively expresses the immediate cost at the current moment and the expected cost at future moments based on the theory of stochastic optimal control;

[0007] S3: Based on the constructed Bellman equation, through backward iteration, calculate the expected cost of each control variable under the optimal expected cost according to the current state variable;

[0008] S4: Dynamically adjust the bitrate under the image resolution regulation according to the control variable corresponding to the optimal total cost of the current state variable.

[0009] Further, in the step S1, the total cost of video transmission includes image quality loss, bandwidth usage, and buffer stability.

[0010] Furthermore, in the step S1,

[0011] State variable is a key parameter used to describe the system at each time step t. In the bitrate control with the optimization goal of minimizing the total cost of video transmission, the key parameters include the bitrate of the current frame and the buffer state ;

[0012] Control variable is an adjustable decision variable used to affect the future development of the system state variable . In the bitrate control with the optimization goal of minimizing the total cost of video transmission, the decision variable is the bitrate of the next frame;

[0013] Random variable is a random noise used to represent the uncertainty of network bandwidth.

[0014] Furthermore, in the step S2, the immediate cost is obtained through the following formula:

[0015]

[0016] In the formula, J is the immediate cost, is the time step corresponding to the current state variable, N is the final time step, is the state variable, is the control variable, is the buffer state, is the bitrate of the current frame, Q is the image perception quality, R is the bitrate, S is the weight of the buffer state, is the cost of the buffer state, is the image quality based on the perceptual distortion degree, is used to measure the visual information retention degree between the encoded video frame and the original video frame, is the video multi-method evaluation for comprehensively considering multiple visual feature evaluation metrics, and are weight parameters used to adjust the importance of different evaluation metrics in the overall quality perception.

[0017] Furthermore, in the step S2, the Bellman equation is expressed as the following formula:

[0018]

[0019] In the formula, is the optimal expected cost starting from the current state variable , is, under the current control variable , the optimal expected cost starting from the state variable Transfer to the next state variable The expected cost, where A and B are parameters dynamically adjusted according to the system.

[0020] Furthermore, in the step S4, the image resolution adjustment is expressed by the following formula:

[0021]

[0022] In the formula, is the adjusted image resolution, is the base resolution, is the reference bit rate.

[0023] The present invention further includes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the bit rate control method are implemented.

[0024] It also includes a device for processing data, including:

[0025] A memory on which a computer program is stored;

[0026] A processor for executing the computer program in the memory to implement the steps of the bit rate control method.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects:

[0028] A method, device and medium for inter-frame image bit rate control based on stochastic optimal control according to the present invention consider the changes in video content and the uncertainty of network conditions by establishing a stochastic model, so as to be able to operate stably in different network environments, automatically adapt to changes, effectively adjust the bit rate of inter-frame images, thereby improving the overall efficiency of video coding and video quality. It avoids video stuttering or image quality degradation caused by network fluctuations and optimizes the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a step diagram of a method for inter-frame image bit rate control based on stochastic optimal control. DETAILED DESCRIPTION OF THE INVENTION

[0030] The following are specific embodiments of the present invention in combination with the accompanying drawings, and the technical solutions of the present invention are further described, but the present invention is not limited to these embodiments.

[0031] A video compression architecture is a system designed to efficiently compress raw video data, including preprocessing and frame analysis stages for preparing and optimizing video frame data. The encoder utilizes inter-frame and intra-frame prediction techniques to reduce data redundancy through motion compensation and spatial-domain prediction. The transformation and quantization steps further compress the prediction errors, while entropy coding techniques are used for the final compression of the data stream. The decoder parses and reconstructs the video data according to the bitstream format, and the quality assessment module evaluates the quality of the decoded video through structured and perceptual assessment methods. The control strategy adjusts the encoding parameters based on the assessment results to optimize video quality and bitrate efficiency. The design of the video compression architecture needs to consider the complexity of video content and real-time requirements to meet the visual experience and performance needs of different application scenarios. Video bitrate control plays a crucial role in video compression. It balances the relationship between video quality and transmission bandwidth by adjusting the bitrate during the video encoding process. This process ensures that the video achieves the best visual quality at a given bitrate, avoiding image quality problems or bandwidth waste caused by too high or too low bitrates. To this end, as Figure 1 shown, the present invention proposes an inter-frame image bitrate control method based on stochastic optimal control, including the steps of:

[0032] S1: Determine system parameters with the optimization goal of minimizing the total cost of video transmission, where the system parameters include state variables, control variables, and random variables;

[0033] S2: Based on the system parameters and the optimization goal, construct a Bellman equation that recursively expresses the immediate cost at the current moment and the expected cost at future moments based on the stochastic optimal control theory;

[0034] S3: Based on the constructed Bellman equation, calculate the expected costs of each control variable under the optimal expected cost according to the current state variables through backward iteration;

[0035] S4: Dynamically adjust the bitrate under image resolution regulation according to the control variable corresponding to the optimal total cost of the current state variable.

[0036] The stochastic optimal control used in the present invention is a mathematical tool for studying and solving the problem of finding an optimal control strategy in an uncertain environment. This control method is a branch of control theory that deals with how to optimally control a dynamic system in a stochastic environment. Such problems can generally be described by a stochastic process for the dynamic behavior of the system, and the goal is to minimize a certain performance metric (such as a cost function or energy consumption).

[0037] Regarding the performance and quality requirements of video compression and transmission, the minimized performance metric in the present invention is the total cost of video transmission, which includes image quality loss, bandwidth usage, and buffer stability. Based on these cost factors, the present invention defines system parameters closely related to these cost factors, including: (1) state variables , in the upper and lower image streams of video bitrate control, the state variable is a key parameter describing the system at each time step t. Generally, in this application environment, this key parameter includes the bitrate of the current frame and the buffer state ; (2) control variable , which is an adjustable decision variable used to affect the future development of the system state variable . In the bitrate control with the optimization goal of minimizing the total cost of video transmission, this decision variable usually represents the bitrate of the next frame; (3) random variable , which is a random noise (t is the time variable) used to represent the uncertainty of network bandwidth.

[0038] Based on these system parameters, the present invention further constructs a Bellman equation that recursively expresses the immediate cost at the current moment and the expected cost at future moments to achieve bitrate regulation. Among them, the Bellman equation is one of the core concepts in dynamic programming theory, proposed by the American mathematician Richard Bellman. It provides a recursive method to solve multi-stage decision-making problems, especially when finding the optimal strategy. The Bellman equation can be applied to discrete-time dynamic programming problems and can also be extended to continuous-time optimal control problems.

[0039] In the discrete dynamic programming of the video compression and transmission application scenario, an optimal strategy is determined through the Bellman equation, and this strategy can minimize the cumulative cost. Then, assume there is a dynamic system, expressed as the state transition equation between inter-frame images:

[0040]

[0041] where, represents the system state at the current time step of the system, is the system state at the next time step of the system, and A and B are parameters adjusted according to the system dynamics.

[0042] And in this system, the immediate cost of the system is expressed as follows:

[0043]

[0044] In the formula, J is the immediate cost, is the time step corresponding to the current state variable, N is the final time step, Q is the perceptual quality of the image, R is the bitrate, S is the weight of the buffer state, is the cost of the buffer state, such as overflow or underflow. And can be expressed as the image quality based on the perceptual distortion, and its formula can be expressed as:

[0045]

[0046] In the formula, is used to measure the visual information retention between the encoded video frame and the original video frame, is the multi-method evaluation of the video that comprehensively considers multiple visual feature evaluation metrics, and is the weight parameter used to adjust the importance of different evaluation metrics in the overall quality perception.

[0047] Then, considering the expected cost of the inter-frame image transfer to the next state our Bellman equation is expressed as the minimization solution for the total cost under different control variables :

[0048]

[0049] In the formula, is the optimal expected cost starting from the current state variable , is the expected cost of transferring from the state variable to the next state variable under the current control variable . This equation expresses a recursive relationship, which shows how to minimize the total future cost through the decisions in the current stage. Specifically, it tells us how to decompose a complex problem into a series of smaller sub-problems and then solve these sub-problems recursively.

[0050] Specifically, the Bellman equation divides the bitrate and buffer size into a finite number of levels and initializes. At the end time step t = N, set , because there is no more cost after the last system state. Then, through backward iteration, for each , calculate:

[0051] , for each possible control variable , update and calculate the total cost, and record the optimal control decision for each state.

[0052] In this way, we can calculate the control variable corresponding to the optimal total cost of the current state variable Here, we first adjust the bit rate according to the control variable Adjust the image resolution of the next frame. Resolution adjustment is usually performed based on current network conditions and the desired video quality level while maintaining the image aspect ratio. The main goal of adjusting resolution is to reduce the amount of data while maintaining the clarity of the video content as much as possible when network bandwidth is limited.

[0053] We first set a base resolution Then we get the optimal control variable according to the Bellman equation , that is, the bit rate is adjusted to obtain the appropriate resolution :

[0054]

[0055] In the formula, is the base bit rate, is the bit rate of the current inter-frame image. This can keep the resolution and bit rate in a certain proportional relationship, thus maintaining a relatively consistent visual effect under different bandwidth conditions. and Applied to the video encoder to ensure that the resolution of the output video is consistent with the desired target.

[0056] In practical applications, we also need to adjust parameters based on real-time video quality assessment such as peak signal-to-noise ratio (PSNR) and buffer status. distribution to adapt to new conditions.

[0057] The present invention also includes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the bit rate control method are implemented.

[0058] Also included is a device for processing data, comprising:

[0059] a memory having a computer program stored thereon;

[0060] A processor is used to execute the computer program in the memory to implement the steps of the bit rate control method.

[0061] In summary, for the inter-frame image bitrate control method, device, and medium based on stochastic optimal control according to the present invention, a stochastic model is established to consider the changes in video content and the uncertainties of network conditions. As a result, it can operate stably in different network environments, automatically adapt to changes, and effectively adjust the bitrate of inter-frame images, thereby improving the overall efficiency of video coding and video quality. It avoids video stuttering or image quality degradation caused by network fluctuations and optimizes the user's viewing experience.

[0062] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the attached drawings). If the specific posture changes, the directional indications will also change accordingly.

[0063] In addition, in the present invention, descriptions such as "first", "second", and "one" are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0064] In the present invention, unless otherwise clearly specified and limited, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] In addition, the technical solutions between various embodiments of the present invention can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

Claims

1. A method for controlling inter-frame image bit rate based on random optimal control, characterized in that: Includes steps: S1: Determine system parameters with minimizing the total cost of video transmission as the optimization goal, wherein the system parameters include state variables, control variables and random variables; S2: According to the system parameters and optimization objectives, the Bellman equation is constructed based on the stochastic optimal control theory to recursively express the immediate cost at the current moment and the expected cost at the future moment; S3: Based on the constructed Bellman equation, the expected cost of each control variable under the optimal expected cost is calculated according to the current state variables through reverse iteration; S4: dynamically adjust the bit rate under image resolution control according to the control variable corresponding to the optimal total cost of the current state variable; In the step S1, State variables is used to describe the key parameters of the system at each time step t. In the bit rate control with the optimization goal of minimizing the total cost of video transmission, the key parameters include the bit rate of the current frame and buffer status ; Control variables To affect the system state variables The adjustable decision variable for future development, in the bit rate control with minimizing the total cost of video transmission as the optimization goal, the decision variable is the bit rate of the next frame; random variable is the random noise used to represent the uncertainty of network bandwidth; In the step S2, the instantaneous cost is obtained by the following formula: Where J is the immediate cost, is the time step corresponding to the current state variable, N is the final time step, is the state variable, is the control variable, is the buffer state, is the bit rate of the current frame, Q is the perceived image quality, R is the bit rate, S is the weight of the buffer state, is the cost of the buffer state, is the image quality based on perceived distortion, It is used to measure the degree of visual information preservation between the encoded video frame and the original video frame. It is a multi-method video evaluation method that comprehensively considers multiple visual feature evaluation indicators. and is a weight parameter used to adjust the importance of different evaluation indicators in the overall quality perception; In the step S2, the Bellman equation is expressed as the following formula: In the formula, is the current state variable The optimal expected cost at the start, For the current control variable Next, from the state variable Transfer to next state variable A and B are parameters adjusted dynamically according to the system.

2. The inter-frame image bit rate control method based on random optimal control according to claim 1, characterized in that: In the step S1, the total cost of video transmission includes image quality loss, bandwidth usage and buffer stability.

3. The inter-frame image bit rate control method based on random optimal control according to claim 1, characterized in that: In step S4, the image resolution control is expressed as the following formula: In the formula, is the image resolution after adjustment, is the base resolution, is the base bit rate.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the rate control method described in any one of claims 1 to 3 are implemented.

5. A device for processing data, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the bit rate control method according to any one of claims 1 to 3.

Images