Video single-exposure compression imaging compression ratio adjusting method based on deep reinforcement learning

Abstract

The invention discloses a video single-exposure compression imaging compression ratio adjusting method based on deep reinforcement learning, and the method comprises the steps: carrying out the Markov dynamic decision process modeling of a video single-exposure compression imaging process, defining a compression measurement value of each time step as a current state, defining the compression ratio of the next time step as an action, and carrying out the dynamic decision process modeling of the Markov dynamic decision process. The compression ratio and the reconstruction quality of the video are considered in the return of the evaluation action; giving a training set, and training an intelligent agent capable of adaptively adjusting the compression ratio at each moment by maximizing cumulative return in the Markov decision-making process of modeling by using a deep reinforcement learning algorithm; and finally, the trained intelligent agent is used for adaptively adjusting the compression ratio in the video single-exposure compression imaging system in a new scene. According to the invention, the compression ratio can be adaptively adjusted only through the compression measurement value, so that the existing single-exposure compression imaging system can dynamically balance the compression ratio and the video reconstruction quality.

Description

Technical Field

[0001] This invention relates to the cross-application of artificial intelligence algorithms in the field of optics, specifically to a method for adjusting the compression rate of single-exposure compression imaging of video based on deep reinforcement learning. Background Technology

[0002] Single-exposure video compression imaging is a novel, multidisciplinary imaging technology that integrates optical compression of dynamic scenes with neural network decoding. Several existing publications have reported on this novel imaging technology, such as: Document 1: Z. Cheng, B. Chen, R. Lu, Z. Wang, H. Zhang, Z. Meng, and X. Yuan, "Recurrent neural networks for snapshot compressive imaging," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 2, pp. 2264–2281, 2022. Document 2: X. Yuan, DJ Brady, and AK Katsaggelos, “Snapshotcompressive imaging: Theory, algorithms, and applications,” IEEE SignalProcessing Magazine, vol. 38, no. 2, pp. 65–88, 2021. Document 3: X. Yuan, Y. Liu, J. Suo, and Q. Dai, “Plug-and-play algorithms for largescale snapshot compressive imaging,” in Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition, pp. 1447–1457, 2020. Single-exposure compressed video imaging technology can be applied to key fields such as space-based computing, autonomous driving, and national defense. Specifically, it consists of a hardware encoder and a software decoder. During encoding, each frame of the dynamic scene to be captured is modulated by a different mask, and the summation is achieved through optical integration. The modulated image is then compressed to form a compressed measurement value. Correspondingly, during decoding, the reconstruction network efficiently reconstructs each frame based on the compressed measurement value captured by the camera and the known mask, ultimately obtaining a high-quality video signal.

[0003] Typically, scenes being compressed are too long to be compressed into a single compression measurement. Therefore, single-exposure compression imaging systems compress this long scene into multiple compression measurements in an orderly fashion. However, existing video single-exposure compression imaging technologies use a fixed compression rate to compress the scene, making it difficult to adapt to dynamic changes in the scene within that time period. With a fixed reconstruction frame rate, the higher the compression rate, the longer the scene time stored in a single compression measurement. Therefore, if the given compression rate is too high, ideal reconstruction results may not be achieved at certain critical nodes in dynamic scenes; if the given compression rate is too low, a large amount of resources are needed to transmit and store the compressed results.

[0004] One way to solve the above problem is to reconstruct the quality of the video, that is, to adaptively adjust the compression rate based on the difference between the reconstructed video and the original scene. For example... Figure 1 As shown in the figure, the video reconstruction quality at different stages and compression rates in the "runner" scene is illustrated. The first box from left to right, representing the moment when the athlete is in a fixed starting position, shows a high compression rate. B =12, Medium compression ratio B =8 and low compression ratio B The reconstructed video obtained with =4 has a similar quality difference, so a high compression ratio can be used for compression at this point; within the second box, B =4 and B The reconstruction effect of =8 is similar and significantly better than... B The reconstruction effect is 12; at this point, the compression ratio should be adjusted to... B =8; within the third box B= The reconstruction effect of 4 is significantly better than B =8 and B= The reconstruction effect is 12. At this point, the compression ratio should be adjusted to... B= 4.

[0005] However, in real-world single-exposure compression systems, we cannot obtain the original scene, only compressed measurements. Therefore, adapting the compression ratio using only these measurements becomes a significant challenge. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a video single-exposure compression imaging compression ratio adjustment method based on deep reinforcement learning. By reconstructing the quality of the video, that is, the difference between the reconstructed video and the original scene, the compression ratio is adaptively adjusted. Based on this, an agent is trained to obtain a trained agent that, when applied to a single-exposure compression imaging system, adaptively adjusts the compression ratio solely based on compression measurements.

[0007] The objective of this invention is achieved through the following technical solution: A method for adjusting the compression ratio of single-exposure compressed imaging of video based on deep reinforcement learning includes the following steps: S1: Model the single-exposure compression imaging process of video using Markov dynamic decision process modeling. Define the compression measurement value at each time step as the current state, define the compression rate at the next time step as the action, and consider both the compression rate and the video reconstruction quality in the evaluation of the action reward. S2: Given a training set, use a deep reinforcement learning algorithm to train an agent that can adaptively adjust the compression rate at each time step by maximizing the cumulative reward in the Markov decision process modeled above. S3: Use the trained agent to adaptively adjust the compression rate in a single-exposure compression imaging system for new scenarios.

[0008] Furthermore, in S1, when modeling the Markov dynamic decision process for the single-exposure compressed imaging process of the video, the calculation formula for the reward of the action reward is as follows: in, B t Indicates time step t Compression ratio at any moment e It is a parameter that needs to be adjusted. L t+1 To reconstruct the video using a given video reconstruction network t Peak signal-to-noise ratio of the reconstructed video at time +1. L bas The initial peak signal-to-noise ratio is given.

[0009] Furthermore, the given video reconstruction network is a hybrid network architecture based on transformer and convolutional neural network.

[0010] Furthermore, the training set in S2 is a video dataset of the main objects moving at different speeds.

[0011] Furthermore, during the training process of S2, the policy network and estimation grid of the Markov decision process are trained simultaneously using a temporal difference method.

[0012] The beneficial effects of this invention are as follows: This invention uses reinforcement learning to train an agent. When the trained agent is applied to a single-exposure video compression system, it can adaptively adjust the compression rate based solely on compression measurements, thereby enabling existing single-exposure compression imaging systems to dynamically balance compression rate and video reconstruction quality. Attached Figure Description

[0013] Figure 1 It refers to the video reconstruction quality at different compression rates at different stages in the "runner" scene.

[0014] Figure 2 This is a flowchart of a video single-exposure compression imaging compression rate adjustment method based on deep reinforcement learning.

[0015] Figure 3 This is a logical diagram of the training process for the intelligent agent.

[0016] Figure 4 It is the result of a trained agent adjusting the compression rate and the corresponding video reconstruction quality simply by adjusting the compression measurement value.

[0017] Figure 5 It is the actual compression measurement value and the compression ratio selected by the agent based on the current measurement value. Detailed Implementation

[0018] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0019] like Figure 2 As shown in this embodiment, the video single-exposure compression imaging compression ratio adjustment method based on deep reinforcement learning includes the following steps: Step 1: Model the single-exposure compression imaging process of video using Markov dynamic decision process modeling. Define the compression result (i.e., the compression measurement value) at each time step as the current state, define the compression rate at the next time step as the action, and consider both the compression rate and the video reconstruction quality in the evaluation of the action reward. The five core parameters of a Markov dynamic decision process are defined as follows: (1) The state in the decision-making process: the time step t Compression measurement value Y at time t As t state of time s t ,Right nows t =Y t ; (2) Actions in the decision-making process: time steps t The compression measurement value Y at time +1 t+1 Corresponding compression ratio B t As t Moment of action a t ,Right now a t = B t ; (3) Reward: A reward indicates that an action is taken. a t The reward after reaching the next state t Momentary Rewards r t The calculation formula is as follows: in, L t+1 To reconstruct the video using a given video reconstruction network U t The peak signal-to-noise ratio (PSNR) of the reconstructed video at time +1. L bas The initial given value for the peak signal-to-noise ratio; e It is a parameter that needs to be adjusted.

[0020] The network structure of the video reconstruction network U is a hybrid network architecture based on transformer and convolutional neural network.

[0021] (4) Transition probability between states: The transition probability P represents the probability of transitioning between states when executing t Moment of action a t At that time, from the state s t arrive s t+1 The state transition probability, because a t = B t， s t =Y t Therefore, the transition probability P is expressed as P(Y). t+1 |Y t , B t ).

[0022] (5) Discount coefficient: γ Define the discount factor for calculating the cumulative return value function.

[0023] Step 2: Given a training set, use a deep reinforcement learning algorithm to train an agent that can adaptively adjust the compression rate at each time step by maximizing the cumulative reward in the Markov decision process modeled above.

[0024] The training data in the training set consists of video datasets of the main object moving at different speeds. These video datasets can be publicly available or downloaded from online sources. Given the training set, a Markov decision process model is first built, and state transition samples are collected. Then, using these state transition samples, the agent is trained through reinforcement learning, as follows: Figure 3 As shown. During training, the policy network and the estimation network are trained simultaneously using the temporal difference method.

[0025] In Markov decision-making, a policy network is a parameterized function approximator used to directly generate the action to be taken in a given state (i.e., the probability distribution of the action or the deterministic action).

[0026] In the video single-exposure compression imaging system of this embodiment, the policy network π State s As input, output a deterministic action.

[0027] Policy networks typically consist of convolutional layers and fully connected layers. The input is a state, i.e., a compressed measurement value, which is processed by weight parameters. θ The learning process maps states to actions. The training objective of the policy network is to adjust the weight parameters to maximize cumulative reward.

[0028] In the Markov decision process, network estimation Q It is a neural network used to approximate the true action-value function. Its core objective is to evaluate the long-term expected reward of each pair of state-action under a specific policy.

[0029] Estimating network Q It also consists of convolutional layers and fully connected layers. The input is a compressed measurement and the action output by the current policy network in response to this compressed measurement. The training objective is to minimize the estimation error of the action-value function of the current policy network.

[0030] The iterative method for estimating network Q is to minimize the following loss function: In this embodiment, the policy network is updated by maximizing... The output value.

[0031] Step 3: Use the trained agent to adaptively adjust the compression rate of single-exposure compression imaging in new scenes.

[0032] Based on the obtained sequence of compressed measurements, the compression ratio sequence, and the mask in the single-exposure compressed imaging system, video reconstruction is performed in the video reconstruction network U to finally obtain the video.

[0033] The video reconstruction network U takes compressed measurements and a mask from a single-exposure compressed imaging system as inputs and outputs the reconstructed video.

[0034] Figure 4 These are test results from a simulation experiment; the figure shows different compression ratios. B The corresponding PSNR of the decoded video and the PSNR of the decoded video corresponding to the compression ratio adaptively adjusted by the policy grid trained through reinforcement learning (RL). It can be seen that when different compression ratios... B When the corresponding PSNR values ​​are relatively close, the trained policy network will choose a higher compression ratio to compress the scene. When the PSNR values ​​of different compression ratios differ significantly, the trained policy network will choose a lower compression ratio to ensure the quality of the decoded video.

[0035] Figure 5 The compression measurements in a real single-exposure compression imaging system and the compression ratio selected by the agent to construct subsequent compression measurements based on these compression measurements. B '.like Figure 5 In (a), when the target is stationary, the scene in the compressed measurement is clear, and the trained policy network will set... B =12; for example Figure 5 In (b), when the target moves slowly, the scene in the compressed measurement is relatively blurry, and the trained policy network will set... B =8; for example Figure 5 In (c), when the target moves quickly, the scene in the compressed measurement is very blurry. The trained policy network will set B'=4, thus proving that the method of the present invention can adaptively conditionally compress the compression rate and dynamically balance the compression rate and video reconstruction quality.

[0036] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for adjusting the compression rate of single-exposure compressed imaging of video based on deep reinforcement learning, characterized in that, Includes the following steps: S1: Model the single-exposure compression imaging process of video using Markov dynamic decision process modeling. Define the compression measurement value at each time step as the current state, define the compression rate at the next time step as the action, and consider both the compression rate and the video reconstruction quality in the evaluation of the action reward. S2: Given a training set, use a deep reinforcement learning algorithm to train an agent that can adaptively adjust the compression rate at each time step by maximizing the cumulative reward in the Markov decision process modeled above. S3: Use the trained agent to adaptively adjust the compression rate in a single-exposure compression imaging system for new scenarios.

2. The video single-exposure compression imaging compression rate adjustment method based on deep reinforcement learning according to claim 1, characterized in that, In step S1, when modeling the Markov dynamic decision process for the single-exposure compressed imaging process of video, the calculation formula for the reward of the action reward is as follows: in, B t Indicates time step t Compression ratio at any moment e It is a parameter that needs to be adjusted. L t+1 To reconstruct the video using a given video reconstruction network t Peak signal-to-noise ratio of the reconstructed video at time +1. L bas The initial peak signal-to-noise ratio is given.

3. The video single-exposure compression imaging compression rate adjustment method based on deep reinforcement learning according to claim 2, characterized in that, The given video reconstruction network is a hybrid network architecture based on transformer and convolutional neural network.

4. The video single-exposure compression imaging compression rate adjustment method based on deep reinforcement learning according to claim 1, characterized in that, The training set in S2 is a video dataset of main objects moving at different speeds.

5. The video single-exposure compression imaging compression rate adjustment method based on deep reinforcement learning according to claim 1, characterized in that, During the training of S2, the policy network and estimation grid of the Markov decision process are trained simultaneously using the temporal difference method.

Images