Video enhancement method and device, computer equipment and storage medium
By constructing a closed-loop feedback mechanism through video state estimation and reinforcement learning controller, and dynamically adjusting policy actions, this solves the problem of lack of automated evaluation and dynamic decision-making in existing video augmentation methods, and achieves autonomous learning and efficient video augmentation.
Patent Information
- Application Number
- CN202511144252.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-15
- Publication Date
- 2025-11-21
AI Technical Summary
Existing video enhancement methods lack automated evaluation and feedback capabilities, rely on ground truth data which is difficult to obtain, cannot achieve globally optimal results, and cannot dynamically make decisions and configure algorithm strategies based on video features.
The video state estimation module extracts state vectors, and reinforcement learning (RL) controllers and actuators are used for video enhancement. A closed-loop feedback mechanism is constructed to dynamically adjust policy actions. The system is then trained in conjunction with a value network to achieve autonomous learning and optimization.
It achieves automatic improvement of video enhancement effects even without ground truth data, adapts to different scenarios and content types, and has high portability and robustness.
Smart Images
Figure CN120997059A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of video enhancement, and in particular to a video enhancement method and device, a computer device and a storage medium. BACKGROUND
[0002] Most conventional video enhancement methods do not have automatic evaluation feedback capability; some systems have feedback capability, but rely on true value data, but true value data is difficult to obtain in actual ultra-high definition content production and operation systems, and the video enhancement effect is poor; at the same time, the conventional method cannot dynamically decide and configure algorithm calling strategies according to video features to achieve a globally optimal effect. SUMMARY
[0003] Therefore, the present application aims to overcome the deficiencies in the prior art and provides a video enhancement method, device, computer device and storage medium for dynamically deciding and configuring algorithm strategies according to video content states to realize differentiated processing of different types of videos.
[0004] The present application provides the following technical solutions: In a first aspect, the present application provides a video enhancement method, comprising: inputting an original video segment into a video state estimation module to obtain a state vector; inputting the state vector into an initial RL controller to obtain a policy action; inputting the policy action into an RL executor to obtain an enhanced video; training the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller; and performing video enhancement through the target RL controller.
[0005] In an embodiment, the step of inputting an original video segment into a video state estimation module to obtain a state vector comprises: inputting the original video segment into the video state estimation module to obtain an average brightness distribution, an average motion intensity, a flicker degree, a semantic scene and a feature code; fusing the average brightness distribution, the average motion intensity, the flicker degree, the semantic scene and the feature code to obtain the state vector.
[0006] In an embodiment, the initial RL controller comprises an initial policy network, and the step of inputting the state vector into an initial RL controller to obtain a policy action comprises: inputting the state vector into the initial policy network to obtain the policy action.
[0007] In an embodiment, the RL executor comprises a policy path decoder, a video enhancement module stack and a video generation module stack, the inputting the policy action into the RL executor to obtain an enhanced video comprises: generating executable instructions according to the policy action by the policy path decoder; obtaining the enhanced video according to the executable instructions by the video enhancement module stack and the video generation module stack.
[0008] In an embodiment, the initial RL controller further comprises a value network, the training the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller comprises: obtaining a fusion reward result according to the enhanced video; taking the state vector, the policy action, the enhanced video and the fusion reward result as a training sample; calculating an action value according to the training sample by the value network; updating the initial policy network according to the action value to obtain a target policy network, taking the target policy network as the target RL controller.
[0009] In an embodiment, the obtaining a fusion reward result according to the enhanced video comprises: obtaining a video quality evaluation result, a running efficiency result and a similarity result according to the enhanced video; obtaining a fusion reward result according to the video quality evaluation result, the running efficiency result and the similarity result.
[0010] In an embodiment, the method further comprises: calculating a TD target value according to the action value, the calculation formula of the TD target value is , wherein is the TD target value, is a fusion reward result, the is a discount factor, is a state vector corresponding to a next original video segment, is a policy action corresponding to the next original video segment, is an action value corresponding to the next original video segment; calculating a loss value according to the action value and the TD target value; updating the value network according to the loss value.
[0011] In a second aspect, the present application provides a video enhancement device, comprising: a state estimation module, configured to input the original video clip into a video state estimation module to obtain a state vector; an action generation module, configured to input the state vector into an initial RL controller to obtain a policy action; a video enhancement module, configured to input the policy action into an RL executor to obtain an enhanced video; an update processing module, configured to train the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller, and perform video enhancement through the target RL controller.
[0012] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the video enhancement method in the first aspect.
[0013] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the video enhancement method in the first aspect.
[0014] The video enhancement method, device, computer device and storage medium disclosed by the present application input an original video clip into a video state estimation module to obtain a state vector, input the state vector into an initial RL controller to obtain a policy action, input the policy action into an RL executor to obtain an enhanced video, train the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller, and perform video enhancement through the target RL controller. In this way, a closed-loop feedback is constructed by using a state vector and a reward function, and the dependence on external true value data is eliminated. The RL controller automatically improves the video enhancement effect through continuous interaction and learning. The policy action can be dynamically adjusted to adapt to different scenes and content types. BRIEF DESCRIPTION OF DRAWINGS
[0015] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of protection of the present application. In each drawing, similar components are denoted by similar reference numerals.
[0016] Figure 1 Fig. 1 shows a flow diagram of the video enhancement method proposed in the present embodiment; Figure 2 Fig. 2 shows another flow diagram of the video enhancement method proposed in the present embodiment; Figure 3 Fig. 3 shows a structural diagram of the video enhancement device proposed in the present embodiment.
[0017] Brief Description of the Drawings 300 - video enhancement device; 301 - state estimation module; 302 - action generation module; 303 - video enhancement module; 304 - update processing module. DETAILED DESCRIPTION
[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application.
[0019] The components of the embodiments of the present application generally described and illustrated herein can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.
[0020] Hereinafter, the terms "include", "have", and their conjugates used in various embodiments of the present application are only intended to denote a certain characteristic, number, step, operation, element, component, or a combination thereof, and should not be construed as excluding the presence or addition of one or more other characteristics, numbers, steps, operations, elements, components, or combinations thereof.
[0021] In addition, the terms "first", "second", "third", and the like are used only to distinguish descriptions, and should not be understood as indicating or implying relative importance.
[0022] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present application belong. The terms (such as those defined in commonly used dictionaries) will be interpreted as having a meaning that is the same as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning, unless clearly defined in various embodiments of the present application.
[0023] Embodiment 1 The embodiments of the present disclosure provide a video enhancement method for dynamically deciding and configuring algorithm strategies according to video content states, to realize differentiated processing of different types of videos.
[0024] Please refer to Figure 1 The video enhancement method includes steps S101-S104, which will be described in detail below.
[0025] Step S101, input the original video clip into the video state estimation module to obtain a state vector.
[0026] In this embodiment, the original video clip is input into the video state estimation module for feature extraction to obtain a state vector for reinforcement learning.
[0027] In a specific embodiment, step S101 includes: inputting the original video clip into the video state estimation module to obtain an average brightness distribution, an average motion intensity, a flicker degree, a semantic scene, and a feature code; and fusing the average brightness distribution, the average motion intensity, the flicker degree, the semantic scene, and the feature code to obtain the state vector.
[0028] In this embodiment, the original video clip is input into the video state estimation module to obtain an average brightness distribution, an average motion intensity, a flicker degree, a semantic scene, and a feature code, and the like. The average brightness distribution is a gray histogram mean and a standard deviation; the average motion intensity is an average optical flow amplitude, which can be obtained using a RAFT, a TV-L1, or the like; the flicker degree is a difference in brightness between adjacent frames; the semantic scene is a type of scene description, which can use a CLIP model to extract a text embedding; and the feature code is an image embedding extracted using a pre-trained model.
[0029] Further, the average brightness distribution, the average motion intensity, the flicker degree, the semantic scene, and the feature code are fused to obtain the state vector. The feature fusion formula is: wherein, .
[0030] Step S102, input the state vector into an initial RL controller to obtain a policy action.
[0031] In this embodiment, the state vector is input into the initial RL controller to obtain a policy action. One action representation includes the following fields: action type, module name, region coordinates, parameter code, whether to repeat, number of repetitions, and whether it is a final action. Action generation can generate a policy through enumeration, or a policy network can be trained using an actor critic architecture, such as A2C (Advantage Actor-Critic), TD3 (Twin Delayed DDPG), and SAC (Soft Actor-Critic).
[0032] In a specific embodiment, the initial RL controller includes an initial policy network, and step 102 includes: inputting the state vector into the initial policy network to obtain the policy action.
[0033] In the embodiment, the state vector is input into the initial policy network to obtain a policy action, so as to map a multi-dimensional continuous state into an optimizable action, and to dynamically determine which enhancement modules to use and the parameter configuration of each module through the policy network, so as to construct a combinable and adjustable enhancement path.
[0034] In step S103, the policy action is input into an RL executor to obtain an enhanced video.
[0035] In the embodiment, the policy action is input into the RL executor, and the RL executor adjusts to obtain the enhanced video according to the action parameter in the policy action.
[0036] In a specific embodiment, the RL executor includes a policy path decoder, a video enhancement module stack and a video generation module stack, and step S103 includes: generating executable instructions through the policy path decoder according to the policy action; and obtaining the enhanced video through the video enhancement module stack and the video generation module stack according to the executable instructions.
[0037] In the embodiment, the policy action is parsed through the policy path decoder to obtain executable instructions. The executable instructions include a video processing sequence (such as whether to interpolate first or to denoise first), a video processing specific process (such as an HDR enhancement degree, a video style effect, etc.), etc.
[0038] Further, the video enhancement module stack and the video generation module stack are decoupled, and the algorithm modules are independent of each other and can be interleaved. The video enhancement module stack and the video generation module stack are called according to the video processing information contained in the executable instructions to generate the enhanced video. The video enhancement module stack includes a super resolution (SR), a denoising (Denoise), an interpolation (Interpolation), a color enhancement (Color Enhancement), a dynamic range expansion (HDR Mapping), etc. The video generation module stack includes a style transfer, a conditional video generation, a text-to-video generation, an image-to-video generation, an action synthesis, a video editing, etc.
[0039] In step S104, the initial RL controller is trained according to the state vector, the policy action and the enhanced video to obtain a target RL controller, and the target RL controller is used for video enhancement.
[0040] In the embodiment, the initial RL controller is trained according to the state vector, the policy action and the enhanced video to obtain the target RL controller, and data is obtained from historical operation data of an ultra-high-definition content production platform, so that the quality of the generated video can be continuously improved without relying on true value data, and a real-time inference enhancement strategy is deployed, which has high portability and system robustness.
[0041] Please refer to Figure 2 In a specific embodiment, the initial RL controller further comprises a value network, and step S104 comprises steps S1041-S1044, which are described in detail as follows.
[0042] In step S1041, a fusion reward result is obtained according to the enhanced video.
[0043] In this embodiment, the enhanced video is evaluated in multiple dimensions to obtain the fusion reward result.
[0044] In a specific embodiment, step S1041 comprises: obtaining a video quality evaluation result, a running efficiency result and a similarity result according to the enhanced video; and obtaining a fusion reward result according to the video quality evaluation result, the running efficiency result and the similarity result.
[0045] In this embodiment, a video quality evaluation result, a running efficiency result and a similarity result are obtained according to the enhanced video. The video quality evaluation result includes human subjective scoring, no-reference quality evaluation (such as natural image quality evaluator, blind reference image spatial quality evaluator), and indirect evaluation indicators (such as video click rate and collection rate); the running efficiency evaluation result evaluates running speed and punishes paths with slow running efficiency, including running speed and parameter quantity; the similarity evaluation evaluates whether the generated result is consistent with the information provided by the user in the generation task, including human subjective scoring.
[0046] Further, a fusion reward result is obtained according to the video quality evaluation result, the running efficiency result and the similarity result. The calculation formula of the fusion reward result is: wherein each scoring item is represented by , .
[0047] The no-reference image quality evaluation, inter-frame consistency score, subjective scoring model and running efficiency are introduced to construct the reward function, so as to realize quality-driven strategy optimization.
[0048] In step S1042, the state vector, the policy action, the enhanced video and the fusion reward result are taken as training samples.
[0049] In this embodiment, the state vector , the policy action , the enhanced video and the fusion reward result are taken as training samples.
[0050] Step S1043, calculating the action value according to the training sample by the value network.
[0051] In this embodiment, the action value is calculated by the value network according to the training sample .
[0052] Step S1044, updating the initial policy network according to the action value to obtain a target policy network, and taking the target policy network as the target RL controller.
[0053] In this embodiment, the initial policy network is updated according to the action value to obtain a target policy network, and the target policy network is taken as the target RL controller. The initial policy network is updated by using the derivative of the action value to select a higher Q value action, and the update formula is: .
[0054] When the target policy network is trained, the target policy network is used as the target RL controller for video enhancement inference. Illustratively, the video to be enhanced is input into the state estimation module to extract the state vector corresponding to the video to be enhanced; the state vector corresponding to the video segment to be enhanced is input into the target RL controller to output the policy action corresponding to the video segment to be enhanced; the RL executor generates the enhanced video segment corresponding to the video to be enhanced according to the policy action corresponding to the video segment to be enhanced; the above steps are repeated until the complete enhanced video is obtained.
[0055] In a specific embodiment, the method further comprises: calculating a TD target value according to the action value; calculating a loss value according to the action value and the TD target value; and updating the value network according to the loss value.
[0056] In this embodiment, the TD target value is calculated according to the action value , and the calculation formula of the TD target value is , is the state vector corresponding to the next original video segment, is the policy action corresponding to the next original video segment.
[0057] Further, the loss value is calculated by using a loss function , and the value network is updated according to the loss value.
[0058] It should be noted that if the initial RL controller only includes the value network, the policy action is generated by enumeration , and the network parameters are updated by using the formula , wherein is the learning rate. The target network, experience replay mechanism can also be used to improve the stability of training.
[0059] It should be noted that the text encoder (such as CLIP) is used to generate text embedding according to the text auxiliary information, and the picture encoder (such as ViT) is used to generate picture embedding according to the image auxiliary information; the text embedding and the picture embedding are input into the RL executor and the reward evaluation feedback module respectively, which are used to assist network training, so as to improve the video enhancement accuracy.
[0060] The video enhancement method provided in the embodiment inputs the original video segment into the video state estimation module to obtain a state vector; inputs the state vector into the initial RL controller to obtain a policy action; inputs the policy action into the RL executor to obtain an enhanced video; trains the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller; and performs video enhancement through the target RL controller. In this way, the state vector and the reward function are used to construct a closed-loop feedback, so as to get rid of the dependence on external true value data; the RL controller automatically improves the video enhancement effect through continuous interaction and learning; and the policy action can be dynamically adjusted to adapt to different scenes and content types.
[0061] Embodiment 2 In addition, the embodiment of the present disclosure provides a video enhancement device 300, please see Figure 3 , comprising: The state estimation module 301 is configured to input the original video segment into the video state estimation module to obtain a state vector. The action generation module 302 is configured to input the state vector into the initial RL controller to obtain a policy action. The video enhancement module 303 is configured to input the policy action into the RL executor to obtain an enhanced video. The update processing module 304 is configured to train the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller; and perform video enhancement through the target RL controller.
[0062] Optionally, the state estimation module 301 is further configured to input the original video segment into the video state estimation module to obtain an average brightness distribution, an average motion intensity, a flicker degree, a semantic scene and a feature code; and fuse the average brightness distribution, the average motion intensity, the flicker degree, the semantic scene and the feature code to obtain the state vector.
[0063] Optionally, the initial RL controller comprises an initial policy network, and the action generation module 302 is further configured to input the state vector into the initial policy network to obtain the policy action.
[0064] Optionally, the RL executor comprises a policy path decoder, a video enhancement module stack and a video generation module stack, and the video enhancement module 303 is further configured to generate executable instructions according to the policy action through the policy path decoder, and obtain the enhanced video according to the executable instructions through the video enhancement module stack and the video generation module stack.
[0065] Optionally, the initial RL controller further comprises a value network, and the update processing module 304 is further configured to obtain a fusion reward result according to the enhanced video, take the state vector, the policy action, the enhanced video and the fusion reward result as a training sample, calculate an action value according to the training sample through the value network, update the initial policy network according to the action value to obtain a target policy network, and take the target policy network as the target RL controller.
[0066] Optionally, the update processing module 304 is further configured to obtain a video quality evaluation result, a running efficiency result and a similarity result according to the enhanced video, and obtain a fusion reward result according to the video quality evaluation result, the running efficiency result and the similarity result.
[0067] Optionally, the update processing module 304 is further configured to calculate a TD target value according to the action value, and a calculation formula of the TD target value is , wherein, is the TD target value, is a fusion reward result, is a discount factor, is a state vector corresponding to a next original video segment, is a policy action corresponding to the next original video segment, is an action value corresponding to the next original video segment; calculate a loss value according to the action value and the TD target value; and update the value network according to the loss value.
[0068] The apparatus provided by the embodiments of the present disclosure can execute the steps of the video enhancement method provided in Embodiment 1, and thus will not be described again.
[0069] The video enhancement device provided in the embodiment inputs an original video segment into a video state estimation module to obtain a state vector, inputs the state vector into an initial RL controller to obtain a policy action, inputs the policy action into an RL executor to obtain an enhanced video, trains the initial RL controller according to the state vector, the policy action and the enhanced video to obtain a target RL controller, and performs video enhancement through the target RL controller. In this way, a closed-loop feedback is constructed by using a state vector and a reward function, and dependence on external true value data is eliminated. The RL controller automatically improves video enhancement effect through continuous interaction and learning. The policy action can be dynamically adjusted to adapt to different scenes and content types.
[0070] Embodiment 3 In addition, the embodiment of the present disclosure provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the video enhancement method in the embodiment 1.
[0071] The device provided in the embodiment of the present disclosure can execute the steps of the video enhancement method provided in the embodiment 1, and thus will not be described again.
[0072] Embodiment 4 The embodiment of the present disclosure provides a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the video enhancement method in the embodiment 1.
[0073] In the embodiment, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0074] The computer readable storage medium provided in the embodiment can implement the video enhancement method provided in the embodiment 1, and thus will not be described again.
[0075] In all the examples shown and described herein, any specific value should be interpreted as merely exemplary and not as a limitation, and thus other examples of the example embodiments can have different values.
[0076] It should be noted that similar reference numbers and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0077] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application.
Claims
1. A video enhancement method, characterized in that, include: The original video clip is input into the video state estimation module to obtain the state vector; The state vector is input to the initial RL controller to obtain the policy action; The strategy action is input into the RL executor to obtain the enhanced video; The initial RL controller is trained based on the state vector, the policy action, and the enhanced video to obtain the target RL controller; video enhancement is then performed using the target RL controller.
2. The video enhancement method according to claim 1, characterized in that, The step of inputting the original video segment into the video state estimation module to obtain the state vector includes: The original video clip is input into the video state estimation module to obtain the average brightness distribution, average motion intensity, flicker level, semantic scene, and feature encoding; The average brightness distribution, the average motion intensity, the flickering degree, the semantic scene, and the feature encoding are fused to obtain the state vector.
3. The video enhancement method according to claim 1, characterized in that, The initial RL controller includes an initial policy network. The step of inputting the state vector into the initial RL controller to obtain policy actions includes: The state vector is input into the initial policy network to obtain the policy action.
4. The video enhancement method according to claim 1, characterized in that, The RL executor includes a policy path decoder, a video enhancement module stack, and a video generation module stack. The step of inputting the policy action into the RL executor to obtain the enhanced video includes: The policy path decoder generates executable instructions based on the policy action. The enhanced video is obtained by the video enhancement module stack and the video generation module stack according to the executable instructions.
5. The video enhancement method according to claim 3, characterized in that, The initial RL controller further includes a value network. The process of training the initial RL controller based on the state vector, the policy action, and the augmented video to obtain the target RL controller includes: The fusion reward result is obtained based on the enhanced video; The state vector, the policy action, the enhanced video, and the fusion reward result are used as training samples. The value network calculates the action value based on the training samples; The initial policy network is updated based on the action value to obtain the target policy network, which is then used as the target RL controller.
6. The video enhancement method according to claim 5, characterized in that, The step of obtaining the fusion reward result based on the enhanced video includes: Based on the enhanced video, video quality evaluation results, operational efficiency results, and similarity results are obtained; The fusion reward result is obtained based on the video quality evaluation result, the operational efficiency result, and the similarity result.
7. The video enhancement method according to claim 5, characterized in that, The method further includes: The TD target value is calculated based on the action value, and the formula for calculating the TD target value is as follows: In the formula, For the TD target value, In order to integrate the reward results, the above, As a discount factor, This is the state vector corresponding to the next original video segment. The strategy action corresponding to the next original video segment. The action value corresponding to the next original video segment; Calculate the loss value based on the action value and the TD target value; The value network is updated based on the loss value.
8. A video enhancement device, characterized in that, include: The state estimation module is used to input the original video clip into the video state estimation module to obtain the state vector; An action generation module is used to input the state vector into the initial RL controller to obtain policy actions; The video enhancement module is used to input the policy action to the RL executor to obtain the enhanced video; An update processing module is used to train the initial RL controller based on the state vector, the policy action, and the enhanced video to obtain a target RL controller; and to perform video enhancement through the target RL controller.
9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the video enhancement method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the video enhancement method as described in any one of claims 1 to 7.