A video compression hyperparameter tuning method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a video compression hyperparameter optimization method, device and equipment and a storage medium. The method comprises: obtaining demand information and a prompt word template of video compression; inputting at least one frame of image of a video to be compressed and the prompt word template into a visual large model to output description information of visual features of the video to be compressed, and converting the description information into a structured feature vector; determining a reward function according to the demand information; inputting the structured feature vector into a pre-trained deep reinforcement learning agent, and taking maximization of a reward value output by the reward function as an optimization target to output optimal compression hyperparameters matched with the video to be compressed; and encoding and compressing the video to be compressed based on the optimal compression hyperparameters. Based on semantic perception of the visual large model, precise identification and differentiated protection of sensitive compression areas are realized, an end-to-end mapping from semantic features to compression parameters is established by deep reinforcement learning, and overall compression efficiency and perception quality are improved.

Description

Technical Field

[0001] This invention relates to the field of video compression technology, and in particular to a video compression hyperparameter tuning method, a video compression hyperparameter tuning device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of 5G / 6G communication and artificial intelligence technologies, new video formats such as 4K / 8K ultra-high-definition video, virtual reality (VR), and augmented reality (AR) are widely used in education, entertainment, transportation, industry, and other fields, resulting in an explosive growth in video data. To alleviate the enormous pressure on transmission bandwidth and storage resources caused by massive amounts of heterogeneous video data, video compression coding technology has become a key supporting element.

[0003] Existing technologies, such as clustering and sub-block encoding replacement, can improve the compression rate to some extent by performing pre-processing on similar images. However, they rely on pixel-level difference indicators for block decisions, which can easily lead to unreasonable block division and local image quality loss in dynamic scenes. Furthermore, the processing latency increases when the computing power of edge devices is limited. In addition, they lack a deep understanding of the semantic content of the video and a key area recognition mechanism, making it difficult to distinguish compression-sensitive areas such as faces, text, and fine textures, which poses a risk of misjudging core information and causing distortion. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a video compression hyperparameter tuning method, a video compression hyperparameter tuning device, an electronic device, and a computer-readable storage medium that overcome or at least partially solve the above problems.

[0005] To address the above problems, a first aspect of this invention provides a video compression hyperparameter tuning method, the method comprising: Obtain video compression requirement information and prompt word templates; the prompt word templates include visual features of the video to be compressed in compression-related dimensions, the visual features including video image attributes and content semantics; the visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate visual feature descriptions in natural language form; The visual big model is input with at least one frame of the video to be compressed and the prompt word template, and the visual feature description information of the video to be compressed is output and the description information is converted into a structured feature vector. The reward function is determined based on the demand information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The structured feature vector is input into a pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as its optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The video to be compressed is encoded and compressed based on the optimal compression hyperparameters.

[0006] According to a second aspect of the present invention, a video compression hyperparameter tuning apparatus is provided, the apparatus comprising: The requirement information acquisition module is used to acquire the requirement information and prompt word template for video compression. The prompt word template includes visual features of the video to be compressed in compression-related dimensions, including video image attributes and content semantics. The visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate a visual feature description in natural language. The description information output module is used to input at least one frame of the video to be compressed and the prompt word template into the visual big model, output the description information of the visual features of the video to be compressed, and convert the description information into a structured feature vector. The reward function determination module is used to determine the reward function based on the requirement information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The compression parameter determination module is used to input the structured feature vector into a pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The video encoding and compression module is used to encode and compress the video to be compressed based on the optimal compression hyperparameters.

[0007] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the video compression hyperparameter tuning method as described in any of the preceding claims.

[0008] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the video compression hyperparameter tuning method as described in any of the preceding claims.

[0009] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a method, apparatus, device, and storage medium for video compression hyperparameter tuning. The method includes: acquiring video compression requirement information and a prompt word template; the prompt word template includes visual features of the video to be compressed in compression-related dimensions, including video image attributes and content semantics; the visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate a visual feature description in natural language; at least one frame of the video to be compressed and the prompt word template are input into the visual model, outputting a description of the visual features of the video to be compressed, and converting the description information into a structured feature vector; determining a reward function based on the requirement information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters; the structured feature vector is input into a pre-trained deep reinforcement learning agent, which optimizes by maximizing the reward value output by the reward function and outputs the optimal compression hyperparameters matching the video to be compressed; and encoding and compressing the video to be compressed based on the optimal compression hyperparameters. By using a large visual model and prompt word templates to guide the model in visual semantic analysis of video images, the system can automatically extract visual features related to compression dimensions, accurately locate key areas with a high risk of distortion after compression, and determine the reward function based on user needs. This allows the compression strategy to be adaptively adjusted according to different application scenarios (such as data center monitoring prioritizing compression rate, and highway lanes prioritizing image quality). By learning the mapping relationship between a large number of video features and hyperparameter combinations, the agent can quickly adjust parameters based on content semantic features when facing dynamic scenes, avoiding the unreasonable division of motion regions by traditional block-based methods. Attached Figure Description

[0010] Figure 1 This is a flowchart of the steps of a video compression hyperparameter tuning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the technical architecture of a video compression hyperparameter tuning method provided in an embodiment of the present invention; Figure 3 This is a flowchart of another video compression hyperparameter tuning method provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating a video compression hyperparameter tuning method provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a video compression hyperparameter tuning device provided in an embodiment of the present invention. Detailed Implementation

[0011] To make the above-mentioned objects, features and advantages of the present invention more readily understood, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0012] Existing technologies, such as clustering and sub-block encoding replacement, can improve the compression rate to some extent by performing pre-processing on similar images. However, they rely on pixel-level difference indicators for block decisions, which can easily lead to unreasonable block division and local image quality loss in dynamic scenes. Furthermore, the processing latency increases when the computing power of edge devices is limited. In addition, they lack a deep understanding of the semantic content of the video and a key area recognition mechanism, making it difficult to distinguish compression-sensitive areas such as faces, text, and fine textures, which poses a risk of misjudging core information and causing distortion.

[0013] One of the core concepts of this invention is that, through a large visual model, prompt word templates guide the large visual model to perform visual semantic analysis on video images, automatically extracting visual features related to compression dimensions, accurately locating key areas with a high risk of distortion after compression, and determining the reward function based on user needs. This allows the compression strategy to be adaptively adjusted according to different application scenarios (such as data center monitoring prioritizing compression rate, and highway lanes prioritizing image quality). By learning the mapping relationship between a large number of video features and hyperparameter combinations, the intelligent agent can quickly adjust parameters based on content semantic features when facing dynamic scenes, avoiding the unreasonable division of motion regions by traditional block-based methods.

[0014] Reference Figure 1 The diagram illustrates a flowchart of a video compression hyperparameter tuning method provided by an embodiment of the present invention. The method specifically includes the following steps: Step 101: Obtain video compression requirement information and prompt word templates; the prompt word templates include visual features of the video to be compressed in compression-related dimensions, the visual features include video image attributes and content semantics; the visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate visual feature descriptions in natural language form. The requirement information is a constrained description provided by the user when initiating a video compression task, outlining the compression goals, application scenarios, and expected results. It represents the user's preference between compression efficiency and video quality and serves as a prerequisite input for the reward function configuration. This primarily includes: Compression efficiency preference, i.e., the user's expected compression rate, such as target bitrate, target file size, or compression ratio requirements, specifically "desiring a 30% reduction in bitrate" or "file size not exceeding 100 megabytes." Video quality preference, i.e., the user's requirements for image fidelity, such as the minimum acceptable peak signal-to-noise ratio or structural similarity index, or subjective quality level, specifically "peak signal-to-noise ratio not lower than 38 dB" or "maintaining no distortion in facial areas." Application scenario type, i.e., the source and purpose of the video, which implicitly imposes specific requirements on the compression strategy; for example, "computer room monitoring" scenarios prioritize compression efficiency, "highway lanes" scenarios prioritize dynamic image quality protection, and "medical imaging" scenarios prioritize the fidelity of critical areas. Special protection requirements, i.e., the key target types for which image quality must be prioritized, such as "faces need protection," "license plates need protection," or "text areas need protection." Constraints, i.e. hardware or time limitations, such as "encoding time not exceeding twice the real-time frame rate" or "running on an edge device".

[0015] To address the diverse video compression needs across various scenarios, a standardized Zero-Shot prompt template was designed. This template is a pre-designed set of standardized natural language instructions used to guide a large-scale visual model in performing specific-dimensional visual semantic analysis of the image content of the video to be compressed. This template is not an arbitrary input natural language question, but rather a structured instruction framework designed specifically for video compression scenarios. Its function is to enable the large-scale visual model to output visual feature descriptions that are relevant to compression decisions and can be subsequently quantified.

[0016] The prompt word template guides the visual model to extract visual features of the video to be compressed in compression-related dimensions. These features include two categories: video image attributes, which are objective technical attributes at the video signal level, such as resolution, frame rate, image complexity, motion intensity, and texture richness. These attributes reflect the signal characteristics of the video itself and are directly related to the selection of compression parameters. For example, high-motion-intensity videos require larger keyframe intervals or finer bitrate allocation to avoid motion blur, while high-texture-complexity videos require finer quantization parameters to preserve details. Content semantics refers to high-level semantic information in the video image, such as key target categories, distribution of sensitive areas, scene type, and distortion risk type. This semantic information reflects "what the content is" and "what is important" of the video, enabling subsequent compression strategies to distinguish between compression-sensitive areas such as faces, text, vehicles, and fine textures from ordinary background areas.

[0017] In this embodiment of the invention, the video compression hyperparameter tuning method first obtains two types of input information: user requirements for video compression and a pre-designed prompt word template. The requirements information is a constraint description provided by the user for the current compression task, including compression efficiency preferences, video quality preferences, application scenario types, special protection requirements, and hardware constraints. This requirements information serves as the basis for subsequent reward function configuration, enabling the compression strategy to be personalized and adaptively adjusted according to the user's diverse needs, rather than adopting a fixed compression strategy. The prompt word template is a pre-designed set of structured natural language instructions specifically designed to guide a large visual model to extract visual features of the video to be compressed in compression-related dimensions. This prompt word template does not rely on training data for a specific task and can directly convert feature extraction requirements into cross-modal instructions understandable by the large visual model under zero-shot conditions, ensuring the universality and adaptability of the solution in different application scenarios.

[0018] Step 102: Input at least one frame of the video to be compressed and the prompt word template into the visual big model, output the description information of the visual features of the video to be compressed, and convert the description information into a structured feature vector; In this embodiment of the invention, after obtaining the demand information and the prompt word template, at least one frame is extracted from the video to be compressed as a keyframe. This keyframe, along with the prompt word template, is input into a large-scale visual model. The large-scale visual model performs visual semantic analysis on the video images and outputs visual feature description information in natural language form. Keyframe selection methods include at least one of the following: extracting the first and last frames of the video, extracting frames at fixed time intervals, and extracting abrupt change frames based on scene transition detection. The specific number of keyframes is determined based on the context window capacity of the selected large-scale visual model. For long videos, keyframe extraction is used for content representation instead of inputting all frames to ensure the processing efficiency of the large-scale visual model and the representativeness of the input information.

[0019] The visual big data model does not perform traditional image classification or object detection tasks. Instead, based on its cross-modal semantic understanding capabilities, it transforms the "seen" image content into "written" text descriptions. This includes image attribute information such as video resolution, frame rate, image complexity, motion intensity, and texture richness, as well as semantic information such as the category and location of key objects, the distribution and proportion of sensitive areas, scene category, and the types of distortion that are prone to occur at low bitrates. This descriptive information is presented in natural language, such as "high image complexity," "medium motion intensity," "contains a face region, accounting for approximately 20%," and "this video is prone to blockiness and blurring at low bitrates." Subsequently, the natural language descriptions output by the large visual model are converted into structured feature vectors. Qualitative descriptions are quantified into numerical forms; for example, "high image complexity" is quantified as a complexity score of 4 (out of 5), "moderate motion intensity" as a motion intensity score of 3 (out of 5), and the proportion of sensitive areas is directly quantified into values ​​between 0 and 1. Scene types are encoded into corresponding category indices, ultimately forming a structured feature vector composed of multiple numerical dimensions, which serves as the standard input format for subsequent deep reinforcement learning agents. Through this step, previously incalculable semantic information is transformed into quantifiable and computable structured representations, providing accurate quantitative basis for refined compression control.

[0020] Step 103: Determine the reward function based on the demand information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The reward function is a mathematical function used to quantitatively evaluate the quality of compression hyperparameter combinations. Its input is the candidate compression hyperparameter combinations output by the deep reinforcement learning agent, and its output is a scalar reward value, reflecting the degree of compression effectiveness under that hyperparameter combination. In the deep reinforcement learning framework, the agent's training and inference objectives are both to maximize the cumulative reward value; therefore, the reward function directly determines whether the agent's optimization direction meets the user's expectations.

[0021] The reward function is constructed based on user requirements and includes: a compression efficiency term, which measures the degree of data reduction in the compressed video, reflecting the effectiveness of the compression hyperparameter combination in saving storage space and transmission bandwidth, typically quantified based on compression ratio; a video quality term, which measures the fidelity of the compressed video, reflecting the effectiveness of the compression hyperparameter combination in maintaining the integrity of visual information, typically quantified based on peak signal-to-noise ratio or structural similarity metrics; and a system penalty term, which penalizes excessively long compression times to prevent excessive reduction in compression efficiency. When the compression time exceeds the baseline time, this term outputs a positive penalty, indicating that while the hyperparameter combination may perform well in terms of compression ratio or image quality, it comes at an excessive time cost, thus guiding the agent to seek a balance between compression ratio, image quality, and time.

[0022] In this embodiment of the invention, after obtaining the structured feature vector of the video to be compressed, a reward function is determined based on the acquired user demand information. This reward function is the core evaluation signal for subsequent deep reinforcement learning agent decision-making; its output value is used to measure the compression effect corresponding to different combinations of compression hyperparameters, guiding the agent to output the optimal hyperparameters in the direction of meeting user needs.

[0023] The requirement information includes at least the user's preferences for compression efficiency, video quality, application scenario type, and special protection requirements. User preference parameters are configured based on this requirement information. For example, when the requirement information indicates "data center monitoring, prioritize compression efficiency," the weight of compression efficiency-related items is configured to a higher value; when the requirement information indicates "highway lanes, prioritize dynamic image quality," the weight of video quality-related items is configured to a higher value; when the requirement information indicates the need to protect faces, license plates, or text areas, additional weights are configured for reward items related to sensitive area protection. Based on the above configuration, a reward function is constructed. This reward function includes at least compression efficiency-related items and video quality-related items, and the two are combined according to the weights determined by the user preference parameters. For scenarios with special protection requirements, the reward function also weights and integrates sensitive area protection-related items. These items are quantified based on the local structural similarity index of the areas where preset key target types are located in the video, so that the image quality fidelity of the compressed sensitive areas receives additional incentives in the reward function.

[0024] Once the reward function is constructed, its output value will serve as an evaluation criterion for assessing candidate hyperparameter combinations for subsequent deep reinforcement learning agent outputs: hyperparameter combinations with better compression effects and those that better meet user needs will receive higher reward values. Thus, through the reward function, the user's subjective needs are transformed into objective, quantitative evaluation signals, enabling the deep reinforcement learning agent's decisions to align with user preferences, rather than employing a fixed compression strategy.

[0025] In some embodiments, the reward function includes a compression efficiency reward component, a video quality reward component, and a system penalty component, and step 103 may include the following sub-steps: Sub-step S11: Configure user preference parameters according to the required information; Sub-step S12: Based on the user preference parameters, determine the weight ratio of compression efficiency and video quality in the reward function; the user preference parameters characterize the user's emphasis on compression efficiency versus video quality; the compression efficiency reward component reflects the degree of data reduction in the compressed video, quantized based on compression ratio; the video quality reward component reflects the fidelity of the compressed video, quantized based on peak signal-to-noise ratio; the system penalty component applies a penalty to cases where the compression time exceeds the baseline time, to constrain the compression efficiency from excessively decreasing; the system penalty component is the difference between the current compression time and the baseline compression time divided by the baseline compression time, where the baseline compression time is the compression time of a traditional encoder compressing source videos under default configuration. Sub-step S13: The compression efficiency reward component and the video quality reward component are weighted and summed according to the weight ratio, and the product of the system penalty component and the corresponding weight coefficient is subtracted to obtain the reward function.

[0026] In this embodiment of the invention, after obtaining the structured feature vector of the video to be compressed, a reward function is determined based on the acquired user demand information. This reward function is the core evaluation signal for subsequent deep reinforcement learning agent decision-making; its output value is used to measure the compression effect corresponding to different combinations of compression hyperparameters, guiding the agent to output the optimal hyperparameters in the direction of meeting user needs. The reward function includes a compression efficiency reward component, a video quality reward component, and a system penalty component.

[0027] Configure user preference parameters based on the required information. This information includes user preferences for compression efficiency, video quality, application scenario type, and special protection requirements. When the required information indicates "data center monitoring, prioritize compression efficiency," configure the user preference parameters to favor compression efficiency; when the required information indicates "highway lanes, prioritize dynamic image quality," configure the user preference parameters to favor video quality; when the required information indicates the need to protect faces, license plates, or text areas, additional parameters related to sensitive area protection are configured.

[0028] Based on the aforementioned user preference parameters, the weight ratios of compression efficiency and video quality in the reward function are determined. The user preference parameters characterize the user's emphasis on either compression efficiency or video quality; the compression efficiency reward component reflects the degree of data reduction in the compressed video, quantized based on the compression ratio; the video quality reward component reflects the fidelity of the compressed video, quantized based on the peak signal-to-noise ratio; and the system penalty component penalizes situations where the compression time exceeds a baseline time, thus preventing an excessive decrease in compression efficiency. Specifically, the system penalty component is the difference between the current compression time and the baseline compression time divided by the baseline compression time, which is the compression time of a traditional encoder (such as H.265) compressing source videos under default configuration. The introduction of the system penalty component prevents the agent from selecting excessively long hyperparameter combinations to pursue the ultimate compression ratio or image quality, ensuring the timeliness of the solution in actual deployment.

[0029] The compression efficiency reward component and the video quality reward component are weighted and summed according to a determined weight ratio. Then, the product of the system penalty component and its corresponding weight coefficient is subtracted to obtain the final reward function. Taking the compression efficiency reward component as CR, the video quality reward component as Q, the system penalty component as P, the user preference weight as α (α∈[0,1]), and the adjustment factor corresponding to the system penalty as β as an example, the reward function is expressed as R=α×CR+(1 α)×Q β×P. By adjusting the value of α, the reward function can smoothly transition between prioritizing compression efficiency and prioritizing video quality; by adjusting the value of β, the influence of the system penalty component on the total reward can be controlled. Thus, the reward function quantifies the user's comprehensive needs for compression efficiency, video quality, and compression time into a unified evaluation signal, providing a clear optimization direction for the deep reinforcement learning agent's optimization decisions.

[0030] In some embodiments, step 103 may further include the following sub-steps: Sub-step S21: When the demand information indicates that image quality protection is required for the preset key target types in the video to be compressed, obtain the sensitive area reward component; the sensitive area reward component is quantified based on the local structural similarity index of the region where the preset key target types are located in the video, and the preset key target types include at least one of face, text, license plate, and fine texture regions; Sub-step S22: The compression efficiency reward component and the video quality reward component are weighted and summed according to the weight ratio, and then the product of the system penalty component and the corresponding weight coefficient is subtracted to obtain the basic reward function; Sub-step S23: Obtain the weight coefficient corresponding to the sensitive region reward component, and superimpose the product of the sensitive region reward component and the weight coefficient onto the basic reward function to obtain the reward function.

[0031] In this embodiment of the invention, when the demand information indicates that image quality protection is required for a preset key target type in the video to be compressed, a sensitive region reward component is obtained. The preset key target types include at least one of faces, text, license plates, and fine texture regions. These target types are predefined by cue word templates and identified and labeled from keyframes of the video to be compressed by a large visual model. The sensitive region reward component is quantified based on the Local Structure Similarity Index (ROI-SSIM) of the region where the preset key target type is located, and is used to measure the structural fidelity of the region after compression. Unlike the structural similarity index based on the entire frame, the local structure similarity index only calculates the structural similarity of image patches within the sensitive region, which can more accurately reflect the degree of image quality damage in the key region and avoid the problem of the entire frame index being diluted by a large background area and masking local distortion.

[0032] The compression efficiency reward component and the video quality reward component are weighted and summed according to a determined weight ratio. Then, the product of the system penalty component and its corresponding weight coefficient is subtracted to obtain the basic reward function. This basic reward function corresponds to the general reward function form when there are no special protection requirements.

[0033] Obtain the weight coefficient corresponding to the reward component of the sensitive region. Multiply the reward component of the sensitive region by this weight coefficient to obtain the incentive amount for the sensitive region. This incentive is then added to the basic reward function to obtain the final reward function. Taking the basic reward function as R_base, the reward component of the sensitive region as S, and the corresponding weight coefficient as λ as an example, the final reward function is expressed as R = R_base + λ × S = α × CR + (1 α)×Q β×P+λ×S. Wherein, the quantification method for the sensitive region reward component S is (current sensitive region ROI - SSIM). (Reference Sensitive Area ROI-SSIM) / (1) The baseline sensitive area ROI-SSIM is the statistical mean of the ROI-SSIM of the same type of video sensitive area in the data acquisition module. By normalizing S, its dimensions are aligned with other components in the basic reward function, which facilitates the unified configuration of the weight coefficient λ.

[0034] Based on the general reward function framework, by weighting and fusing the reward components of sensitive areas, the reward function can provide additional incentives for the image quality fidelity of the compressed sensitive areas. This guides the deep reinforcement learning agent to give higher protection priority to the areas where the user-specified key target types are located during the optimization process, thus achieving an adaptive balance between compression efficiency and regionally differentiated image quality protection.

[0035] Step 104: Input the structured feature vector into the pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The "deep reinforcement learning agent" is the core decision-making module for hyperparameter optimization. It takes a structured feature vector as input, aims to maximize the reward function, and outputs the optimal compression hyperparameters that match the video to be compressed. The deep reinforcement learning agent is a decision-making model trained using deep reinforcement learning algorithms. Essentially, it's a policy network that maps input state information to optimal action outputs. The agent's input state is the structured feature vector of the video to be compressed, and its output action is a combination of compression hyperparameters (such as quantization parameters QP, bitrate, GOP size, etc.). The training objective is to maximize the cumulative reward value by taking actions given a state. Unlike traditional optimization methods (such as grid search and Bayesian optimization), the deep reinforcement learning agent learns through interaction with the environment (i.e., the encoder), capturing the complex nonlinear mapping relationship between video features and optimal hyperparameters. It can output the optimal parameter combination with only a small amount of forward computation during the inference phase, avoiding the high computational overhead of online optimization.

[0036] State Input: The agent's input is a structured feature vector, containing video image attribute features and content semantic features. Video image attributes include signal-level features directly related to compression parameter selection, such as resolution, frame rate, image complexity, motion intensity, and texture richness. Content semantics includes high-level semantic information such as key target categories, sensitive area distribution, scene type, and low bitrate distortion risk type. These features together constitute the complete information foundation for the agent's decision-making, enabling the agent to not only perceive the objective signal attributes of the video but also understand its semantic content. Action Output: The agent's output is a combination of compression hyperparameters, specifically including adjustable parameters such as quantization parameters (QP), target bitrate, and GOP size. The combination of these parameters directly determines the compression ratio, image quality, and compression time after encoding and compression, and are the core variables affecting the compression effect.

[0037] Deep reinforcement learning agents are pre-trained offline, rather than online for each compression task. The training process involves using structured feature vectors from a large number of historical videos as input state samples, candidate compression hyperparameter combinations as output actions, and a reward function as the evaluation signal. Deep reinforcement learning algorithms are used for iterative training, allowing the agent to gradually learn the mapping relationship of "which compression hyperparameter action will yield the maximum reward value given a structured feature vector state." This mapping relationship is essentially an end-to-end modeling of the causal link of "features → hyperparameters → compression effect." In actual training, multiple agents are trained using various deep reinforcement learning algorithms, each corresponding to different video types and / or resolutions, forming a pre-trained model library to achieve accurate adaptation to different video features.

[0038] The agent's reasoning process revolves around maximizing the reward function as its core optimization objective. When a structured feature vector is input into the agent, the policy network outputs a candidate hyperparameter combination based on the current state. Subsequently, the compression effect of this candidate hyperparameter combination is evaluated by the reward function, which outputs a scalar reward value. The agent adjusts its policy direction based on this reward value, repeating the process for a finite number of iterations until the hyperparameter combination with the highest reward value is selected as the final output. In this process, the reward function acts as an evaluation signal, defining "what constitutes a good compression effect," and the agent's task is to find the optimal parameter configuration under this evaluation criterion. By mapping user demand information to the specific composition of the reward function, the agent's optimization direction is aligned with user preferences.

[0039] During the inference phase, the agent does not need to search for optimal parameters from scratch. Instead, it starts from a pre-trained policy network and converges to the optimal hyperparameter combination in just a finite number of iterations (e.g., 3 to 5). The pre-trained model has already learned a large number of mapping relationships between video features and hyperparameters. The inference process for new video input is essentially a rapid adaptation to the existing policy, rather than relearning. This allows for a balance between tuning effectiveness and response speed in practical deployments, making it suitable for video compression scenarios with high real-time requirements.

[0040] This invention employs multiple deep reinforcement learning agents, each trained using different deep reinforcement learning algorithms (such as SAC, PPO, and DDPG). Each agent corresponds to a different video type and / or resolution. During the inference phase, based on the video type and resolution features extracted from the structured feature vector, a matching agent is automatically selected from the multiple agents for invocation. Agents using different algorithms possess different characteristics: the SAC algorithm is highly exploratory and suitable for hyperparameter search of unknown video types; the PPO algorithm balances speed and stability, making it suitable for general scenarios; and the DDPG algorithm provides stable output, making it suitable for high-quality offline transcoding. Through the collaboration and automatic matching of multiple algorithm agents, this solution can adapt to diverse video compression needs.

[0041] In this embodiment of the invention, the input to the agent is a structured feature vector, which contains video image attribute features and content semantic features of the video to be compressed. During the inference phase, the agent's policy network outputs a candidate compression hyperparameter combination as the initial action based on the input structured feature vector. Subsequently, the compression effect corresponding to this candidate hyperparameter combination is evaluated using a determined reward function to obtain a scalar reward value. This reward value reflects the quality of compression effect achieved by adopting this hyperparameter combination under the current structured feature vector. The agent adjusts the output direction of its policy network based on this reward value, repeating the iterative process of "outputting candidate hyperparameters, evaluating the reward function, and adjusting the policy direction." After a finite number of iterations, the compression hyperparameter combination with the highest cumulative reward value is determined as the optimal compression hyperparameter and output. Since the agent has learned a large number of mapping relationships between video features and optimal hyperparameters through offline pre-training, the inference phase only requires a few iterations to converge, eliminating the need for a search from scratch.

[0042] In some embodiments, the structured feature vector includes video parameter features and video content features; step 104 may include the following sub-steps: Sub-step S31: Select a matching deep reinforcement learning agent from multiple deep reinforcement learning agents based on the video parameter features and / or video content features in the structured feature vector; the multiple deep reinforcement learning agents correspond to different video parameter features and / or video content features respectively. Sub-step S32: Input the structured feature vector into the matching deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed.

[0043] In this embodiment of the invention, video parameter features include resolution and / or frame rate, and video content features include at least one of image complexity, motion intensity, texture richness, key target type, and scene category. Based on the video parameter features and / or video content features in the structured feature vector, a matching deep reinforcement learning agent is selected from multiple deep reinforcement learning agents. These multiple deep reinforcement learning agents are pre-trained based on different deep reinforcement learning algorithms, and each agent corresponds to different video parameter features and / or video content features, used to characterize the mapping relationship between the structured feature vector and the optimal compression hyperparameter combination. The training method for each agent is as follows: using the structured feature vector of historical videos as input, the compression hyperparameter combination as output, and the reward function as the reward signal, iterative training is performed using the corresponding deep reinforcement learning algorithm until the policy network converges. During the inference phase, video parameter features and video content features are extracted from the output structured feature vector and matched with the video parameter features and / or video content features corresponding to each agent. Agents with consistent labels are selected as the matching deep reinforcement learning agents.

[0044] The structured feature vector is input into the selected deep reinforcement learning agent, which aims to maximize the reward value output by a defined reward function, and outputs the optimal compression hyperparameters that match the video to be compressed. The agent's policy network outputs candidate hyperparameter combinations based on the input structured feature vector. The reward function evaluates these combinations and outputs a reward value. The agent adjusts its policy direction based on the reward value. After a finite number of iterations, the hyperparameter combination with the highest reward value is determined as the optimal compression hyperparameters and output. This method quickly selects the decision model best suited to the current video from multiple specialized agents, ensuring the agent's tuning accuracy for specific video types while avoiding the computational overhead of online training through the reuse of pre-trained models, thus achieving a balance between tuning performance and inference efficiency.

[0045] In some embodiments, step S32 may include the following sub-steps: Sub-step S321: The structured feature vector is used as the current state input of the deep reinforcement learning agent, and the deep reinforcement learning agent outputs the initial compressed hyperparameter combination according to the corresponding policy network. Sub-step S322: Using the reward value output by the reward function as the evaluation signal, after N rounds of iterative optimization, the compression hyperparameter combination with the highest cumulative reward value is taken as the optimal compression hyperparameter output; N is a positive integer, and the value of N is determined according to the similarity between the structured feature vector and the feature distribution of the training dataset corresponding to the deep reinforcement learning agent.

[0046] In this embodiment of the invention, the structured feature vector is used as the current state input of the deep reinforcement learning agent. The agent then outputs an initial compressed hyperparameter combination based on its corresponding policy network. The policy network is a deep neural network whose input layer dimension is the same as the dimension of the structured feature vector, and whose output layer dimension is the same as the dimension of the compressed hyperparameter combination. When the structured feature vector is input, the policy network calculates through forward propagation and outputs a set of hyperparameter combinations corresponding to the current state as an initial action. This initial action represents the agent's preliminary estimate of the optimal hyperparameters under the current policy.

[0047] Using the reward value output by the reward function as the evaluation signal, after N rounds of iterative optimization, the compression hyperparameter combination with the highest cumulative reward value is taken as the optimal compression hyperparameter output. Here, N is a positive integer, and its value is determined based on the similarity between the structured feature vector and the feature distribution of the training dataset corresponding to the deep reinforcement learning agent. When the similarity between the structured feature vector of the video to be compressed and the feature distribution of samples in the agent's training dataset is high, it indicates that the current video is relatively close to the training data, and the agent has sufficient prior knowledge. In this case, a smaller value for N is sufficient for rapid convergence. When the similarity is low, it indicates that the current video differs significantly from the training data, requiring more rounds of iterative exploration to adapt to the features of the current video. In this case, a larger value for N is used. This similarity can be measured by calculating the Euclidean distance or cosine similarity between the structured feature vector and the feature centers of the training dataset. The smaller the distance or the higher the similarity, the smaller the value of N, and vice versa.

[0048] Step 105: Encode and compress the video to be compressed based on the optimal compression hyperparameters.

[0049] In this embodiment of the invention, the optimal combination of compression hyperparameters output, including quantization parameters, target bitrate, GOP size, and other encoding parameters, is used as the configuration input of the encoder. This is used to perform the actual encoding and compression operation on the video to be compressed, generating a compressed video file. This completes a closed loop from video feature extraction, user demand analysis, reward function configuration to agent decision-making, achieving adaptive tuning of compression hyperparameters and full-process intelligent video encoding and compression.

[0050] In some embodiments, after extracting the descriptive information into a structured feature vector, the method further includes the following steps: Step 11, obtain benchmark data; the benchmark data includes the mean of each dimension of the structured feature vectors of multiple historical videos that match the video to be compressed; Step 12: Compare the feature values ​​of each dimension of the structured feature vector with the mean values ​​of each dimension of the benchmark data to determine the deviation value of each dimension. Step 13: If the deviation value of any dimension exceeds the corresponding preset deviation threshold, the feature verification is determined to be unsuccessful, and a correction prompt word is generated. Step 14: Re-input the corrected prompt word and at least one frame of the video to be compressed into the visual big model, output the corrected visual feature description through the visual big model, and re-extract the corrected visual feature description into a corrected structured feature vector until the deviation value of all feature dimensions of the structured feature vector does not exceed the preset deviation threshold.

[0051] In this embodiment of the invention, benchmark data is obtained. This benchmark data originates from the data acquisition module and is constructed as follows: raw video data covering different scene types, resolutions, and content types is collected; compression result data of each raw video under different hyperparameter configurations and a hyperparameter benchmark configuration library of mainstream compression standards are collected; the collected data undergoes deduplication and anomaly removal processing; scene and complexity labels are applied to each video; and the statistical mean and standard deviation of each feature of videos of the same category are calculated. The statistical interval (mean ± standard deviation) is used as the benchmark data. When verifying the structured feature vector of the current video to be compressed, based on the scene type or video type label of the video to be compressed, the mean of each dimension of the structured feature vector of multiple historical videos matching that type is obtained from the benchmark data and used as a comparison benchmark.

[0052] The eigenvalues ​​of each dimension of the structured feature vector are compared with the mean of the corresponding dimension in the benchmark data, and the deviation value of each dimension is calculated. The deviation value can be measured by absolute deviation or relative deviation, such as the absolute value of the difference between a eigenvalue of a certain dimension and the benchmark mean of that dimension, or the ratio of the difference to the benchmark mean.

[0053] If the deviation value of any dimension exceeds the preset deviation threshold for that dimension, the feature verification is deemed unsuccessful, and a correction prompt is generated. The correction prompt is automatically generated by the rule engine based on the feature deviation type, video metadata, and the statistical range of the benchmark data. For example, in a data center monitoring video scenario, if the deviation of the feature value of the image complexity dimension from the benchmark mean exceeds the preset threshold of 1.5 points, the rule engine can generate the following correction prompt: "This video contains small devices located in shadowy corners; texture richness should be higher than 2 points. Please re-evaluate." In a highway lane video scenario, the rule engine can generate: "This video contains fast-moving car scenes; image complexity should be higher than 3 points. Please re-evaluate."

[0054] The generated correction prompts and at least one frame of the video to be compressed are re-input into the large-scale visual model. The large-scale visual model outputs the corrected visual feature descriptions, which are then re-extracted into corrected structured feature vectors. Subsequently, the process returns to perform a deviation comparison verification. If any feature dimension deviation exceeds a preset threshold, steps 13 and 14 are iterated until the deviation values ​​of all feature dimensions of the structured feature vector do not exceed the corresponding preset deviation thresholds. At this point, the feature verification is confirmed to be successful, and the final structured feature vector output is used as the state input for subsequent deep reinforcement learning agents. By using benchmark data to verify the quality of the structured feature vectors extracted by the large-scale visual model, and automatically generating correction prompts to drive iterative optimization of the large-scale visual model when verification fails, a closed-loop feedback mechanism of "extraction → verification → correction → re-verification" is formed. This effectively overcomes the feature extraction deviation problem that may occur in specific scenarios, ensuring the accuracy and reliability of the structured feature vectors.

[0055] Reference Figure 2 This diagram illustrates the technical architecture of a video compression hyperparameter tuning method provided by an embodiment of the present invention. The left half shows the feature extraction and verification process based on a large visual model: prompt word templates (containing non-semantic and semantic feature prompts) and the video to be compressed are input into the large visual model, outputting a structured feature vector. This vector covers non-semantic features such as resolution, frame rate, image complexity, motion intensity, and texture richness, as well as semantic features such as key target type, sensitive area proportion, low bitrate distortion risk type, and scene category. After extraction, the structured feature vector is compared with benchmark data. If the deviation exceeds a threshold, a correction prompt word is generated for iterative correction to ensure feature accuracy. The middle part shows the reward function configuration: Based on user needs (prioritizing compression efficiency, prioritizing quality, or a trade-off between the two), the reward function R = α×CR+(1 α)×Q β×P, where CR is based on compression ratio quantization, Q is based on PSNR quantization, and P is the compression time penalty; when there are scenarios where specific key targets need to be protected, the sensitive area excitation S is additionally weighted and fused.

[0056] The right half describes deep reinforcement learning hyperparameter tuning and closed-loop feedback: structured feature vectors serve as state input to the DRL agent. The agent comprises an Actor network (outputting compressed bitrate, GOP, and other actions), a Critic network (evaluating action value), and a Replay Buffer (experience replay). The method set includes three algorithms: SAC (highly exploratory), PPO (balancing speed and stability), and DDPG (offline high-quality transcoding). The agent outputs optimal hyperparameters with the goal of maximizing the reward function. After encoding and compression, the results are evaluated. If the evaluation meets the criteria, the parameters are updated in the database; otherwise, feature correction or reward function adjustment is triggered, and incremental fine-tuning is performed, forming a closed-loop optimization.

[0057] Reference Figure 3 The diagram illustrates a flowchart of another video compression hyperparameter tuning method provided by an embodiment of the present invention. The method specifically includes the following steps: Step 201: Obtain the compression effect evaluation result of the video to be compressed after encoding and compression based on the optimal compression hyperparameters; the compression effect evaluation result includes at least one of compression ratio, peak signal-to-noise ratio or structural similarity index, and compression time; In this embodiment of the invention, the compression effect evaluation result of the video to be compressed after encoding and compression based on the optimal compression hyperparameters is obtained. This evaluation result includes at least one of compression ratio, peak signal-to-noise ratio (PSNR) or structural similarity index (SSIM), and compression time, used to comprehensively measure the compression effect from three dimensions: data reduction degree, image fidelity, and processing timeliness. Compression ratio reflects the degree of data reduction in the compressed video, and can be quantified by the ratio of the compressed file size to the original file size or the percentage reduction in bitrate; it is a core indicator for measuring compression efficiency. Peak signal-to-noise ratio (PSNR) or structural similarity index (SSIM) reflects the image fidelity of the compressed video. PSNR is quantified based on pixel-level error, while SSIM is quantified based on structural similarity; both are core indicators for measuring video quality. Compression time reflects the time consumed from the start of encoding to its completion, and is an important indicator for evaluating the practical feasibility of deploying the compression scheme.

[0058] The evaluation results are obtained by collecting and calculating the output of the encoder compression process: compression ratio and compression time are directly obtained from the encoder's output logs or system monitoring, while PSNR or SSIM is calculated by comparing the original video with the compressed video frame by frame. Once the compression effect evaluation results are obtained, they will serve as input for subsequent closed-loop feedback optimization, used to determine whether the current optimization results meet user needs and how to make adjustments.

[0059] Step 202: If the compression effect evaluation result meets the preset conditions, the structured feature vector of the video to be compressed, the optimal compression hyperparameters, the compression effect evaluation result, and the demand information are associated and stored as sample data for incremental training of the deep reinforcement learning agent; the preset conditions are that all evaluation indicators in the compression effect evaluation result reach or exceed the corresponding preset indicator thresholds. In this embodiment of the invention, if the compression effect evaluation result meets the preset conditions, the structured feature vector of the video to be compressed, the optimal compression hyperparameters, the compression effect evaluation result, and the requirement information are associated and stored as sample data for incremental training of the deep reinforcement learning agent. The preset conditions are that all evaluation indicators in the compression effect evaluation result reach or exceed the corresponding preset indicator thresholds. For example, the compression rate reaches the user-set target value (e.g., bitrate reduction of no less than 20%), the peak signal-to-noise ratio or structural similarity indicator reaches the user-set minimum quality requirement (e.g., peak signal-to-noise ratio of no less than 38 dB), and the compression time does not exceed the user-set time limit constraint (e.g., no more than twice the real-time frame rate).

[0060] When all the above indicators meet the standards, it indicates that the current optimization results are effective and the deep reinforcement learning agent's decisions under the given video features and user needs are correct. Therefore, all relevant data for the current task, including structured feature vectors, optimal compression hyperparameters, compression effect evaluation results, and demand information, are stored as complete sample data in the training sample set of the data acquisition module. This sample data contains a complete information chain from "video features" to "user needs" to "hyperparameter decisions" to "compression effect," providing high-quality positive samples for subsequent incremental training of the deep reinforcement learning agent.

[0061] Through the aforementioned associated storage, the training sample set is continuously expanded using effective optimization cases verified in actual compression tasks. Based on the new samples, the deep reinforcement learning agent is incrementally fine-tuned, enabling the agent to continuously accumulate new successful experiences. As the number of uses increases, the decision accuracy of the agent continues to improve, forming a positive accumulation effect of "the more it is used, the better the effect".

[0062] Step 203: If the compression effect evaluation result does not meet the preset conditions, then adjust the parameters of the reward function according to the deviation direction and deviation amount between the compression effect evaluation result and the corresponding threshold, and optimize the deep reinforcement learning agent.

[0063] In this embodiment of the invention, the deviation direction is used to indicate whether the current evaluation result is biased towards insufficient compression efficiency, insufficient video quality, or excessive compression time. For example, when the compression rate fails to meet the standard, the deviation direction is "insufficient compression efficiency," indicating that the incentive strength of the compression efficiency-related term in the reward function is insufficient and its weight needs to be increased. When the peak signal-to-noise ratio fails to meet the standard, the deviation direction is "insufficient video quality," indicating that the incentive strength of the video quality-related term in the reward function is insufficient and its weight needs to be increased. When the compression time exceeds the time limit, the deviation direction is "excessive compression time," indicating that the penalty strength of the system penalty term is insufficient or the adjustment factor needs to be increased. The deviation amount is used to indicate the degree of the above deviations. For example, if the bitrate is only reduced by 15% while the target is 20%, the deviation amount is a difference of 5 percentage points. This deviation amount is used to determine the adjustment range of the reward function parameters: the larger the deviation, the larger the adjustment step size of the corresponding parameters; the smaller the deviation, the smaller the adjustment step size.

[0064] After determining the adjustment direction and magnitude of the reward function parameters, the adjusted reward function is reapplied to the deep reinforcement learning agent's inference process. This drives the agent to re-output the compression hyperparameter combination under the updated evaluation signal, and the new compression effect is compared with the preset conditions again. If the conditions are still not met, the adjustment process continues iteratively until the evaluation result meets the preset conditions. This allows the reward function to be gradually calibrated based on the gap between the actual compression effect and the user's expectations, until the output of the deep reinforcement learning agent meets the user's needs. When the evaluation result finally meets the criteria, the adjusted reward function configuration in this iteration is stored as an empirically effective configuration for the current scenario, for rapid configuration reuse in subsequent similar scenarios. The final optimized sample (structured feature vector, optimal hyperparameters, evaluation results, and requirement information) is associated and stored in the training sample set for incremental training of the agent.

[0065] This embodiment uses data center surveillance video compression as an example to illustrate the application of the present invention in scenarios where compression efficiency is paramount. Data center surveillance videos are characterized by a large number of cameras and long recording times (24 / 7), consuming significant storage space and transmission bandwidth. Therefore, the compression scheme must prioritize a high compression rate while protecting the image quality of critical areas such as equipment and faces to ensure that details remain traceable after compression.

[0066] First, we obtain the user's video compression requirements and prompt word template. The requirements indicate that the current scenario is data center monitoring, prioritizing compression efficiency, and requiring the protection of faces and equipment areas. The prompt word template contains prompts to guide the large visual model in extracting non-semantic features and prompts to extract semantic features.

[0067] Subsequently, keyframes are extracted from the video to be compressed, and these keyframes, along with prompt word templates, are input into a large-scale visual model (such as InstructBLIP). The large-scale visual model performs visual semantic parsing and outputs a natural language description of visual features, including image attributes such as resolution, frame rate, image complexity, motion intensity, and texture richness, as well as semantic information such as face proportion, device proportion, scene category, and low bitrate distortion risk level. This descriptive information is converted into a structured feature vector, for example: [resolution (1080P encoded as 2), frame rate (30fps encoded as 1), image complexity 3 points, motion intensity 1 point, texture richness 2 points, face proportion 0.2, device proportion 0.8, scene category (server room encoded as 0), distortion risk level 3].

[0068] The extracted structured feature vectors were compared with the benchmark data. All deviations in each dimension were within the preset threshold range (e.g., the image complexity score did not deviate from the benchmark by more than 1.5 points), thus the feature verification passed. The reward function was determined based on the requirements. The requirements indicated that data center monitoring prioritized compression efficiency and needed to protect faces and equipment areas. Therefore, the user preference weight α was configured to 0.6, giving compression efficiency-related items a higher weight in the reward function. Simultaneously, the requirements indicated the need for image quality protection for preset key target types (faces, equipment). Therefore, the sensitive area reward component S was weighted and fused in the reward function, resulting in the final reward function: R = 0.6 × CR + 0.4 × Q. β×P+λ×S, where CR is the compression efficiency reward based on compression rate quantization, Q is the video quality reward based on peak signal-to-noise ratio quantization, and P is the system penalty based on compression time quantization.

[0069] The structured feature vectors are input into a pre-trained deep reinforcement learning agent. The system extracts video type and resolution features from the structured feature vectors and automatically matches them to the corresponding pre-trained agent. The agent optimizes by maximizing the reward value output by the reward function, and outputs the optimal combination of compression hyperparameters after a finite number of iterations, such as quantization parameter QP=28, bitrate=1200kbps, and GOP size=200. Based on the above optimal hyperparameters, the monitoring video of the computer room is encoded and compressed. After compression, the compression effect evaluation results are obtained: the compression ratio reaches the expected target (bitrate reduction of more than 20%), the peak signal-to-noise ratio and structural similarity indicators meet the standards, and the compression time meets the real-time constraints. All indicators reach or exceed the preset thresholds, confirming the effectiveness of the optimization results. Subsequently, the structured feature vectors, the optimal hyperparameter combination, the evaluation results, and the requirement information are associated and stored as positive samples for incremental training of the deep reinforcement learning agent.

[0070] Reference Figure 4This diagram illustrates a flowchart of a video compression hyperparameter tuning method provided by an embodiment of the present invention. The process begins with obtaining prompt word templates and video compression requirements. Then, video frames and prompt word templates are input into a large visual model, which outputs descriptive information of visual features and converts it into structured feature vectors. Simultaneously, a reward function is determined based on the requirements. After the structured feature vectors are validated, they are input into a pre-trained deep reinforcement learning agent. The agent optimizes by maximizing the reward function and outputs the optimal compression hyperparameters that match the video to be compressed. Finally, the video to be compressed is encoded based on the optimal hyperparameters, and the compression effect evaluation results are obtained, providing a basis for subsequent closed-loop feedback. This achieves end-to-end intelligent tuning from video content perception to compression parameter output.

[0071] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0072] Reference Figure 5 The diagram shows a structural block diagram of a video compression hyperparameter tuning device provided in an embodiment of the present invention. The device specifically includes the following modules: The requirement information acquisition module 301 is used to acquire the requirement information and prompt word template for video compression; the prompt word template includes visual features of the video to be compressed in compression-related dimensions, the visual features include video image attributes and content semantics; the visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate a visual feature description in natural language form. The description information output module 302 is used to input at least one frame of the video to be compressed and the prompt word template into the visual big model, output the description information of the visual features of the video to be compressed, and convert the description information into a structured feature vector. The reward function determination module 303 is used to determine the reward function based on the demand information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The compression parameter determination module 304 is used to input the structured feature vector into a pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The video encoding and compression module 305 is used to encode and compress the video to be compressed based on the optimal compression hyperparameters.

[0073] In some embodiments, after extracting the descriptive information into a structured feature vector, the method further includes: A benchmark data acquisition module is used to acquire benchmark data; the benchmark data includes the mean values ​​of each dimension of the structured feature vectors of multiple historical videos that match the video to be compressed. The dimensional feature comparison module is used to compare the feature values ​​of each dimension of the structured feature vector with the mean values ​​of each dimension of the benchmark data to determine the deviation value of each dimension. The correction prompt word generation module is used to determine that feature verification fails and generate correction prompt words if the deviation value of any dimension exceeds the corresponding preset deviation threshold. The feature vector correction module is used to re-input the correction prompt word and at least one frame of the video to be compressed into the visual big model, output the corrected visual feature description through the visual big model, and re-extract the corrected visual feature description into a corrected structured feature vector until the deviation value of all feature dimensions of the structured feature vector does not exceed the preset deviation threshold.

[0074] In some embodiments, the structured feature vector includes video parameter features and video content features; the compression parameter determination module 304 includes: The agent matching submodule is used to select a matching deep reinforcement learning agent from multiple deep reinforcement learning agents based on the video parameter features and / or video content features in the structured feature vector; the multiple deep reinforcement learning agents correspond to different video parameter features and / or video content features respectively; The compression hyperparameter determination submodule is used to input the structured feature vector into the matching deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed.

[0075] In some embodiments, the compression hyperparameter determination submodule includes: The initial parameter determination unit is used to take the structured feature vector as the current state input of the deep reinforcement learning agent, and the deep reinforcement learning agent outputs the initial compressed hyperparameter combination according to the corresponding policy network. The optimal parameter determination unit is used to take the reward value output by the reward function as the evaluation signal, and after N rounds of iterative optimization, take the compression hyperparameter combination with the highest cumulative reward value as the optimal compression hyperparameter output; N is a positive integer, and the value of N is determined according to the similarity between the structured feature vector and the feature distribution of the training dataset corresponding to the deep reinforcement learning agent.

[0076] In some embodiments, the reward function includes a compression efficiency reward component, a video quality reward component, and a system penalty component; the reward function determination module 303 includes: The preference parameter acquisition submodule is used to configure user preference parameters based on the requirement information. The weight ratio determination submodule is used to determine the weight ratio of compression efficiency and video quality in the reward function based on the user preference parameters. The user preference parameters characterize the user's emphasis on compression efficiency versus video quality. The compression efficiency reward component reflects the degree of data reduction in the compressed video and is quantized based on the compression ratio. The video quality reward component reflects the fidelity of the compressed video and is quantized based on the peak signal-to-noise ratio. The system penalty component is used to penalize cases where the compression time exceeds the baseline time, in order to constrain the compression efficiency from decreasing excessively. The system penalty component is the difference between the current compression time and the baseline compression time divided by the baseline compression time, where the baseline compression time is the compression time of a traditional encoder compressing the same source video under default configuration. The reward function determination submodule is used to perform a weighted summation of the compression efficiency reward component and the video quality reward component according to the weight ratio, and then subtract the product of the system penalty component and the corresponding weight coefficient to obtain the reward function.

[0077] In some embodiments, the reward function determination module 303 further includes: The reward component acquisition submodule is used to acquire the sensitive region reward component when the demand information indicates that the preset key target type in the video to be compressed needs to be protected in terms of image quality; the sensitive region reward component is quantified based on the local structural similarity index of the region where the preset key target type is located in the video, and the preset key target type includes at least one of face, text, license plate, and fine texture region. The basic function acquisition submodule is used to perform a weighted summation of the compression efficiency reward component and the video quality reward component according to the weight ratio, and then subtract the product of the system penalty component and the corresponding weight coefficient to obtain the basic reward function; The reward function generation submodule is used to obtain the weight coefficients corresponding to the reward components of the sensitive regions, and to superimpose the product of the reward components of the sensitive regions and the weight coefficients onto the basic reward function to obtain the reward function.

[0078] In some embodiments, the apparatus further includes: The evaluation result acquisition module is used to acquire the compression effect evaluation result of the video to be compressed after encoding and compression based on the optimal compression hyperparameters; the compression effect evaluation result includes at least one of compression ratio, peak signal-to-noise ratio or structural similarity index, and compression time; The sample data determination module is used to associate and store the structured feature vector of the video to be compressed, the optimal compression hyperparameters, the compression effect evaluation result, and the demand information as sample data for incremental training of the deep reinforcement learning agent if the compression effect evaluation result meets the preset conditions; the preset conditions are that all evaluation indicators in the compression effect evaluation result reach or exceed the corresponding preset indicator thresholds. The agent optimization module is used to adjust the parameters of the reward function and optimize the deep reinforcement learning agent if the compression effect evaluation result does not meet the preset conditions, based on the deviation direction and deviation amount between the compression effect evaluation result and the corresponding threshold.

[0079] As the apparatus embodiment is basically similar to the method embodiment, it is described in a relatively simple manner. For relevant details, please refer to the description of the method embodiment.

[0080] This invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the video compression hyperparameter tuning method embodiments described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0081] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the video compression hyperparameter tuning method embodiments described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0082] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0083] Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present invention is not limited to performing functions in the order shown or discussed. It may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0085] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for optimizing hyperparameters in video compression, characterized in that, The method includes: Obtain video compression requirement information and prompt word templates; the prompt word templates include visual features of the video to be compressed in compression-related dimensions, the visual features including video image attributes and content semantics; the visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate visual feature descriptions in natural language form; The visual big model is input with at least one frame of the video to be compressed and the prompt word template, and the visual feature description information of the video to be compressed is output and the description information is converted into a structured feature vector. The reward function is determined based on the demand information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The structured feature vector is input into a pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as its optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The video to be compressed is encoded and compressed based on the optimal compression hyperparameters.

2. The video compression hyperparameter tuning method according to claim 1, characterized in that, After extracting the descriptive information into a structured feature vector, the method further includes: Obtain benchmark data; the benchmark data includes the mean values ​​of each dimension of the structured feature vectors of multiple historical videos that match the video to be compressed; The feature values ​​of each dimension of the structured feature vector are compared with the mean values ​​of each dimension of the benchmark data to determine the deviation value of each dimension. If the deviation value of any dimension exceeds the corresponding preset deviation threshold, the feature verification is determined to be unsuccessful, and a correction prompt word is generated. The corrected prompt word and at least one frame of the video to be compressed are re-input into the visual big model. The visual big model outputs a corrected visual feature description and extracts the corrected visual feature description into a corrected structured feature vector until the deviation of all feature dimensions of the structured feature vector does not exceed the preset deviation threshold.

3. The video compression hyperparameter tuning method according to claim 1, characterized in that, The structured feature vector includes video parameter features and video content features; the structured feature vector is input into a pre-trained deep reinforcement learning agent, which optimizes by maximizing the reward value output by the reward function and outputs optimal compression hyperparameters matching the video to be compressed, including: Based on the video parameter features and / or video content features in the structured feature vector, a matching deep reinforcement learning agent is selected from multiple deep reinforcement learning agents; the multiple deep reinforcement learning agents correspond to different video parameter features and / or video content features respectively; The structured feature vector is input into the matching deep reinforcement learning agent, which optimizes by maximizing the reward value output by the reward function and outputs the optimal compression hyperparameters that match the video to be compressed.

4. The video compression hyperparameter tuning method according to claim 3, characterized in that, The deep reinforcement learning agent that inputs the structured feature vector into the matching algorithm aims to maximize the reward value output by the reward function and outputs optimal compression hyperparameters that match the video to be compressed, including: The structured feature vector is used as the current state input of the deep reinforcement learning agent, and the deep reinforcement learning agent outputs the initial compressed hyperparameter combination according to the corresponding policy network. Using the reward value output by the reward function as the evaluation signal, after N rounds of iterative optimization, the compression hyperparameter combination with the highest cumulative reward value is taken as the optimal compression hyperparameter output; N is a positive integer, and the value of N is determined based on the similarity between the structured feature vector and the feature distribution of the training dataset corresponding to the deep reinforcement learning agent.

5. The video compression hyperparameter tuning method according to claim 1, characterized in that, The reward function includes a compression efficiency reward component, a video quality reward component, and a system penalty component; Determining the reward function based on the demand information includes: Configure user preference parameters based on the aforementioned requirement information; Based on the user preference parameters, the weight ratios of compression efficiency and video quality in the reward function are determined. The user preference parameters characterize the user's emphasis on either compression efficiency or video quality. The compression efficiency reward component reflects the degree of data reduction in the compressed video, quantized based on the compression ratio. The video quality reward component reflects the fidelity of the compressed video, quantized based on the peak signal-to-noise ratio. The system penalty component penalizes situations where the compression time exceeds a baseline time, thus preventing excessive reduction in compression efficiency. The system penalty component is the difference between the current compression time and the baseline compression time divided by the baseline compression time, where the baseline compression time is the compression time of a traditional encoder compressing source videos under default configuration. The compression efficiency reward component and the video quality reward component are weighted and summed according to the weight ratio, and the product of the system penalty component and its corresponding weight coefficient is subtracted to obtain the reward function.

6. The video compression hyperparameter tuning method according to claim 5, characterized in that, The step of determining the reward function based on the demand information further includes: When the demand information indicates that image quality protection is required for a preset key target type in the video to be compressed, a sensitive region reward component is obtained; the sensitive region reward component is quantified based on the local structural similarity index of the region where the preset key target type is located in the video, and the preset key target type includes at least one of face, text, license plate, and fine texture region. The compression efficiency reward component and the video quality reward component are weighted and summed according to the weight ratio, and then the product of the system penalty component and the corresponding weight coefficient is subtracted to obtain the basic reward function. Obtain the weight coefficient corresponding to the reward component of the sensitive region, and superimpose the product of the reward component of the sensitive region and the weight coefficient onto the basic reward function to obtain the reward function.

7. The video compression hyperparameter tuning method according to claim 1, characterized in that, The method further includes: Obtain the compression effect evaluation result of the video to be compressed after encoding and compression based on the optimal compression hyperparameters; the compression effect evaluation result includes at least one of compression ratio, peak signal-to-noise ratio or structural similarity index, and compression time; If the compression effect evaluation result meets the preset conditions, the structured feature vector of the video to be compressed, the optimal compression hyperparameters, the compression effect evaluation result, and the demand information are associated and stored as sample data for incremental training of the deep reinforcement learning agent; the preset conditions are that all evaluation indicators in the compression effect evaluation result reach or exceed the corresponding preset indicator thresholds. If the compression effect evaluation result does not meet the preset conditions, the parameters of the reward function are adjusted according to the deviation direction and deviation amount between the compression effect evaluation result and the corresponding threshold, and the deep reinforcement learning agent is optimized.

8. A video compression hyperparameter tuning device, characterized in that, The device includes: The requirement information acquisition module is used to acquire the requirement information and prompt word template for video compression. The prompt word template includes visual features of the video to be compressed in compression-related dimensions, including video image attributes and content semantics. The visual model is used to perform visual semantic analysis on the images of the video to be compressed and generate a visual feature description in natural language. The description information output module is used to input at least one frame of the video to be compressed and the prompt word template into the visual big model, output the description information of the visual features of the video to be compressed, and convert the description information into a structured feature vector. The reward function determination module is used to determine the reward function based on the requirement information; the output value of the reward function is used to measure the compression effect corresponding to different compression hyperparameters. The compression parameter determination module is used to input the structured feature vector into a pre-trained deep reinforcement learning agent. The deep reinforcement learning agent takes maximizing the reward value output by the reward function as the optimization objective and outputs the optimal compression hyperparameters that match the video to be compressed. The video encoding and compression module is used to encode and compress the video to be compressed based on the optimal compression hyperparameters.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the video compression hyperparameter tuning method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the video compression hyperparameter tuning method as described in any one of claims 1-7.

Images