An adaptive compression method and system for video streams
By constructing a prospective feature set and dynamically determining the number of fused frames during video stream compression, the adaptability problem of video content changes is solved, achieving efficient compression of video streams and preservation of key information.
Patent Information
- Application Number
- CN202611205146.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-10
- Publication Date
- 2026-11-10
AI Technical Summary
Existing video stream compression methods struggle to balance the dynamic nature of video content changes, the adaptability of compression parameters to different video segments, and the degree of data compression with the preservation of key information, resulting in low bandwidth and storage resource utilization or loss of video content details.
By obtaining the current processing position of the video stream to be processed, the current video segment and the subsequent look-ahead video interval are determined, look-ahead sampling frames are sampled and a look-ahead feature set is constructed, video frame fusion and feature extraction are performed, and the target fusion frame number is dynamically determined based on the current and subsequent video content using a policy network, and the compressed video stream is generated through iterative processing.
It improves the compatibility between compression parameters and video segments, reduces the amount of video stream data and resource consumption, and minimizes the loss of details in important video content, achieving a balance between data compression and the preservation of key information.
Smart Images

Figure CN123075192A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video processing technology, and in particular to an adaptive compression method and system for video streams. Background Technology
[0002] With the development of digital media and network communication technologies, video data is widely used in scenarios such as video surveillance, live streaming, remote interaction, industrial production, and multimedia information services. Compared to text and images, video streams consist of a large number of continuous video frames, characterized by large data volume, strong temporal continuity, and complex content changes. Especially with the trend towards high-definition and ultra-high-definition, video streams are increasingly consuming more network bandwidth, storage capacity, and computing resources. Therefore, video streams typically require compression to reduce the transmission and storage overhead of video data.
[0003] Existing video stream compression methods typically reduce video data volume by eliminating redundant information within and between video frames, and can adjust compression parameters according to preset bitrates, frame rates, or local changes in video content. However, the complexity of scenes, motion states, and effective information density often vary across different time periods in a video stream. Using a relatively fixed compression method makes it difficult to meet the processing needs of different video segments. Even if some methods can adjust compression parameters based on acquired video frames, these adjustments usually rely on a limited range of video features. When video content undergoes continuous changes or local abrupt changes, the compression level may mismatch with the actual content, leading to low bandwidth and storage resource utilization, or loss of detail in high-value video content. Furthermore, the presentation of key information in a video stream is diverse; processing based solely on a single image change or motion feature may fail to accurately distinguish the importance of different video content.
[0004] Therefore, in video stream compression processing, the dynamic nature of video content changes, the insufficient adaptability of compression parameters to different video segments, and the difficulty in balancing data compression and the preservation of key information have become urgent problems to be solved. Summary of the Invention
[0005] This application provides a video stream adaptive compression method and system, aiming to solve the problems in existing video stream compression processing, such as the strong dynamic changes in video content, insufficient adaptability of compression parameters to different video segments, and the difficulty in balancing the degree of data compression with the preservation of key information.
[0006] In a first aspect, a video stream adaptive compression method is provided, the method comprising: Obtain the video stream to be processed, and based on the current processing position of the video stream to be processed, determine the current video segment and the look-ahead video interval following the current video segment; At least one forward-looking sampled frame is sampled from the forward-looking video interval, forward-looking features of each forward-looking sampled frame are extracted, and a forward-looking feature set is constructed based on the temporal position of each forward-looking sampled frame and the corresponding forward-looking features. Frame fusion is performed on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment, and the current state features of the representative frame are extracted. Based on the temporal position of the current video segment, target prospective features are determined from the prospective feature set, and the current state features and the target prospective features are fused to obtain the strategy input features; The policy input features are input into the policy network to obtain the target fusion frame number corresponding to the next video segment; Based on the target fusion frame number, the next video segment is determined from the video stream following the current video segment, and the video frames in the next video segment are fused to obtain the representative frame corresponding to the next video segment; The next video segment is taken as the new current video segment, and the steps of current state feature extraction, target prospective feature determination, target fusion frame number determination and video frame fusion are iteratively executed to obtain the representative frame corresponding to each subsequent video segment. Organize the representative frames corresponding to each video segment according to their temporal order to generate a compressed video stream.
[0007] Optionally, in the above scheme, sampling at least one look-ahead sampling frame from the look-ahead video interval includes: Based on the start and end time positions of the look-ahead video interval, a candidate frame sequence is determined from the video stream to be processed; Based on the temporal range corresponding to the candidate frame sequence, at least one sampling temporal position is determined according to a preset time interval, random sampling rules, or sampling rules based on content complexity, to obtain a set of sampling positions; The video frames corresponding to each sampling time position in the sampling position set are extracted from the candidate frame sequence to obtain the look-ahead sampling frames.
[0008] Optionally, in the above scheme, the step of extracting the prospective features of each prospective sampling frame and constructing a prospective feature set based on the temporal position of each prospective sampling frame and the corresponding prospective features includes: Each of the prospective sampling frames is input into a feature extractor to extract the visual or semantic features of each prospective sampling frame, thereby obtaining the prospective features corresponding to each prospective sampling frame. Each of the aforementioned forward-looking features is associated with the temporal position of the corresponding forward-looking sampling frame to obtain multiple forward-looking feature entries; According to the temporal position corresponding to each of the aforementioned prospective feature entries, the prospective feature set is obtained by temporally organizing the multiple prospective feature entries; When the current processing position reaches the preset update position, the updated look-ahead video interval is determined based on the updated current processing position, and supplementary look-ahead sampling frames are sampled from the updated look-ahead video interval to obtain supplementary look-ahead features. Delete prospective feature entries whose temporal positions are no later than the updated current processing position from the prospective feature set, and add the supplementary prospective features and their temporal positions to the prospective feature set to obtain the updated prospective feature set.
[0009] Optionally, in the above scheme, the step of performing frame fusion on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment includes: Obtain the pixel data corresponding to each video frame in the current video segment to obtain a set of pixel data to be fused; The fusion weights corresponding to each video frame are determined based on their frame order, content similarity, or frame importance. Based on the fusion weights corresponding to each video frame, the pixel values at corresponding pixel positions in each video frame are weighted and fused to obtain fused pixel data; The representative frame corresponding to the current video segment is generated based on the fused pixel data.
[0010] Optionally, in the above scheme, determining the fusion weight corresponding to each video frame based on the content similarity of each video frame includes: Frame-level representation features are extracted from each video frame in the current video segment to obtain a set of frame-level representation features; The frame-level representation features in the frame-level representation feature set are aggregated to obtain the segment aggregated features corresponding to the current video segment; The degree of correlation between each frame-level representation feature and the segment aggregation feature is determined to obtain the content association value corresponding to each video frame; The content association values are input into the attention weight generation model, and the output of the attention weight generation model is normalized to obtain the normalized attention weights corresponding to each video frame. The normalized attention weights are determined as the fusion weights for the corresponding video frames.
[0011] Optionally, in the above scheme, determining the target prospective features from the prospective feature set based on the temporal position of the current video segment includes: Based on the end timing position of the current video segment, determine the reference timing position corresponding to the current video segment; From the prospective feature set, prospective feature entries whose temporal positions are later than the reference temporal positions are selected to obtain a candidate prospective feature entry set; The temporal distance between the temporal position of each candidate prospective feature entry in the candidate prospective feature entry set and the reference temporal position is determined respectively, and the temporal distance corresponding to each candidate prospective feature entry is obtained. The target prospective feature entry with the smallest corresponding temporal distance is determined from the set of candidate prospective feature entries, and the prospective features contained in the target prospective feature entry are determined as the target prospective features.
[0012] Optionally, in the above scheme, the step of fusing the current state features and the target prospective features to obtain policy input features, and inputting the policy input features into the policy network to obtain the target fusion frame number corresponding to the next video segment, includes: The current state features and the target prospective features are concatenated to obtain the policy input features; The policy input features are input into the policy network to obtain the fused frame number decision result; Based on the fusion frame number decision result, a target candidate fusion frame number is determined from a preset fusion frame number candidate set; The target candidate fusion frame number is determined as the target fusion frame number corresponding to the next video segment.
[0013] In the above scheme, optionally, the policy network includes at least one of a feedforward neural network, a recurrent neural network, or a Transformer network, and the step of determining the target candidate fusion frame number from the preset candidate fusion frame number set based on the fusion frame number decision result includes: The policy input features are encoded using the policy network to obtain a policy decision representation; The strategy decision representation is mapped to each candidate fusion frame number in the preset fusion frame number candidate set to obtain the decision value corresponding to each candidate fusion frame number; The decision values corresponding to each of the candidate fusion frame numbers are compared to obtain the maximum decision value; The number of candidate fusion frames corresponding to the maximum decision value is determined as the target number of candidate fusion frames.
[0014] Optionally, the above scheme may also include: Obtain the number of fusion frames for determining each video segment, wherein the number of fusion frames corresponding to the initial video segment is a preset initial fusion frame number, and the number of fusion frames corresponding to each video segment after the initial video segment is the target fusion frame number output in the previous processing cycle. Based on the segment fusion frame number corresponding to each video segment and the preset importance mapping relationship, the segment importance score corresponding to each video segment is determined, wherein the segment importance score is negatively correlated with the corresponding segment fusion frame number; Assign the segment importance score corresponding to each video segment to the original video frame in the corresponding video segment to obtain the frame importance score corresponding to each original video frame; The original video frames in the video stream to be processed are sorted according to the importance scores of each frame to obtain the video frame sorting result; Based on the video frame sorting results, a preset number of target video frames are selected, and each target video frame is organized temporally according to its original temporal position in the video stream to be processed, thereby generating a condensed video.
[0015] Secondly, a video stream adaptive compression system, the system comprising: The video stream processing module is used to acquire the video stream to be processed, and based on the current processing position of the video stream to be processed, determine the current video segment and the look-ahead video interval following the current video segment; The prospective feature construction module is used to sample at least one prospective sampling frame from the prospective video interval, extract prospective features of each prospective sampling frame, and construct a prospective feature set based on the temporal position of each prospective sampling frame and the corresponding prospective features. The dynamic frame fusion and feature extraction module is used to perform frame fusion on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment, and extract the current state features of the representative frame. The strategy feature generation module is used to determine the target prospective features from the prospective feature set according to the temporal position of the current video segment, and to perform feature fusion between the current state features and the target prospective features to obtain the strategy input features; The strategy decision module is used to input the strategy input features into the strategy network to obtain the target fusion frame number corresponding to the next video segment; An adaptive compression control module is used to determine the next video segment from the video stream following the current video segment based on the target fusion frame number, take the next video segment as the new current video segment, and control the dynamic frame fusion and feature extraction module, the strategy feature generation module and the strategy decision module to run iteratively to obtain representative frames corresponding to each subsequent video segment. The compressed video generation module is used to organize the representative frames corresponding to each video segment according to the temporal order of each video segment to generate a compressed video stream.
[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research of existing technical problems, this application recognizes that in video stream compression processing, video content changes are highly dynamic, the adaptability between compression parameters and different video segments is insufficient, and it is difficult to balance the degree of data compression with the preservation of key information. This application determines the current video segment and its subsequent look-ahead video interval based on the current processing position of the video stream to be processed. Look-ahead sampling frames are sampled from the look-ahead video interval to construct a look-ahead feature set, enabling compression decisions to not only reflect the current video content but also obtain forward-looking information about subsequent video content. Frame fusion is performed on video frames in the current video segment to obtain representative frames and extract current state features. Then, based on the temporal position of the current video segment, target look-ahead features are determined from the look-ahead feature set, establishing a correspondence between the current content state and temporally related subsequent content features. By linking the current state features with the target look-ahead features… After feature fusion, the data is input into the policy network, enabling the network to determine the target fusion frame number for the next video segment based on the current video content and its subsequent trends. The target fusion frame number is then used to dynamically determine the number of video frames in the next video segment. Through iterative processing, different video segments are made to adopt a frame fusion degree adapted to their content changes. Thus, during periods of low video content change, the number of fusion frames is increased to reduce the number of representative frames; during periods of high video content change or high information value, the number of fusion frames is decreased to retain more temporal details. Finally, representative frames from each video segment are organized chronologically to generate a compressed video stream, thereby improving the adaptability of compression parameters to different video segments. This reduces the amount of video stream data, bandwidth, and storage resources consumed while minimizing the loss of details in important video content, achieving a balance between video data compression and the preservation of key information. Attached Figure Description
[0017] Figure 1 A flowchart illustrating an embodiment of the adaptive compression method for video streams provided in this application; Figure 2 This application provides a schematic diagram of the architecture of a video stream adaptive compression system based on policy networks and forward-looking features, as one embodiment of the present application. Figure 3 This application provides a flowchart illustrating an embodiment of an adaptive compression method for video streams based on policy networks and forward-looking features. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0019] In one embodiment, such as Figure 1 and Figure 3 As shown, an adaptive compression method for video streams is provided. This method can be applied to stored video, video-on-demand, real-time video streams with buffers, or other video data where limited look-ahead video content is available. For real-time video streams, a look-ahead buffer can be set up at the video receiver to store video frames received earlier but not yet processed in the current position. This allows the processing device to determine the look-ahead video range from the video content after the current processing position. For stored video, video frames after the current processing position can be read directly based on the video frame sequence number or timestamp.
[0020] Before step S101 or at the start of video stream processing, the video stream to be processed is acquired. The video stream to be processed may include raw video frames arranged consecutively in frame order, timestamps of each raw video frame, and video attributes such as video frame rate, image size, and color format. The current processing position may be represented by the frame number of the first video frame to be processed, the current playback time, or the video timestamp.
[0021] The current video segment is determined based on the current processing position. During the initial processing, a preset initial fusion frame count can be used to determine the current video segment. For example, if the preset initial fusion frame count is 3, then 3 video frames are read consecutively from the current processing position to form the initial current video segment. In subsequent processing, the number of video frames contained in the current video segment is determined by the target fusion frame count generated in the previous processing cycle.
[0022] Simultaneously, a look-ahead video interval is determined based on the end position of the current video segment. This look-ahead video interval is located after the current video segment, and its length can be determined according to a preset look-ahead duration or a preset look-ahead frame count. For example, the look-ahead video interval could be defined as 10 seconds of video content or 300 video frames following the current video segment. The length of the look-ahead video interval can be preset based on the video frame rate, buffer capacity, the computing power of the processing device, or the real-time requirements of the application scenario.
[0023] In step S101, at least one look-ahead sampled frame is sampled from the look-ahead video interval. Look-ahead sampling can be performed at the beginning of video stream processing or repeatedly during processing at a preset period. The look-ahead sampled frame is used to sparsely represent subsequent video content, so that the processing device does not need to analyze all video frames in the look-ahead video interval frame by frame.
[0024] In step S102, prospective features are extracted from each prospective sampled frame. These prospective features can be visual, motion, or semantic features. The feature extractor can employ a convolutional neural network, taking the resized and pixel-normalized prospective sampled frames as input and outputting the corresponding feature vectors. Alternatively, a visual Transformer or other image feature extraction models can be used. Each prospective feature is associated with and stored in relation to the timestamp or frame number of its corresponding prospective sampled frame, and organized according to temporal position to form a prospective feature set.
[0025] In step S103, the current video segment is received. The current video segment includes one or more video frames starting from the current processing position. When the current video segment includes only one video frame, that video frame can be directly used as the representative frame; when the current video segment includes multiple video frames, frame fusion is performed on the multiple video frames.
[0026] In step S104, frame fusion is performed on the video frames in the current video segment to obtain a representative frame corresponding to the current video segment. Frame fusion can be achieved by averaging corresponding pixel positions, weighted averaging corresponding pixel positions, or pixel fusion based on attention weights. In other embodiments, frame-level features of each video frame can be extracted first, the frame-level features can be fused, and then a representative frame can be generated through a feature reconstruction model. The representative frame centrally represents the main visual information in the current video segment. By replacing multiple original video frames in the current video segment with a single representative frame, data compression in the time dimension is achieved.
[0027] In step S105, the representative frame is input to the feature extractor to obtain the current state features. The feature extractor used to extract the current state features can share the network structure and model parameters with the feature extractor used to extract the prospective features, so that the current state features and prospective features have consistent feature dimensions and feature spaces; alternatively, different feature extractors can be used, and the two types of features can be transformed to the same or concatenable feature spaces through a linear mapping layer.
[0028] In step S106, target forward-looking features are determined from the look-ahead feature set based on the temporal position of the current video segment. Specifically, the end temporal position of the current video segment can be used as a reference temporal position, and the forward-looking feature that is located after the reference temporal position and has the smallest temporal distance from the reference temporal position is selected from the look-ahead feature set. Thus, the target forward-looking feature can represent the subsequent video content that is closest to the current video segment in time.
[0029] In step S107, the current state features and the target prospective features are concatenated to obtain the policy input features. When the two features have different dimensions, they can be transformed using a fully connected layer or a linear projection layer, and then concatenated in a preset order. The policy input features simultaneously contain the content state of the current video segment and prospective information of subsequent video content.
[0030] In step S108, the policy input features are input into a pre-trained policy network. In step S109, the policy network encodes and makes decisions on the policy input features, outputting the target fusion frame number corresponding to the next video segment. The policy network can be a multilayer perceptron, a recurrent neural network, or a Transformer network. The target fusion frame number can be selected from a preset fusion frame number candidate set, such as 1, 3, 5, and 7.
[0031] In one training implementation, training videos including smooth scenes, moving scenes, camera transition scenes, and key event scenes are collected. Current state features and target prospective features are generated for each training video segment, and corresponding policy input features are constructed. For each candidate fusion frame in a preset candidate set of fusion frames, frame fusion is performed on subsequent training video segments to obtain the corresponding compression result. Based on the compressed data volume, the visual quality difference between the representative frame and the original video content, and the degree of key content retention, a comprehensive evaluation result for each candidate fusion frame is determined, and the candidate fusion frame with the better comprehensive evaluation result is used as the training objective. Alternatively, reinforcement learning can be used to construct a reward value based on compression gains and content fidelity, and the policy network is trained by maximizing the cumulative reward.
[0032] When the policy network determines that subsequent video content changes significantly, has high information density, or contains important events, it can output a smaller number of target fusion frames. This results in the next video segment containing fewer original video frames, thereby generating more representative frames and retaining more temporal details. Conversely, when the policy network determines that subsequent video content changes little or has a high degree of repetition, it can output a larger number of target fusion frames. This results in the next video segment containing more original video frames, thereby reducing the number of representative frames and improving compression.
[0033] In step S110, the next video segment is determined from the video stream following the current video segment based on the target fusion frame number. For example, when the target fusion frame number is 1, one video frame is read from the current video segment to form the next video segment; when the target fusion frame number is 7, 7 video frames are read consecutively to form the next video segment. Frame fusion is then performed on the next video segment to obtain the representative frame corresponding to the next video segment.
[0034] The next video segment is used as the new current video segment. Current state features are extracted again, target prospective features are determined, policy input features are generated, and a new target fusion frame number is determined. This process is repeated cyclically along the video timeline to obtain representative frames for each subsequent video segment. During processing, the prospective video interval and prospective feature set can be updated synchronously to ensure that the prospective feature set always covers a preset range after the current processing position.
[0035] Representative frames are organized according to the temporal order of each video segment in the video stream to be processed, generating a compressed video stream. The representative frames can be directly encapsulated into video files at a preset frame rate, or they can be further sent to H.264, H.265, or other video encoders for spatial compression and bitstream encapsulation, thereby forming a transmittable or storable compressed video stream. Figure 3 Steps S111 to S114 are optional importance scoring and condensed video generation processes; after all video frames have been processed, the processing ends in step S115.
[0036] Through the above implementation method, the forward-looking features in the look-ahead video interval are used to supplement the state information of the current video segment, enabling the policy network to dynamically determine the target fusion frame number of the next video segment by combining the current content and subsequent content. The compression degree is adjusted by changing the number of original video frames contained in each video segment, thereby improving the adaptability between compression parameters and changes in video content. When the video content is flat, the amount of data is reduced, and when the video content is important or changes drastically, more temporal details are retained, thus balancing the degree of video data compression and the effect of retaining key information.
[0037] In one embodiment, video frames within the time range of the look-ahead video interval are read from the video stream to be processed, forming a candidate frame sequence, based on the start and end time positions of that interval. The time positions can be represented by video frame numbers or video timestamps. The video frames in the candidate frame sequence are arranged in their original time order.
[0038] When sampling using a preset time interval, the sampling timing position can be determined according to a fixed frame interval or a fixed time interval. For example, if the lookahead video interval contains 300 video frames, a sampling timing position can be determined every 10 frames; if the duration of the lookahead video interval is 10 seconds, a sampling timing position can also be determined every 0.5 seconds. The sampling timing positions form a set of sampling positions.
[0039] When using random sampling rules, multiple non-repeating sampling time positions can be randomly generated within the look-ahead video interval according to a preset sampling number. To ensure that different time segments within the look-ahead video interval have a chance to be sampled, the look-ahead video interval can be divided into multiple sub-intervals, and at least one sampling time position can be randomly selected from each sub-interval.
[0040] When using content complexity-based sampling rules, we can first calculate the pixel differences, feature differences, motion amplitude, or image information entropy between adjacent video frames in the candidate frame sequence to obtain the content complexity sequence corresponding to the candidate frame sequence. Smaller sampling intervals are set for periods with higher content complexity, and larger sampling intervals are set for periods with lower content complexity, thereby obtaining a set of sampling positions that matches the degree of content change.
[0041] Based on the set of sampling locations, the corresponding video frames are extracted from the candidate frame sequence to obtain the lookahead sampling frames. For cases where the sampling location does not completely coincide with the timestamp of the actual video frame, the video frame with the smallest time distance from that sampling location can be selected as the lookahead sampling frame.
[0042] At the start of video stream processing, an initial sampling can be performed. During processing, when the current processing position moves backward by a preset number of frames, a preset duration has elapsed, or the number of remaining valid look-ahead sampled frames is less than a preset number, look-ahead sampling can be performed again, so that the newly sampled look-ahead sampled frames cover a new look-ahead video interval.
[0043] Through the above implementation method, candidate frame sequences can be determined according to the temporal range of the prospective video interval, and representative prospective sampled frames can be obtained by using uniform sampling, random sampling, or sampling methods adapted to the content complexity. This reduces the amount of prospective video content analysis while retaining the change information of subsequent video content, providing a stable data foundation for prospective feature extraction and subsequent compression decisions.
[0044] In one embodiment, after obtaining each look-ahead sampled frame, the look-ahead sampled frames are resized, converted in color space, and normalized in pixels to make each look-ahead sampled frame meet the input requirements of the feature extractor. The processed look-ahead sampled frames are then input into the feature extractor to obtain the look-ahead features corresponding to each look-ahead sampled frame.
[0045] Feature extractors can employ convolutional neural networks pre-trained for tasks such as image classification, object recognition, or video understanding. Forward-looking features can be visual features output from intermediate layers of the convolutional neural network, or high-level semantic features output from the network's endpoints. Visual features are used to represent visual information such as color, texture, edges, and object shape, while high-level semantic features are used to represent scene categories, object categories, or event semantics.
[0046] Each prospective feature is normalized to ensure a consistent numerical range. Each prospective feature is then associated with the frame number or timestamp of its corresponding prospective sampling frame to form a prospective feature entry. A prospective feature entry includes at least a temporal location field and a feature data field, and may also include fields such as sampling method, feature extraction time, or feature validity status.
[0047] Multiple prospective feature entries are sorted according to their temporal position, and a temporal index is created to obtain the prospective feature set. The prospective feature set can be stored in memory, a cache database, or a feature store. Based on the temporal index, prospective feature entries located after a specified temporal position can be quickly retrieved.
[0048] As the current processing position moves forward, the temporal position of some look-ahead feature entries will be no later than the current processing position, rendering these look-ahead feature entries useless. Therefore, a preset update position can be set. The preset update position can be the position after processing a preset number of video segments, or the position when the temporal distance between the current processing position and the last look-ahead feature entry in the look-ahead feature set is less than a preset distance.
[0049] When the current processing position reaches the preset update position, the look-ahead video interval is redefined based on the updated current processing position. Following the sampling method described in the above embodiments, supplementary look-ahead sampling frames are sampled from the updated look-ahead video interval, and the corresponding supplementary look-ahead features are extracted.
[0050] Remove prospective feature entries whose temporal positions are no later than the updated current processing position from the original prospective feature set to obtain retained prospective feature entries. Associate supplementary prospective features with their corresponding temporal positions to form supplementary prospective feature entries. Add the supplementary prospective feature entries to the retained prospective feature sets and reorganize them according to temporal positions to obtain the updated prospective feature set.
[0051] Through the above implementation method, the feature data of the look-ahead sampling frame is stored in correspondence with the temporal position. As the current processing position changes, invalid features are deleted and new look-ahead features are added, so that the look-ahead feature set continuously represents the video content after the current processing position. This can not only improve the accuracy of target look-ahead feature query, but also avoid invalid features occupying storage space for a long time.
[0052] In one embodiment, pixel data corresponding to each video frame in the current video segment is obtained. To ensure that the corresponding pixel positions of different video frames have consistent meaning, each video frame can be adjusted to the same image size and color space, and the pixel data of each video frame can be normalized to obtain a set of pixel data to be fused.
[0053] When camera shake or object displacement exists in the current video clip, image registration can be performed before frame fusion. Specifically, based on feature point matching results, optical flow results, or camera motion parameters, each video frame can be transformed to a unified spatial coordinate system, so that the same scene location has a corresponding pixel position in each video frame.
[0054] The fusion weight for each video frame is determined based on its frame order, content similarity, or frame importance. When determining fusion weights based on frame order, frames closer to the center of the current video segment receive higher fusion weights. When determining fusion weights based on content similarity, frames more closely related to the overall content of the current video segment receive higher fusion weights. When determining fusion weights based on frame importance, frames containing significant objects, moving targets, or scene changes receive higher fusion weights.
[0055] The sum of the fusion weights corresponding to each video frame can be normalized to 1. For each pixel location, the pixel value of that location is read from each video frame and weighted according to the fusion weights of each video frame to obtain the fused pixel value for that location. This weighting process is performed on all pixel locations to obtain the fused pixel data.
[0056] Representative frames are generated based on the fused pixel data, according to the image size and color format of the original video frames. When the current video segment contains only one video frame, the fusion weight of that video frame is 1, and it can be directly identified as the representative frame. When multiple video frames have the same fusion weight, this process corresponds to averaging the corresponding pixel values of each video frame.
[0057] After generating the representative frame, it can be input into the current state feature extractor to extract the current state features corresponding to the current video segment. Simultaneously, the representative frame, along with the start and end time positions of the corresponding video segment, is stored together for subsequent generation of a compressed video stream in temporal order.
[0058] By implementing the above methods, the pixel data at the corresponding positions of each video frame in the current video segment are weighted and fused. This can retain the common visual information of multiple video frames while replacing multiple original video frames with a representative frame. Furthermore, by using fusion weights, the video frames with higher information value are highlighted, thereby reducing the time redundancy and data size of the video stream.
[0059] In one embodiment, each video frame in the current video segment is input into a frame-level feature extractor to obtain the frame-level representation features corresponding to each video frame. The frame-level representation features can be feature vectors output by a convolutional neural network, used to describe the scene content, object information, and local visual structure of the video frame. A set of frame-level representation features is formed by these individual frame-level representation features.
[0060] The frame-level representation features in the frame-level representation feature set are aggregated to obtain the segment aggregated features corresponding to the current video segment. Aggregation processing can employ feature averaging, max pooling, or a learnable feature aggregation network. The segment aggregated features are used to represent the overall content of the current video segment.
[0061] The correlation between each frame-level representation feature and the segment aggregation feature is determined to obtain the content correlation value for each video frame. The correlation degree can be determined using vector dot product, cosine similarity, or a correlation function learned by a neural network. The larger the content correlation value, the higher the correlation between the corresponding video frame and the overall content of the current video segment.
[0062] The content association values are input into the attention weight generation model. The attention weight generation model can include fully connected layers and normalization layers. The fully connected layers perform non-linear mapping on the content association values, and the normalization layers convert each output value into a normalized attention weight, so that the sum of the normalized attention weights of all video frames is 1.
[0063] Each normalized attention weight is determined as the fusion weight for the corresponding video frame, and the video frames in the current video segment are weighted and fused according to the pixel fusion method described above. Video frames with a high degree of correlation with the overall content of the current video segment contribute more to the representative frame, while video frames with a low degree of correlation or containing occasional noise contribute less to the representative frame.
[0064] In another embodiment, frame sequence position features, motion amplitude features, or scene change features can be input together with frame-level representation features into the attention weight generation model, so that the fused weights can simultaneously reflect the content relevance, temporal position, and degree of motion change of the video frames.
[0065] Through the above implementation method, normalized attention weights are generated by utilizing the degree of correlation between each video frame and the overall content of the segment. This enables the frame fusion process to adaptively distinguish the contribution of different video frames to the main content of the segment, reduce the weakening of important visual information caused by average fusion, and improve the ability of representative frames to represent the main content of the current video segment.
[0066] In one embodiment, the end-time position of the current video segment is obtained and determined as the reference temporal position. Using the end-time position as the reference temporal position ensures that the selected target forward-looking features are located after the current video segment, avoiding the misinterpretation of video content within the current video segment or already processed as forward-looking information.
[0067] Based on the temporal positions of each prospective feature entry in the prospective feature set, prospective feature entries whose temporal positions are later than the reference temporal position are selected to obtain a candidate prospective feature entry set. When the prospective feature set is established with a temporal index, the first prospective feature entry after the reference temporal position can be directly located based on the temporal index, and this entry, along with the prospective feature entries following it, can be identified as candidate prospective feature entries.
[0068] Calculate the temporal distance between the temporal position of each candidate look-ahead feature entry and the reference temporal position. When using frame number to represent the temporal position, the temporal distance is the difference between the corresponding frame numbers; when using timestamp to represent the temporal position, the temporal distance is the difference between the corresponding timestamps.
[0069] The temporal distances of each candidate look-ahead feature entry are compared, and the candidate look-ahead feature entry with the smallest temporal distance is determined as the target look-ahead feature entry. The look-ahead features in the target look-ahead feature entry are then read to obtain the target look-ahead feature. This target look-ahead feature corresponds to the look-ahead sampling frame that is temporally closest to the current video segment after the current video segment.
[0070] If there are no look-ahead feature entries in the look-ahead feature set whose temporal position is later than the reference temporal position, the look-ahead feature set update process can be triggered, and the target look-ahead features can be re-determined from the updated look-ahead feature set. If a complete look-ahead video interval cannot be formed at the end of the video stream to be processed, the look-ahead video interval can be shortened, and look-ahead sampling frames can be sampled from the remaining unprocessed video frames; when there are no remaining video frames, the iterative processing can be terminated.
[0071] By using the above implementation method, based on the end position of the current video segment, the forward-looking feature that is located after it and has the smallest temporal distance is selected from the forward-looking feature set. This enables the target forward-looking feature to accurately reflect the subsequent video content adjacent to the current video segment and reduces the decision-making bias caused by the excessive time span of the forward-looking feature.
[0072] In one embodiment, current state features and target prospective features are obtained. The current state features represent the main visual content obtained after frame fusion of the current video segment, and the target prospective features represent the subsequent video content after the current video segment.
[0073] When the current state features and the target prospective features have different dimensions, they are respectively input into the corresponding feature projection layer, which transforms the two features into a preset dimension. The feature projection layer can be a fully connected layer, or a feature transformation structure that includes a fully connected layer, an activation layer, and a normalization layer.
[0074] The current state features and target prospective features, after dimensionality adjustment, are concatenated in a preset order to obtain the policy input features. The policy input features can retain the boundary between the current feature part and the prospective feature part, enabling the policy network to learn the changing relationship between the current video content and subsequent video content.
[0075] The policy input features are input into the policy network to obtain the fusion frame number decision result. The fusion frame number decision result can be a set of decision values that correspond one-to-one with each candidate fusion frame number in the preset fusion frame number candidate set, or it can be a set of selection probabilities corresponding to each candidate fusion frame number.
[0076] The preset candidate set of fusion frame numbers can be pre-configured based on the video frame rate and compression requirements. For example, the preset candidate set of fusion frame numbers can include 1, 3, 5, and 7. The smaller the candidate fusion frame number, the fewer the original video frames in the next video segment, and the more representative frames generated per unit time; the larger the candidate fusion frame number, the more original video frames in the next video segment, and the fewer representative frames generated per unit time.
[0077] Based on the fusion frame number decision result, a target candidate fusion frame number is selected from the preset fusion frame number candidate set, and this target candidate fusion frame number is determined as the target fusion frame number for the next video segment. Based on the target fusion frame number, the corresponding number of video frames are continuously read from the current video segment to form the next video segment.
[0078] Through the above implementation method, the current state features of the current video segment and the target prospective features of the subsequent video content are combined to form the policy input features. The policy network then determines the target fusion frame number from multiple candidate fusion frame numbers, so that the time span of the next video segment can be adaptively adjusted according to the changes in the current content and subsequent content, thereby improving the targeting and continuity of compression control.
[0079] In one embodiment, the policy network can employ a feedforward neural network. The feedforward neural network may include multiple fully connected layers, which are used to map the policy input features layer by layer to obtain a policy decision representation. Feedforward neural networks are suitable for making rapid decisions based on the current state features and forward-looking features of the target within a single processing cycle.
[0080] Recurrent neural networks (RNNs) can also be used as policy networks. In addition to receiving the policy input features of the current processing cycle, RNNs can also receive the hidden states of the previous processing cycle, generating policy decision representations based on changes in video content over multiple consecutive processing cycles. Therefore, the determination of the target fusion frame number can simultaneously consider current and future information, as well as the changing trends of multiple previous video segments.
[0081] The policy network can also employ a Transformer network. A Transformer network can use current state features, target prospective features, and features from one or more historical processing cycles as a feature sequence, and utilize a self-attention mechanism to calculate the correlation between these temporal features, thus obtaining a policy decision representation. This approach is suitable for video streams with long spans of content changes.
[0082] The policy decision representation is input into the decision mapping layer. The output dimension of the decision mapping layer is consistent with the number of candidate fusion frames in the preset fusion frame candidate set. Each output position corresponds to a candidate fusion frame, thereby obtaining the decision value corresponding to each candidate fusion frame.
[0083] The decision values corresponding to each candidate fusion frame number are compared to determine the maximum decision value, and the candidate fusion frame number corresponding to the maximum decision value is determined as the target candidate fusion frame number. When outputting the fusion frame number decision result in the form of selection probability, each decision value can be normalized first to obtain the selection probability corresponding to each candidate fusion frame number, and then the candidate fusion frame number corresponding to the maximum selection probability can be determined as the target candidate fusion frame number.
[0084] When training the policy network, a smaller number of candidate fusion frames can correspond to a lower level of compression and a higher level of content preservation, while a larger number of candidate fusion frames can correspond to a higher level of compression and a lower data footprint. The policy network parameters are adjusted based on the content complexity, compression quality, and data compression rate of different training video segments, enabling the policy network to learn to select the appropriate number of target fusion frames under different video content conditions.
[0085] Through the above implementation methods, the policy input features are encoded using a feedforward neural network, a recurrent neural network, or a Transformer network, and the target candidate fusion frame number is determined by comparing the decision values of the candidate fusion frame number. This ensures that the output of the fusion frame number has a clear candidate range and selection rules, guaranteeing that the adaptive compression decision can be stably converted into executable video segmentation parameters.
[0086] In one embodiment, such as Figure 2 and Figure 3 As shown, in addition to completing the adaptive compression of the video stream, importance scores can be generated based on the number of fused frames of each video segment, and then a condensed video can be generated. Figure 3 Step S111 is used to determine whether to perform the importance scoring and condensed video generation process; when this process is performed, steps S112 to S114 are executed sequentially.
[0087] The number of fused frames used to determine each video segment is obtained. For the initial video segment, the number of fused frames is a preset initial fused frame number; for each video segment after the initial video segment, the number of fused frames is the output of the policy network from the previous processing cycle, used to determine the target fused frame number for that video segment. Thus, there is a clear temporal correspondence between the number of fused frames and the video segments formed by the segmentation.
[0088] In step S112, the segment importance score for each video segment is determined based on the segment fusion frame number corresponding to each video segment and a preset importance mapping relationship. The preset importance mapping relationship makes the segment importance score negatively correlated with the segment fusion frame number.
[0089] For example, the reciprocal of the number of fused frames, or its normalized reciprocal, can be used to determine the segment importance score. Alternatively, a lookup table can be set up to map fused frame numbers of 1, 3, 5, and 7 to different importance levels or scores. A smaller number of fused frames indicates that the policy network believes the corresponding video segment needs to retain more temporal details, thus resulting in a higher segment importance score; conversely, a larger number of fused frames results in a lower segment importance score.
[0090] Each video segment is assigned a segment importance score to each original video frame within that segment, resulting in a frame importance score for each original video frame. In one implementation, all original video frames within the same video segment have the same frame importance score. In another implementation, the segment importance score can be locally adjusted based on the frame order position, salience, or similarity of the original video frame within the video segment to obtain different frame importance scores.
[0091] In step S113, after processing the preset video interval or the entire video stream to be processed, the frame importance score corresponding to each original video frame is collected, and the frame sequence number, timestamp and frame importance score of the original video frame are associated and stored.
[0092] In step S114, the original video frames in the video stream to be processed are sorted in descending order according to their importance scores to obtain a video frame sorting result. Based on the video frame sorting result, a preset number of target video frames are selected from the top-ranked original video frames. The preset number can be a fixed number of frames, or it can be determined based on the target condensed video duration, the original video duration, or a preset condensation ratio.
[0093] After selecting the target video frames, they are not directly arranged according to their importance scores. Instead, they are reorganized in ascending order based on their original temporal positions within the video stream to be processed, preserving the original development sequence of the video content and generating a condensed video. The condensed video can be composed of discrete target video frames, or a predetermined number of adjacent video frames can be retained before and after each target video frame to form multiple key video segments, which are then combined according to their original temporal order.
[0094] Through the above implementation method, the number of fused frames used to determine each video segment is used as the evaluation criterion for the importance of the content. The compression decision is converted into an importance score of the video segment and the original video frame. Based on the importance score, key video frames are selected to generate a condensed video. This allows the decision results of the same strategy network to simultaneously serve adaptive compression and key information extraction, reducing the additional key frame analysis process and improving the ability of the condensed video to retain important video content.
[0095] In one embodiment, such as Figure 2 As shown, this embodiment provides a video stream adaptive compression system based on policy networks and forward-looking features. The system includes a video stream processing module, a forward-looking feature construction module, a dynamic frame fusion and feature extraction module, a policy feature generation module, a policy decision-making module, an adaptive compression control module, and a compressed video generation module. Each module can be implemented through software programs or by a processor, graphics processor, neural network accelerator, memory, and communication interface. Multiple modules can be deployed in the same computing device or distributed across video acquisition terminals, edge computing nodes, or servers. Figure 2 The data flow of the module shown corresponds to the method processing procedure.
[0096] The video stream processing module acquires the video stream to be processed and determines the current video segment and the look-ahead video interval based on the current processing position of the video stream. The video stream processing module may include a video receiving unit, a buffering unit, and a segmentation unit. The video receiving unit receives the raw video data stream, the buffering unit buffers video frames after the current processing position, and the segmentation unit divides the video stream to be processed into continuous video segments according to a preset initial fusion frame number or the target fusion frame number output by the strategy decision module.
[0097] The look-ahead feature construction module is used to sample look-ahead frames from the look-ahead video interval, extract look-ahead features, and construct a look-ahead feature set. The look-ahead feature construction module may include a look-ahead sampling unit, a feature extraction unit, and a feature management unit. The look-ahead sampling unit determines the look-ahead frames using uniform sampling, random sampling, or content complexity-based sampling methods; the feature extraction unit generates look-ahead features using feature extractors such as convolutional neural networks; and the feature management unit stores the look-ahead features in association with their temporal positions and performs deletion, supplementation, and updates to the look-ahead feature set.
[0098] The dynamic frame fusion and feature extraction module receives the current video segment, performs frame fusion on one or more video frames within the segment to generate a representative frame, and extracts current state features from the representative frame. The module can determine fusion weights based on the frame order, content similarity, or frame importance of each video frame, and then weights and fuses the pixel values at corresponding pixel positions according to these weights.
[0099] The strategy feature generation module is used to determine the target forward-looking features from the forward-looking feature set based on the temporal position of the current video segment, and to fuse the current state features and the target forward-looking features. The strategy feature generation module may include a temporal matching unit, a dimensionality transformation unit, and a feature concatenation unit. The temporal matching unit selects the forward-looking feature that is closest in temporal distance to the current video segment; the dimensionality transformation unit adjusts the dimensions of the two types of features; and the feature concatenation unit generates the strategy input features.
[0100] The policy decision module inputs policy input features into the policy network to obtain the target fusion frame number for the next video segment. The policy network can be a multilayer perceptron, a recurrent neural network, or a Transformer network. The policy decision module maps the policy decision representation output by the policy network to each candidate fusion frame number, obtaining the decision value corresponding to each candidate fusion frame number, and determines the candidate fusion frame number corresponding to the maximum decision value as the target fusion frame number.
[0101] The adaptive compression control module receives the target fusion frame number output by the strategy decision module and sends a segment division control command to the video stream processing module, so that the video stream processing module reads the number of video frames corresponding to the target fusion frame number from the video stream after the current video segment to form the next video segment.
[0102] The adaptive compression control module also sets the next video segment as the new current video segment and controls the dynamic frame fusion and feature extraction module, the strategy feature generation module, and the strategy decision module to run sequentially. Through loop control, the system obtains representative frames of each subsequent video segment along the timeline of the video stream to be processed.
[0103] The compressed video generation module receives representative frames and their corresponding timing positions for each video segment, organizes the representative frames according to the timing order of the video segments, and generates a compressed video stream. The compressed video generation module can further call a video encoder to perform intra-frame encoding, inter-frame encoding, and bitstream encapsulation on the representative frames, outputting a compressed video file or compressed video bitstream suitable for transmission or storage.
[0104] In an optional implementation, the system may further include an importance scoring module and a condensed video generation module. The importance scoring module generates segment importance scores and frame importance scores based on the number of fused frames for each video segment; the condensed video generation module selects a preset number of target video frames based on the frame importance scores and generates a condensed video according to the original temporal positions of the target video frames. Figure 1 The importance rating branch and the condensed video output branch can be set in parallel with the compressed video stream output branch, enabling the system to output compressed video stream and condensed video simultaneously.
[0105] Through data transmission and control coordination among the above system modules, the forward-looking feature construction module can provide sparse feature representations of subsequent video content, the strategy decision module can determine the target fusion frame number by combining the current state features and the target forward-looking features, and the adaptive compression control module can dynamically adjust the number of frames of subsequent video segments according to the target fusion frame number, thereby forming a continuous closed-loop adaptive compression process for the video stream, improving the adaptability between the degree of video compression and content changes, and maintaining the detailed information of important video content while reducing bandwidth and storage resource consumption.
[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A video stream adaptive compression method, characterized in that, The method includes: Obtain the video stream to be processed, and based on the current processing position of the video stream to be processed, determine the current video segment and the look-ahead video interval following the current video segment; At least one forward-looking sampled frame is sampled from the forward-looking video interval, forward-looking features of each forward-looking sampled frame are extracted, and a forward-looking feature set is constructed based on the temporal position of each forward-looking sampled frame and the corresponding forward-looking features. Frame fusion is performed on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment, and the current state features of the representative frame are extracted. Based on the temporal position of the current video segment, target prospective features are determined from the prospective feature set, and the current state features and the target prospective features are fused to obtain the strategy input features; The policy input features are input into the policy network to obtain the target fusion frame number corresponding to the next video segment; Based on the target fusion frame number, the next video segment is determined from the video stream following the current video segment, and the video frames in the next video segment are fused to obtain the representative frame corresponding to the next video segment; The next video segment is taken as the new current video segment, and the steps of current state feature extraction, target prospective feature determination, target fusion frame number determination and video frame fusion are iteratively executed to obtain the representative frame corresponding to each subsequent video segment. Organize the representative frames corresponding to each video segment according to their temporal order to generate a compressed video stream.
2. The video stream adaptive compression method according to claim 1, characterized in that, The step of sampling at least one look-ahead sampled frame from the look-ahead video interval includes: Based on the start and end time positions of the look-ahead video interval, a candidate frame sequence is determined from the video stream to be processed; Based on the temporal range corresponding to the candidate frame sequence, at least one sampling temporal position is determined according to a preset time interval, random sampling rules, or sampling rules based on content complexity, to obtain a set of sampling positions; The video frames corresponding to each sampling time position in the sampling position set are extracted from the candidate frame sequence to obtain the look-ahead sampling frames.
3. The video stream adaptive compression method according to claim 1, characterized in that, The step of extracting prospective features from each of the prospective sampling frames and constructing a prospective feature set based on the temporal position of each prospective sampling frame and its corresponding prospective features includes: Each of the prospective sampling frames is input into a feature extractor to extract the visual or semantic features of each prospective sampling frame, thereby obtaining the prospective features corresponding to each prospective sampling frame. Each of the aforementioned forward-looking features is associated with the temporal position of the corresponding forward-looking sampling frame to obtain multiple forward-looking feature entries; According to the temporal position corresponding to each of the aforementioned prospective feature entries, the prospective feature set is obtained by temporally organizing the multiple prospective feature entries; When the current processing position reaches the preset update position, the updated look-ahead video interval is determined based on the updated current processing position, and supplementary look-ahead sampling frames are sampled from the updated look-ahead video interval to obtain supplementary look-ahead features. Delete prospective feature entries whose temporal positions are no later than the updated current processing position from the prospective feature set, and add the supplementary prospective features and their temporal positions to the prospective feature set to obtain the updated prospective feature set.
4. The video stream adaptive compression method according to claim 1, characterized in that, The step of performing frame fusion on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment includes: Obtain the pixel data corresponding to each video frame in the current video segment to obtain a set of pixel data to be fused; The fusion weights corresponding to each video frame are determined based on their frame order, content similarity, or frame importance. Based on the fusion weights corresponding to each video frame, the pixel values at corresponding pixel positions in each video frame are weighted and fused to obtain fused pixel data; The representative frame corresponding to the current video segment is generated based on the fused pixel data.
5. The video stream adaptive compression method according to claim 4, characterized in that, The step of determining the fusion weight corresponding to each video frame based on the content similarity of each video frame includes: Frame-level representation features are extracted from each video frame in the current video segment to obtain a set of frame-level representation features; The frame-level representation features in the frame-level representation feature set are aggregated to obtain the segment aggregated features corresponding to the current video segment; The degree of correlation between each frame-level representation feature and the segment aggregation feature is determined to obtain the content association value corresponding to each video frame; The content association values are input into the attention weight generation model, and the output of the attention weight generation model is normalized to obtain the normalized attention weights corresponding to each video frame. The normalized attention weights are determined as the fusion weights for the corresponding video frames.
6. The video stream adaptive compression method according to claim 1, characterized in that, The step of determining the target prospective features from the prospective feature set based on the temporal position of the current video segment includes: Based on the end timing position of the current video segment, determine the reference timing position corresponding to the current video segment; From the prospective feature set, prospective feature entries whose temporal positions are later than the reference temporal positions are selected to obtain a candidate prospective feature entry set; The temporal distance between the temporal position of each candidate prospective feature entry in the candidate prospective feature entry set and the reference temporal position is determined respectively, and the temporal distance corresponding to each candidate prospective feature entry is obtained. The target prospective feature entry with the smallest corresponding temporal distance is determined from the set of candidate prospective feature entries, and the prospective features contained in the target prospective feature entry are determined as the target prospective features.
7. The video stream adaptive compression method according to claim 1, characterized in that, The step of fusing the current state features and the target prospective features to obtain policy input features, and then inputting the policy input features into the policy network to obtain the target fused frame number corresponding to the next video segment, includes: The current state features and the target prospective features are concatenated to obtain the policy input features; The policy input features are input into the policy network to obtain the fused frame number decision result; Based on the fusion frame number decision result, a target candidate fusion frame number is determined from a preset fusion frame number candidate set; The target candidate fusion frame number is determined as the target fusion frame number corresponding to the next video segment.
8. The video stream adaptive compression method according to claim 7, characterized in that, The policy network includes at least one of a feedforward neural network, a recurrent neural network, or a Transformer network. The step of determining the target candidate fusion frame number from a preset candidate fusion frame number set based on the fusion frame number decision result includes: The policy input features are encoded using the policy network to obtain a policy decision representation; The strategy decision representation is mapped to each candidate fusion frame number in the preset fusion frame number candidate set to obtain the decision value corresponding to each candidate fusion frame number; The decision values corresponding to each of the candidate fusion frame numbers are compared to obtain the maximum decision value; The number of candidate fusion frames corresponding to the maximum decision value is determined as the target number of candidate fusion frames.
9. The video stream adaptive compression method according to claim 1, characterized in that, Also includes: Obtain the number of fusion frames for determining each video segment, wherein the number of fusion frames corresponding to the initial video segment is a preset initial fusion frame number, and the number of fusion frames corresponding to each video segment after the initial video segment is the target fusion frame number output in the previous processing cycle. Based on the segment fusion frame number corresponding to each video segment and the preset importance mapping relationship, the segment importance score corresponding to each video segment is determined, wherein the segment importance score is negatively correlated with the corresponding segment fusion frame number; Assign the segment importance score corresponding to each video segment to the original video frame in the corresponding video segment to obtain the frame importance score corresponding to each original video frame; The original video frames in the video stream to be processed are sorted according to the importance scores of each frame to obtain the video frame sorting result; Based on the video frame sorting results, a preset number of target video frames are selected, and each target video frame is organized temporally according to its original temporal position in the video stream to be processed, thereby generating a condensed video.
10. A video stream adaptive compression system, characterized in that, The system includes: The video stream processing module is used to acquire the video stream to be processed, and based on the current processing position of the video stream to be processed, determine the current video segment and the look-ahead video interval following the current video segment; The prospective feature construction module is used to sample at least one prospective sampling frame from the prospective video interval, extract prospective features of each prospective sampling frame, and construct a prospective feature set based on the temporal position of each prospective sampling frame and the corresponding prospective features. The dynamic frame fusion and feature extraction module is used to perform frame fusion on the video frames in the current video segment to obtain the representative frame corresponding to the current video segment, and extract the current state features of the representative frame. The strategy feature generation module is used to determine the target prospective features from the prospective feature set according to the temporal position of the current video segment, and to perform feature fusion between the current state features and the target prospective features to obtain the strategy input features; The strategy decision module is used to input the strategy input features into the strategy network to obtain the target fusion frame number corresponding to the next video segment; An adaptive compression control module is used to determine the next video segment from the video stream following the current video segment based on the target fusion frame number, take the next video segment as the new current video segment, and control the dynamic frame fusion and feature extraction module, the strategy feature generation module and the strategy decision module to run iteratively to obtain representative frames corresponding to each subsequent video segment. The compressed video generation module is used to organize the representative frames corresponding to each video segment according to the temporal order of each video segment to generate a compressed video stream.
