A lightweight video coding method and system based on machine learning
By generating a context credibility graph and region partitioning, combined with an improved deep context video compression model, the problems of repetitive computation and difficulty in coordinating encoding quality and computational complexity in existing video coding methods are solved, and the coding stability and efficiency are improved in complex dynamic scenarios.
Patent Information
- Application Number
- CN202611315086.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-27
- Publication Date
- 2026-11-17
AI Technical Summary
Existing machine learning-based video coding methods suffer from problems such as repetitive computation, difficulty in coordinating coding quality and computational complexity when dealing with high-resolution and dynamic scenes, and the entropy coding probabilistic model fails to dynamically adjust according to contextual credibility and motion correction error, resulting in inconsistencies between encoding and decoding.
By acquiring the video to be encoded, the target encoding quality, and the device's computing power, a context credibility graph is generated, and stable reuse, alignment correction, and local refresh regions are divided. An improved deep context video compression model is used for differential quantization and entropy coding, a joint execution routing graph is generated, and the context credibility parameters are updated.
It improves encoding stability in complex and dynamic scenarios, adapts to mobile terminals and edge devices, reduces invalid calculations and reconstruction quality degradation, and ensures encoding and decoding consistency and video compression efficiency.
Smart Images

Figure CN123123577A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of video coding and machine learning technology, specifically to a lightweight video coding method and system based on machine learning. Background Technology
[0002] With the rapid development of mobile terminals, video surveillance, remote conferencing, short video platforms, and edge intelligent devices, the scale of video data generation continues to increase. Video data is typically characterized by high resolution, high frame rate, and large data volume. Effective encoding is required during transmission, storage, and real-time processing to reduce the bandwidth and storage space occupied by the video stream. Traditional video coding methods mainly remove spatial and temporal redundancy between video frames through processes such as intra-frame prediction, inter-frame prediction, motion estimation, motion compensation, transform quantization, and entropy coding. Although these methods have relatively mature encoding workflows, their encoding tool combinations are complex and often consume significant computational resources in high-resolution video and dynamic scenes.
[0003] In recent years, machine learning-based video coding methods have extracted video frame features through neural networks and used the learned motion information, contextual information, and probability distributions to complete video compression. Compared to fixed coding tools, machine learning models can learn more flexible feature representations and probability models based on video content, improving video bitrate distortion performance. Among them, deep contextual video compression models generate temporal context features by reconstructing preceding frames and use these features to assist in encoding the current frame, reducing redundant information between adjacent video frames.
[0004] However, existing deep contextual video compression methods typically employ a fixed network structure to process the entire video frame, performing the same depth of network computation on texture-stable regions, local motion regions, newly added target regions, and occlusion recovery regions. For stable regions with minimal contextual changes, the fixed coding structure easily leads to redundant computation; for regions with invalid context or abrupt content changes, continuing to use historical context for prediction can easily introduce erroneous information, causing reconstruction errors to accumulate and propagate between consecutive video frames. Meanwhile, some lightweight processing methods mainly employ network pruning, parameter sharing, low-precision quantization, or depthwise separable convolution to reduce model computation, without dynamically adjusting the coding path based on the contextual reliability of different regions within the video frame, making it difficult to simultaneously balance coding quality, bitrate, and computational complexity.
[0005] Furthermore, existing methods typically perform separate bitrate control and model lightweighting control, lacking unified coordination between bitrate budget and device computing power. When the computing power of the encoding device changes, a fixed network structure struggles to dynamically adjust the number of network layers, feature channels, and feature scales according to the current device state. When a dynamic network structure is used at the encoding end, inconsistencies in encoding and decoding may occur if the decoding end cannot accurately recover the corresponding execution path. Existing entropy coding probabilistic models also have a relatively fixed dependence on historical context, failing to dynamically adjust the probability distribution based on contextual reliability and motion correction errors, which can easily reduce the accuracy of probability estimation for quantized latent features.
[0006] Therefore, how to provide a lightweight video coding method and system based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] One objective of this invention is to propose a lightweight video coding method and system based on machine learning. This invention acquires the video to be encoded, the target coding quality, and the device's computing power; completes frame group partitioning and scene analysis; and establishes a bitrate budget and a computational budget. Based on preceding reconstructed frames, historical reconstruction distortion, and motion matching information, it extracts multi-scale temporal context features and generates a context credibility map. According to the context credibility map and the degree of local change in the current frame, the current frame is divided into a stable reuse region, an alignment correction region, and a local refresh region, forming a three-path coding mapping map. The current frame features, temporal context features, the three-path coding mapping map, and the two types of budgets are input into an improved deep context video compression model to generate a joint execution routing map, compression potential features, and routing assistance information. Based on the context credibility map and the joint execution routing map, the basic probability distribution of the compression potential features is corrected, and differentiated quantization and entropy coding are performed. Based on the routing assistance information, synchronous decoding and video frame reconstruction are completed, the context drift is calculated, and the context credibility parameters and context memory are updated, thereby achieving lightweight video coding under the collaborative constraints of video content, coding bitrate, and device computing power.
[0008] A lightweight video coding method and system based on machine learning according to an embodiment of the present invention includes the following steps: S1. Obtain the video to be encoded, the target encoding quality, and the device's computing power; complete frame group division and scene analysis; and establish bitrate budget and computational cost budget. S2. Based on the preceding reconstructed frames, historical reconstruction distortion, and motion matching information, extract multi-scale temporal context features and generate a context credibility map; S3. Based on the context credibility graph and the degree of local change in the current frame, divide the current frame into a stable reuse region, an alignment correction region, and a local refresh region, and configure the corresponding coding branches to form a three-path coding mapping graph. S4. Using the current frame features, temporal context features, three-path coding mapping map, and two types of budget inputs to improve the deep context video compression model, generate a joint execution routing map, compression latent features, and routing assistance information. S5. Based on the context confidence graph and the joint execution routing graph, correct the basic probability distribution of the compression potential features, and perform differential quantization and entropy coding in combination with the regional bit rate sub-budget to generate the current frame compressed bitstream; S6. Based on the routing assistance information, complete the synchronous decoding and current frame reconstruction, calculate the context drift, and update the context confidence parameter and context memory.
[0009] Optionally, step S2 includes the following steps: S21. Select reliable previous reconstructed frames associated with the current frame from the reconstructed frame buffer, and extract historical reconstruction features at different spatial scales. S22. Calculate the distortion propagation factor at each location based on the pixel reconstruction error, structural reconstruction error, motion matching confidence, and occlusion ratio corresponding to the historical reconstruction features. S23. Perform prediction consistency analysis on historical reconstruction features at different scales, identify texture abrupt changes, motion discontinuities and occlusion recovery locations, and calculate prediction uncertainty. S24. By fusing distortion propagation factor, prediction uncertainty, motion matching error and scene complexity features, multi-scale temporal context features and a context credibility map representing the reusability of historical context are obtained.
[0010] Optionally, step S3 includes the following steps: S31. Determine the region segmentation threshold based on contextual credibility, local texture differences, and motion continuity, and divide the current frame into high-credibility regions, medium-credibility regions, and low-credibility regions. S32. Set the high-confidence region as a stable reuse region, retain the corresponding historical context features and skip repeated texture encoding; S33. Set the medium confidence region as the alignment correction region, estimate the sparse motion offset field, and perform variable position sampling and local residual correction on the historical context features. S34. Set the low-confidence region as a local refresh region, cut off the corresponding historical context reference, and perform local intra-frame refresh coding. S35. Record the location, encoding path and refresh identifier of each area, and generate a three-path encoding mapping map.
[0011] Optionally, step S4 includes the following steps: S41. Read the path type, area, texture complexity, motion amplitude, and contextual credibility of each region to form a region encoding state vector; S42. Based on the region coding state vector, target coding quality, code rate budget, and computational budget, allocate a code rate sub-budget, a computational quantum budget, and an allowable distortion limit to each region. S43. Calculate the rate distortion gain and computational cost of candidate network layers, candidate feature channels, candidate feature scales, and coding branches; S44. Based on the sub-budget of each region and the upper limit of allowable distortion, select the effective network layer, effective feature channel, effective feature scale and effective coding branch, and generate a joint execution routing diagram; S45. Perform context reuse, motion alignment correction and local refresh coding according to the joint execution routing graph, and stitch together and fuse the boundaries of the region's potential features to obtain compressed potential features and routing assistance information.
[0012] Optionally, S44 includes the following steps: S441. Based on the bitrate distortion gain corresponding to the unit computational overhead, the candidate network layers, candidate feature channels, and candidate feature scales are jointly ranked. S442. Reduce the network execution depth and the number of feature channels for stable reuse regions, while retaining low-cost feature scales for texture correction; S443. Enable motion correction layer and context fusion layer for alignment correction region, and limit local feature range according to sparse motion offset field; S444. Enable independent coding layers and high-resolution local feature branches for locally refreshed regions, and restrict additional computation outside the regions; S445. Convert the effective network layers, effective feature channels, effective feature scales, and effective coding branches selected in each region into deterministic routing indices, and combine them to form a joint execution routing graph.
[0013] Optionally, step S5 includes the following steps: S51. Based on compressed latent features, multi-scale temporal context features, and quantified adjacent latent features, predict the basic probability distribution parameters of each region. S52. Based on contextual confidence, region coding path, motion correction residual, and historical context drift, calculate the probability center correction and probability interval expansion. S53. Use the probability center correction and probability interval expansion to correct the basic probability distribution parameters and generate a three-path conditional probability distribution. S54. Determine the regional quantization step size and potential feature preservation accuracy based on the regional code rate sub-budget, context confidence, and allowable distortion upper limit; S55. Perform entropy encoding on the quantized compressed latent features according to the conditional probability distribution, and encapsulate the three-path encoding mapping relationship, deterministic routing index and quantization parameters into routing auxiliary information.
[0014] Optionally, step S53 includes the following steps: S531. For stable reuse regions, historical context conditions are used as priors, and the probability prediction interval is reduced based on high context confidence. S532. Adjust the center of the basic probability distribution by combining the alignment correction region with the motion correction residual, and expand the probability prediction interval of low confidence positions. S533. Reduce the prior weight of historical context conditions for local refresh areas and generate independent probability distribution parameters based on the local refresh features of the current frame. S534. Perform boundary continuity correction and normalization on the probability distribution parameters at the boundary locations of the three types of regions to generate a conditional probability distribution for entropy coding.
[0015] Optionally, step S6 includes the following steps: S61. Parse the quantization latent features, three-path coding mapping relationship, deterministic routing index and regional quantization parameters from the current frame compressed bitstream; S62. Based on the deterministic routing index, recover the effective network layer, effective feature channel, effective feature scale, and effective coding branch used by the encoding end, and establish a synchronous decoding path; S63. Decode the stable reuse region, alignment correction region and local refresh region respectively according to the synchronous decoding path, and perform boundary fusion and texture correction. S64. Compare the differences between the current reconstructed features and the multi-scale temporal context features in terms of texture structure, motion direction and local brightness, and calculate the context drift of each region. S65. Update the context confidence parameter, region path status, reconstruction reliability, and reconstruction frame buffer according to the context drift amount to form a context memory for subsequent video frame encoding.
[0016] Optionally, S64 includes the following steps: S641. Calculate the texture difference, structural difference, motion direction difference, and brightness difference between the current reconstructed features and the multi-scale temporal context features; S642. Perform time accumulation and attenuation processing on various differences in the corresponding regions of consecutive video frames to distinguish between instantaneous reconstruction disturbances and continuous context offsets. S643. Determine the distortion propagation area and drift level based on the cumulative magnitude and duration of the persistent context offset; S644. Based on the drift level, reduce the historical context reference weight of high drift areas, clear the local context memory of areas exceeding the drift threshold, and increase the local refresh priority; S645. Adjust the credibility threshold, three-path selection parameters, and region budget allocation parameters of the next video frame based on the updated drift level.
[0017] A lightweight video coding system based on machine learning according to an embodiment of the present invention includes the following modules: The video feature processing module is used to acquire the video to be encoded, the target encoding quality parameters, and the computing power parameters of the encoding device, complete frame group division and scene feature extraction, and establish bitrate budget and computational cost budget. The context credibility analysis module is used to extract multi-scale temporal context features and generate a context credibility map; The three-path partitioning module is used to partition the stable reuse region, alignment correction region, and local refresh region based on the context credibility graph, and generate a three-path encoding mapping graph. The lightweight compression module incorporates an improved deep context video compression model to select network layers, feature channels, and region coding branches for computation, and to generate a joint execution routing graph, compress latent features, and routing assistance information. The probability correction and entropy coding module is used to correct the conditional probability distribution based on contextual confidence and regional coding path, perform differential quantization and entropy coding, and generate the compressed bitstream of the current frame. The synchronous decoding module is used to parse routing auxiliary information, restore the network execution path of the encoding end, and complete the reconstruction of the current frame; The drift feedback module is used to calculate the context drift amount, update the context confidence parameters, rebuild the frame buffer and context memory.
[0018] The beneficial effects of this invention are as follows: This invention generates a context reliability map by using preceding reconstructed frames, historical reconstruction distortion, motion matching errors, and texture change information. This enables the video coding model to identify the reusability of historical context in different regions of the current frame, avoiding the direct use of distorted, occluded, or motion-misaligned historical features for current frame prediction, and reducing the possibility of erroneous context accumulating and propagating between consecutive video frames. By dividing the current frame into stable reuse regions, alignment correction regions, and local refresh regions, this invention can employ differentiated coding processing for different regions. The stable reuse region directly utilizes reliable historical context to reduce redundant texture calculations; the alignment correction region maintains temporal prediction accuracy through motion offset estimation and local feature correction; and the local refresh region cuts off low-reliability historical context and performs independent coding, thereby improving coding stability in complex dynamic scenes, target addition scenes, and occlusion recovery scenes.
[0019] This invention employs an improved deep contextual video compression model that jointly analyzes regional coding paths, bitrate budgets, and computational budgets. Based on the texture complexity, motion amplitude, contextual reliability, and permissible distortion levels of different regions, it dynamically selects the network layers, feature channels, feature scales, and coding branches involved in the computation. Compared to processing entire video frames with a fixed network structure, this invention reduces the network execution depth and the number of invalid feature channels in stable regions, prioritizing the allocation of limited computational resources to motion-complex regions and locally refreshed areas. While maintaining reconstruction quality, it reduces the number of model parameter calls and encoding computations, making the video encoding process adaptable to mobile terminals, edge devices, and other application environments with limited computing power.
[0020] This invention generates a joint execution routing graph and encapsulates deterministic routing indices related to network execution paths into routing assistance information. This enables the decoder to accurately recover the effective network layers, effective feature channels, effective feature scales, and region coding branches used by the encoder, ensuring consistency in the encoding and decoding process under dynamic network structures. The joint execution routing graph is constrained by both bitrate and computational budgets, avoiding the problems of increased computational burden from simply reducing the bitrate, or significant degradation in video quality from simply reducing computational load. This achieves coordinated control between encoding bitrate, reconstruction quality, and device computational complexity.
[0021] This invention performs regionalized corrections on the basic probability distribution of compressed latent features based on contextual credibility, regional coding paths, motion correction residuals, and historical context drift. For stable reuse regions, the probability prediction range is narrowed using high-credibility historical context; for alignment correction regions, the probability distribution center and prediction interval are adjusted based on motion correction residuals; for locally refreshed regions, the influence of prior historical context conditions is reduced and independent probability parameters are generated. This improves the accuracy of latent feature probability estimation under different coding paths, reduces redundant codewords caused by probability prediction biases during entropy coding, and improves video compression efficiency.
[0022] After reconstructing the current frame, this invention calculates the context drift based on the texture, structural, and motion direction differences between the reconstructed features and the temporal context features. It then uses this context drift to update the confidence parameters, regional path states, and context memory. By distinguishing between instantaneous reconstruction perturbations and persistent context offsets, this invention reduces the historical context reference weight in high-drift regions and clears the corresponding local context memory when the drift exceeds a threshold, thereby increasing the priority of local refresh encoding in subsequent video frames. This forms a closed-loop processing mechanism that interconnects context confidence evaluation, three-path encoding, joint execution routing, conditional probability correction, and drift feedback updates, achieving dynamic optimization of the lightweight video encoding process. Attached Figure Description
[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a lightweight video coding method based on machine learning proposed in this invention; Figure 2 This is a schematic diagram of the improved deep context video compression model proposed in this invention; Figure 3 This is a structural diagram of the lightweight video coding system based on context drift feedback proposed in this invention. Detailed Implementation
[0024] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0025] refer to Figures 1-3 A lightweight video coding method and system based on machine learning, comprising the following steps: S1. Obtain the video to be encoded, the target encoding quality, and the device's computing power; complete frame group division and scene analysis; and establish bitrate budget and computational cost budget. S2. Based on the preceding reconstructed frames, historical reconstruction distortion, and motion matching information, extract multi-scale temporal context features and generate a context credibility map; S3. Based on the context credibility graph and the degree of local change in the current frame, divide the current frame into a stable reuse region, an alignment correction region, and a local refresh region, and configure the corresponding coding branches to form a three-path coding mapping graph. S4. Using the current frame features, temporal context features, three-path coding mapping map, and two types of budget inputs to improve the deep context video compression model, generate a joint execution routing map, compression latent features, and routing assistance information. S5. Based on the context confidence graph and the joint execution routing graph, correct the basic probability distribution of the compression potential features, and perform differential quantization and entropy coding in combination with the regional bit rate sub-budget to generate the current frame compressed bitstream; S6. Based on the routing assistance information, complete the synchronous decoding and current frame reconstruction, calculate the context drift, and update the context confidence parameter and context memory.
[0026] In this embodiment, S1 includes: The system acquires the video to be encoded, the target encoding quality, and the device's computing power. It then performs preprocessing, frame grouping, and scene complexity analysis on the video to be encoded, and establishes the bitrate budget and computation budget corresponding to the current encoding task.
[0027] The encoding system receives the video to be encoded. This video can be captured in real-time by a camera device or retrieved from local storage, a network video stream, or a cloud video library. The video data includes consecutive video frames, resolution, frame rate, color format, timestamps, and video source identifiers. For multiple video inputs, the encoding system establishes an independent encoding task for each video stream and maintains separate reconstructed frame buffers, region path states, and context memories.
[0028] Before encoding, resolution adaptation, color space conversion, pixel normalization, and timestamp correction are performed on the video frames. Resolution adaptation ensures that the input video frames meet the model's downsampling factor requirements. When the width or height of a video frame is not divisible by the model's downsampling factor, padding pixels are added to the edges of the video frame, and the corresponding padding area is cropped after decoding.
[0029] Color space conversion transforms the input video frame from a red-green-blue color space to a color space that separates luminance and chrominance, allowing luminance and chrominance features to be encoded with different precisions. Pixel normalization maps pixel values from different video sources to a uniform numerical range, reducing the impact of input device differences on feature extraction results.
[0030] The encoding system divides frames into groups based on scene changes. Each frame group includes keyframes and prediction frames. Keyframes are encoded independently and used to initialize the reconstruction frame buffer and context memory; prediction frames utilize preceding reconstruction information for context-aided encoding. When a camera switch, large-area occlusion, sudden change in global brightness, or new content exceeding a preset coverage ratio is detected, the current frame group is terminated early, and subsequent video frames are set as new keyframes.
[0031] Scene complexity analysis includes texture complexity analysis, motion complexity analysis, brightness change analysis, and region update ratio analysis. Texture complexity is determined based on edge density, local gradient changes, the proportion of high-frequency features, and texture repetition. Motion complexity is determined based on the coarse-grained motion amplitude, motion direction dispersion, and motion region coverage between adjacent video frames. Brightness change is determined based on the difference in the average pixel value and brightness distribution changes of corresponding regions in adjacent video frames. The region update ratio represents the proportion of the area in the current frame that has undergone significant content changes compared to the previous frame.
[0032] The target coding quality can be preset by the user or automatically determined based on transmission bandwidth, storage capacity, and service priority. Target coding quality can be expressed as target bitrate, allowable distortion range, target peak signal-to-noise ratio, target structural similarity, target visual quality level, or a combination of the aforementioned parameters.
[0033] The encoding system establishes a bitrate budget based on the target encoding quality, video resolution, frame rate, frame group length, and transmission bandwidth. The bitrate budget includes the total bitrate budget for the frame group and the bitrate budget for the current frame, which is used to limit the size of the compressed bitstream generated in the current encoding process.
[0034] Device computing power parameters include processor type, available video memory, available system memory, allowable single-frame encoding latency, target processing frame rate, and device power consumption level. The device resource configuration is read upon startup, and the available resource status is updated periodically during encoding. When the device executes other tasks simultaneously, causing a decrease in available resources, the computational budget for subsequent video frames is adjusted.
[0035] The computational budget is used to limit the network depth level, number of channel groups, feature scale, and region coding branches that can be enabled in the current frame. The computational budget can be represented by allowing floating-point operations, allowing single-frame execution latency, allowing video memory usage, or a combination of multiple resource constraints.
[0036] Bitrate budget and computational cost budget jointly constrain the subsequent encoding process. The encoding system provides feedback adjustments to both budgets based on the actual bitrate, actual encoding latency, and reconstruction quality of the most recent video frames. When the actual encoding latency exceeds the target latency, the optional network depth and the number of enhancement channel groups in subsequent frames are reduced; when the actual bitrate continuously exceeds the target bitrate, the quantization level in low-importance stable regions is increased; when the reconstruction quality is lower than the target quality, the resource allocation ratio in motion-complex regions and locally refreshed regions is prioritized.
[0037] To avoid drastic changes in budget parameters between adjacent video frames, time smoothing is applied to the budget update results. The corresponding budget tier is only adjusted when the actual bitrate or actual latency deviates from the target range for multiple consecutive video frames. Short-term resource fluctuations in a single frame do not immediately alter the joint execution routing structure.
[0038] In this embodiment, S2 includes: Read the preceding reconstructed frames, historical reconstruction distortion information, and motion matching information corresponding to the current frame, extract multi-scale temporal context features, and generate a context credibility map to characterize the reusability of historical context.
[0039] When the current frame is a predicted frame, the encoder reads the preceding reconstructed frame from the reconstructed frame buffer. Instead of using the uncompressed original preceding video frame to construct the historical reference, the encoder uses the same preceding reconstructed frame as the decoder, enabling both the encoder and decoder to generate subsequent context based on the same historical information.
[0040] The reconstruction frame buffer can store the most recent reconstruction result or multiple reconstruction results. When using a multi-frame buffer, reference frames are selected based on temporal distance, historical reconstruction quality, motion correlation, and regional drift level. Reference frames that are closer in time but have lower reconstruction quality can have lower reference priority than reference frames that are slightly farther in time but have higher reconstruction quality.
[0041] The preceding reconstructed frame input is a multi-scale temporal context extraction unit. This unit employs a progressive downsampling structure to extract shallow texture features, mid-level structural features, and deep region correlation features. The feature extraction layer utilizes a lightweight structure composed of depthwise separable convolutions, grouped convolutions, and pointwise convolutions to reduce the number of parameters and computational load associated with traditional convolutions.
[0042] Shallow temporal context features are used to describe local edges, brightness, and fine-grained texture; mid-level temporal context features are used to describe target contours, local structures, and motion regions; and deep temporal context features are used to describe large-scale scene relationships and regional semantic associations. After scale mapping and channel alignment, features at different scales are combined to form multi-scale temporal context features.
[0043] Historical reconstruction distortion information is recorded during the encoding and decoding process of preceding video frames, including region quantization level, reconstruction residual level, boundary correction magnitude, and historical drift level. The encoder calculates the distortion propagation risk based on the reconstruction state of the corresponding region in the preceding frame. Regions with high quantization intensity, large reconstruction differences, or multiple consecutive frames at high drift levels have a higher risk of distortion propagation.
[0044] Motion matching information is generated by region matching between shallow features of the current frame and features reconstructed in the preceding frame. The encoder first determines candidate motion ranges in the low-resolution feature space, then performs local refinement matching on texture variation regions, target edges, and occlusion boundaries. Stable background regions retain only coarse-grained matching results, thereby reducing the computational overhead of motion estimation.
[0045] Motion matching confidence is determined based on the uniqueness of candidate matching locations, the similarity of matching features, and the continuity of local motion. Regions where a clear and unique corresponding location can be found have high motion matching confidence; regions with repetitive textures, image blur, occlusion, or multiple similar candidate locations have low motion matching confidence.
[0046] Motion matching error is used to represent the local difference between historical reconstructed features after candidate motion offset and shallow features of the current frame. A low motion matching error indicates that the historical context can still describe the current region after spatial adjustment; a high motion matching error indicates that the current region has added content, undergone rapid deformation, or that the historical context has become invalid.
[0047] The context credibility analysis unit integrates distortion propagation risk, motion matching confidence, motion matching error, texture change degree, occlusion status, and historical drift level to calculate a continuous context credibility map. Higher context credibility indicates that the historical context can stably describe the current region, while lower context credibility indicates that the historical context is faulty or at risk of error propagation.
[0048] The context confidence graph is statistically analyzed at the region granularity corresponding to each routing block. The pixel-level confidence within each routing block is weighted and aggregated to form a region confidence value. Target edges, motion boundaries, and high-frequency texture locations have higher statistical weights to prevent large flat backgrounds from obscuring important local changes.
[0049] The high and low confidence thresholds are jointly determined based on the confidence distribution of the current frame, the average confidence of the regions in the previous few frames, and the computational cost estimate. When the scene is generally stable, the proportion of stable reuse regions is increased; when the scene changes significantly or historical drift increases, the range of stable reuse is reduced, and the proportion of local refresh regions is increased.
[0050] The high confidence threshold is greater than the low confidence threshold. When the regional confidence is higher than the high confidence threshold, the historical context can be directly reused; when the regional confidence is between the two thresholds, the historical context has alignment and correction value; when the regional confidence is lower than the low confidence threshold, the historical context is not suitable as the main basis for prediction of the current region.
[0051] To reduce frequent path switching, a state preservation mechanism is used for reliability judgment. When a region first meets the path switching conditions, the state of the candidate path is recorded, and the actual path is changed only after a preset number of frames have been reached. When a new target appears, the camera changes, or there is large-area occlusion, the corresponding region is allowed to immediately switch to a locally refreshed path.
[0052] Abnormal regions with an area smaller than the minimum routing block are processed based on neighborhood confidence and duration. Short-lived and small abnormal regions are merged into neighboring paths; continuous abnormal regions or those located at the edge of the target are expanded into the complete routing block to ensure that subsequent block encoding has a regular spatial range.
[0053] A continuous context confidence map is retained at the encoding end for three-path region partitioning, bitrate resource allocation, and quantization level determination. The decoding end does not restore the continuous confidence values; instead, it performs subsequent decoding based on the three-path encoding mapping relationship and historical drift levels in the compressed bitstream.
[0054] In this embodiment, S3 includes: Based on the context credibility graph and the degree of local change in the current frame, the stable reuse region, alignment correction region, and local refresh region are divided. The encoding path corresponding to each region is configured, and a three-path encoding mapping graph is formed.
[0055] The three-path partitioning unit initially classifies each routing block based on a high-confidence threshold and a low-confidence threshold. Routing blocks with context confidence higher than the high-confidence threshold and low local change are set as stable reuse areas; routing blocks with context confidence between the two thresholds, or with correctable motion offsets, are set as alignment correction areas; and routing blocks with context confidence lower than the low-confidence threshold, or with added content or high historical drift levels, are set as local refresh areas.
[0056] When a high-confidence region exhibits significant motion, it is not directly designated as a stable reuse region. The encoder further determines whether the historical context can maintain structural consistency after motion alignment. Regions with significant positional changes but high motion matching confidence are entered into the alignment correction path to avoid direct reuse that could cause target ghosting and edge misalignment.
[0057] Stable multiplexing regions typically correspond to static backgrounds, slowly changing textures, or internal regions of a target that remain stable across multiple consecutive video frames. The stable multiplexing coding branch skips full motion estimation, deep feature transformation, and high-resolution refresh operations, performing only low-cost luminance correction, low-frequency texture correction, and boundary fusion.
[0058] For regions with high contextual reliability and stable performance across multiple consecutive video frames, some historical reconstruction features can be directly reused, encoding only minor brightness variations, texture corrections, and boundary changes. Stable reuse paths reduce redundant feature extraction and latent feature transmission.
[0059] Alignment correction areas typically correspond to areas of target movement, local deformation, camera shake, or relative background displacement. The historical context still contains valid content, but its spatial position or local texture has changed, thus requiring motion alignment and local correction.
[0060] The alignment correction coding branch generates sparse motion offsets before quantization based on the motion matching results. The sparse motion offsets only set motion nodes at motion boundaries, locally high-discrepancy locations, and locations with low matching confidence. Other locations are propagated locally based on neighboring motion nodes, thereby reducing the coding and computational load caused by dense motion fields.
[0061] Multi-scale temporal context features are sampled at variable positions using sparse motion offsets. After motion alignment, the residual difference between the current frame's shallow features and the alignment context features is calculated, and this residual difference is mapped to local correction features. These local correction features are used to describe texture, shape, and brightness variations that cannot be explained by motion alignment.
[0062] Local refresh areas typically correspond to areas with newly added targets, unblocking, rapid deformation, camera transitions, sudden changes in strong light, or areas of continuous distortion propagation. The local refresh coding branch severs the main reference relationships between the corresponding region and the low-confidence historical context, and independently generates refresh potential features based on the local features of the current frame.
[0063] The local refresh range includes low-reliability routing blocks and boundary-extended routing blocks. Boundary extension is used to ensure the continuity of texture, structure, and brightness between the refresh area and adjacent stable reused areas or alignment correction areas. When the area of the local refresh region exceeds the set proportion of the current frame, the current frame is adjusted to a key frame to avoid increasing the bitstream burden with a large amount of scattered path information.
[0064] After initial classification, spatially adjacent route blocks with the same path type are merged into a route region. Route regions preferentially use regular rectangular boundaries. If the target edge cannot be completely covered, a small number of boundary route blocks are allowed to be added so that the corresponding encoding branches can be executed in subsequent block batch processing.
[0065] The three-path coding map records the spatial location, path type, reference frame index, motion assist information status, and boundary extension status of each routing area. Stable reuse paths use the first path identifier, alignment correction paths use the second path identifier, and partial refresh paths use the third path identifier.
[0066] The three-path coding map is a discretized result of the continuous context reliability map and is also path control information shared by the encoder and decoder. The encoder compresses the mapping relationship and writes it into the routing auxiliary information, while the decoder recovers the corresponding decoding branch based on the path identifier without recalculating the path state using the original current frame.
[0067] When the three-path encoding mapping relationship of adjacent video frames is similar, only the changed routing area and the changed path identifier are recorded. When a shot change or a large-scale path change occurs, the complete three-path encoding mapping map is retransmitted.
[0068] In this embodiment, S4 includes: The current frame features, multi-scale temporal context features, three-path coding map, bitrate budget, and computational budget are input into the improved deep context video compression model to generate a joint execution routing graph, compression latent features, and routing assistance information.
[0069] The improved deep context video compression model reads the shallow features of the current frame, multi-scale temporal context features, three-path coding map, bitrate budget, and computational budget. The jointly executed routing unit establishes the region coding state based on the path type, region area, texture complexity, motion amplitude, contextual confidence, historical drift level, and target importance of each routing region.
[0070] The importance of a target can be determined based on saliency detection results, target recognition results, or business configuration. In video surveillance scenarios, people, vehicles, and abnormal targets can be assigned a higher importance level, while ordinary static backgrounds can be assigned a lower importance level. When no business targets are configured, the target importance is determined based on texture complexity, motion saliency, and region update ratio.
[0071] The bitrate budget is allocated to each routing region according to the regional coding status, forming a regional bitrate sub-budget. The stable multiplexing region mainly carries correction information, and its regional bitrate sub-budget is relatively low; the alignment correction region needs to carry quantization motion assistance information and potential local correction features, and its regional bitrate sub-budget is moderate; the local refresh region needs to independently describe the current content, and its regional bitrate sub-budget increases according to the region area, texture complexity, and target importance.
[0072] The computational budget is allocated as a regional computational quantum budget based on the regional coding status. The regional computational quantum budget limits the selectable network depth level, number of channel groups, and feature scale. When device resources are limited, priority is given to reducing enhanced channel groups in stable reuse regions, while necessary local refresh branches in high-drift regions are not shut down.
[0073] The improved deep contextual video compression model internally incorporates multiple candidate coding operations, including shallow feature transformation, mid-level context fusion, deep texture enhancement, motion correction, low-resolution feature branch, high-resolution feature branch, basic channel group, enhancement channel group, and fine channel group. Each candidate coding operation has a pre-calibrated computational cost.
[0074] The joint execution routing unit evaluates the benefits of candidate coding operations. First, it predicts the region reconstruction distortion and expected bitstream without candidate coding operations, then it predicts the region reconstruction distortion and expected bitstream after candidate coding operations are enabled, and calculates the distortion reduction, bitstream change, and computational overhead corresponding to the candidate coding operations.
[0075] The overall benefit of candidate coding operations is determined based on the amount of distortion reduction, the amount of bitstream change, and the computational cost. Candidate coding operations are ranked from highest to lowest according to the overall benefit per unit of computational cost, and priority is given to candidate coding operations that can significantly reduce reconstruction distortion or compress the bitstream with lower computational cost.
[0076] The joint execution routing unit sequentially selects candidate coding operations and accumulates the estimated computational overhead and estimated bitstream. Adding new candidate coding operations stops when adding more would cause the regional computational load to exceed the regional computational quantum budget, cause the regional bitstream to exceed the regional bitrate sub-budget, or when the current region has reached the allowable distortion range.
[0077] When multiple candidate encoding operations have the same overall benefit, priority should be given to operations with lower actual execution latency, higher memory access continuity, or those that adopt the same network structure as adjacent routing areas, in order to reduce the additional overhead caused by frequent branch switching and feature reorganization.
[0078] Stable reuse regions are selected starting from shallow network levels, with the basic channel group and low-resolution correction branch enabled by default. When historical reconstruction quality deteriorates or regional texture complexity is high, an enhancement channel group or mid-level texture correction operation is added. Regions that remain stable across multiple consecutive video frames can retain only the basic channel group.
[0079] The alignment correction region enables motion correction branches and mid-level context fusion branches by default. When the motion complexity is low, fewer motion nodes and basic channel groups are used; when the motion direction is discrete or the local deformation is significant, the density of motion nodes is increased and the channel groups are enhanced, but it is still constrained by the quantum budget of the region computation.
[0080] Independent refresh branches are enabled for locally refreshed regions. Low-importance locally refreshed regions use mid-level network settings, while important target regions or high-frequency texture regions can use full network settings and fine channel groups. Independent refresh branches only operate within the locally refreshed region and its boundary extension range, and do not call high-cost networks for the entire feature map.
[0081] To improve the actual running efficiency of the model, regions with the same routing structure are merged into batch processing groups. Routing regions with the same network depth, channel group, and feature scale are enabled for combined execution, reducing model branch switching, redundant memory allocation, and irregular feature access.
[0082] The joint execution routing map records the actual network depth level, channel group number, feature scale number, and coding branch number used in each routing area. The joint execution routing map is further converted into a deterministic routing index. The encoder and decoder pre-share candidate operation numbers and network structure, so there is no need to transmit network parameters in the compressed bitstream.
[0083] The current frame features and multi-scale temporal context features are entered into the corresponding coding branches according to the joint execution routing graph. The stable multiplexing branch outputs context correction latent features; the alignment correction branch outputs pre-quantization motion assistance information and locally corrected latent features; and the local refresh branch outputs independently refreshed latent features.
[0084] After scale unification and channel mapping, latent features from different regions are combined according to their original spatial locations. Boundary fusion units perform feature smoothing, texture correction, and brightness continuity processing on the boundary locations of adjacent regions based on the path type and boundary expansion state, forming compressed latent features.
[0085] The motion assistance information before quantization is quantized through motion parameters to form quantized motion assistance information that can be reproduced at both the encoding and decoding ends. The unquantized motion error is only used for path analysis and resource allocation at the encoding end and is not used as input to the conditional probability model at the decoding end.
[0086] The routing assistance information includes three-path coding mapping relationships, deterministic routing indexes, quantized motion assistance information, area boundary information, and channel group recovery indexes. The routing assistance information is compressed using differential coding, run-length coding, or probabilistic coding methods.
[0087] In this embodiment, S5 includes: Based on the three-path coding map, the joint execution routing map, and the context information that can be consistently obtained at both ends of the encoding and decoding, the conditional probability distribution of the compression potential features is corrected, differential quantization and entropy coding are performed, and the compressed bitstream of the current frame is generated.
[0088] The encoder generates super-prior latent features based on the compressed latent features, and performs quantization and entropy encoding on the super-prior latent features. The decoder recovers the super-prior features before parsing the main latent features, so that the encoder and decoder can obtain consistent probability prediction conditions.
[0089] The base probability distribution is predicted jointly by prior reconstructed features, quantized super-prior features, and quantized adjacent latent features. Alignment correction regions also utilize quantized motion-aided information as a condition. Three-path coding mappings and deterministic routing indices are used to select the probability prediction branch corresponding to the current region.
[0090] The continuous-form context confidence graph is used only at the encoder end to determine the regional code rate sub-budget and quantization level, and is not directly input into the entropy probability prediction network that needs to be consistent between encoding and decoding. Entropy probability prediction is accomplished using three-path identifiers, historical drift levels, and other reproducible information at both ends.
[0091] The stable reuse region selects the historical context conditional probability branch. This branch predicts the probability center and probability range based on the preceding reconstructed features and the decoded prior features. When the historical drift level of the stable reuse region is low, a more concentrated probability range is adopted; when the historical drift level increases, the probability range is expanded to reduce the impact of historical context prediction bias.
[0092] The alignment correction region selects the motion conditional probability branch. This branch performs the same motion mapping on the preceding reconstructed features based on quantized motion-aided information, and then combines the decoded super-prior features and adjacent latent features to predict the probability distribution. When the quantized motion amplitude is large or the historical drift level is high, the probability distribution tolerance is increased.
[0093] The local refresh region selects the intra-frame conditional probability branch. This branch reduces the influence of historical context and mainly predicts the probability distribution based on super-prior features and decoded adjacent latent features. The historical context probability branch can be turned off in the first prediction frame after a shot change.
[0094] The location of the region boundary is determined based on the three-path coding mapping map, without using the continuous confidence value within the encoder. The boundary probability distribution is generated by the probability parameters corresponding to adjacent paths and the preset boundary fusion weights, and the encoder and decoder use the same fusion rules.
[0095] The differentiated quantization unit determines the quantization level of a region based on the regional bitrate sub-budget, path type, target importance, and encoder context reliability. A preset finite number of quantization levels are used, and the corresponding quantization parameters are written into the compressed bitstream. The decoder then performs inverse quantization based on the same quantization parameters.
[0096] Stable reuse regions typically use larger or medium quantization steps, alignment correction regions use medium quantization steps, and local refresh regions use medium or smaller quantization steps depending on the importance of the target and the texture complexity. Quantization priority is higher for important target regions than for less important background regions.
[0097] During the encoding process, the used bitrate is counted in real time. When the actual bitrate is lower than the current allocation progress, the remaining bitrate is prioritized for allocation to high-complexity alignment correction regions and important local refresh regions; when the actual bitrate is close to the budget limit, the quantization level of low-importance stable reuse regions is prioritized to be increased.
[0098] The quantized compressed latent features are entropy encoded according to the modified conditional probability distribution. The encoding order, region order, channel group order, and probability branch selection are all determined by the three-path encoding mapping relationship and the deterministic routing index. The decoding end recovers the quantized latent features in the same order.
[0099] The current frame compressed bitstream includes frame type identifier, prior bitstream, quantization latent feature bitstream, three-path coding mapping relationship, deterministic routing index, quantization motion assistance information, region quantization parameters, and region boundary information.
[0100] To reduce the bitrate occupied by routing auxiliary information, when adjacent video frames have the same routing status, only the changed routing regions and the changed routing indices are recorded. When the same routing regions are continuously distributed, the region range is recorded using a run-length encoding method. The decoding end reconstructs the complete joint execution routing graph based on the routing status of the previous frame and the change information of the current frame.
[0101] In this embodiment, S6 includes: Based on the routing assistance information, the network execution path used by the encoding end is restored, the synchronous decoding of the compression potential features and the reconstruction of the current frame are completed, the context drift is calculated, and the context confidence parameters and context memory are updated.
[0102] After receiving the compressed bitstream of the current frame, the decoding end parses the frame type, the prior bitstream, the three-path coding mapping relationship, the deterministic routing index, the quantization motion assistance information, and the regional quantization parameters.
[0103] The decoder recovers the network depth level, channel group number, feature scale, and decoding branch used for each routing area based on the deterministic routing index. The encoder and decoder use the same candidate operation number, network parameters, and area processing order to ensure consistency in the dynamic network execution structure.
[0104] The decoder utilizes the decoded prior features, pre-order reconstructed features, three-path encoding mapping relationships, quantized motion-aid information, decoded adjacent latent features, and historical drift levels to generate a conditional probability distribution consistent with that of the encoder, and completes the entropy decoding of the quantized latent features.
[0105] Stable reuse regions read corresponding historical reconstruction features from local context memory and combine them with context-corrected latent features obtained from decoding to recover the current region. Regions using direct context reuse reuse corresponding reconstruction features based on path identifiers and perform brightness correction and boundary fusion.
[0106] The alignment correction region maps the historical reconstructed features to their positions based on quantized motion-aided information, and then combines this with locally corrected latent features to complete the region reconstruction. The encoder and decoder use the same quantized motion-aided information and variable position sampling rules to obtain consistent motion alignment results.
[0107] The locally refreshed region invokes an independent refresh decoding branch, performing inverse quantization, feature upsampling, and region image reconstruction based on the latent refresh features. The locally refreshed region does not use low-confidence historical context as the primary source of content, thus preventing historical errors from continuing into the current reconstruction result.
[0108] The three types of regions are spatially combined according to a three-path coding mapping map. The region boundary fusion unit performs texture smoothing, structural correction, and brightness continuity processing based on the boundary information in the compressed bitstream and preset fusion rules. The fused reconstructed features are mapped to the video pixel space to form the current reconstructed frame.
[0109] After generating the compressed bitstream, the encoder performs the same synchronous decoding process as the decoder, using quantized latent features, quantized motion assist information, and deterministic routing indexes, to form the current reconstructed frame for the encoder. Both the encoder and decoder write the current reconstructed frame to their respective reconstructed frame buffers, without using the original current frame before encoding to update the historical context.
[0110] The context drift feedback unit calculates single-frame region differences based on the current reconstructed features, previous reconstructed features, region path state, and decoded motion information. Single-frame region differences include texture preservation differences, structural change differences, motion continuity differences, and region boundary differences.
[0111] The stable reuse region primarily detects continuous texture loss and structural shift. When continuous texture blurring, contour shift, or cumulative brightness error occurs in the stable reuse region, it indicates a decrease in the reliability of direct reuse of historical context, requiring improvement of the drift state of the corresponding region.
[0112] The alignment correction region primarily detects structural differences that persist after motion mapping. When local differences decrease frame by frame after motion alignment, the likelihood of the corresponding region entering a stable reuse path increases; when motion offset is unstable or local differences continue to increase, the priority of the corresponding region entering a local refresh path is increased.
[0113] The primary function of the local refresh region is to detect whether the reconstructed features after the refresh can form a stable new historical context. If the refreshed region can maintain a stable matching relationship in subsequent video frames, the historical context reference weight of the corresponding region is gradually restored; if there are still significant changes, a lower historical context reference weight is maintained.
[0114] The current frame's single-frame regional difference is weighted and updated with the historical drift state of the previous frame to form the current historical drift value. When the current single-frame difference is high, the update weight of the current difference is increased; when the current single-frame difference is low, the historical drift value is gradually decreased according to the attenuation coefficient.
[0115] If a region's single-frame difference only occasionally exceeds the drift trigger threshold and the next video frame returns to the normal range, it is identified as an instantaneous reconstruction disturbance, and the region path is not immediately changed. If the number of frames in which the region's difference continuously reaches the drift trigger threshold exceeds a set value, it is identified as a persistent context offset, and the region drift level is increased.
[0116] The historical drift levels are set to low drift, medium drift, and high drift. Low drift regions retain the historical context reference state; medium drift regions reduce the historical context reference weight and increase the selection priority of alignment correction paths; high drift regions clear the corresponding local context features and increase the selection priority of local refresh paths.
[0117] After the historical drift value of a region remains below the recovery threshold for a preset number of frames, the drift level is gradually reduced. When the recovery threshold is lower than the drift trigger threshold, a path state hysteresis interval is formed, preventing frequent switching of drift level and region coding path between adjacent video frames.
[0118] Context memory updates include reconstructed frame buffer updates, historical reconstruction quality updates, regional path status updates, and historical drift level updates. The encoder and decoder maintain historical states based on the same reconstructed frames and the same drift update rules, ensuring consistent probability prediction conditions for subsequent video frames.
[0119] The continuous context confidence map is recalculated by the encoder after the next video frame arrives, referencing the updated historical drift levels. Context confidence in high-drift regions is suppressed, while context confidence in long-term stable regions is gradually restored. The decoder does not store the continuous confidence map; it only stores the path states and drift levels that can be reproduced in the compressed bitstream.
[0120] Context drift feedback is also used to adjust the region budget allocation for subsequent video frames. High-drift regions receive higher local refresh resource priority, while long-term stable regions have reduced network execution depth and channel group number. When device computing resources decrease, additional coding operations in low-drift stable regions are preferentially reduced, while necessary local refresh capabilities are preserved in high-drift regions.
[0121] This implementation forms a closed-loop technology chain that interconnects contextual credibility analysis, three-path coding, dual-budget joint routing for bitrate and computational complexity, reproducible probability distribution correction for encoding and decoding, synchronous decoding, and context drift feedback. Contextual credibility determines the region coding path, the three-path coding map determines the region coding branch and the probability prediction branch, the joint execution routing map determines the actual network depth, channel group, and feature scale involved in the computation, and the historical drift level formed by the reconstruction result affects the credibility judgment and resource allocation of subsequent video frames.
[0122] This implementation does not perform single-network pruning on a typical deep context video compression model. Instead, it employs differentiated coding based on the historical context reliability of different regions within a video frame and the benefits of candidate network operations, using regular routing blocks, preset channel groups, and network depth levels. Identical routing regions are processed in batches, reducing branch switching and memory access overhead caused by irregular dynamic networks. This translates the theoretical reduction in computational load into a reduction in actual encoding latency.
[0123] In video surveillance scenarios, fixed background areas can use stable reuse paths, moving targets can use alignment correction paths, and newly entering personnel, vehicles, and obstruction recovery areas can use local refresh paths. In drone or vehicle-mounted video transmission scenarios, road targets, obstacles, and vehicle areas can be assigned higher importance levels, and the bitrate and computation budgets can be adjusted synchronously based on wireless bandwidth and terminal computing power.
[0124] In mobile terminal video transmission scenarios, when device temperature rises or available computing resources decrease, the joint execution routing unit reduces the network execution depth and enhances the number of channel groups in stable background areas, while preserving the encoding quality of important dynamic areas. For stored video encoding tasks, the single-frame encoding latency limit can be relaxed, and the network execution level in local refresh areas and important texture areas can be increased.
[0125] For different application environments, the routing block size, frame group length, number of reference frames, context confidence threshold, number of drift trigger frames, number of quantization levels, and number of candidate network operations can be adjusted, but the core relationship between context confidence-driven three-path coding, dual-budget joint routing of bit rate and computational complexity, reproducible conditional probability correction for encoding and decoding, and context drift feedback updates remains unchanged.
[0126] Example 1: To verify the feasibility of this invention in practice, it was applied to a high-definition video encoding scenario on an edge computing device. Simulated encoding verification was performed on video segments of fixed monitoring, fast-moving scenes, and mixed scenes involving occlusion and new targets. The resolution of the test videos was uniformly set to 1920×1080, the frame rate to 30 frames per second, and each test segment contained 300 frames. The original depth context video compression model was used as a control scheme. The test device was an edge computing terminal with graphics processing capabilities, recording indicators such as average bitrate, peak signal-to-noise ratio, single-frame encoding time, and peak video memory usage. The following data are simulated verification data used to illustrate the technical effect and do not represent actual production data for a specific enterprise or project.
[0127] Fixed monitoring scenarios mainly include long-term stable backgrounds and a small number of slowly moving targets. Fast-moving scenarios mainly include rapid target displacement, camera shake, and rapid changes in local textures. Mixed scenarios of occlusion and target addition mainly include targets that reappear after short-term occlusion, new targets added at the edge of the frame, and local brightness changes. These three types of scenarios are used to verify the impact of stable context reuse, motion alignment correction, and local refresh coding on the performance of lightweight video coding.
[0128] During testing, the encoder generates a context credibility map based on preceding reconstructed frames, historical reconstruction distortion, and motion matching information. It then divides the current frame into stable reuse regions, alignment correction regions, and local refresh regions according to credibility. The improved deep context video compression model combines bitrate and computation budgets to dynamically configure the network layers, feature channels, and coding branches involved in the computation for each region. Subsequently, it corrects the probability distribution of latent features based on context credibility and completes differential quantization and entropy coding. The decoder recovers the corresponding execution path according to the routing assistance information in the bitstream, completes the reconstruction of the current frame, and updates the context memory of subsequent frames using the context drift.
[0129] To ensure consistency in the comparison results, both encoding schemes used the same video clips, frame group structures, target encoding quality, and test equipment conditions. Each video group was run five times consecutively, and the average of the remaining three results was calculated after removing the highest and lowest values. A lower average bitrate indicates a smaller compressed bitstream generated per unit time; a higher peak signal-to-noise ratio indicates better reconstructed video quality; and lower single-frame encoding time and peak video memory usage indicate a more significant lightweight effect of the model on limited devices.
[0130] Table 1 Comparison of encoding performance in different video scenarios (simulation verification data)
[0131] As shown in Table 1, in a fixed monitoring scenario, the average bitrate of this invention decreased from 2.84 Mbit / s to 2.61 Mbit / s, the single-frame encoding time decreased from 46.7 ms to 30.5 ms, and the peak video memory usage decreased from 1830 MB to 1260 MB. In this scenario, most background areas have high contextual reliability, allowing the improved model to prioritize stable reuse paths, reducing repetitive texture extraction and high-cost motion estimation. Therefore, the reduction in encoding time and video memory usage is significant. The PSNR changed from 39.18 dB to 39.12 dB, a relatively small change, indicating that reducing repeated computation in stable areas did not cause significant reconstruction quality loss.
[0132] In fast-moving scenarios, this invention reduces the average bit rate from 4.72 Mbit / s to 4.45 Mbit / s, increases the PSNR from 37.06 dB to 37.15 dB, and reduces the single-frame coding time from 53.8 ms to 38.9 ms. Fast-moving regions exhibit significant feature offsets, and directly reusing historical context can easily lead to prediction errors. This invention assigns these regions to alignment correction paths and adjusts the conditional probability distribution based on motion correction residuals, thereby reducing invalid full-frame motion calculations and improving the accuracy of latent feature probability estimation.
[0133] In scenarios involving mixed occlusion and target addition, the average bitrate of this invention decreased from 4.18 Mbit / s to 3.92 Mbit / s, the PSNR increased from 37.61 dB to 37.72 dB, and the single-frame coding time decreased from 51.2 ms to 36.6 ms. The historical context of occlusion recovery locations and newly added target locations has low reliability. The model cuts off invalid contexts through local refresh paths to prevent reconstruction errors from propagating to subsequent video frames. Other stable regions still use context reuse paths, thus balancing local coding quality with overall computational overhead.
[0134] In summary, this invention reduces average bitrate, single-frame encoding time, and peak video memory usage under simulated verification conditions across three scenarios. These results demonstrate that the context confidence graph provides an effective basis for three-path encoding, the execution route formed by the combined bitrate budget and computational budget can configure the model's computational structure based on regional content, and probability distribution correction and context drift feedback can improve encoding stability in dynamic scenarios. Therefore, this invention reduces the computational burden of video encoding while essentially maintaining or improving reconstruction quality, making it suitable for mobile terminals, edge computing devices, and real-time video transmission scenarios.
[0135] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A lightweight video coding method based on machine learning, characterized in that, Includes the following steps: S1. Obtain the video to be encoded, the target encoding quality, and the device's computing power; complete frame group division and scene analysis; and establish bitrate budget and computational cost budget. S2. Based on the preceding reconstructed frames, historical reconstruction distortion, and motion matching information, extract multi-scale temporal context features and generate a context credibility map; S3. Based on the context credibility graph and the degree of local change in the current frame, divide the current frame into a stable reuse region, an alignment correction region, and a local refresh region, and configure the corresponding coding branches to form a three-path coding mapping graph. S4. Using the current frame features, temporal context features, three-path coding mapping map, and two types of budget inputs to improve the deep context video compression model, generate a joint execution routing map, compression latent features, and routing assistance information. S5. Based on the context confidence graph and the joint execution routing graph, correct the basic probability distribution of the compression potential features, and perform differential quantization and entropy coding in combination with the regional bit rate sub-budget to generate the current frame compressed bitstream; S6. Based on the routing assistance information, complete the synchronous decoding and current frame reconstruction, calculate the context drift, and update the context confidence parameter and context memory.
2. The lightweight video coding method based on machine learning according to claim 1, characterized in that, S2 includes the following steps: S21. Select reliable previous reconstructed frames associated with the current frame from the reconstructed frame buffer, and extract historical reconstruction features at different spatial scales. S22. Calculate the distortion propagation factor at each location based on the pixel reconstruction error, structural reconstruction error, motion matching confidence, and occlusion ratio corresponding to the historical reconstruction features. S23. Perform prediction consistency analysis on historical reconstruction features at different scales, identify texture abrupt changes, motion discontinuities and occlusion recovery locations, and calculate prediction uncertainty. S24. By fusing distortion propagation factor, prediction uncertainty, motion matching error and scene complexity features, multi-scale temporal context features and a context credibility map representing the reusability of historical context are obtained.
3. The lightweight video coding method based on machine learning according to claim 1, characterized in that, S3 includes the following steps: S31. Determine the region segmentation threshold based on contextual credibility, local texture differences, and motion continuity, and divide the current frame into high-credibility regions, medium-credibility regions, and low-credibility regions. S32. Set the high-confidence region as a stable reuse region, retain the corresponding historical context features and skip repeated texture encoding; S33. Set the medium confidence region as the alignment correction region, estimate the sparse motion offset field, and perform variable position sampling and local residual correction on the historical context features. S34. Set the low-confidence region as a local refresh region, cut off the corresponding historical context reference, and perform local intra-frame refresh coding. S35. Record the location, encoding path and refresh identifier of each area, and generate a three-path encoding mapping map.
4. The lightweight video coding method based on machine learning according to claim 1, characterized in that, S4 includes the following steps: S41. Read the path type, area, texture complexity, motion amplitude, and contextual credibility of each region to form a region encoding state vector; S42. Based on the region coding state vector, target coding quality, code rate budget, and computational budget, allocate a code rate sub-budget, a computational quantum budget, and an allowable distortion limit to each region. S43. Calculate the rate distortion gain and computational cost of candidate network layers, candidate feature channels, candidate feature scales, and coding branches; S44. Based on the sub-budget of each region and the upper limit of allowable distortion, select the effective network layer, effective feature channel, effective feature scale and effective coding branch, and generate a joint execution routing diagram; S45. Perform context reuse, motion alignment correction and local refresh coding according to the joint execution routing graph, and stitch together and fuse the boundaries of the region's potential features to obtain compressed potential features and routing assistance information.
5. The lightweight video coding method based on machine learning according to claim 4, characterized in that, S44 includes the following steps: S441. Based on the bitrate distortion gain corresponding to the unit computational overhead, the candidate network layers, candidate feature channels, and candidate feature scales are jointly ranked. S442. Reduce the network execution depth and the number of feature channels for stable reuse regions, while retaining low-cost feature scales for texture correction; S443. Enable motion correction layer and context fusion layer for alignment correction region, and limit local feature range according to sparse motion offset field; S444. Enable independent coding layers and high-resolution local feature branches for locally refreshed regions, and restrict additional computation outside the regions; S445. Convert the effective network layers, effective feature channels, effective feature scales, and effective coding branches selected in each region into deterministic routing indices, and combine them to form a joint execution routing graph.
6. The lightweight video coding method based on machine learning according to claim 1, characterized in that, S5 includes the following steps: S51. Based on compressed latent features, multi-scale temporal context features, and quantified adjacent latent features, predict the basic probability distribution parameters of each region. S52. Based on contextual confidence, region coding path, motion correction residual, and historical context drift, calculate the probability center correction and probability interval expansion. S53. Use the probability center correction and probability interval expansion to correct the basic probability distribution parameters and generate a three-path conditional probability distribution. S54. Determine the regional quantization step size and potential feature preservation accuracy based on the regional code rate sub-budget, context confidence, and allowable distortion upper limit; S55. Perform entropy encoding on the quantized compressed latent features according to the conditional probability distribution, and encapsulate the three-path encoding mapping relationship, deterministic routing index and quantization parameters into routing auxiliary information.
7. The lightweight video coding method based on machine learning according to claim 6, characterized in that, S53 includes the following steps: S531. For stable reuse regions, historical context conditions are used as priors, and the probability prediction interval is reduced based on high context confidence. S532. Adjust the center of the basic probability distribution by combining the alignment correction region with the motion correction residual, and expand the probability prediction interval of low confidence positions. S533. Reduce the prior weight of historical context conditions for local refresh areas and generate independent probability distribution parameters based on the local refresh features of the current frame. S534. Perform boundary continuity correction and normalization on the probability distribution parameters at the boundary locations of the three types of regions to generate a conditional probability distribution for entropy coding.
8. The lightweight video coding method based on machine learning according to claim 1, characterized in that, S6 includes the following steps: S61. Parse the quantization latent features, three-path coding mapping relationship, deterministic routing index and regional quantization parameters from the current frame compressed bitstream; S62. Based on the deterministic routing index, recover the effective network layer, effective feature channel, effective feature scale, and effective coding branch used by the encoding end, and establish a synchronous decoding path; S63. Decode the stable reuse region, alignment correction region and local refresh region respectively according to the synchronous decoding path, and perform boundary fusion and texture correction. S64. Compare the differences between the current reconstructed features and the multi-scale temporal context features in terms of texture structure, motion direction and local brightness, and calculate the context drift of each region. S65. Update the context confidence parameter, region path status, reconstruction reliability, and reconstruction frame buffer according to the context drift amount to form a context memory for subsequent video frame encoding.
9. A lightweight video coding method based on machine learning according to claim 8, characterized in that, S64 includes the following steps: S641. Calculate the texture difference, structural difference, motion direction difference, and brightness difference between the current reconstructed features and the multi-scale temporal context features; S642. Perform time accumulation and attenuation processing on various differences in the corresponding regions of consecutive video frames to distinguish between instantaneous reconstruction disturbances and continuous context offsets. S643. Determine the distortion propagation area and drift level based on the cumulative magnitude and duration of the persistent context offset; S644. Based on the drift level, reduce the historical context reference weight of high drift areas, clear the local context memory of areas exceeding the drift threshold, and increase the local refresh priority; S645. Adjust the credibility threshold, three-path selection parameters, and region budget allocation parameters of the next video frame based on the updated drift level.
10. A lightweight video coding system based on machine learning, characterized in that, A lightweight video coding method based on machine learning as described in any one of claims 1 to 9, characterized in that it comprises the following modules: The video feature processing module is used to acquire the video to be encoded, the target encoding quality parameters, and the computing power parameters of the encoding device, complete frame group division and scene feature extraction, and establish bitrate budget and computational cost budget. The context credibility analysis module is used to extract multi-scale temporal context features and generate a context credibility map; The three-path partitioning module is used to partition the stable reuse region, alignment correction region, and local refresh region based on the context credibility graph, and generate a three-path encoding mapping graph. The lightweight compression module incorporates an improved deep context video compression model to select network layers, feature channels, and region coding branches for computation, and to generate a joint execution routing graph, compress latent features, and routing assistance information. The probability correction and entropy coding module is used to correct the conditional probability distribution based on contextual confidence and regional coding path, perform differential quantization and entropy coding, and generate the compressed bitstream of the current frame. The synchronous decoding module is used to parse routing auxiliary information, restore the network execution path of the encoding end, and complete the reconstruction of the current frame; The drift feedback module is used to calculate the context drift amount, update the context confidence parameters, rebuild the frame buffer and context memory.
