Ultra-high-definition video dynamic compression processing method considering context information

Abstract

This invention relates to the technical field of video compression, specifically to a dynamic compression processing method for ultra-high-definition video that considers contextual information. The invention obtains the regional sensitivity of each image frame in the video to be compressed by identifying the corresponding region; divides the video into multiple scene segments; determines multiple filter reference frames for the main features of the scene based on the distortion cost and corresponding regional sensitivity of each image frame in each scene segment as a compression reference frame; reorders the filter reference frames based on the selection results during compression, according to the scene segments corresponding to these multiple filter reference frames; smooths the bitrate fluctuations caused by compression of different filter reference frames in each scene segment; and outputs the compressed video stream. This technical solution comprehensively utilizes human visual characteristics, scene content continuity, and an adaptive reference optimization mechanism, balancing the dynamic compression efficiency and accuracy of ultra-high-definition video.

Description

Technical Field

[0001] This invention relates to the technical field of video compression, and more specifically to a dynamic compression processing method for ultra-high-definition video that takes into account context information. Background Technology

[0002] Ultra-high-definition (UHD) video refers to video with 4K (3840×2160 pixels) or 8K (7680×4320 pixels) resolution, meeting technical requirements such as high frame rate, high bit depth, wide color gamut, and high dynamic range. Context-based dynamic compression can greatly improve the encoding efficiency of UHD video. By intelligently analyzing scene content, motion characteristics, and visual saliency, the compression algorithm can allocate high bit rates to complex areas and low bit rates to simple areas. This compression method can significantly reduce the bit rate while ensuring no significant degradation in subjective image quality, saving storage and bandwidth costs, and adapting to various network environments to achieve smoother streaming media transmission.

[0003] However, traditional compression methods only use the first frame of a scene segment as a reference frame and add it to the reference frame sequence. They do not take into account the representativeness of the first frame for the entire scene segment. When compressing and analyzing subsequent frames, the first frame cannot achieve good reference, resulting in a higher bitrate for subsequent frames and lower video compression efficiency. Summary of the Invention

[0004] To address the technical challenge of balancing efficiency and accuracy in dynamic compression of ultra-high-definition video, this invention aims to provide a dynamic compression method for ultra-high-definition video that considers contextual information. The specific technical solution adopted is as follows:

[0005] This invention provides a method for dynamic compression processing of ultra-high-definition video that considers context information. The method includes:

[0006] Identify human eye-sensitive features in different regions of each image frame of the video to be compressed in order to obtain the regional sensitivity of the corresponding region of each image frame.

[0007] Scene similarity analysis is performed on each image frame of the video to be compressed in order to divide the video into multiple scene segments;

[0008] Based on the distortion cost and corresponding regional sensitivity of each image frame in each scene segment as a compression reference frame, multiple filter reference frames representing the scene image features of each scene segment are determined.

[0009] Based on the compressed scene segments corresponding to multiple selected reference frames, the selected reference frames are reordered during the compression process based on the selection results of each selected reference frame.

[0010] The bitrate fluctuations caused by compression of different selected reference frames in each scene segment are smoothed, and the compressed video stream is output.

[0011] In one optional embodiment, identifying human eye-sensitive features in different regions of each image frame of the video to be compressed to obtain the regional sensitivity of the corresponding region of each image frame includes:

[0012] Convert the RGB images represented by each frame of the video to be compressed into LAB images and scale them to the same size;

[0013] All LAB images of the same size are input into the trained deep learning model to obtain multiple human feature locations in the LAB images, including facial feature locations, eye feature locations, and limb feature locations.

[0014] Predict the significance value of each person's feature location in each LAB image based on the preset regression model;

[0015] Gaussian smoothing and normalization were applied to all significant values ​​to obtain the region sensitivity of the corresponding region for each image frame.

[0016] In one optional embodiment, scene similarity analysis is performed on each image frame of the video to be compressed to divide the video into multiple scene segments, including:

[0017] The current frame of the video to be compressed is compared with the preset comparison items to obtain the image similarity between the current frame and the adjacent frames, the minimum similarity of all adjacent frames, and the scene similarity between the current frame and the scene start frame.

[0018] Based on image similarity, minimum similarity, and scene similarity, the reasonableness of the current frame in the corresponding scene is obtained;

[0019] The current starting frame with a reasonableness level greater than the degree threshold is determined as the segmentation frame of the corresponding scene segment, and the video to be compressed is divided into multiple scene segments based on all segmentation frames.

[0020] In one optional embodiment, the reasonableness of the current frame's corresponding scene is obtained based on image similarity, minimum similarity, and scene similarity, including:

[0021] Based on image similarity and minimum similarity, a first coefficient representing the intensity of local change between the current frame and adjacent frames is obtained;

[0022] The reciprocal of the scene similarity is determined as the second coefficient that characterizes the degree of difference between the current frame and the current scene as a whole;

[0023] Based on the first and second coefficients, the reasonableness of the scene corresponding to the current frame is obtained.

[0024] In one optional embodiment, based on the distortion cost and corresponding region sensitivity of each image frame in each scene segment as a compression reference frame, multiple filter reference frames representing the scene image features of each scene segment are determined, including:

[0025] Motion estimation and motion compensation are performed on the current frame of each scene segment as a compression reference frame to obtain the residual block of the current frame;

[0026] The residual block of the current frame is processed to obtain the distortion cost of the current frame relative to each image frame of the corresponding scene segment;

[0027] Based on the average sensitivity of all regions in the current frame, the maximum sensitivity of all image frames, and all distortion costs incurred by the current frame as a reference frame for the corresponding scene segment, the degree of conformity of the current frame as a reference frame for the corresponding scene segment is obtained.

[0028] Multiple image frames that meet the criteria for each scene segment with a degree greater than the threshold are identified as multiple filtering reference frames.

[0029] In one optional embodiment, the degree of conformity of the current frame as a reference frame for the corresponding scene segment is obtained based on the average sensitivity of all regions of the current frame, the maximum sensitivity of all image frames, and all distortion costs incurred by the current frame as a reference frame for the corresponding scene segment, including:

[0030] The ratio of average sensitivity to maximum sensitivity is determined as the visual importance of the current frame;

[0031] The product of visual importance and all distortion costs is used to determine the degree of conformity of the current frame as a reference frame for the corresponding scene segment.

[0032] In an optional embodiment, during the compression process, the selected reference frames are reordered based on the selection results of each selected reference frame, including:

[0033] Obtain the number of times the current frame to be compressed and the previous filtering reference frame are used and the number of frames between them, as well as the image similarity between the previous filtering reference frame and the filtering reference frame corresponding to the current frame;

[0034] The conformity difference is obtained by comparing the conformity of the pre-selected reference frame with the maximum conformity of the corresponding scene segment.

[0035] Based on the number of times it is used, the number of frames between them, the difference in image similarity and conformity, the degree of bias of the pre-screening reference frame relative to the corresponding scene segment is obtained;

[0036] The pre-selected reference frames are reordered according to the bias of a preset number of pre-selected reference frames in order to improve the compression coding efficiency of the corresponding scene segments.

[0037] In one optional embodiment, the bias of the preceding screening reference frame relative to the corresponding scene segment is obtained based on the number of times it is used, the number of frames between them, image similarity, and the difference in conformity, including:

[0038] The usage activity of the preceding filter reference frame is obtained by the ratio of the number of times it is used to the number of frames between them.

[0039] Based on the difference between image similarity and matching degree, the scene matching degree between the pre-screening reference frame and the scene segment is obtained;

[0040] The bias of the pre-screening reference frame relative to the corresponding scene segment is obtained by multiplying the usage activity and scene matching degree.

[0041] In one optional embodiment, smoothing is performed based on the bitrate fluctuations caused by compression of different selected reference frames in each scene segment, including:

[0042] Get the compression bitrate of the current frame and the previous frame in each scene segment, the current required bitrate and the corresponding current distortion cost of the current frame using the filter reference frames, and the sorting order of the current frame using the filter reference frames.

[0043] Based on the compression bitrate, the current required bitrate, the current distortion cost, and the sorting order, obtain the reference bias index for the current frame to be compressed using the filtered reference frame.

[0044] The reference frames corresponding to the reference bias indicators that are greater than the indicator threshold among all reference bias indicators are determined as the target reference frames of the current frame, and the compression processing of the current frame is performed based on the target reference frames.

[0045] In one optional embodiment, a reference bias index for compressing the current frame using the selected reference frame is obtained based on the compression bitrate, the currently required bitrate, the current distortion cost, and the sorting order, including:

[0046] Based on the difference between the compressed bitrate and the current required bitrate, the bitrate stability coefficient of the current frame compressed using the filtered reference frame is obtained.

[0047] Based on the bit rate stability coefficient, the first reciprocal of the current distortion cost, and the second reciprocal of the sorting order, the reference bias index for compression of the current frame using the filtered reference frame is obtained.

[0048] The present invention has the following beneficial effects:

[0049] The technical solution of this invention identifies the human eye's visually sensitive features in different regions of each image frame in the video to be compressed, obtaining the regional sensitivity of the corresponding region in each image frame; performs scene similarity analysis on each image frame of the video to be compressed to divide the video into multiple scene segments; based on the dynamic compression principle combining visual characteristics and content similarity, determines multiple filter reference frames that can represent the main features of the scene based on the distortion cost and corresponding regional sensitivity of each image frame in each scene segment as compression reference frames; according to the compression scene segments corresponding to the multiple filter reference frames, the filter reference frames are reordered based on the selection results of each filter reference frame during the compression process to adaptively adjust the priority of the filter reference frames during the compression process; the bitrate fluctuations caused by the compression of different filter reference frames in each scene segment are smoothed, and the compressed video stream is output. This technical solution comprehensively utilizes human visual characteristics, scene content continuity, and adaptive reference optimization mechanisms, enabling ultra-high-definition video to be compressed faster and smaller while ensuring smoothness and viewing effect, occupying less storage space, thus balancing the dynamic compression efficiency and accuracy of ultra-high-definition video. Attached Figure Description

[0050] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a method for dynamic compression processing of ultra-high-definition video that takes context information into account, according to an embodiment of the present invention.

[0052] Figure 2 A flowchart illustrating the division of a video to be compressed into multiple scene segments, provided as an embodiment of the present invention;

[0053] Figure 3 This is a flowchart for determining multiple filtering reference frames provided in one embodiment of the present invention;

[0054] Figure 4 This is a flowchart of reordering the screening reference frames provided in one embodiment of the present invention. Detailed Implementation

[0055] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an ultra-high-definition video dynamic compression processing method considering context information proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0057] Traditional methods for compressing ultra-high-definition video use the first frame of a scene for compression. However, the representativeness of the first frame is limited compared to the entire scene, resulting in high bitrates and low efficiency. This invention, based on dynamic compression with context information, significantly improves encoding efficiency in ultra-high-definition video compression. By intelligently analyzing scene content, motion characteristics, and visual saliency, the processing algorithm allocates high bitrates to complex areas and low bitrates to simple areas. This drastically reduces the bitrate without significantly degrading subjective image quality, saving video storage space and bandwidth costs, and adapting to diverse network environments for smoother streaming media transmission. The specific scheme of this invention's dynamic compression method for ultra-high-definition video considering context information will be described in detail below with reference to the accompanying drawings.

[0058] Please see Figure 1 This document illustrates a flowchart of a dynamic compression processing method for ultra-high-definition video that considers context information, provided by an embodiment of the present invention. This method can be applied to computer devices or server devices to compress ultra-high-definition video; any device capable of running this method is acceptable, and no specific limitations are placed on the operating terminal. The processing method includes:

[0059] S11. Identify the human eye-sensitive features of different regions in each image frame of the video to be compressed, so as to obtain the regional sensitivity of the corresponding region of each image frame.

[0060] Specifically, the video to be compressed is an ultra-high-definition video requiring compression processing. Ultra-high-definition videos occupy a large amount of storage space, and the bandwidth required for playback is also significant. Keyframes or continuous frame sequences can be extracted from the video to be compressed, converting the original RGB image into a color image that better matches human visual perception. Physical attributes of the color image, such as color, texture, brightness, and contrast, can be extracted. Different image features have varying sensitivities to human visual recognition. For example, when an image frame represents a person, the face and eyes are more likely to attract the viewer's attention, while the background is more easily overlooked. Based on this difference in focus, sensitivity analysis can be performed to determine the regional sensitivity of different areas within each image frame. Regional sensitivity quantifies the degree of sensitivity of different regions of an image frame to human visual recognition. A recognition model can be set to determine the regional sensitivity of different regions in each image frame, providing a weight reference for subsequent adaptive compression.

[0061] For example, step S11 includes sub-steps S11-1 to S11-4, which are described in detail below:

[0062] S11-1. Convert the RGB images represented by each frame of the video to be compressed into LAB images and scale them to the same size. Converting RGB images to LAB images better matches the color space perception of human vision. Scale all LAB images to a uniform size, which can accelerate subsequent calculations and ensure the comparability of different image frames.

[0063] S11-2. Input all LAB images of the same size into the trained deep learning model to obtain multiple human feature locations in the LAB images. These multiple human feature locations include facial feature locations, eye feature locations, and limb feature locations, and may also include other foreground features that are more easily noticed. The deep learning model can be configured as YOLO (You Only Look Once Real-Time Object Detection Model) or MTCNN (Multi-task Cascaded Convolutional Networks). The deep learning model can identify features that are easily noticed by the eye, such as faces and human bodies, in the image frames.

[0064] S11-3. Predict the saliency value of each person's feature location in each LAB image based on the preset regression model. The result of feature recognition can be used as input, with the multidimensional features (color, texture, motion, etc.) of each pixel or each image block. A regression model (such as SVR) or classifier can be used to predict the saliency value of the corresponding location in the image frame. The saliency value can quantify the degree to which different person's feature locations are noticed by the human eye.

[0065] S11-4. Perform Gaussian smoothing and normalization on all significant values ​​to obtain the regional sensitivity of the corresponding area in each image frame. Normalization ensures that all significant values ​​are within a preset range. By setting a threshold or using a clustering algorithm, the image is divided into different levels of sensitivity regions. In essence, this method quantifies the importance of visually sensitive parts of the video before compression, simulating the human visual system and marking important image regions (faces, title text) in the image.

[0066] At this point, the region sensitivity of each image frame has been obtained, and we proceed to step S12.

[0067] S12. Perform scene similarity analysis on each image frame of the video to be compressed to divide the video into multiple scene segments.

[0068] Specifically, the video to be compressed may involve multiple scene changes, such as switching from indoor people to outdoor street scenes. The features that need attention will change accordingly in different scenes. Therefore, scene similarity analysis can be performed to divide the video to be compressed into multiple scene segments, with each scene segment corresponding to a video scene. Dividing scenes into frame segments can significantly improve video compression efficiency and subjective visual quality. By accurately identifying scene boundaries and inserting keyframes at these boundaries, the encoder can reset the prediction context, completely avoiding invalid inter-frame predictions across different scenes, thereby greatly reducing data redundancy.

[0069] Structural similarity (SSIM) can be calculated between adjacent frames of the video to be compressed to determine the degree of drastic change in the scene. When the similarity drops to a certain threshold, it indicates that there is a significant change in the scene content, such as a scene change or a large camera movement. The corresponding image frame is marked as the starting frame of the new scene segment. Based on each starting frame, the video to be compressed is divided into multiple scene segments, and each scene segment is a part of the video content to be compressed.

[0070] For example, please refer to Figure 2 Step S12 includes sub-steps S12-1 to S12-3, which are described in detail below:

[0071] S12-1. Based on preset comparison items, perform a similarity comparison on the current frame of the video to be compressed to obtain the image similarity between the current frame and adjacent frames, the minimum similarity among all adjacent frames, and the scene similarity between the current frame and the scene start frame. Let the current frame be j, the previous frame be j-1, and the scene start frame be p. By comparison, the similarity between the current frame j and the previous frame j-1 can be obtained, denoted as the image similarity. Perform similarity comparisons on all adjacent image frames, and denot the minimum similarity as... The similarity between the current frame j and the scene's starting frame p is denoted as the scene similarity. Similarity calculation can be performed based on the first frame of the video to be compressed as the starting frame of the first scene, that is, scene segments are divided according to the playback order of the video to be compressed.

[0072] S12-2. Based on image similarity, minimum similarity, and scene similarity, determine the reasonableness of the current frame within the corresponding scene. This can be understood as the difference between image similarity and minimum similarity. The smaller the value, the greater the similarity between the current frame j and the scene's starting frame p. The smaller the size, the less similar the current frame j is to the previous frame segment in the current frame segment, and the greater the difference in scene. Therefore, the current frame j should be used as the starting frame of the new frame segment (i.e., the new scene segment).

[0073] Therefore, based on image similarity and minimum similarity, a first coefficient representing the intensity of local changes between the current frame and adjacent frames can be obtained; the reciprocal of scene similarity is determined as a second coefficient representing the overall difference between the current frame and the current scene; further, the product of the first and second coefficients is used to derive the reasonableness of the scene corresponding to the current frame. This reasonableness is denoted as... There exists a formula: In the formula, exp represents the number of exponents to the base natural constant. The max-min normalization method can be used to... After normalization, we get Its range is [0,1].

[0074] S12-3. The current starting frame with a reasonableness level greater than the reasonableness threshold is determined as the segmentation frame for the corresponding scene segment. Based on all segmentation frames, the video to be compressed is divided into multiple scene segments. The reasonableness threshold can be set based on actual needs, for example, set to 0.8. When the current frame j is determined to be the starting frame of the new frame segment, the above method is used to judge each frame of the image frame to be compressed, and the frame that meets the requirements is used as the starting frame of the new frame segment (i.e., the segmentation frame). The video to be compressed is divided into multiple scene segments for reference when obtaining reference frames later.

[0075] At this point, the segmented frames of each scene have been obtained, and the video to be compressed has been divided into multiple scene segments. Proceed to step S13.

[0076] S13. Based on the distortion cost and corresponding regional sensitivity of each image frame in each scene segment as a compression reference frame, determine multiple filter reference frames that represent the scene image features of each scene segment.

[0077] Specifically, when dynamically compressing ultra-high-definition video, to improve encoding accuracy and compression efficiency, it is necessary to select several reference frames from each scene segment that represent the main features of that segment. These are referred to as the selection reference frames. Within each scene segment, all image frames can be analyzed individually to calculate the distortion cost of each image frame as a compression reference. Combined with the previously obtained region sensitivity, the visual fidelity of the image frame can be evaluated. The distortion cost can be obtained by using the image frame as a reference frame, performing predictive coding on other frames within the same scene, calculating their prediction residuals and corresponding bitrates, and thus obtaining the corresponding distortion cost. Region sensitivity characterizes the degree of detail preservation of an image frame in highly sensitive areas (such as human faces and subject edges). These two indicators can be combined with certain weights to form a comprehensive evaluation value, for example, through a weighted or normalized product. Based on this comprehensive evaluation value, all frames within the scene segment are ranked, and the top-scoring frames are selected as selection reference frames. This method not only avoids video distortion caused by using redundant or poor-quality image frames in compression, but also significantly improves the efficiency of the encoding process and the visual consistency of video restoration, providing an accurate reference basis for the entire dynamic compression process.

[0078] In video compression, establishing a reference frame sequence is the core mechanism for achieving efficient inter-frame compression. Leveraging the high temporal correlation of videos, the content of subsequent frames is predicted by referencing already encoded frames (reference frames). This allows encoding only the differences between frames (residuals) and motion vectors, rather than complete pixel information, significantly improving compression efficiency. Furthermore, the smaller the difference between the reference frame and subsequent frames, the more representative it is, and the more easily recognizable features it contains. Using this reference frame as the reference frame results in a lower bitrate for subsequent frames and higher compression efficiency. Please refer to [link to relevant documentation]. Figure 3 Step S13 includes sub-steps S13-1 to S13-4, which are described in detail below:

[0079] S13-1. For each scene segment, the current frame is used as the compression reference frame for motion estimation and motion compensation to obtain the residual block of the current frame. Adjacent frames in the video to be compressed are usually very similar, possibly differing only in object position movement or minor changes in image details. When compressing the video, it is unnecessary to repeatedly store the entire frame; only the changed parts need to be recorded. Motion estimation can be used to find a block change in the current frame, which is then used as the result of motion estimation to obtain a motion vector. Based on the motion vector, the predicted block for the corresponding region in the current frame can be derived. The difference between the predicted block and the true block is the residual block. For the current frame j in a single scene segment k, it is used as the reference frame. Using this current frame j as the reference frame, motion estimation and compensation are performed on image frames m in scene segment k to generate residual blocks.

[0080] S13-2. Process the residual block of the current frame to obtain the distortion cost of the current frame relative to each image frame in the corresponding scene segment. This can involve transforming, quantizing, and dequantizing the residual, and calculating the distortion D between the reconstructed block and the original block (commonly calculated using SAD, SATD, or SSE). Simultaneously, estimate the bit rate R required to encode the motion vectors and residual coefficients. Finally, multiply the distortion D and the bit rate R by Lagrange multipliers. The results are added together to obtain the distortion cost of the current frame j for image frame m. ,Right now .

[0081] S13-3. Based on the average sensitivity of all regions in the current frame, the maximum sensitivity of all image frames, and all distortion costs incurred by the current frame as a reference frame for the corresponding scene segment, the conformity degree of the current frame as a reference frame for the corresponding scene segment is obtained. The conformity degree characterizes the degree to which the current frame, as a reference frame for the corresponding scene segment, can meet the requirement of compression without distortion. The ratio of the average sensitivity to the maximum sensitivity can be determined as the visual importance of the current frame; the product of the visual importance and all distortion costs is determined as the conformity degree of the current frame as a reference frame for the corresponding scene segment.

[0082] Taking the current frame j as an example, the average human eye recognition sensitivity of all regions in the current frame j is obtained. The average sensitivity of human eye recognition sensitivity obtained from all frames is then compared. and maximum sensitivity The sum of the distortion costs of the current frame j for all image frames in scene segment k is... ,in Let be the number of all image frames in scene segment k. The greater the sum of distortion costs, the higher the average sensitivity of human eye recognition in all regions of the current frame j. With maximum sensitivity (Greater than 0, otherwise meaningless) ratio The larger the value, the greater the similarity of the current frame j to all image frames in scene segment k, and the more representative it is. Furthermore, the overall human visual sensitivity to the reference frame is high. When subsequent frames use it as a reference frame, the corresponding supplementary distinguishing regions may have lower human visual sensitivity, thus allowing for a lower bitrate and greater compression efficiency. Therefore, the degree of similarity between the current frame j and the reference frame in scene segment k can be obtained. for: This method is used to calculate the degree of conformity for each image frame in scene segment k as a reference frame. Here, norm represents linear normalization.

[0083] S13-4. Multiple image frames with a compliance level greater than a threshold for each scene segment are identified as multiple filtering reference frames. Each scene segment has a reference frame sequence composed of multiple filtering reference frames. Video compression processing is performed on each scene segment based on the reference frame sequence. When calculating using the above method, the image frame with the highest compliance level can be selected as the filtering reference frame for this scene segment and added to the reference frame sequence for compression reference of other subsequent image frames.

[0084] At this point, multiple filtering reference frames for each scene segment have been obtained, and we proceed to step S14.

[0085] S14. Based on the compression scene segments corresponding to multiple selected reference frames, the selected reference frames are reordered during the compression process based on the selection results of each selected reference frame.

[0086] Specifically, each image frame is compressed using a similar selected reference frame from the reference frame sequence. If the selected reference frames in the reference frame sequence are always sorted according to their order of addition, the image frame may not be able to quickly find a suitable selected reference frame through sequential comparison, thus increasing the compression time. Therefore, the order of different selected reference frames in the reference frame sequence can be adjusted based on their frequency of use in the current time period and their degree of compatibility, so that suitable selected reference frames can be found more quickly.

[0087] For example, please refer to Figure 4 Step S14 includes sub-steps S14-1 to S14-4, which are described in detail below:

[0088] S14-1. Obtain the number of times the current frame to be compressed and the previous filtering reference frame are used, the number of frames between them, and the image similarity between the previous filtering reference frame and the corresponding filtering reference frame of the current frame. Taking the current frame j as an example, when compressing the current frame j, obtain the number of times the filtering reference frame v is used in the reference frame sequence when compressing 20 image frames prior to the current frame j, denoted as v. Calculate the number of frames between the last image frame used in the filtering reference frame v and the current frame j, denoted as . The similarity between the reference frame v and the current frame j is denoted as the image similarity. .

[0089] S14-2. Obtain the conformity difference based on the difference between the conformity of the pre-selected reference frame and the maximum conformity of the corresponding scene segment. Obtain the conformity of the reference frame for the selected reference frame v. The same formula for calculating the degree of conformity is used; the maximum degree of conformity among all reference frames is obtained by comparison. The conformity difference was obtained. .

[0090] S14-3. Based on the number of uses, the number of frames between them, the image similarity, and the difference in conformity, obtain the degree of bias of the preceding screening reference frame relative to the corresponding scene segment. A calculation model can be formulated based on the relationship between each parameter and the degree of bias, and the degree of bias of the preceding screening reference frame relative to the corresponding scene segment can be obtained through the calculation model.

[0091] It's understandable that the number of times it's used... The more frames the last used image frame was, the more frames it was between the current frame j and the current frame j. The smaller the value, the better the match with the selected reference frame v. With maximum degree of conformity The difference in conformity The smaller the value, the higher the image similarity between the selected reference frame v and the corresponding reference frame in the scene segment where the current frame j is located. The larger the value, the more frequently the filter reference frame is used, and the greater its similarity to subsequent frames in the current scene segment. To reduce the traversal time of the filter reference frame, this filter reference frame v can be placed at a earlier position in the sequence.

[0092] Therefore, we can obtain the degree of bias in the order of reference frame v in the sequence when compressing the current frame j: , To the degree of bias, The ratio of the number of times it is used to the number of frames between them represents the activity level of the preceding filter reference frame. It represents the scene matching degree between the pre-screened reference frame and the scene segment it is in. The purpose of setting the coefficient 1 is also to avoid the denominator being 0.

[0093] S14-4. The pre-selected reference frames are reordered according to the bias of a preset number of pre-selected reference frames to improve the compression coding efficiency of the corresponding scene segment. The bias of each selected reference frame in the reference frame sequence is obtained, and the frames are reordered from largest to smallest bias. This ensures that subsequent frames select the selected reference frames faster, thereby effectively improving the compression efficiency of ultra-high-definition video.

[0094] At this point, the reordering of the selected reference frames has been completed, and we proceed to step S15.

[0095] S15. Smooth the bitrate fluctuations caused by different selected reference frames in each scene segment and output the compressed video stream.

[0096] Specifically, during ultra-high-definition video compression, the compressed video exhibits significant bitrate fluctuations. High bitrate frames can instantly overwhelm network bandwidth and buffers, easily causing the receiving buffer to overflow due to rapid data consumption, resulting in playback stuttering, or the sending buffer to overflow due to excessive data influx, leading to data packet loss. Visually, if the encoder is not properly controlled, viewers will perceive frequent jumps in image quality between sharpness and blurriness, a phenomenon known as the "ping-pong effect," severely impacting the viewing experience. Simultaneously, excessively high instantaneous bitrates also pose a challenge to the real-time decoding capabilities of terminal devices. This step smooths out bitrate fluctuations based on different selected reference frames within each scene segment, resulting in more stable bitrate changes between compressed frames.

[0097] To smooth bitrate fluctuations, the actual bitrate of each frame after using different filtered reference frames can be recorded during the compression stage, and the bitrate variation within adjacent frames or adjacent time windows can be calculated. By setting a target bitrate range or average bitrate curve, fluctuations exceeding a threshold can be adjusted. For example, methods such as moving average, exponential weighting, or adaptive filtering can be used to smooth the bitrate curve, weakening short-term bitrate peaks and appropriately compensating for low bitrate segments. Alternatively, bitrate smoothing can be implemented based on feedback control, using the deviation between the current frame bitrate and the target bitrate as a control variable to dynamically adjust the quantization parameters or reference frame weights of subsequent frames, thereby achieving bitrate smoothing.

[0098] After the above smoothing process, the output bitstream of the entire video is more continuous in the time dimension, improving the bandwidth adaptability of the playback end and avoiding buffering or distortion problems caused by excessively high instantaneous bitrate. The final generated video compressed stream maintains the overall image quality consistency while taking into account compression stability and transmission efficiency, thus ensuring that ultra-high-definition video still has a smooth and balanced viewing experience under high compression rate conditions.

[0099] For example, step S15 includes sub-steps S15-1 to S15-3, which are described in detail below:

[0100] S15-1. Obtain the compression bitrate of the current frame and the previous frame in each scene segment, the current required bitrate and corresponding current distortion cost of the current frame compressed using the selected reference frames, and the sorting order of the current frame using the selected reference frames. Record the bitrate of the current frame j and the previous image frame j-1 as follows: ; Calculate the current required bitrate for the current frame j when using the filtered reference frame m in the reference frame sequence. ; and the corresponding current distortion cost. ; Obtain the current sort order of reference frame m in the reference frame sequence. .

[0101] S15-2. Based on the compression bitrate, the currently required bitrate, the current distortion cost, and the sorting order, obtain the reference bias index for compressing the current frame using the selected reference frame. Compression bitrate With the current required bitrate The difference The smaller the value, the lower the current distortion cost. The smaller the value, the better the sorting order of the reference frame m in the sequence. The smaller the value, the smaller the bitrate spike when using the filter reference frame m, and the fewer traversals are required, making it more suitable as the filter reference frame for the current frame j.

[0102] Therefore, the reference bias index for selecting reference frame m as the compression reference for the current frame j can be obtained. It can be done through the formula: exp represents an exponential function with the natural constant as its base, and the reference bias index of the current frame j is calculated using the filtered reference frame compression m. The max-min normalization method is then used to... After normalization, we get Its range is [0,1].

[0103] It should be noted that in the above formula, the first term is derived from the difference between the compressed bitrate and the current required bitrate, which represents the bitrate stability coefficient of the current frame when using the selected reference frame for compression, ensuring that the compression result is stable in terms of bitrate; the second term ensures that the compression achieves the optimal trade-off between distortion and bitrate; the third term maintains the temporal continuity and historical stability of the selection of reference frames, and the reference bias index of the current frame when using the selected reference frame for compression is derived based on the product of the three terms.

[0104] By using bitrate variation, distortion cost, and the sorting position of the selected reference frames as metrics for stationarity, compression efficiency, and historical priority, respectively, a reference bias index is calculated for each reference frame. A larger reference bias index indicates that the selected reference frame has greater advantages in compression smoothness, image quality preservation, and prediction continuity.

[0105] S15-3. Select the reference frames corresponding to the reference bias indices that are greater than the index threshold from all reference bias indices, and determine them as the target reference frames for the current frame. Perform compression processing on the current frame based on the target reference frames. The index threshold can be set based on the experience of technical personnel or on calibration experiments, for example, set to 0.8. Calculate the reference bias index for each selected reference frame in the reference frame sequence as the compression reference for the current frame j. When the obtained... When the traversal stops, the corresponding filter reference frame is used as the target reference frame for compression of the current frame j. This reduces bitrate fluctuations and redundant information, thereby significantly improving the compression efficiency of ultra-high-definition video.

[0106] The above method is used to obtain different scene segments of ultra-high-definition video. For each scene segment, a representative reference frame for that scene segment is obtained. The selected reference frames are added to the reference frame sequence (those added later undergo traversal analysis first). Each image frame selects the target reference frame that maximizes the compression efficiency of the current frame through traversal analysis of the frames in the reference frame sequence.

[0107] The acquired target reference frame and corresponding image frame are transmitted and stored in the database. The original image block in the image frame is subtracted from the predicted block found in the target reference frame to obtain residual data. A Discrete Cosine Transform (DCT) is performed on the residual data to concentrate energy on a few coefficients. Then, the transform coefficients are quantized. All lateral information (reference frame index, motion vector) and the quantized residual coefficients are entropy-coded together to generate the final compressed video stream.

[0108] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0109] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for dynamic compression of ultra-high definition video considering context information, characterized in that, The method comprises: identifying human eye recognition sensitive features in different regions of each image frame of the video to be compressed to obtain regional sensitivity of the corresponding regions of each image frame; performing scene similarity analysis on each image frame of the video to be compressed to divide the video to be compressed into multiple scene segments; determining multiple screening reference frames representing scene picture features of each scene segment according to distortion cost of each image frame of the scene segment as a compression reference frame and corresponding regional sensitivity; reordering the screening reference frames based on selection results of each screening reference frame in the compression process according to corresponding compression scene segments of the multiple screening reference frames; smoothing code rate fluctuations caused by compression of different screening reference frames of each scene segment and outputting a compressed video stream code; the reordering of the screening reference frames based on the selection results of each screening reference frame in the compression process comprises: obtaining the number of uses and the number of frames apart of a current frame to be compressed and a preceding screening reference frame, and image similarity of the preceding screening reference frame and a corresponding screening reference frame of the current frame; obtaining a difference value of the degree of coincidence according to the difference value of the degree of coincidence of the preceding screening reference frame and the maximum degree of coincidence of the corresponding scene segment; obtaining a degree of bias of the preceding screening reference frame relative to the corresponding scene segment according to the number of uses, the number of frames apart, the image similarity and the difference value of the degree of coincidence; reordering each preceding screening reference frame according to the degree of bias of a preset number of preceding screening reference frames to improve compression and encoding efficiency of the corresponding scene segment; the method for obtaining the degree of bias comprises: obtaining a use activity of the preceding screening reference frame according to a ratio of the number of uses and the number of frames apart; obtaining a scene matching degree of the preceding screening reference frame and the scene segment according to the image similarity and the difference value of the degree of coincidence; obtaining the degree of bias of the preceding screening reference frame relative to the corresponding scene segment according to a product of the use activity and the scene matching degree.

2. The method of claim 1, wherein the method further comprises: the identification of the human eye recognition sensitive features in different regions of each image frame of the video to be compressed to obtain the regional sensitivity of the corresponding regions of each image frame comprises: converting RGB images represented by each image frame of the video to be compressed into LAB images and scaling to the same specification; inputting all LAB images of the same specification into a deep learning model to obtain multiple human feature positions in the LAB images, wherein the multiple human feature positions include face feature positions, eye feature positions and limb feature positions; predicting a saliency value of each human feature position in each LAB image according to a preset regression model; performing Gaussian smoothing and normalization processing on all saliency values to obtain the regional sensitivity of the corresponding regions of each image frame.

3. The method of claim 1, wherein the method further comprises: the scene similarity analysis on each image frame of the video to be compressed to divide the video to be compressed into multiple scene segments comprises: performing similarity comparison on a current frame of the video to be compressed according to a preset comparison item to obtain image similarity of the current frame and adjacent frames, minimum similarity of all adjacent frames, and scene similarity of the current frame and a scene starting frame; According to the image similarity, the minimum similarity and the scene similarity, a rationality degree of the current frame in the corresponding scene is obtained; If the rationality degree of the current starting frame is greater than a degree threshold, the current starting frame is determined as a split frame of the corresponding scene segment, and the to-be-compressed video is divided into multiple scene segments according to all the split frames.

4. The method of claim 3, wherein the context-aware dynamic compression processing of ultra-high definition video is performed by a video encoder. The obtaining of the rationality degree of the current frame in the corresponding scene according to the image similarity, the minimum similarity and the scene similarity comprises: According to the image similarity and the minimum similarity, a first coefficient representing a local change intensity between the current frame and the adjacent frame is obtained; The reciprocal of the scene similarity is determined as a second coefficient representing a difference degree of the current frame from the whole corresponding scene; According to the first coefficient and the second coefficient, the rationality degree of the current frame in the corresponding scene is obtained. 5.The method of claim 1, wherein, The determining of the multiple screening reference frames representing the scene picture features of each scene segment according to the distortion cost of each image frame of the scene segment as a compression reference frame and the corresponding region sensitivity comprises: Motion estimation and motion compensation are performed on the current frame of each scene segment as a compression reference frame to obtain a residual block of the current frame; The residual block of the current frame is processed to obtain a distortion cost of the current frame relative to each image frame of the corresponding scene segment; According to an average sensitivity of all the region sensitivities of the current frame, a maximum sensitivity of all the image frames and all the distortion costs generated by the current frame as the reference frame of the corresponding scene segment, a compliance degree of the current frame as the reference frame of the corresponding scene segment is obtained; The multiple image frames of each scene segment with a compliance degree greater than a degree threshold are determined as the multiple screening reference frames.

6. The method of claim 5, wherein the context-aware dynamic compression processing of ultra-high definition video is performed by a video encoder. The obtaining of the compliance degree of the current frame as the reference frame of the corresponding scene segment according to the average sensitivity of all the region sensitivities of the current frame, the maximum sensitivity of all the image frames and all the distortion costs generated by the current frame as the reference frame of the corresponding scene segment comprises: The ratio of the average sensitivity to the maximum sensitivity is determined as a visual importance of the current frame; The product of the visual importance and the all the distortion costs is determined as the compliance degree of the current frame as the reference frame of the corresponding scene segment.

7. The method for dynamic compression processing of ultra-high definition video considering context information according to claim 1, characterized in that, The smoothing processing according to the code rate fluctuation caused by the compression of different screening reference frames of each scene segment comprises: The compression code rate of the current frame and the previous frame of each scene segment, the current required code rate and the corresponding current distortion cost of the current frame compressed by using the screening reference frame and the arrangement order of the screening reference frame used by the current frame are obtained; According to the compression code rate, the current required code rate, the current distortion cost and the arrangement order, a reference bias index of the current frame compressed by using the screening reference frame is obtained; The screening reference frame corresponding to the reference bias index greater than an index threshold in all the reference bias indexes is determined as a target reference frame of the current frame, and the compression processing of the current frame is implemented according to the target reference frame.

8. The method for dynamic compression processing of ultra-high definition video considering context information according to claim 7, characterized in that, The obtaining of the reference bias index of the current frame compressed by using the screening reference frame according to the compression code rate, the current required code rate, the current distortion cost and the arrangement order comprises: obtaining a code rate smoothing coefficient of the current frame using screened reference frame compression according to the difference between the compression code rate and the current required code rate; obtaining a reference bias index of the current frame using screened reference frame compression according to the code rate smoothing coefficient, a first reciprocal of the current distortion cost and a second reciprocal of the arrangement order.

Images