Video quality enhancement method and system based on edge perception and branch-specific fusion

Abstract

The application discloses a video quality enhancement method and system based on edge perception and branch-specific fusion, relates to the technical field of video processing, and comprises the following steps: acquiring a low-quality video frame sequence and extracting shallow-level features, performing enhancement processing through a double-branch network model, and extracting single-frame edge features and multi-frame dynamic features; branch-specific distillation modules are used to differentially process the double-branch features, and then the edge perception branch-specific fusion module is used for adaptive fusion; through the edge perception reconstruction module, basic feature refinement and edge perception high-low frequency interaction reconstruction are performed; and finally, high-quality video frames after enhancement are output. Through the edge perception and double-branch specific fusion strategy, the detail definition and dynamic consistency of the screen content video are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and specifically to a video quality enhancement method and system based on edge perception and branch-specific fusion. Background Technology

[0002] With the popularization of online collaboration, cloud gaming, and virtual education, screen content videos (SCVs) are widely used in digital interaction and have become an important part of internet video traffic. Screen content videos contain a large amount of high-frequency details such as text and graphics, and have characteristics such as frequent scene changes (such as slideshows and window transitions) and fast motion (such as game screens), which are significantly different from natural videos.

[0003] To reduce transmission bandwidth, screen content video is typically compressed using the High Efficiency Video Coding-Screen Content Coding (HEVC-SCC) standard. However, this compression process inevitably introduces artifacts such as blockiness, blurring, and edge distortion. These artifacts are particularly noticeable in text-dense areas and during scene transitions, severely impacting the user's visual experience and adversely affecting subsequent tasks such as image recognition and object detection.

[0004] Existing video quality enhancement methods are mainly divided into two categories: single-frame enhancement and multi-frame enhancement. Single-frame methods only utilize the spatial information of a single frame, making it difficult to handle dynamic content and frame reconstruction after scene changes. While multi-frame methods utilize inter-frame temporal information, they lack adaptability when processing multi-scale edge features and frequent scene changes in screen content videos, and are prone to alignment errors and feature fusion errors. Enhancement methods specifically designed for screen content videos still suffer from problems such as insufficient multi-scale edge capture, weak anti-interference ability of spatiotemporal fusion, and loss of details during branch feature fusion, making it difficult to meet the needs of high-quality enhancement for screen content videos.

[0005] Therefore, there is an urgent need for a screen content video quality enhancement solution that can accurately capture multi-scale edge details, robustly handle spatiotemporal dynamic changes, and efficiently fuse branch features, so as to effectively remove compression artifacts and restore clear video details. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a video quality enhancement method and system based on edge perception and branch-specific fusion. By constructing an edge-aware dual-branch network and combining branch-specific distillation, edge-aware branch-specific fusion, and high- and low-frequency interactive reconstruction, this method effectively removes compression artifacts from screen content videos and improves detail clarity and dynamic consistency using techniques such as multi-scale multi-morphological convolution and deformable alignment.

[0007] On the one hand, video quality enhancement methods based on edge awareness and branch-specific fusion include:

[0008] S1, Construct and train a screen content video quality enhancement model to obtain a trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module.

[0009] S2, obtain the low-quality frame sequence of the video content to be enhanced, and extract edge-aware features from the low-quality frame sequence to obtain shallow features;

[0010] S3: Input the shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames.

[0011] Specifically, shallow features are input into single-frame edge branches and multi-frame dynamic branches. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are input into a branch-specific distillation module, which performs feature transformation, local attention, and global calibration to obtain distilled branch features. The distilled branch features are input into an edge-aware branch-specific fusion module, which performs channel feature selection and spatial weight adaptive fusion to obtain fused features. The fused features are input into an edge-aware reconstruction module, which performs basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing to obtain enhanced high-quality video frames.

[0012] Furthermore, the extraction of single-frame edge features by edge-aware multi-scale multi-morphological convolutional blocks is divided into an edge-aware multi-scale convolution stage and an edge-aware multi-morphological convolution stage.

[0013] The edge-aware multi-scale convolution stage uses 1×1, 3×3, 5×5 and 7×7 convolution kernels to extract multi-scale features, which are then concatenated and connected with the initial input features to establish scale residual connections to obtain multi-scale features.

[0014] The edge-aware multimorphic convolution stage uses 1×9, 9×1, 3×3, 3×1 and 1×3 directional convolution kernels to extract directional features. After channel attention weighting and convolutional projection, edge residual connections are established with multi-scale features. Then, after two 3×3 convolutions and element-wise addition operations, single-frame edge features are output.

[0015] Furthermore, the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion, specifically including:

[0016] The multi-scale deformable alignment process specifically involves constructing a three-level feature pyramid and calculating the offset and mask at each scale through a deformable convolutional network to achieve feature alignment across multiple frames.

[0017] The temporal relevance enhancement process specifically involves calculating the content similarity between the current frame and adjacent frames through feature embedding, generating a temporal attention map, and then fusing the aligned multi-frame features after weighting to obtain multi-frame aggregated features.

[0018] The dynamic spatiotemporal information fusion process specifically involves calculating the difference features between the aggregated features of multiple frames and the features of the current frame, performing convolutional transformation based on the difference features to obtain dynamic spatiotemporal information, and outputting multi-frame dynamic features.

[0019] Furthermore, the single-frame edge features and multi-frame dynamic features are input into the branch-specific distillation module. After feature transformation, local attention, and global calibration, the distilled branch features are obtained, specifically including:

[0020] Branch-specific distillation module for single-frame edge features and multi-frame dynamic features A three-stage specialized distillation process is performed, consisting of branch-specific differentiated feature transformation, branch-independent local attention enhancement, and cross-branch global weight calibration, to obtain distilled branch features that retain branch characteristics. The calculation formula is as follows:

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] in, Indicates multi-frame dynamic features; Represents edge features of a single frame; Represents the dynamic features of multiple frames after transformation; This represents the sigmoid activation function; This represents the edge features of a single frame after transformation; This represents a 5×5 convolution operation; This represents a 3×3 convolution operation; Indicates average pooling; This represents the dynamic features of multiple frames after specific attention processing across multiple frames; This represents the edge features of a single frame after specific attention processing for a single frame branch; This represents a multi-frame branch-specific attention convolution; This represents a single-frame branch-specific attention convolution; Represents global convolution; Indicates the global relevance weight across branches; This represents the multi-frame dynamic distillation features after distillation. This represents the edge distillation features of a single frame after distillation.

[0029] Furthermore, in S3, the fused feature input to the edge-aware reconstruction module undergoes basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing to obtain enhanced high-quality video frames, specifically including:

[0030] Fusion features Deep feature refinement is performed using multi-layer residual blocks to output basic features that include complete structural information. ;

[0031] Next, the basic features Classified into high-frequency components according to channels. with low-frequency components Based on high frequency components with low-frequency components The enhanced high-quality video frame is calculated using the following formula:

[0032] ;

[0033] ;

[0034] ;

[0035] );

[0036] ;

[0037] in, This represents a full-resolution high-frequency feature processing convolution sequence; This represents the initial high-frequency features after high-frequency convolution processing; This indicates the low-frequency features after initial downsampling and low-frequency convolution, followed by upsampling to restore the size; f high_final This represents the final high-frequency characteristics after f is fused with low-frequency information; low_final This represents the final low-frequency characteristics after fusing high-frequency information and then restoring the size; This represents a low-frequency convolutional sequence; and These represent downsampling and upsampling, respectively. The channel mapping function representing the transformation from low-frequency features to high-frequency features; A channel mapping function representing high-frequency features to low-frequency features; This represents the channel splicing and fusion function; This indicates that feature concatenation is performed at the channel dimension;

[0038] Finally, the reconstructed high and low frequency fusion features are analyzed. With basic features Establish residual connections to ultimately output enhanced, high-quality video frames. .

[0039] Furthermore, the screen content video quality enhancement model uses a weighted combination of MS-SSIM loss and L1 loss as its loss function. The calculation formula is as follows:

[0040] ;

[0041] in, These are weight parameters; Used to measure the multi-scale structural similarity between enhanced frames and original frames; Used to measure pixel-level differences; This represents the weight Gaussian kernel at the Mth scale obtained from downsampling in the MS-SSIM loss function.

[0042] On the other hand, video quality enhancement systems based on edge awareness and branch-specific fusion include:

[0043] The model training module is used to construct and train the screen content video quality enhancement model to obtain the trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module.

[0044] The feature extraction module is used to acquire low-quality frame sequences of the video content to be enhanced, and to perform edge-aware feature extraction on the low-quality frame sequences to obtain shallow features.

[0045] The enhancement module is used to input shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames.

[0046] Specifically, shallow features are input into single-frame edge branches and multi-frame dynamic branches. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are input into a branch-specific distillation module, which performs feature transformation, local attention, and global calibration to obtain distilled branch features. The distilled branch features are input into an edge-aware branch-specific fusion module, which performs channel feature selection and spatial weight adaptive fusion to obtain fused features. The fused features are input into an edge-aware reconstruction module, which performs basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing to obtain enhanced high-quality video frames.

[0047] The present invention adopts the above technical solution and has the following beneficial effects:

[0048] (1) This invention extracts multi-scale and multi-directional edge features through edge-aware multi-scale and multi-morphological convolutional blocks in single-frame edge branches, thereby enhancing the ability to capture details in high-frequency areas such as text and graphics, and thus improving the clarity of video frames.

[0049] (2) This invention uses multi-scale deformable alignment, time correlation enhancement and dynamic spatiotemporal information fusion through multi-frame dynamic branching, which can robustly handle dynamic changes such as fast motion and scene switching in screen content video, reduce alignment errors and enhance inter-frame temporal consistency.

[0050] (3) The present invention performs differentiated feature transformation, local attention enhancement and global weight calibration through the branch-specific distillation module to preserve branch characteristics; and then performs adaptive fusion of channel and spatial dimensions through the edge-aware branch-specific fusion module, which effectively avoids loss of details and improves the expressive power of fused features. Attached Figure Description

[0051] Figure 1 This is a flowchart of a video quality enhancement method based on edge perception and branch-specific fusion according to an embodiment of the present invention.

[0052] Figure 2 This is a schematic diagram of the screen content video quality enhancement model according to an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of an edge-aware multi-scale multi-morphological convolutional block according to an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the branch-specific distillation module according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the edge-aware reconstruction module according to an embodiment of the present invention;

[0056] Figure 6 This is a diagram of a video quality enhancement system based on edge perception and branch-specific fusion, according to an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0058] like Figure 1 As shown, the present invention provides a video quality enhancement method based on edge awareness and branch-specific fusion, comprising:

[0059] S1. Construct and train a screen content video quality enhancement model to obtain a trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module.

[0060] S2, acquire the low-quality frame sequence of the video content to be enhanced, extract edge-aware features from the low-quality frame sequence to obtain shallow features.

[0061] Specifically, in this embodiment, the video content to be enhanced is a video compressed in LDP mode by a VTM12.1 encoder, with quantization parameters (QP) selectable as 22, 27, 32, or 37. Feature extraction is performed on each low-quality frame using five stacked residual blocks. Each residual block contains a convolutional layer, an activation function, and a batch normalization layer, outputting shallow features with a dimension of 64. These features preserve the basic structural information of the original frame, providing a foundation for subsequent branch processing.

[0062] Specifically, in this embodiment, each frame in the low-quality frame sequence is processed using five stacked residual blocks. This transforms the original pixel data into shallow features that retain basic structural information, laying the foundation for subsequent branch feature extraction. In this specific embodiment, the video content to be enhanced is a video compressed in LDP mode using a VTM12.1 encoder, with quantization parameters (QP) selectable at 22, 27, 32, or 37. Feature extraction is performed on each low-quality frame using five stacked residual blocks. Each residual block contains a convolutional layer, an activation function, and a batch normalization layer, outputting shallow features with a dimension of 64. These features retain the basic structural information of the original frame, providing a basis for subsequent branch processing.

[0063] S3 inputs shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames.

[0064] Specifically, shallow features are input into single-frame edge branches and multi-frame dynamic branches. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are input into a branch-specific distillation module, which performs feature transformation, local attention, and global calibration to obtain distilled branch features. The distilled branch features are input into an edge-aware branch-specific fusion module, which performs channel feature selection and spatial weight adaptive fusion to obtain fused features. The fused features are input into an edge-aware reconstruction module, which performs basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing to obtain enhanced high-quality video frames.

[0065] Specifically, the extraction of single-frame edge features by edge-aware multi-scale multi-morphological convolutional blocks is divided into an edge-aware multi-scale convolution stage and an edge-aware multi-morphological convolution stage.

[0066] The edge-aware multi-scale convolution stage uses 1×1, 3×3, 5×5 and 7×7 convolution kernels to extract multi-scale features, which are then concatenated and connected with the initial input features to establish scale residual connections to obtain multi-scale features.

[0067] The edge-aware multimorphic convolution stage uses 1×9, 9×1, 3×3, 3×1 and 1×3 directional convolution kernels to extract directional features. After channel attention weighting and convolutional projection, edge residual connections are established with multi-scale features. Then, after two 3×3 convolutions and element-wise addition operations, single-frame edge features are output.

[0068] Specifically, the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion, including:

[0069] The multi-scale deformable alignment process specifically involves constructing a three-level feature pyramid and calculating the offset and mask at each scale through a deformable convolutional network to achieve feature alignment across multiple frames.

[0070] The temporal relevance enhancement process specifically involves calculating the content similarity between the current frame and adjacent frames through feature embedding, generating a temporal attention map, and then fusing the aligned multi-frame features after weighting to obtain multi-frame aggregated features.

[0071] The dynamic spatiotemporal information fusion process specifically involves calculating the difference features between the aggregated features of multiple frames and the features of the current frame, performing convolutional transformation based on the difference features to obtain dynamic spatiotemporal information, and outputting multi-frame dynamic features.

[0072] Specifically, such as Figure 2The diagram shows the overall structure of the screen content video quality enhancement model according to an embodiment of the present invention. The model input is a target low-quality frame of the screen content video to be enhanced, along with its preceding and following frames, totaling five video frames. The final output is an enhanced target high-quality frame. The model consists of a shallow feature extraction section, a parallel dual-branch structure, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module, in sequence. The input five low-quality video frames are first subjected to shallow feature extraction to obtain shallow features. These shallow features are simultaneously fed into a single-frame edge branch and a multi-frame dynamic branch to extract single-frame edge features and multi-frame dynamic features, respectively. The dual-branch output features are fed into the branch-specific distillation module for feature optimization, and then undergo adaptive feature fusion through the edge-aware branch-specific fusion module. The fused features are input into the edge-aware reconstruction module for feature refinement, generating an enhancement residual mapping. This residual mapping is then added element-wise to the target low-quality frame to be enhanced, ultimately outputting an enhanced target video frame with clear edges, complete details, and excellent temporal consistency.

[0073] Specifically, such as Figure 3 As shown, in this embodiment, the single-frame edge branch includes two cascaded edge-aware multi-scale multi-morphological convolutional blocks (EA-MSMC). This module first extracts multi-scale features using 1×1, 3×3, 5×5, and 7×7 convolutional kernels through edge-aware multi-scale convolution (EA-MSConv), and establishes scale residual connections with the input features after concatenation. Then, it extracts directional features using 1×9, 9×1, 3×3, 3×1, and 1×3 directional convolutional kernels through edge-aware multi-morphological convolution (EA-MMConv), and establishes edge residual connections with the multi-scale features after channel attention (CA) weighting and convolutional projection. Finally, it outputs the single-frame edge features through two 3×3 convolutions and element-wise summation. The multi-frame dynamic branch includes a multi-scale deformable alignment module (MSDA), a temporal correlation enhancement module (TCR), and a dynamic spatiotemporal information fusion module (DSTIF). MSDA constructs a three-level feature pyramid (L1, L2, L3), calculates offsets and masks at each scale using DCNv2, and achieves multi-frame feature alignment; TCR generates a temporal attention map through feature embedding and similarity calculation, and weights and fuses the aligned features; DSTIF concatenates the offsets, masks, temporal attention maps and feature difference information, and obtains dynamic spatiotemporal features through convolution transformation, outputting multi-frame dynamic features.

[0074] Specifically, the branch-specific distillation module analyzes single-frame edge features. and multi-frame dynamic features A three-stage specialized distillation process is performed, consisting of branch-specific differentiated feature transformation, branch-independent local attention enhancement, and cross-branch global weight calibration, to obtain distilled branch features that retain branch characteristics. The calculation formula is as follows:

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in, Indicates multi-frame dynamic features; Represents edge features of a single frame; Represents the dynamic features of multiple frames after transformation; This represents the sigmoid activation function; This represents the edge features of a single frame after transformation; This represents a 5×5 convolution operation; This represents a 3×3 convolution operation; Indicates average pooling; This represents the dynamic features of multiple frames after specific attention processing across multiple frames; This represents the edge features of a single frame after specific attention processing for a single frame branch; This represents a multi-frame branch-specific attention convolution; This represents a single-frame branch-specific attention convolution; Represents global convolution; Indicates the global relevance weight across branches; This represents the multi-frame dynamic distillation features after distillation. This represents the edge distillation features of a single frame after distillation.

[0083] Specifically, in the branch-specific distillation module, feature transformation uses convolutional sequences with different receptive fields to adapt to multi-frame dynamic features and single-frame edge features respectively. Multi-frame dynamic features use a 1×1 convolution + 5×5 convolution sequence, while single-frame edge features use a 1×1 convolution + 3×3 convolution sequence. Local attention generates attention weights through branch-specific channel attention and global average pooling to enhance the transformed features. Global calibration generates cross-branch global weights through global convolutional layers and sigmoid activation functions to weight the features processed by local attention, resulting in the distilled branch features.

[0084] Specifically, such as Figure 4The diagram illustrates the Branch-Specific Distillation (BSD) module in this embodiment, which includes feature transformation, local attention, and global calibration. Feature transformation employs different convolutional sequences to adapt to the dual-branch features; local attention enhances key features through channel attention and global average pooling; global calibration generates cross-branch weights through global convolution, balancing the global correlation between the two branches and outputting the distilled branch features.

[0085] The Edge-Aware Branch-Specific Fusion Module (EABSF) includes Channel Feature Selection (CFS) and Spatial Weight Adaptation (SWA). CFS generates channel weights based on global statistical information, differentiating the importance of dual-branch channels; SWA generates a spatial mask based on dynamic spatiotemporal information, adjusting the spatial weights according to the regional motion intensity to achieve adaptive fusion. Figure 5 As shown, the edge-aware reconstruction module includes basic feature refinement, edge-aware high-low frequency interactive reconstruction (EA-HLFIR), and residual output. Basic feature refinement enhances the expressive power of fused features through five stacked residual blocks; EA-HLFIR divides features into high-frequency and low-frequency components, processes high-frequency components at full resolution, optimizes low-frequency components through downsampling, and promotes information complementarity through cross-frequency domain interaction; finally, the channel dimension is mapped by 3×3 convolution, and residual connections are established with the original low-quality frame to output a high-quality video frame.

[0086] Specifically, channel feature selection generates channel weights based on global statistical information to distinguish the importance of bi-branch features in different channels; spatial weight adaptation generates spatial masks based on dynamic spatiotemporal information and dynamically adjusts the spatial weights of bi-branch features according to regional motion intensity to achieve adaptive fusion of channel and spatial dimensions and enhance edge feature preservation.

[0087] Specifically, regarding fusion features Deep feature refinement is performed using multi-layer residual blocks to output basic features that include complete structural information. ;

[0088] Next, the basic features Classified into high-frequency components according to channels. with low-frequency components Based on high frequency components with low-frequency components The enhanced high-quality video frame is calculated using the following formula:

[0089] ;

[0090] ;

[0091] ;

[0092] );

[0093] ;

[0094] in, This represents a full-resolution high-frequency feature processing convolution sequence; This represents the initial high-frequency features after high-frequency convolution processing; This indicates the low-frequency features after initial downsampling and low-frequency convolution, followed by upsampling to restore the size; f high_final This represents the final high-frequency characteristics after f is fused with low-frequency information; low_final This represents the final low-frequency characteristics after fusing high-frequency information and then restoring the size; This represents a low-frequency convolutional sequence; and These represent downsampling and upsampling, respectively. The channel mapping function representing the transformation from low-frequency features to high-frequency features; A channel mapping function representing high-frequency features to low-frequency features; This represents the channel splicing and fusion function; This indicates that feature concatenation is performed at the channel dimension;

[0095] Finally, the reconstructed high and low frequency fusion features are analyzed. With basic features Establish residual connections to ultimately output enhanced, high-quality video frames. .

[0096] Specifically, the screen content video quality enhancement model uses a weighted combination of MS-SSIM loss and L1 loss as the loss function. The calculation formula is as follows:

[0097] ;

[0098] in, These are weight parameters; Used to measure the multi-scale structural similarity between enhanced frames and original frames; Used to measure pixel-level differences; This represents the weight Gaussian kernel at the Mth scale obtained from downsampling in the MS-SSIM loss function.

[0099] Specifically, in this embodiment, basic feature refinement enhances the expressive power of fused features through five stacked residual blocks; the edge-aware high-low frequency interactive reconstruction module (EA-HLFIR) divides features into high-frequency and low-frequency components for separate processing, achieving complementary high- and low-frequency information through cross-frequency domain interaction; finally, the channel dimension is mapped by 3×3 convolution, and residual connections are established with the original low-quality frames to output high-quality video frames. The edge-aware multi-scale multi-morphological convolution block includes an edge-aware multi-scale convolution stage and an edge-aware multi-morphological convolution stage. Multi-scale convolution captures features at different resolutions, multi-morphological convolution accurately extracts directional edges, and dual residual connections ensure feature integrity. Multi-frame dynamic branches solve inter-frame misalignment caused by complex motion through multi-scale deformable alignment, enhance temporal correlation to highlight effective temporal information, and quantify the intensity of scene changes through dynamic spatiotemporal information fusion, improving the model's adaptability to scene switching and rapid motion. The branch-specific distillation module reduces feature redundancy and improves fusion efficiency while preserving single-frame edge details and multi-frame dynamic information through differentiated feature transformation, branch-specific local attention, and cross-branch global calibration. The edge-aware high- and low-frequency interactive reconstruction module uses full-resolution processing for high-frequency components to maximize the preservation of details such as text edges, and downsampling optimization for low-frequency components to reduce computational load. The cross-frequency domain interaction mechanism promotes the complementarity of high- and low-frequency information and improves the overall reconstruction quality.

[0100] Specifically, in this embodiment, four existing algorithms—ARCNN, STDF-R3L, QECF, and STA—were selected for comparative testing on the SCV82 dataset with four quantization parameters QP=22 / 27 / 32 / 37. Here, QP represents the quantization parameter, used to control the quality and bit rate of video encoding. The evaluation metrics are PSNR and SSIM, where PSNR is Peak Signal-to-Noise Ratio and SSIM is the Structural Similarity Index. These two metrics are widely used in compressed video enhancement. APSNR>0 and ∆SSIM>0 indicate improved compressed video quality.

[0101] Specifically, our method was compared with four existing methods (ARCNN, STDF-R3L, QECF, and STA) under different quantization parameters (QP=22, 27, 32, 37). The results show that our method achieved the highest average gain under all QP conditions: 0.275 dB / 4.8 × 10⁻² at QP=22, 0.388 dB / 10.2 × 10⁻² at QP=27, 0.483 dB / 21.3 × 10⁻² at QP=32, and 0.567 dB / 34.9 × 10⁻² at QP=37. In contrast, the best-performing method among the comparisons, STA, only achieved 0.556 dB / 30.9 × 10⁻² at QP=37. As can be seen, the method of this application is significantly superior to existing methods under different compression intensities, especially under strong compression (high QP) conditions, where its advantages are more prominent. It can effectively improve the detail clarity and structural integrity of screen content video. Compared with existing mainstream compression screen content video enhancement methods, at QP=22, the average ∆PSNR of this method reaches 0.275dB, ∆SSIM reaches 4.8×10⁻², and BD-BR is reduced by 7.01%, showing excellent performance in text edge restoration and scene switching processing.

[0102] In summary, the present invention also has the following beneficial effects:

[0103] (1) Design edge-aware multi-scale and multi-morphological convolutional blocks. Through the synergistic effect of multi-scale convolution and multi-morphological directional convolution, accurately capture fine-scale text edges and large-scale structural edges in screen content videos, effectively solve the problem of insufficient multi-scale edge capture, and lay the foundation for subsequent detail restoration.

[0104] (2) Multi-frame dynamic branch integration integrates multi-scale deformable alignment, temporal correlation enhancement and dynamic spatiotemporal information fusion, which can robustly handle frequent scene switching and fast movement of screen content video, reduce the introduction of errors in the spatiotemporal fusion process and improve the efficiency of inter-frame information utilization.

[0105] (3) The branch-specific distillation module improves the quality of feature fusion by using differentiated feature adaptation, local attention enhancement and global calibration to retain the uniqueness of single-frame edge details and multi-frame dynamic information, while reducing feature redundancy and avoiding detail dilution during branch feature fusion.

[0106] (4) The edge perception reconstruction module adopts high- and low-frequency differential processing and cross-frequency domain interaction mechanism. High-frequency component full-resolution processing ensures accurate restoration of details such as text edges, low-frequency component optimization reduces computational cost, and cross-frequency domain interaction promotes information complementarity, effectively balancing detail restoration and computational efficiency.

[0107] (5) Through the collaborative design of each module, experiments on the SCV82 dataset show that the method outperforms the existing mainstream methods in terms of ∆PSNR and ∆SSIM, and can significantly remove compression artifacts, restore the clear edges and details of the screen content video, and improve the user's visual experience.

[0108] like Figure 6 As shown, this embodiment also discloses a video quality enhancement system based on edge perception and branch-specific fusion, including:

[0109] The model training module 61 is used to construct and train the screen content video quality enhancement model to obtain the trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module.

[0110] Feature extraction module 62 is used to acquire a low-quality frame sequence of the video content to be enhanced, and to perform edge-aware feature extraction on the low-quality frame sequence to obtain shallow features.

[0111] Enhancement module 63 is used to input shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames;

[0112] Specifically, shallow features are input into single-frame edge branches and multi-frame dynamic branches. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are input into a branch-specific distillation module, which performs feature transformation, local attention, and global calibration to obtain distilled branch features. The distilled branch features are input into an edge-aware branch-specific fusion module, which performs channel feature selection and spatial weight adaptive fusion to obtain fused features. The fused features are input into an edge-aware reconstruction module, which performs basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing to obtain enhanced high-quality video frames.

[0113] The specific implementation of the video quality enhancement system based on edge perception and branch-specific fusion is the same as that of the video quality enhancement method based on edge perception and branch-specific fusion, and will not be described again in this embodiment.

[0114] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A video quality enhancement method based on edge perception and branch-specific fusion, characterized in that, Includes the following steps: S1, Construct and train a screen content video quality enhancement model to obtain a trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module. S2, obtain the low-quality frame sequence of the video content to be enhanced, and extract edge-aware features from the low-quality frame sequence to obtain shallow features; S3: Input the shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames. Specifically, shallow features are input into the single-frame edge branch and the multi-frame dynamic branch. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale and multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are then input into the branch-specific distillation module. After feature transformation, local attention, and global calibration, distilled branch features are obtained. The distilled branch features are then input into the edge-aware branch-specific fusion module, where channel feature selection and spatial weight adaptive fusion are used to obtain fused features. The fused features are then input into the edge-aware reconstruction module, where basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing are used to obtain enhanced high-quality video frames. Single-frame edge features and multi-frame dynamic features are input into the branch-specific distillation module. After feature transformation, local attention, and global calibration, the distilled branch features are obtained, specifically including: Branch-specific distillation module for single-frame edge features and multi-frame dynamic features A three-stage specialized distillation process is performed, consisting of branch-specific differentiated feature transformation, branch-independent local attention enhancement, and cross-branch global weight calibration, to obtain distilled branch features that retain branch characteristics. The calculation formula is as follows: ； ； ； ； ； ； ； in, Indicates multi-frame dynamic features; Represents edge features of a single frame; Represents the dynamic features of multiple frames after transformation; This represents the sigmoid activation function; Represents the edge features of a single frame after transformation; This represents a 5×5 convolution operation; This represents a 3×3 convolution operation; Indicates average pooling; This represents the dynamic features of multiple frames after specific attention processing across multiple frames; This represents the edge features of a single frame after specific attention processing for a single frame branch; This represents a multi-frame branch-specific attention convolution; This represents a single-frame branch-specific attention convolution; Represents global convolution; Indicates the global relevance weight across branches; This represents the dynamic distillation features across multiple frames after distillation. This represents the edge distillation features of a single frame after distillation.

2. The video quality enhancement method based on edge perception and branch-specific fusion according to claim 1, characterized in that, In S3, the edge-aware multi-scale multi-morphic convolutional block extracts single-frame edge features in two stages: edge-aware multi-scale convolution and edge-aware multi-morphic convolution. The edge-aware multi-scale convolution stage uses 1×1, 3×3, 5×5 and 7×7 convolution kernels to extract multi-scale features, which are then concatenated and connected with the initial input features to establish scale residual connections to obtain multi-scale features. The edge-aware multimorphic convolution stage uses 1×9, 9×1, 3×3, 3×1 and 1×3 directional convolution kernels to extract directional features. After channel attention weighting and convolutional projection, edge residual connections are established with multi-scale features. Then, after two 3×3 convolutions and element-wise addition operations, single-frame edge features are output.

3. The video quality enhancement method based on edge perception and branch-specific fusion according to claim 1, characterized in that, In S3, the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion, specifically including: The multi-scale deformable alignment process specifically involves constructing a three-level feature pyramid and calculating the offset and mask at each scale through a deformable convolutional network to achieve feature alignment across multiple frames. The temporal relevance enhancement process specifically involves calculating the content similarity between the current frame and adjacent frames through feature embedding, generating a temporal attention map, and then fusing the aligned multi-frame features after weighting to obtain multi-frame aggregated features. The dynamic spatiotemporal information fusion process specifically involves calculating the difference features between the aggregated features of multiple frames and the features of the current frame, performing convolutional transformation based on the difference features to obtain dynamic spatiotemporal information, and outputting multi-frame dynamic features.

4. The video quality enhancement method based on edge perception and branch-specific fusion according to claim 1, characterized in that, In S3, the fused features are input into the edge-aware reconstruction module. After basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing, enhanced high-quality video frames are obtained, specifically including: Fusion features Deep feature refinement is performed using multi-layer residual blocks to output basic features that include complete structural information. ; Next, the basic features Classified into high-frequency components according to channels. with low-frequency components Based on high frequency components with low-frequency components The enhanced high-quality video frame is calculated using the following formula: ； ； ； )； ； in, This represents a full-resolution high-frequency feature processing convolution sequence; This represents the initial high-frequency features after high-frequency convolution processing; This indicates the low-frequency features after initial downsampling and low-frequency convolution, followed by upsampling to restore the size; f high_final This represents the final high-frequency characteristics after f fusion of low-frequency information; low_final This represents the final low-frequency characteristics after fusing high-frequency information and then restoring the size; This represents a low-frequency convolutional sequence; and These represent downsampling and upsampling, respectively. The channel mapping function representing the transformation from low-frequency features to high-frequency features; A channel mapping function representing high-frequency features to low-frequency features; This represents the channel splicing and fusion function; This indicates that feature concatenation is performed at the channel dimension; Finally, the reconstructed high and low frequency fusion features are analyzed. With basic features Establish residual connections to ultimately output enhanced, high-quality video frames. .

5. The video quality enhancement method based on edge perception and branch-specific fusion according to claim 1, characterized in that, This also includes a screen content video quality enhancement model that uses a weighted combination of MS-SSIM loss and L1 loss as the loss function. The calculation formula is as follows: ； in, These are weight parameters; Used to measure the multi-scale structural similarity between enhanced frames and original frames; Used to measure pixel-level differences; This represents the weight Gaussian kernel at the Mth scale obtained from downsampling in the MS-SSIM loss function.

6. A video quality enhancement system based on edge perception and branch-specific fusion, characterized in that, include: The model training module is used to construct and train the screen content video quality enhancement model to obtain the trained screen content video quality enhancement model. The screen content video quality enhancement model includes a single-frame edge branch, a multi-frame dynamic branch, a branch-specific distillation module, an edge-aware branch-specific fusion module, and an edge-aware reconstruction module. The feature extraction module is used to acquire low-quality frame sequences of the video content to be enhanced, and to perform edge-aware feature extraction on the low-quality frame sequences to obtain shallow features. The enhancement module is used to input shallow features into the trained screen content video quality enhancement model to obtain enhanced high-quality video frames. Specifically, shallow features are input into the single-frame edge branch and the multi-frame dynamic branch. The single-frame edge branch extracts single-frame edge features through edge-aware multi-scale and multi-morphological convolutional blocks, while the multi-frame dynamic branch extracts multi-frame dynamic features through multi-scale deformable alignment, temporal correlation enhancement, and dynamic spatiotemporal information fusion. The single-frame edge features and multi-frame dynamic features are then input into the branch-specific distillation module. After feature transformation, local attention, and global calibration, distilled branch features are obtained. The distilled branch features are then input into the edge-aware branch-specific fusion module, where channel feature selection and spatial weight adaptive fusion are used to obtain fused features. The fused features are then input into the edge-aware reconstruction module, where basic feature refinement, edge-aware high- and low-frequency interactive reconstruction, and residual output processing are used to obtain enhanced high-quality video frames. Single-frame edge features and multi-frame dynamic features are input into the branch-specific distillation module. After feature transformation, local attention, and global calibration, the distilled branch features are obtained, specifically including: Branch-specific distillation module for single-frame edge features and multi-frame dynamic features A three-stage specialized distillation process is performed, consisting of branch-specific differentiated feature transformation, branch-independent local attention enhancement, and cross-branch global weight calibration, to obtain distilled branch features that retain branch characteristics. The calculation formula is as follows: ； ； ； ； ； ； ； in, Indicates multi-frame dynamic features; Represents edge features of a single frame; Represents the dynamic features of multiple frames after transformation; This represents the sigmoid activation function; Represents the edge features of a single frame after transformation; This represents a 5×5 convolution operation; This represents a 3×3 convolution operation; Indicates average pooling; This represents the dynamic features of multiple frames after specific attention processing across multiple frames; This represents the edge features of a single frame after specific attention processing for a single frame branch; This represents a multi-frame branch-specific attention convolution; This represents a single-frame branch-specific attention convolution; Represents global convolution; Indicates the global relevance weight across branches; This represents the dynamic distillation features across multiple frames after distillation. This represents the edge distillation features of a single frame after distillation.

Images