Real-time video sharpening system based on deep artifact recognition

Abstract

This invention discloses a real-time video clarity restoration system based on depth artifact recognition, belonging to the field of computer vision technology. Specifically, it includes: a decoding extraction module, a feature extraction module, a mask generation module, a dynamic allocation module, and an image reconstruction module. The system decodes the original video stream, extracting the luminance and chrominance channel data of each video frame. By comparing the transform coefficient information obtained from the decoding side with cross-frame chrominance data, a dual-channel fused chrominance residual feature map is calculated, constructing a multi-channel input feature tensor. Based on a lightweight shallow convolutional neural network, a pixel-wise artifact probability mask is output through cross-channel feature fusion and nonlinear mapping. The video frame is divided into high-probability artifact regions and low-probability smooth regions, and these regions are processed separately. Using spatial fusion weights, the restored region features are weighted and linearly combined with the original video frame data, and a high-definition video frame is output after processing.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a real-time video clarity restoration system based on depth artifact recognition. Background Technology

[0002] With the rapid popularization of ultra-high-definition video technology and streaming media services, the transmission bandwidth and storage costs of video data are facing enormous pressure. To alleviate this contradiction, modern video compression standards have adopted lossy compression technology, which significantly reduces the bit rate through discrete cosine transform, quantization, and motion compensation.

[0003] However, in scenarios with low bitrates or high compression ratios, significant compression artifacts often occur after video decoding and reconstruction, mainly manifesting in the following forms: Block artifacts: Due to block-based transform quantization, a discontinuous, step-like appearance appears at macroblock boundaries; Ringing effect: ripple noise generated near strong edges of an image due to the loss of high-frequency components; Color bleeding and blurring: Traditional downsampling and quantization cause color edges to lose sharpness, resulting in color bleeding.

[0004] Current video restoration solutions mainly fall into two categories: (1) Traditional loop filtering or post-processing algorithms have low computational complexity but limited repair capabilities, which can easily cause excessive smoothing of images and loss of texture details. (2) Although the repair methods based on deep neural networks far surpass traditional algorithms in repair quality, most of them have the following drawbacks: Performing a uniform, high-complexity convolution operation on all regions of the entire image is insufficient to meet the real-time processing requirements of high-definition video streams. Failure to fully utilize the transform coefficients and residual information provided by the decoding end leads to false enhancement of originally smooth and lossless regions, and even introduces new artifacts; Existing algorithms mostly focus on repairing the luminance channel, neglecting the importance of the chrominance channel in human vision, resulting in unnatural color performance in the repaired video.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a real-time video clarity restoration system based on depth artifact recognition, so as to solve the problems in the background art mentioned above.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a real-time video clarity restoration system based on depth artifact recognition, specifically including: a decoding extraction module, a feature extraction module, a mask generation module, a dynamic allocation module, and an image reconstruction module; Decoding and extraction module: Decodes the original video stream and extracts the luminance and chrominance channel data of its video frames; Feature extraction module: Extracts high-frequency edge features based on the luminance channel data, and calculates the dual-channel fused chroma residual feature map by comparing the transform coefficient information obtained from the decoding side with the cross-frame chroma, and constructs a multi-channel input feature tensor; Mask generation module: Input the high-frequency components of the edges and the chromaticity residual data into a preset lightweight shallow convolutional neural network, and output a pixel-by-pixel artifact probability mask through cross-channel feature fusion and nonlinear mapping; Dynamic allocation module: Based on the weight distribution of the artifact probability mask, the video frame is divided into a high-probability artifact region and a low-probability smooth region. The high-probability artifact region is reconstructed by calling a repair operator based on a depth residual structure, and the low-probability smooth region is processed to preserve pixels. Image reconstruction module: Using the artifact probability mask as spatial fusion weight, the repaired regional features are combined linearly with the original video frame data in a weighted manner, and after inverse normalization and tone correction, high-definition video frames are output.

[0008] As a preferred embodiment of the real-time video clarity restoration system based on depth artifact recognition described in this invention, wherein: Receive the raw video stream; The original video stream is entropy decoded, dequantized, and inverse transformed by a video decoder to obtain the pixel data of the reconstructed video frames. The pixel data is converted from the RGB color space to the YUV color space, where the Y channel is the luminance component and the U and V channels are the chrominance components. The luminance channel data and chrominance channel data of the video frame are extracted and separated respectively, and stored as independent three-dimensional tensors; The extracted luminance channel data and chrominance channel data are normalized to uniformly map the pixel value range to the [0,1] interval; The processed luminance channel data and chrominance channel data are simultaneously transmitted to the feature extraction module.

[0009] As a preferred embodiment of the real-time video clarity restoration system based on depth artifact recognition described in this invention, wherein: Extracting the high-frequency edge components of the video frame using a high-pass filter operator specifically includes: The gradient magnitudes of the brightness channels in the horizontal and vertical directions are calculated using the Sobel operator, and then processed by a nonlinear activation function to generate single-channel edge feature maps. The calculation of the dual-channel fused chroma residual feature map specifically includes: (1) Perform pixel-by-pixel subtraction between the chroma channel data of the current video frame and the smoothed chroma version of the reference frame after low-pass filtering to obtain the original chroma residual; (2) Based on the transform coefficient residual information obtained from the decoder side, the original chromaticity residual is weighted and adjusted; (3) Normalize the residuals of the luminance channel data and the chrominance channel data respectively, and then stitch the channels together to construct a dual-channel fused chrominance residual feature map; The residuals of the extracted edge high-frequency components and chroma channel data are concatenated along the channel dimension to construct a multi-channel input feature tensor. The multi-channel input feature tensor is subjected to contrast stretching. The processed multi-channel feature tensor is synchronously output to the mask generation module.

[0010] As a preferred embodiment of the real-time video clarity restoration system based on depth artifact recognition described in this invention, wherein: The multi-channel feature tensor is received and input into a preset lightweight shallow convolutional neural network; Cross-channel feature fusion and nonlinear mapping are performed through the forward propagation of the lightweight shallow convolutional neural network to generate a single-channel artifact probability map that is completely consistent with the spatial resolution of the input video frame. Each pixel value represents the probability of compression artifacts appearing at that position (0 represents low probability, 1 represents high probability). Post-process the output artifact probability map; The generated pixel-by-pixel artifact probability mask is synchronously output to the dynamic allocation module and the image reconstruction module.

[0011] As a preferred embodiment of the real-time video clarity restoration system based on depth artifact recognition described in this invention, wherein: The pixel-wise artifact probability mask output by the mask generation module is received. Its size is consistent with the spatial resolution of the video frame, and its value range is [0,1]. Each pixel value represents the probability of compression artifacts occurring at that position. The preset probability threshold is 0.7. Binarization segmentation is performed on the artifact probability mask, specifically including: (1) Define the region in the mask whose pixel value is greater than or equal to the threshold as a high-probability artifact region; (2) Define the region in the mask where the pixel value is less than the threshold as a low-probability smoothing region; (3) Generate high-probability region masks and low-probability region masks; Region segmentation and feature preparation for high-probability artifact regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the high probability region mask to extract the feature tensor of the high probability artifact region; (2) The extracted high-probability region features are input into the repair operator based on the deep residual structure. The repair operator performs texture reconstruction operation through the residual attention unit constructed by the multi-layer residual block to restore the high-frequency details and texture information lost in compression. Processing low-probability smoothing regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the low probability region mask to extract the feature tensor of the low probability smooth region; (2) Perform pixel preservation processing on this area, and perform light smoothing or edge softening; The features of high-probability artifact regions reconstructed by the repair operator are spatially stitched together with the features of low-probability smooth regions after preservation processing to generate a complete repaired video frame feature tensor. The processed complete video frame feature tensor and the original artifact probability mask are synchronously output to the image reconstruction module.

[0012] As a preferred embodiment of the real-time video clarity restoration system based on depth artifact recognition described in this invention, wherein: Channel separation is performed on the feature tensor of the repaired video frame to obtain the repaired luminance channel data and the repaired chrominance channel data respectively; Linear weighted fusion is performed using artifact probability masks as fusion weights; Post-processing is performed on the fused Y, U, and V channel data, including: (1) Inverse normalization process, which maps pixel values ​​from the [0,1] range back to the original pixel value range; (2) Mild global smoothing and tone correction; The processed fusion channel data is reassembled into complete video frames, and the repaired high-definition video frames are output as the final output of the system.

[0013] On the other hand, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements the steps of the real-time video clarity restoration system based on depth artifact recognition as described above.

[0014] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements the steps of the real-time video clarity restoration system based on depth artifact recognition as described above.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: (1) The probability of artifacts per pixel is predicted by the mask generation module, and an asymmetric repair strategy is implemented by the dynamic allocation module.

[0016] For high-probability artifact regions, a high-complexity depth residual operator is invoked, while for low-probability smooth regions, simple pixel-preserving processing is performed.

[0017] By avoiding indiscriminate depth convolution operations on the entire image, while ensuring that key damaged areas are deeply repaired, the overall computational pressure is significantly reduced, ensuring real-time processing of ultra-high-definition video streams.

[0018] (2) Unlike traditional image restoration algorithms, the feature extraction module introduces the transform coefficient residual information from the decoder side.

[0019] By quantitatively assessing the degree of compression damage using transform domain information, artifact recognition no longer relies solely on spatial domain features. This effectively distinguishes between the complex textures of the image itself and the block effects produced by compression, preventing the accidental erasure of original details and improving the physical accuracy of restoration.

[0020] (3) Construct a dual-channel fused chroma residual feature map and perform cross-frame comparison with the reference frame.

[0021] We have made specific optimizations to address common issues in video compression such as chroma leakage and blurred color edges, restoring clear color boundaries and resulting in videos with more natural color transitions and higher saturation fidelity that are perceived by the human eye.

[0022] (4) Solve the problem of visual discontinuity caused by local repair. By introducing a spatial fusion weighting mechanism in the image reconstruction module, the original pixels and the repaired features are linearly weighted and fused using a continuous value domain artifact probability mask.

[0023] It effectively eliminates seams or visual breaks that may occur at the boundary between high- and low-probability areas, ensuring the smoothness and visual continuity of the repaired video frames in the spatial dimension. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0025] Figure 1 This is a flowchart of the real-time video clarity restoration system based on depth artifact recognition according to the present invention.

[0026] Figure 2 This is a schematic diagram of the modules of the real-time video clarity restoration system based on depth artifact recognition of the present invention.

[0027] Table 1 is a summary table comparing the objective indicators of different restoration schemes of the real-time video clarity restoration system based on depth artifact recognition of the present invention when the quantization parameter is 37.

[0028] Table 2 is a summary table comparing the processing speeds of different restoration schemes in the real-time video clarity restoration system based on depth artifact recognition of the present invention. Detailed Implementation

[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0030] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a real-time video clarity restoration system based on depth artifact recognition, specifically including: a decoding extraction module, a feature extraction module, a mask generation module, a dynamic allocation module, and an image reconstruction module; Decoding and extraction module: Decodes the original video stream and extracts the luminance and chrominance channel data of its video frames; Receive the raw video stream; The original video stream is entropy decoded, dequantized, and inverse transformed by a video decoder to obtain the pixel data of the reconstructed video frames. The pixel data is converted from the RGB color space to the YUV color space, where the Y channel is the luminance component and the U and V channels are the chrominance components. The luminance channel data and chrominance channel data of the video frame are extracted and separated respectively, and stored as independent three-dimensional tensors; The extracted luminance channel data and chrominance channel data are normalized to uniformly map the pixel value range to the [0,1] interval; The processed luminance channel data and chrominance channel data are simultaneously transmitted to the feature extraction module.

[0031] Feature extraction module: Extracts high-frequency edge features based on the luminance channel data, and calculates the dual-channel fused chroma residual feature map by comparing the transform coefficient information obtained from the decoding side with the cross-frame chroma, and constructs a multi-channel input feature tensor; Extracting the high-frequency edge components of the video frame using a high-pass filter operator specifically includes: The gradient magnitudes of the brightness channels in the horizontal and vertical directions are calculated using the Sobel operator, and then processed by a nonlinear activation function to generate single-channel edge feature maps. The calculation of the dual-channel fused chroma residual feature map specifically includes: (1) Perform pixel-by-pixel subtraction between the chroma channel data of the current video frame and the smoothed chroma version of the reference frame after low-pass filtering to obtain the original chroma residual; (2) Based on the transform coefficient residual information obtained from the decoder side, the original chromaticity residual is weighted and adjusted; (3) Normalize the residuals of the luminance channel data and the chrominance channel data respectively, and then stitch the channels together to construct a dual-channel fused chrominance residual feature map; The residuals of the extracted edge high-frequency components and chroma channel data are concatenated along the channel dimension to construct a multi-channel input feature tensor. The multi-channel input feature tensor is subjected to contrast stretching. The processed multi-channel feature tensor is synchronously output to the mask generation module.

[0032] Mask generation module: Input the high-frequency components of the edges and the chromaticity residual data into a preset lightweight shallow convolutional neural network, and output a pixel-by-pixel artifact probability mask through cross-channel feature fusion and nonlinear mapping; The multi-channel feature tensor is received and input into a preset lightweight shallow convolutional neural network; Cross-channel feature fusion and nonlinear mapping are performed through the forward propagation of the lightweight shallow convolutional neural network to generate a single-channel artifact probability map that is completely consistent with the spatial resolution of the input video frame. Each pixel value represents the probability of compression artifacts appearing at that position (0 represents low probability, 1 represents high probability). Post-process the output artifact probability map; The generated pixel-by-pixel artifact probability mask is synchronously output to the dynamic allocation module and the image reconstruction module.

[0033] Dynamic allocation module: Based on the weight distribution of the artifact probability mask, the video frame is divided into a high-probability artifact region and a low-probability smooth region. The high-probability artifact region is reconstructed by calling a repair operator based on a depth residual structure, and the low-probability smooth region is processed to preserve pixels. The pixel-wise artifact probability mask output by the mask generation module is received. Its size is consistent with the spatial resolution of the video frame, and its value range is [0,1]. Each pixel value represents the probability of compression artifacts occurring at that position. The preset probability threshold is 0.7. Binarization segmentation is performed on the artifact probability mask, specifically including: (1) Define the region in the mask whose pixel value is greater than or equal to the threshold as a high-probability artifact region; (2) Define the region in the mask where the pixel value is less than the threshold as a low-probability smoothing region; (3) Generate high-probability region masks and low-probability region masks; Region segmentation and feature preparation for high-probability artifact regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the high probability region mask to extract the feature tensor of the high probability artifact region; (2) The extracted high-probability region features are input into the repair operator based on the deep residual structure. The repair operator performs texture reconstruction operation through the residual attention unit constructed by the multi-layer residual block to restore the high-frequency details and texture information lost in compression. Processing low-probability smoothing regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the low probability region mask to extract the feature tensor of the low probability smooth region; (2) Perform pixel preservation processing on this area, and perform light smoothing or edge softening; The features of high-probability artifact regions reconstructed by the repair operator are spatially stitched together with the features of low-probability smooth regions after preservation processing to generate a complete repaired video frame feature tensor. The processed complete video frame feature tensor and the original artifact probability mask are synchronously output to the image reconstruction module.

[0034] Image reconstruction module: Using the artifact probability mask as spatial fusion weight, the repaired regional features are combined linearly with the original video frame data in a weighted manner, and after inverse normalization and tone correction, high-definition video frames are output. Channel separation is performed on the feature tensor of the repaired video frame to obtain the repaired luminance channel data and the repaired chrominance channel data respectively; Linear weighted fusion is performed using artifact probability masks as fusion weights; Post-processing is performed on the fused Y, U, and V channel data, including: (1) Inverse normalization process, which maps pixel values ​​from the [0,1] range back to the original pixel value range; (2) Mild global smoothing and tone correction; The processed fusion channel data is reassembled into complete video frames, and the repaired high-definition video frames are output as the final output of the system. Example

[0035] The following is another embodiment of the present invention, which provides a real-time video clarity restoration system based on depth artifact recognition. In order to verify the beneficial effects of the present invention, a simulation experiment is conducted for scientific demonstration.

[0036] This embodiment verifies the technical advantages of the "Real-time Video Clarity Restoration System Based on Depth Artifact Recognition" described in this invention through simulation experiments in improving video clarity, reducing compression artifacts, and maintaining real-time processing performance.

[0037] Standard video test sequences were selected (including BQTerrace, Cactus, and BasketballDrive), with resolutions covering 1080P and 4K.

[0038] Compression environment: The original video was compressed using an H.265 / HEVC encoder, with quantization parameters set to 32, 37, and 42 to simulate different degrees of compression impairment.

[0039] The comparison schemes include: Control group A: Traditional HEVC deblocking filter.

[0040] Control group B: The entire image is uniformly repaired using a fully convolutional neural network, and does not include the artifact probability mask and dynamic allocation module of this invention.

[0041] Experimental group: The real-time video clarity restoration system described in this invention.

[0042] The video stream to be processed is input into the system of the present invention, and the decoding and extraction module outputs the normalized YUV three-dimensional tensor. Based on the feature extraction module, luminance edges are extracted using the Sobel operator, and a fused chroma residual feature map is constructed by combining the decoder-side transform coefficient residuals. The mask generation module outputs an artifact probability map, which automatically identifies high-probability damage areas and activates the repair operator based on the residual attention unit only for these areas. The image reconstruction module uses a linear combination based on mask weights to generate repaired high-definition video frames.

[0043] The analysis of experimental results includes: The objective evaluation of image quality is shown in Table 1, comparing the restoration effects of different schemes on the test sequence under the condition of quantization parameter 37:

[0044] As shown in Table 1, the peak signal-to-noise ratio and structural similarity index of the experimental group of this invention were significantly higher than those of the control group A and the control group B on all test sequences.

[0045] Compared to control group B, which lacks artifact recognition capabilities, the present invention improves the average peak signal-to-noise ratio by 0.59 dB and the structural similarity by 0.022.

[0046] Experimental data demonstrate that by introducing the residual information of the transform coefficients and the chroma residual features on the decoding side, this system can more accurately compensate for quantization loss. In particular, the restoration gain is most significant in 4K high-resolution sequences, proving the superior performance of this invention in the restoration of complex textures on a large scale.

[0047] The improved universality of structural similarity means that while eliminating artifacts, the present invention effectively protects the structural features of the original video through dynamic allocation modules, avoiding the edge degradation problem common in the prior art.

[0048] The processing timeliness is shown in Table 2. The evaluation assesses the processing speed for 4K resolution video under the same hardware environment.

[0049] As shown in Table 2, at 4K ultra-high definition resolution, the processing speed of control group B was only 11.2 FPS, far below the standard for industrial-grade real-time processing. In contrast, the processing speed of the experimental group in this invention reached 48.6 FPS, fully meeting the requirements for real-time online restoration of 4K video.

[0050] As the resolution increases, the performance improvement of this invention compared to control group B increases from 3.69 times to 6.08 times. This indicates that the higher the resolution, the larger the proportion of smooth regions in the image. This invention uses a dynamically allocated module to shield the depth calculation of low-probability artifact regions, resulting in increasingly significant energy savings and speed-up effects.

[0051] When processing video streams of the same length, the GPU memory usage of the system of this invention was reduced by approximately 45% compared to the control group B. This demonstrates that the lightweight convolutional network described in this invention, combined with the dynamic repair operator, greatly optimizes the allocation of computing resources and has the potential for deployment on mobile or edge computing devices.

[0052] The probability of artifacts per pixel is predicted by a mask generation module, and an asymmetric repair strategy is implemented using a dynamic allocation module.

[0053] For high-probability artifact regions, a high-complexity depth residual operator is invoked, while for low-probability smooth regions, simple pixel-preserving processing is performed.

[0054] By avoiding indiscriminate depth convolution operations on the entire image, while ensuring that key damaged areas are deeply repaired, the overall computational pressure is significantly reduced, ensuring real-time processing of ultra-high-definition video streams.

[0055] Unlike traditional image restoration algorithms, the feature extraction module incorporates the transform coefficient residual information from the decoder side.

[0056] By quantitatively assessing the degree of compression damage using transform domain information, artifact recognition no longer relies solely on spatial domain features. This effectively distinguishes between the complex textures of the image itself and the block effects produced by compression, preventing the accidental erasure of original details and improving the physical accuracy of restoration.

[0057] Construct a dual-channel fused chroma residual feature map and perform cross-frame comparison with a reference frame.

[0058] We have made specific optimizations to address common issues in video compression such as chroma leakage and blurred color edges, restoring clear color boundaries and resulting in videos with more natural color transitions and higher saturation fidelity that are perceived by the human eye.

[0059] To address the visual discontinuity issues caused by partial restorations. By introducing a spatial fusion weighting mechanism in the image reconstruction module, the original pixels and the repaired features are linearly weighted and fused using a continuous value domain artifact probability mask.

[0060] It effectively eliminates seams or visual breaks that may occur at the boundary between high- and low-probability areas, ensuring the smoothness and visual continuity of the repaired video frames in the spatial dimension.

[0061] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A real-time video clarity restoration system based on depth artifact recognition, characterized in that, Specifically, it includes: The system includes a decoding and extraction module, a feature extraction module, a mask generation module, a dynamic allocation module, and an image reconstruction module. Decoding and extraction module: Decodes the original video stream and extracts the luminance and chrominance channel data of its video frames; Feature extraction module: Extracts high-frequency edge features based on the luminance channel data, and calculates the dual-channel fused chroma residual feature map by comparing the transform coefficient information obtained from the decoding side with the cross-frame chroma, and constructs a multi-channel input feature tensor; Mask generation module: Input the high-frequency components of the edges and the chromaticity residual data into a preset lightweight shallow convolutional neural network, and output a pixel-by-pixel artifact probability mask through cross-channel feature fusion and nonlinear mapping; Dynamic allocation module: Based on the weight distribution of the artifact probability mask, the video frame is divided into a high-probability artifact region and a low-probability smooth region. The high-probability artifact region is reconstructed by calling a repair operator based on a depth residual structure, and the low-probability smooth region is processed to preserve pixels. Image reconstruction module: Using the artifact probability mask as spatial fusion weight, the repaired regional features are combined linearly with the original video frame data in a weighted manner, and after inverse normalization and tone correction, high-definition video frames are output.

2. The real-time video clarity restoration system based on depth artifact recognition according to claim 1, characterized in that: The decoding and extraction module receives the original video stream; The original video stream is entropy decoded, dequantized, and inverse transformed by a video decoder to obtain the pixel data of the reconstructed video frames. The pixel data is converted from the RGB color space to the YUV color space, where the Y channel is the luminance component and the U and V channels are the chrominance components. The luminance channel data and chrominance channel data of the video frame are extracted and separated respectively, and stored as independent three-dimensional tensors; The extracted luminance channel data and chrominance channel data are normalized to uniformly map the pixel value range to the [0,1] interval; The processed luminance channel data and chrominance channel data are simultaneously transmitted to the feature extraction module.

3. The real-time video clarity restoration system based on depth artifact recognition according to claim 1, characterized in that: In the feature extraction module, the high-frequency edge components of the video frame are extracted using a high-pass filter operator, specifically including: The gradient magnitudes of the brightness channels in the horizontal and vertical directions are calculated using the Sobel operator, and then processed by a nonlinear activation function to generate single-channel edge feature maps. The calculation of the dual-channel fused chroma residual feature map specifically includes: (1) Perform pixel-by-pixel subtraction between the chroma channel data of the current video frame and the smoothed chroma version of the reference frame after low-pass filtering to obtain the original chroma residual; (2) Based on the transform coefficient residual information obtained from the decoder side, the original chromaticity residual is weighted and adjusted; (3) Normalize the residuals of the luminance channel data and the chrominance channel data respectively, and then stitch the channels together to construct a dual-channel fused chrominance residual feature map; The residuals of the extracted edge high-frequency components and chroma channel data are concatenated along the channel dimension to construct a multi-channel input feature tensor. The multi-channel input feature tensor is subjected to contrast stretching. The processed multi-channel feature tensor is synchronously output to the mask generation module.

4. The real-time video clarity restoration system based on depth artifact recognition according to claim 1, characterized in that: In the mask generation module, the multi-channel feature tensor is received and input into a preset lightweight shallow convolutional neural network; Cross-channel feature fusion and nonlinear mapping are performed through the forward propagation of the lightweight shallow convolutional neural network to generate a single-channel artifact probability map that is completely consistent with the spatial resolution of the input video frame. Each pixel value represents the probability of compression artifacts appearing at that position (0 represents low probability, 1 represents high probability). Post-process the output artifact probability map; The generated pixel-by-pixel artifact probability mask is synchronously output to the dynamic allocation module and the image reconstruction module.

5. The real-time video clarity restoration system based on depth artifact recognition according to claim 1, characterized in that: In the dynamic allocation module, a pixel-by-pixel artifact probability mask is received from the mask generation module. Its size is consistent with the video frame spatial resolution, and its value range is [0,1]. Each pixel value represents the probability of compression artifacts appearing at that position. The preset probability threshold is 0.

7. Binarization segmentation is performed on the artifact probability mask, specifically including: (1) Define the region in the mask whose pixel value is greater than or equal to the threshold as a high-probability artifact region; (2) Define the region in the mask where the pixel value is less than the threshold as a low-probability smoothing region; (3) Generate high-probability region masks and low-probability region masks; Region segmentation and feature preparation for high-probability artifact regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the high probability region mask to extract the feature tensor of the high probability artifact region; (2) The extracted high-probability region features are input into the repair operator based on the deep residual structure. The repair operator performs texture reconstruction operation through the residual attention unit constructed by the multi-layer residual block to restore the high-frequency details and texture information lost in compression. Processing low-probability smoothing regions: (1) Multiply the luminance channel data and chrominance channel data of the original video frame element by element with the low probability region mask to extract the feature tensor of the low probability smooth region; (2) Perform pixel preservation processing on this area, and perform light smoothing or edge softening; The features of high-probability artifact regions reconstructed by the repair operator are spatially stitched together with the features of low-probability smooth regions after preservation processing to generate a complete repaired video frame feature tensor. The processed complete video frame feature tensor and the original artifact probability mask are synchronously output to the image reconstruction module.

6. The real-time video clarity restoration system based on depth artifact recognition according to claim 1, characterized in that: In the image reconstruction module, channel separation is performed on the feature tensor of the repaired video frame to obtain the repaired luminance channel data and the repaired chrominance channel data respectively. Linear weighted fusion is performed using artifact probability masks as fusion weights; Post-processing is performed on the fused Y, U, and V channel data, including: (1) Inverse normalization process, which maps pixel values ​​from the [0,1] range back to the original pixel value range; (2) Mild global smoothing and tone correction; The processed fusion channel data is reassembled into complete video frames, and the repaired high-definition video frames are output as the final output of the system.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the module of the real-time video clarity restoration system based on depth artifact recognition as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the module of the real-time video clarity restoration system based on depth artifact recognition as described in any one of claims 1 to 6.

Images