A video repair strategy automatic generation method and system based on multi-dimensional degradation identification

Abstract

The application discloses a video repair strategy automatic generation method and system based on multi-dimensional degradation identification, and relates to the technical field of video repair and image quality evaluation. The method comprises the following steps: dividing an input video into segments, extracting key frames and continuous sampling frames, performing multi-dimensional degradation detection on the segments in the space, time, exposure color, structure damage and key area, and generating a degradation image containing various degradation intensities, confidence, damage masks and protection constraints; selecting a candidate repair module according to the degradation image, generating a directed repair graph according to the processing dependency constraint, and generating node parameters; selecting a preview segment to perform trial processing, outputting a repair plan after quality gating verification and adjustment, and executing the repair plan, and realizing strategy closed-loop optimization through index writing back. The application can automatically adapt to videos of different degradation types, improve repair accuracy and efficiency, and guarantee the integrity of key area content.

Description

Technical Field

[0001] This invention relates to the field of video restoration and image quality evaluation technology, specifically to a method and system for automatically generating video restoration strategies based on multi-dimensional degradation recognition. Background Technology

[0002] With the continued growth in demand for digital image restoration and low-quality video enhancement, videos from scenarios such as old film, low-bitrate compression, and damaged transmission often suffer from multiple types and dimensions of degradation problems, including noise, compression blocks, blur, low light, color cast, flicker, jitter, scratches, and blemishes. Most existing video restoration technologies employ a pre-defined, fixed-order call to restoration models, failing to adaptively arrange the restoration process based on the actual type and degree of degradation. This easily leads to errors in the restoration logic order; for example, performing super-resolution before noise reduction amplifies noise features, or performing image enhancement before scratch removal increases the difficulty of defect restoration, ultimately resulting in degraded restoration effects and wasted computational resources. Furthermore, existing technologies typically use a single degradation label to describe the video's damaged state, failing to comprehensively cover spatial degradation, temporal degradation, structural damage, and the protection constraints of semantically important regions. They also struggle to accurately quantify the intensity and detection confidence of various degradation types, thus failing to provide reliable data support for the automated generation of restoration strategies. In addition, conventional repair processes lack pre-processing quality checks and closed-loop feedback mechanisms, making it impossible to predict the repair effect and dynamically adjust strategies before full-screen processing. It is also difficult to record and trace the repair process in a versioned manner, which fails to meet the reliability, reproducibility, and optimizability requirements of engineered batch repair. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for automatically generating video restoration strategies based on multi-dimensional degradation identification. By performing multi-dimensional identification of spatial degradation, temporal degradation, exposure and color degradation, structural damage, and key areas of video segments, a degradation profile with intensity and confidence is formed. Based on the processing dependencies between degradations, risk constraints, and trial processing quality thresholds, an executable restoration map is automatically generated and updated, thereby achieving adaptive orchestration and closed-loop quality control of the video restoration chain.

[0004] To achieve the above objectives, the embodiments of this invention provide the following technical solutions:

[0005] This application provides an automatic generation method for video restoration strategies based on multi-dimensional degradation recognition, comprising: dividing the input video into at least one video segment, and extracting keyframes and continuous sampling frames for each segment; performing spatial degradation detection, temporal degradation detection, exposure and color detection, structural damage detection, and key area detection on the segments respectively, generating a degradation profile containing various degradation intensities and their confidence levels, structural damage masks, and key area protection constraints; determining primary and secondary degradation based on the degradation profile, selecting candidate restoration modules from a degradation-processing module mapping table, and generating a directed restoration graph composed of restoration nodes based on processing dependency constraints; generating parameters for each node in the restoration graph, selecting a preview segment to execute the restoration graph and calculating quality gating indicators, adjusting the node parameters of the restoration graph or replacing modules if the threshold is not met, until the threshold is met and then outputting a restoration plan; performing video restoration according to the restoration plan, and writing back the actual quality indicators to update the strategy.

[0006] Furthermore, the spatial degradation detection includes: a blur score obtained by fusing Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; a noise score that estimates the high-frequency residual variance in flat texture regions and excludes subtitles and edge regions; a block effect score that statistically analyzes brightness abrupt changes on both sides of the coding block boundary; and a ringing score that detects alternating overshoot and undershoot patterns on both sides of strong edges. The temporal degradation detection includes: estimating the transformation of adjacent frames through feature point matching, optical flow, or a global motion model, and using abnormal global translation, rotation, or affine changes as jitter scores after removing foreground subject motion; and a flicker score obtained by statistically analyzing the brightness difference, color difference, or feature difference of the corresponding region after motion compensation. The structural damage detection includes: scratch detection that generates scratch masks using elongated linear morphology, cross-frame positional stability, and directional continuity; blemish detection that generates blemish masks using small-area isolated anomalies and temporal persistence; and missing regions determined by occlusion detection, black block detection, or uncertainty output of a repair model. The mask includes pixel position, category, confidence level, and cross-frame association identifier.

[0007] Furthermore, the key area detection includes the detection of face areas, text areas, and logo / watermark areas; for face areas, feature vectors, key points, and skin color statistics are recorded; for text areas, OCR text, character boxes, and confidence scores are recorded; for logo / watermark areas, boundary and transparency estimates are recorded; the key area protection constraint is used to control the subsequent repair module from performing free-generative repair on the protected content.

[0008] Furthermore, the fusion intensity of various degradations in the degraded image is determined according to the following formula: ;in, Let be the fusion strength of the i-th type of degradation. Let j be the score of the j-th detection method for the i-th type of degradation. For the corresponding confidence level, These are the weighting coefficients.

[0009] Furthermore, the primary degradation is the degradation type where the score exceeds the primary threshold and ranks in the top K, where K is 1 to 4, and the secondary degradation uses a threshold lower than the primary threshold; the processing dependency constraints include: the image stabilization node precedes the temporal fusion node, the strong noise processing node precedes the super-resolution node, and the mask repair node precedes the module that amplifies defects; when the text region accounts for a high proportion, the free generation node is restricted and an OCR verification node is added.

[0010] Furthermore, the node parameters are generated from the degradation intensity via a piecewise function or table lookup, including: the denoising intensity is... ,in The fusion intensity of the noise category in the degraded image. The denoising intensity is limited to 0 to 100, and the number of pixels in the scratch mask dilation varies between 1 and 12 depending on the fusion intensity of the scratch category in the degraded image. The super-resolution ratio is determined by the input and target resolutions, ranging from 1.5 to 4. The upper limit of the generation intensity of the text region is lower than that of the background region. The quality gating indicators include sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker indicators. When the OCR confidence decreases, the facial feature similarity is below the threshold, or flicker increases, the intensity of the corresponding node is reduced, a conservative model is replaced, or a manually reviewed node is inserted. The repair plan includes fragment identifiers, degradation vectors, node lists, topological relationships, model versions, parameters, region mask references, quality thresholds, failure retry strategies, and result storage paths. The execution engine runs according to the repair plan and writes back the actual indicators to update the subsequent strategies for similar fragments.

[0011] Accordingly, this application also provides an automatic generation system for video restoration strategies based on multi-dimensional degradation recognition, comprising: a video sampling module, used to divide the input video into at least one video segment and extract keyframes and continuous sampling frames; a multi-dimensional spatial degradation analysis module, connected to the video sampling module, used to perform spatial degradation detection on the segment, including at least blur detection, noise detection, block effect detection, and ringing detection; a temporal degradation analysis module, connected to the video sampling module, used to perform temporal degradation detection on the segment, including at least jitter detection and flicker detection; an exposure and color analysis module, connected to the video sampling module, used to perform exposure and color detection on the segment, including at least low light detection, overexposure detection, and color cast detection; a structural damage mask module, connected to the video sampling module, used to perform structural damage detection on the segment, including at least scratch detection, spot detection, and missing area detection, and generate a corresponding damage mask; and a key area protection module, connected to the video sampling module, used to detect key areas and generate protection constraints, wherein the key areas... The domain includes at least a face region, a text region, and a logo / watermark region; a degradation image fusion module, connected to the multi-dimensional spatial degradation analysis module, temporal degradation analysis module, exposure color analysis module, structural damage mask module, and key area protection module, is used to fuse the outputs of each module to generate a degradation image containing various degradation intensities and their confidence levels, damage masks, and protection constraints; a restoration map generation module, connected to the degradation image fusion module, has a built-in degradation-processing module mapping table, used to select candidate modules based on the degradation image and generate a directed restoration map based on processing dependency constraints; it is used to update the degradation-processing module mapping table or node parameter generation strategy; a parameter and trial processing module, connected to the restoration map generation module, is used to generate parameters for each node in the restoration map, select a preview segment to execute the restoration map and calculate quality gate indicators, and adjust parameters or replace modules if the threshold is not met; a restoration execution record module, connected to the parameter and trial processing module, is used to output and execute a restoration plan after the threshold is met, and write the actual quality indicators back to the restoration map generation module.

[0012] Furthermore, in the multidimensional spatial degradation analysis module: the fuzzy score is obtained by fusing Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; the noise score estimates the high-frequency residual variance in flat texture regions and excludes subtitles and edge regions; the block effect score statistically analyzes the brightness abrupt changes on both sides of the coding block boundary; the ringing score detects alternating overshoot and undershoot patterns on both sides of strong edges; the temporal degradation analysis module estimates the transformation of adjacent frames through feature point matching or optical flow, removes foreground motion to obtain a jitter score, and statistically analyzes the brightness difference or color difference of the corresponding region after motion compensation to obtain a flicker score; the structural damage mask module generates scratch masks using linear morphology and cross-frame continuity, generates blemish masks using small-area isolated anomalies and temporal persistence, and generates damage masks containing pixel position, category, confidence level, and cross-frame association identifiers.

[0013] Furthermore, the parameter and test processing module includes: a parameter generation unit, used to generate denoising intensity according to a piecewise function. ,in The fusion intensity of the noise category in the degraded image. The denoising intensity is limited to 0 to 100, which is a rounding function; the number of scratch mask dilation pixels varies between 1 and 12 depending on the fusion intensity of the scratch category in the degraded image; the super-resolution factor is determined by the input and target resolution, ranging from 1.5 to 4, and the upper limit of the generation intensity of the text region is lower than the upper limit of the background region; the quality gating unit is used to calculate the sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker index on the preview segment, and outputs parameter downgrading or model replacement instructions to the restoration image generation module when any protection index fails to meet the threshold; the manual review unit is connected to the quality gating unit and is used to display the synchronous comparison, local magnification, and mask overlay of the original and restoration results in low confidence / high risk scenarios, receive annotations from reviewers, and generate new policy constraints.

[0014] Furthermore, the data structure of the degraded profile includes: profile identifier, video identifier, segment identifier, start timestamp, end timestamp, sampling plan, spatial score, temporal score, color score, damage mask, salient region, confidence score, detector version, reference style, generation time, and profile version; the data structure of the repair plan includes: segment identifier, degradation vector, node list, topological relationship, model version, parameters, region mask reference, quality threshold, failure retry strategy, and result storage path; the video frames, features, masks, strategies, task descriptions, or logs transmitted between modules are all associated with video stream identifiers, timestamps, and configuration versions to ensure video processing order, traceability, and recovery capability after failure.

[0015] The beneficial effects of this invention are as follows: by forming a confidence-based degradation profile through multi-dimensional degradation identification and weighted fusion, the repair map is automatically arranged according to processing dependency constraints, and closed-loop adjustment is achieved through trial processing quality gating. This realizes the technical effect of automatically generating video repair strategies from manual experience-driven to data-driven, avoiding the problems of defect amplification and semantic destruction caused by fixed repair order. Attached Figure Description

[0016] Figure 1 A flowchart illustrating an automatic video restoration strategy generation method based on multi-dimensional degradation recognition, provided in an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of the structure of an automatic video restoration strategy generation system based on multi-dimensional degradation recognition, provided in an embodiment of this application. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0019] In this invention, the terms "system" and "network" are used interchangeably. "Multiple" refers to two or more; therefore, in this invention, "multiple" can also be understood as "at least two." "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that in the description of this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0020] Example 1:

[0021] like Figure 1As shown in the figure, this application provides an automatic generation method for video restoration strategies based on multi-dimensional degradation recognition, including: dividing the input video into at least one video segment, and extracting keyframes and continuous sampling frames for each segment; performing spatial degradation detection, temporal degradation detection, exposure and color detection, structural damage detection, and key area detection on the segments respectively, generating a degradation profile containing various degradation intensities and their confidence levels, structural damage masks, and key area protection constraints; determining primary and secondary degradation based on the degradation profile, selecting candidate restoration modules from a degradation-processing module mapping table, and generating a directed restoration graph composed of restoration nodes based on processing dependency constraints; generating parameters for each node in the restoration graph, selecting a preview segment to execute the restoration graph and calculating quality gating indicators, adjusting the node parameters of the restoration graph or replacing modules if the threshold is not met, until the threshold is met and then outputting a restoration plan; performing video restoration according to the restoration plan, and writing back the actual quality indicators to update the strategy.

[0022] In another possible embodiment, the input video is first divided into at least one video segment, and keyframes and consecutive sampled frames are extracted from each segment. Specifically, the encoding attributes of the input video (including resolution, frame rate, encoding format, bitrate, color space, whether it is interlaced, etc.) and timestamp information are parsed, and the video is divided into several segments according to shot boundary detection or a fixed time window (selectable within the range of 0.5 seconds to 10 seconds). For each segment, the first frame, the last frame, and uniformly distributed keyframes are extracted for spatial degradation and color degradation detection; at the same time, consecutive sampled frames are extracted at fixed intervals to identify temporal degradation such as flickering, jitter, and motion discontinuity. For interlaced video, the preprocessing module first performs deinterlacing; for variable frame rate video, timestamp normalization is first performed; for video with subtitles, the subtitle area is first marked. Then, spatial degradation detection, temporal degradation detection, exposure and color detection, structural damage detection, and key area detection are performed on the segments respectively to generate a degradation profile containing various degradation intensities and their confidence levels, structural damage masks, and key area protection constraints. Spatial degradation detection includes blur, noise, compression block effect, and ringing detection; temporal degradation detection includes jitter and flicker detection; exposure and color detection includes low light, overexposure, and color cast detection; structural damage detection includes scratches, blemishes, and occlusion defects detection; and key area detection includes face, text, and logo / watermark areas. Each detector outputs an intensity score between 0 and 1 and a corresponding confidence level. The structural damage detector outputs a pixel-level mask, and the key area detector outputs protection constraints. Based on the degradation profile, primary and secondary degradations are determined. Candidate repair modules are selected from the degradation-processing module mapping table, and a directed repair graph consisting of repair nodes is generated based on processing dependency constraints. Primary degradation is defined as degradation types with scores exceeding the primary threshold and ranking in the top K (K ranges from 1 to 4). Secondary degradations are selected using thresholds lower than the primary threshold. The degradation-processing module mapping table pre-stores the correspondence between degradation categories and repair modules. For example, jitter corresponds to the image stabilization module, flicker corresponds to the brightness timing correction module, compression blocks correspond to deblocking and deringing modules, scratches correspond to the mask repair module, low light corresponds to the conservative exposure recovery module, and low resolution corresponds to the super-resolution module. After the strategy generator retrieves candidate modules from the mapping table, it constructs a directed acyclic graph (DAG) based on pre-, post-, mutually exclusive, and parallelizable constraints. For example, the image stabilization node must be located before the timing fusion node, the strong noise processing node must be located before the super-resolution node, and the mask repair node must be located before the module that amplifies defects. Finally, parameters are generated for each node in the repair graph. A preview segment is selected to execute the repair graph and calculate the quality gating index. If the threshold is not met, the node parameters of the repair graph are adjusted or the module is replaced until the threshold is met, then the repair plan is output and executed. The node parameters are obtained from the degradation intensity through a piecewise function or a lookup table.Representative preview clips ranging from 0.5 to 5 seconds are selected (covering each type of degradation combination, the beginning and end of the shot, and key areas) to perform candidate restoration. Quality gating metrics such as sharpness improvement, noise retention, OCR confidence change, facial feature similarity, color difference, and temporal flicker are calculated. If the preview metrics do not meet the rules (e.g., decreased OCR confidence, facial feature similarity below the threshold, or increased flicker), the strength of the corresponding node is reduced, a conservative model is replaced, or a manually reviewed node is inserted, and the process is retried until it passes. After passing the threshold, a restoration plan (RepairPlan) is output. The execution engine runs the full-screen restoration according to the plan and writes back the actual metrics, enabling subsequent strategies for similar clips to be updated based on historical data.

[0023] By dividing the video into segments and performing degradation detection from five dimensions—space, time, exposure and color, structural damage, and key areas—a degradation profile with intensity and confidence is formed. Then, a directed restoration map is automatically generated based on processing dependency constraints and adjusted through a closed-loop trial processing quality gate. This achieves a shift in video restoration strategy from a fixed order to adaptive arrangement, avoiding the amplification of defects and destruction of semantic regions caused by manually setting the restoration order. At the same time, reproducible and traceable engineering deployment is achieved through preview verification and index write-back.

[0024] In the embodiments of this application, the spatial degradation detection includes: a blur score obtained by fusing Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; a noise score that estimates the high-frequency residual variance in flat texture regions and excludes subtitles and edge regions; a block effect score that statistically analyzes the brightness abrupt changes on both sides of the coding block boundary; a ringing score that detects alternating overshoot and undershoot patterns on both sides of strong edges; the temporal degradation detection includes: estimating the transformation of adjacent frames through feature point matching, optical flow, or a global motion model, and using abnormal global translation, rotation, or affine changes as jitter scores after removing the motion of the foreground subject; a flicker score obtained by statistically analyzing the brightness difference, color difference, or feature difference of the corresponding region after motion compensation; the structural damage detection includes: scratch detection that generates scratch masks using elongated linear morphology, cross-frame positional stability, and directional continuity; blemish detection that generates blemish masks using small-area isolated anomalies and temporal persistence; missing regions that are determined by occlusion detection, black block detection, or uncertainty output of a repair model; the mask includes pixel position, category, confidence level, and cross-frame association identifier.

[0025] In another possible embodiment, spatial degradation detection is first performed. Spatial degradation detection includes four sub-items: fuzziness scoring, noise scoring, block effect scoring, and ringing scoring.

[0026] The calculation process for the blur score is as follows: The image is divided into non-overlapping image blocks. After excluding textureless sky areas and solid-color background areas, the Laplacian variance (reflecting the dispersion of the second derivative of the image; the smaller the variance, the more blurred the image), gradient magnitude (Sobel or Scharr operator response), and high-frequency energy (the proportion of high-frequency components after Fourier transform) of each block are calculated. These three traditional indicators are then weighted and fused with the output score of a learned fuzzy classifier (a fuzzy / sharp binary or multi-class classification model trained on a convolutional neural network) to obtain the final blur score. Locally blurred areas are saved as block-level heatmaps to prevent the blur score from obscuring the sharp subject. The calculation process for the noise score is as follows: Flat texture areas (i.e., areas where the local gradient is less than a threshold) are identified in the image. In these areas, the high-frequency residual variance (i.e., the variance of the difference between the image and the low-pass filter result) is estimated as the noise intensity estimate. Simultaneously, subtitle areas and strong edge areas are excluded to avoid misclassifying text strokes or edges as noise. Further improvements in accuracy can be made by combining wavelet high-frequency subband coefficients, chroma channel variance, or self-supervised noise estimation models. To avoid misclassifying artistic elements such as film grain, rain, snow, and smoke as noise, a judgment is made based on temporal stability and scene category; grain can be selectively retained for videos with a clear style. The block effect score is calculated by statistically analyzing the brightness abrupt change amplitude on both sides of the boundary of 8×8, 16×16, or predicted coding blocks. Specifically, brightness values ​​of several pixels on both sides of the block boundary are taken, the brightness jump variable at the boundary is calculated, and the average and maximum jump of all block boundaries in the entire frame are statistically analyzed as the block effect score. If the input bitstream is accessible, quantization parameters (QP), GOP structure, and packet loss markers can be used to enhance the judgment. The ringing score is calculated by detecting alternating overshoot and undershoot patterns on both sides of strong edges (edges where the gradient amplitude exceeds a threshold). Specifically, brightness profiles on both sides of the edge are extracted along the edge normal direction, and oscillation patterns appearing near the edge, first with overshoot peaks higher than the local mean and then with undershoot valleys lower than the local mean, are detected. The larger the oscillation amplitude and range, the higher the ringing score. Next, temporal degradation detection is performed. Temporal degradation detection includes two sub-items: jitter score and flicker score. The jitter score is calculated as follows: the transformation relationship between adjacent frames (including translation, rotation, and affine transformation parameters) is estimated through feature point matching (such as ORB, SIFT features), optical flow methods (such as Farneback dense optical flow), or global motion models. The RANSAC algorithm is used to remove outliers belonging to the moving foreground subject, retaining inliers belonging to the background to estimate global motion. The estimated global motion parameters are high-pass filtered to extract high-frequency, narrative-insensitive (including unintentional panning, zooming, etc.) abnormal global displacements, rotations, or affine changes as jitter candidates; their amplitude and frequency are combined to form the jitter score. The image stabilization strategy should also set an upper limit on the cropping ratio to prevent excessive cropping of effective content for image stabilization.The flicker score calculation process is as follows: First, motion compensation is performed on adjacent frames (adjacent frames are aligned using the aforementioned optical flow or global motion estimation). Then, the brightness difference, color difference, or feature difference is calculated in the corresponding region after motion compensation. Median and quantile statistics are used to exclude outliers caused by local flashes, screen transitions, or stage lighting effects. The final statistical measure is used as the flicker score. For film flicker, it can be estimated separately according to low-frequency brightness trends (such as slow exposure changes) and high-frequency jitter (such as rapid inter-frame fluctuations), and then processed separately by different correction modules. Then, structural damage detection is performed. Structural damage detection includes three sub-items: scratch detection, spot detection, and missing area detection. The scratch detection process is as follows: the long, thin linear morphological features of scratches (including aspect ratios exceeding a threshold), cross-frame positional stability (including minimal positional change of the same scratch in adjacent frames), and directional continuity (such as scratches usually being distributed approximately perpendicularly along the film movement direction) are used for detection. Specifically, morphological processing (such as top-hat transformation and line segment detection operators) is first performed on a single frame to extract linear candidates. Then, the scratch mask is propagated between adjacent frames via optical flow and fused with the detection results of the current frame. In case of conflict, the region with higher confidence is retained, and finally, a scratch mask containing pixel location, category, confidence, and cross-frame association identifier is generated. The process of taint detection is as follows: detection is performed using the small-area isolated anomalous features of taints (such as a local area that is significantly different from surrounding pixels but has an area smaller than a threshold) and temporal persistence (such as the same position appearing continuously in multiple frames). For small-area anomalous regions detected in a single frame, their persistence is verified temporally, random noise points are excluded, and a taint mask is generated. The process of missing region detection is as follows: missing regions are determined by occlusion detection (such as identifying regions that are largely covered or have missing content), black block detection (such as identifying rectangular regions that are completely black or nearly completely black), or the uncertainty output of the repair model (such as determining it as missing when the prediction variance of the repair model for a certain region exceeds a threshold). Missing regions are also mapped using masks that include pixel location, category, confidence level, and cross-frame association identifiers. Finally, the detection results from all these types of data are aggregated to form intermediate data containing spatial scores, temporal scores, and structural damage masks, which is then input into the degradation profile fusion module.

[0027] By employing multi-source fusion of Laplacian variance / gradient / frequency domain energy and learned classifiers for fuzz detection, noise detection based on residual variance in flat regions, block effect detection based on block boundary brightness statistics, ringing detection based on edge overshoot and undershoot patterns, jitter detection based on motion estimation and RANSAC foreground culling, flicker detection based on brightness difference statistics after motion compensation, and scratch / stain / missing item detection based on morphology and temporal verification, comprehensive and accurate quantification of spatial and temporal degradation and structural damage has been achieved. This provides multi-source evidence support for degradation profiling and improves the accuracy and robustness of degradation identification.

[0028] In the embodiments of this application, the key area detection includes the detection of face areas, text areas, and logo / watermark areas; for face areas, feature vectors, key points, and skin color statistics are recorded; for text areas, OCR text, character boxes, and confidence scores are recorded; for logo / watermark areas, boundary and transparency estimates are recorded; the key area protection constraint is used to control the subsequent repair module from performing free-generative repair on the protected content.

[0029] In another possible implementation, firstly, face region detection is performed. Face detectors (such as deep learning-based MTCNN, RetinaFace, etc.) are used to detect face locations on keyframes and consecutively sampled frames. For each detected face, the following information is extracted: face feature vector (a high-dimensional embedding vector extracted by a face recognition model such as ArcFace, used for identity consistency comparison before and after restoration), face landmarks (including coordinates of dozens of landmarks such as eyes, eyebrows, nose, corners of the mouth, and facial contours, used to monitor facial feature position shifts after restoration), and skin color statistics (such as calculating the mean and variance of each channel in the RGB or Lab color space in the face region, used to monitor whether abnormal changes in skin color occur after restoration). Face region information is saved with the degraded image and used as a strong protective constraint in subsequent restorations. Then, text region detection is performed. Text detectors (such as EAST, DBNet, etc.) and OCR engines (such as PaddleOCR, Tesseract, etc.) are used to detect text regions on keyframes, recording the text content recognized by the OCR, the bounding box coordinates of each character, and the confidence score. Text regions are prioritized for protection over general image regions because errors in text generation (such as stroke distortion and character replacement) directly affect information delivery. For fixed-position subtitles in the video, their regions are marked during preprocessing, and conservative parameters are used in noise reduction and super-resolution processes. Logo / watermark region detection is then performed. Logo and watermark positions in the video are detected using a logo detection model or template matching method, and their bounding box coordinates and transparency estimates (Alpha channel estimation) are recorded. Logo / watermark regions typically have fixed positions and appearances; they should not be treated as defects to be repaired and altered or covered up during the restoration process. Finally, the information on face, text, and logo / watermark regions is summarized into key region protection constraints and written into the degraded image. The protective constraints play the following roles in subsequent inpainting generation and parameter settings: First, they restrict the use of freely generated nodes in text regions (e.g., disabling GAN generative inpainting) and replacing it with edge restoration or conservative super-resolution; second, they reduce the intensity of generative enhancement in face regions, requiring the calculation of feature vector similarity, keypoint offset, and skin color distribution difference after inpainting, and backing down if thresholds are not met; third, they preserve the original pixels in logo / watermark regions. When the processing area of ​​the inpainting model overlaps with the protected area, the protective constraints take precedence over the inpainting parameters, ensuring that the protected content is not treated as a texture to be freely generated.

[0030] By recording feature vectors, key points, and skin color statistics for face regions, recording OCR text, character boxes, and confidence levels for text regions, and recording boundaries and transparency for logo / watermark regions, and using the above information as protective constraints throughout the entire process of restoration image generation, parameter setting, and quality gating, precise protection of semantically important regions is achieved. This avoids erroneous details or identity drift in face, text, and logo regions, thus improving the credibility and usability of the restoration results.

[0031] In the embodiments of this application, the fusion intensity of various degradations in the degraded image is determined according to the following formula: ;in, Let be the fusion intensity of the i-th type of degradation. Let j be the score of the j-th detection method for the i-th type of degradation. For the corresponding confidence level, These are the weighting coefficients.

[0032] In another possible implementation, firstly, the multi-source data structure for degradation detection is defined. For the i-th type of degradation (such as blur, noise, compression blocks, ringing, low light, overexposure, color cast, flicker, jitter, motion discontinuity, scratches, blemishes, occlusion, etc.), the system uses j different detection methods for evaluation. Each detection method outputs two values: a score. (Values ​​range from 0 to 1, representing the degradation intensity determined by this method) and confidence level (Values ​​range from 0 to 1, indicating the reliability of the method for this scoring result). For example, for fuzzy degradation, Method 1 uses Laplace variance, and the output... and Method 2 is the high-frequency energy output in the frequency domain. and Method 3 is a learned fuzzy classifier, outputting... and Then, weights are assigned to each detection method. Weight The ability and reliability of the j-th detection method to discriminate the i-th type of degradation can be preset based on the detector's precision, recall, or historical performance on the validation set, or learned on calibration data through Bayesian optimization or grid search. Detectors that perform stably on specific degradation types are given higher weights, while detectors easily affected by scene content are given lower weights. Then, a weighted fusion calculation is performed. The fusion strength for the i-th type of degradation is... Determined by the following formula:

[0033] Where i is the degradation category number and j is the detection method number; The fusion intensity for the i-th type of degradation, with a value ranging from 0 to 1; Let be the score of the j-th detection method for the i-th type of degradation, with a value ranging from 0 to 1; Let be the confidence level of the j-th detection method for the i-th type of degradation, with a value ranging from 0 to 1; Let be the weight of the j-th detection method for the i-th type of degradation, and be a non-negative real number; This indicates a summation operation performed on the multiple detection methods corresponding to the subscript j.

[0034] Finally, the fusion module not only outputs the fusion strength It also outputs a confidence interval or uncertainty estimate. The conclusions are consistent across different detectors (each...). Increase confidence for degradation types with smaller variance; increase confidence for degradation types with obvious conflicts (each For degradation types with high variance, the candidate degradation set is retained and the priority of strong repair modules is reduced. In addition to weighted averaging, Bayesian fusion, Dempster-Shafer evidence fusion, or learning-based fusion networks can also be used for fusion. Defects with low confidence but high risk (such as suspected facial scratches or missing captions) are moved to the conservative preview or manual review branch; degradations with low confidence and minor impact are handled with low-intensity general processing. "Undetermined" is itself a processable state for the policy generator, rather than a forced execution of high-risk generative repair.

[0035] By adopting a triple-weighted normalized fusion formula, the method reliability weight, score value, and current confidence level of the detector are uniformly incorporated into the fusion calculation, realizing adaptive weighting of multi-source degradation detection results. High-reliability and high-confidence detection results dominate the fusion intensity, while low-reliability or low-confidence results are automatically suppressed. At the same time, the uncertainty estimate is output to guide subsequent conservative processing, improving the accuracy and robustness of degradation profiles in complex video scenarios.

[0036] In the embodiments of this application, the primary degradation is the degradation type whose score exceeds the primary threshold and ranks in the top K, where K is 1 to 4, and the secondary degradation is selected using a threshold lower than the primary threshold; the processing dependency constraints include: the image stabilization node is before the temporal fusion node, the strong noise processing node is before the super-resolution node, and the mask repair node is before the module that amplifies defects; when the text region accounts for a high proportion, the free generation node is restricted and an OCR verification node is added.

[0037] In another possible implementation, firstly, primary and secondary degradations are determined. After the degradation profile is generated, the system fuses the intensity of each type of degradation. Sort from highest to lowest. The primary degradation is defined as a degradation type that simultaneously satisfies the following two conditions: firstly, the fusion strength... The degradation is categorized into two main types: 1) exceeding the primary threshold (which can be preset based on video type and restoration target, e.g., 0.5); and 2) ranking in the top K positions, where K ranges from 1 to 4 (i.e., a maximum of 4 main degradations are selected). For example, in a 576P old movie clip, scratches (dscratch=0.82), flicker (dflicker=0.69), jitter (dshake=0.58), and noise (dnoise=0.54) all exceed the primary threshold and rank in the top four, thus being identified as primary degradations. Spots (dspot=0.43), color cast (dcast=0.47), and blur (dblur=0.35) do not exceed the primary threshold and are judged as secondary degradations using a lower threshold (e.g., 0.3). Then, candidate restoration modules are selected from the degradation-processing module mapping table. The mapping table is indexed by degradation category and pre-stores one or more restoration modules corresponding to each type of degradation. For example: jitter corresponds to the image stabilization module; flicker corresponds to the brightness timing correction module; compression blocks and ringing correspond to the deblocking and deringing modules; scratches and blemishes correspond to the mask repair module (based on image inpainting technology); low light corresponds to the conservative exposure recovery module; color cast corresponds to the color recovery module (the lens or user-specified reference image can be used as the color reference); low resolution corresponds to the super-resolution module; noise corresponds to the noise reduction module; blur corresponds to the deblurring module. The modules corresponding to the main degradation are mandatory nodes, and the modules corresponding to the secondary degradation are optional nodes. Whether to include them in the repair graph depends on the computing resources and quality requirements. Then, a directed repair graph is established based on the processing dependency constraints. The processing dependency constraints include the following three categories: The first category is the precondition (HardPrecedence), that is, a certain node must be executed before another node. Specifically, it includes: (1) The image stabilization node must be executed before the temporal fusion node, because after the image stabilization eliminates the global motion between frames, the cross-frame temporal fusion can accurately align the pixels; if the temporal fusion is done first and then the image stabilization is done, the motion compensation error will cause ghosting in the fusion result. (2) The strong noise processing node must be executed before the super-resolution node, because super-resolution amplifies the noise while increasing the image resolution. Noise reduction first can avoid the noise being difficult to remove after super-resolution. (3) The mask repair node must be executed before the modules that amplify defects (such as super-resolution, deblurring, and sharpening), because if structural damage such as scratches and blemishes is not repaired first, super-resolution and sharpening will amplify the defects and generate artifacts that are difficult to eliminate. The second type is post-requisite, which means that certain nodes must be followed by specific nodes. For example, after the deblocking and deringing nodes, a timing consistency check node should be connected to verify that the deblocking process has not introduced inter-frame inconsistencies; after the color restoration node, a style preservation check node should be connected to verify that the color correction has not deviated from the original tone. The third type is mutual exclusion and parallelizable constraints. Mutual exclusion constraints mean that certain nodes cannot be enabled at the same time. For example, two different noise reduction models should not be used in series; parallelizable constraints mean that certain nodes have no data dependency and can be executed at the same time on different threads or devices to improve efficiency. For example, spatial noise reduction and color restoration can be executed in parallel after image stabilization is completed.Furthermore, when the text region accounts for a high percentage (the text region area exceeds a threshold, such as 15%), the system restricts the use of freely generated nodes (such as GAN generative inpainting and generative super-resolution) and adds an OCR verification node to verify the consistency of text content after inpainting. When the face region area accounts for a high percentage, a face preservation verification node is added. Finally, the policy generator uses a rule engine, constraint satisfaction problem (CSP) solving, dynamic programming, or heuristic search algorithms to construct the optimal or near-optimal directed acyclic inpainting graph from candidate nodes, while satisfying the above dependency constraints. A typical inpainting node sequence is: decoding and color space normalization - image stabilization - deflickering - scratch / blemish masking inpainting - deblocking and deringing - noise reduction - exposure and color correction - deblurring - super-resolution - region refinement - temporal consistency verification - encoding output. Each node in the graph records the input format, output format, parameter range, resource cost, failure fallback scheme, and quality threshold.

[0038] By employing a primary threshold plus a selection mechanism for the top K primary degradations and a selection mechanism for secondary degradations with lower thresholds, precise selection of repair modules is achieved, avoiding resource waste and over-repair caused by indiscriminately performing full processing on all degradations. The repair node topology is standardized through four types of dependency constraints: pre-processing, post-processing, mutual exclusion, and parallelism. In particular, the three hard constraints of image stabilization before temporal fusion, noise reduction before super-resolution, and mask repair before amplification modules ensure the logical correctness of the repair chain and avoid defect amplification and artifact generation caused by incorrect execution order.

[0039] In embodiments of this application, the node parameters are generated from the degradation intensity via a piecewise function or table lookup, including: denoising intensity of... ,in The fusion intensity of the noise category in the degraded image. The denoising intensity is limited to 0 to 100, and the number of pixels in the scratch mask dilation varies between 1 and 12 depending on the fusion intensity of the scratch category in the degraded image. The super-resolution ratio is determined by the input and target resolutions, ranging from 1.5 to 4. The upper limit of the generation intensity of the text region is lower than that of the background region. The quality gating indicators include sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker indicators. When the OCR confidence decreases, the facial feature similarity is below the threshold, or flicker increases, the intensity of the corresponding node is reduced, a conservative model is replaced, or a manually reviewed node is inserted. The repair plan includes fragment identifiers, degradation vectors, node lists, topological relationships, model versions, parameters, region mask references, quality thresholds, failure retry strategies, and result storage paths. The execution engine runs according to the repair plan and writes back the actual indicators to update the subsequent strategies for similar fragments.

[0040] In another possible embodiment, firstly, parameters are generated for each node in the repair graph. The parameters are obtained from the degradation intensity via a piecewise function or by looking up a table, and specifically include the following parameters: (1) Denoising intensity parameter: the denoising intensity is ,in The fusion strength of the noise category in the degraded image (value from 0 to 1). This is the rounding function. When... When the noise reduction intensity is 20 (lowest level), The noise reduction intensity is 80 (highest level), and the final noise reduction intensity is limited to between 0 and 100. This piecewise linear mapping makes the noise reduction filter stronger as the noise intensity increases. (2) Scratch mask dilation parameter: The number of dilated pixels of the scratch mask varies with It varies between 1 and 12, among which The fusion intensity of scratches in the degraded image. The greater the scratch intensity, the more pixels the mask expands outward to ensure that the scratch area is completely covered and has sufficient context information during repair. The dilation operation uses a morphological dilation operator, and the size of the dilation kernel corresponds to the number of dilated pixels. (3) Super-resolution ratio parameter: The super-resolution ratio is determined by the input video resolution and the target output resolution, ranging from 1.5 to 4 times. For example, when the input is 576P (720×576) and the target is 1080P (1920×1080), the ratio is about 2 times; when the input is 360P and the target is 1080P, the ratio is 3 times. The upper limit of the ratio is 4 times to avoid over-generating false details. (4) Region differentiation parameter: The upper limit of the generation intensity of the text region is lower than that of the background region. For example, the super-resolution generation intensity of the background region can be set to 0.8 (high-fidelity generation), and the text region is limited to 0.3 (conservative interpolation) to prevent the text strokes from being tampered with by the generative model. The generative enhancement intensity of the face region is also limited. The system records the source degradation dimension, mapping version, and final truncation value during parameter generation to ensure the parameter selection process is reproducible. Then, trial processing is performed and quality gating metrics are calculated. For each segment or each degradation combination, the system selects a 0.5 to 5 second preview clip (covering the beginning and end of the shot, the most severely degraded area, and key areas) and processes it according to the candidate restoration map. After processing, calculate the following six quality gating indicators: (1) Sharpness improvement: compare the Laplacian variance or high-frequency energy of the images before and after restoration, and the sharpness should be improved; (2) Noise residue: measure the high-frequency residual variance after restoration in a flat area, and the noise residue should be lower than the threshold; (3) OCR confidence change: compare the OCR recognition confidence of the text area before and after restoration, and the confidence should not decrease; (4) Facial feature similarity: calculate the cosine similarity of the ArcFace feature vector of the face area before and after restoration, and the similarity should be higher than the threshold (e.g., 0.9); (5) Color difference: calculate the color difference in the Lab color space before and after restoration, and the color difference should be within the allowable range and conform to the original style constraints; (6) Temporal flicker index: compare the brightness difference statistics of adjacent frames after motion compensation before and after restoration, and the flicker should not increase. Then adjust the strategy according to the gating results. The gating judgment logic is: if the sharpness is improved but the OCR decreases or the flicker increases, it is not considered as passing. Specifically, when OCR confidence decreases, reduce the super-resolution / deblurring intensity of text regions or replace it with a conservative model; when facial feature similarity is below a threshold, reduce the generative enhancement intensity of face regions or revert to original pixels; when flicker increases, reduce the temporal processing intensity or increase temporal consistency constraints; when sharpness improvement is insufficient, appropriately increase the deblurring or super-resolution intensity. After adjustments, re-execute trial processing until all gating indicators meet the requirements. For low-confidence / high-risk scenarios (such as suspected facial scratches), a manual review node can be inserted for manual confirmation of the repair strategy. Finally, output the RepairPlan.The remediation plan includes the following fields: fragment identifier and degradation vector (fusion strength of various degradations). The repair module generates a comprehensive set of data, including: confidence level, node list (module name and version for each node in the repair diagram), topology (directed connections between nodes), model version (model version number used by each repair module), parameters (final parameter values ​​for each node), region mask references (storage path references for damage masks and key region masks to avoid directly writing large masks into the policy text), quality thresholds (thresholds for various gating indicators), failure retry strategy (number of retries and fallback plan when a node fails), and result storage path (storage location for repaired video and logs). The execution engine runs the full-screen repair according to the RepairPlan and writes back the actual running metrics (processing time, actual quality score, failed nodes, etc.) to the policy generation module, enabling subsequent policies for similar segments to be updated and optimized based on historical data. Video frames, features, masks, policies, task descriptions, and logs passed between modules are all associated with video stream identifiers, timestamps, and configuration versions to ensure processing order, traceability, and post-failure recovery capabilities.

[0041] By adopting The piecewise function linearly maps the degradation intensity to repair parameters and sets differential generation intensity upper limits for text and face regions, achieving adaptive quantization of parameters and regional differentiation protection. Through preview verification of six types of quality gating indicators and a closed-loop mechanism of adjustment upon failure, the repair strategy is automatically verified and risk intercepted before full-scale execution. By outputting a RepairPlan containing complete fields and version information and writing back the actual indicators, the repair process is made reproducible, traceable, and continuously optimized.

[0042] Example 2:

[0043] Reference Figure 2This application also provides an automatic video restoration strategy generation system based on multi-dimensional degradation recognition, comprising: a video sampling module for dividing an input video into at least one video segment and extracting keyframes and continuous sampling frames; a multi-dimensional spatial degradation analysis module connected to the video sampling module for performing spatial degradation detection on the segment, including at least blur detection, noise detection, block effect detection, and ringing detection; a temporal degradation analysis module connected to the video sampling module for performing temporal degradation detection on the segment, including at least jitter detection and flicker detection; an exposure and color analysis module connected to the video sampling module for performing exposure and color detection on the segment, including at least low light detection, overexposure detection, and color cast detection; a structural damage mask module connected to the video sampling module for performing structural damage detection on the segment, including at least scratch detection, spot detection, and missing area detection, and generating a corresponding damage mask; and a key area protection module connected to the video sampling module for detecting key areas and generating protection constraints, wherein the key areas are... The system includes at least a face region, a text region, and a logo / watermark region; a degradation image fusion module, connected to the multi-dimensional spatial degradation analysis module, temporal degradation analysis module, exposure color analysis module, structural damage mask module, and key area protection module, respectively, to fuse the outputs of each module and generate a degradation image containing various degradation intensities and their confidence levels, damage masks, and protection constraints; a restoration image generation module, connected to the degradation image fusion module, with a built-in degradation-processing module mapping table, used to select candidate modules based on the degradation image and generate a directed restoration image according to processing dependency constraints; used to update the degradation-processing module mapping table or node parameter generation strategy; a parameter and trial processing module, connected to the restoration image generation module, used to generate parameters for each node in the restoration image, select a preview segment to execute the restoration image and calculate quality gate indicators, and adjust parameters or replace modules if the threshold is not met; a restoration execution record module, connected to the parameter and trial processing module, used to output and execute a restoration plan after the threshold is met, and write the actual quality indicators back to the restoration image generation module.

[0044] In another possible embodiment, the system first includes a video sampling module. This module is used to parse the encoded attributes and timestamps of the input video, divide the input video into at least one video segment according to shot boundary detection or a fixed time window, and extract the first frame, last frame, uniform keyframe, and continuous sampling frames for each segment. The video sampling module is also responsible for deinterlacing interlaced video, timestamp normalization of variable frame rate video, and pre-marking of subtitle regions. The output of the video sampling module is connected to a multidimensional spatial degradation analysis module, a temporal degradation analysis module, an exposure color analysis module, a structural damage masking module, and a key area protection module, respectively, distributing the sampled frames to each detection module.

[0045] The system then comprises five parallel detection modules:

[0046] The multidimensional spatial degradation analysis module, connected to the video sampling module, is used to perform spatial degradation detection on segments, including at least blur detection, noise detection, block effect detection, and ringing detection, and outputs a spatial score (including the intensity and confidence of various degradation types and multi-source evidence citations).

[0047] The time degradation analysis module, connected to the video sampling module, is used to perform time degradation detection on segments, including at least jitter detection and flicker detection, and outputs a time score.

[0048] The exposure and color analysis module, connected to the video sampling module, performs exposure and color detection on video clips, including at least low-light detection, overexposure detection, and color cast detection. The low-light score is obtained from the average brightness, the proportion of pixels in dark areas, and the signal-to-noise ratio in dark areas; the overexposure score is obtained from the proportion of pixels near maximum brightness and the loss of highlight detail; the color cast score is obtained from grayscale world deviation, skin tone distribution deviation, or the difference in the average color channels. For videos with a defined style, reference shots or user-specified reference images can be used as color benchmarks to avoid misjudging cinematic color grading as color cast. This module outputs a color score.

[0049] The structural damage mask module, connected to the video sampling module, is used to perform structural damage detection on the segments, including at least scratch detection, spot detection, and missing region detection, and to generate a damage mask containing pixel location, category, confidence level, and cross-frame association identifier.

[0050] The key area protection module, connected to the video sampling module, is used to detect face areas, text areas, and logo / watermark areas and generate protection constraints.

[0051] Next, a degradation image fusion module is set up. This module is connected to the multi-dimensional spatial degradation analysis module, the temporal degradation analysis module, the exposure color analysis module, the structural damage mask module, and the key area protection module, respectively. It receives the outputs of the five detection modules and processes them according to the formula. By fusing the intensity and confidence of various degradations, and summarizing damage masks and protection constraints, a structured degradation profile is generated. The data structure of the degradation profile includes: profile_id, video_id, segment_id, start_pts, end_pts, sampling_plan, spatial_scores, temporal_scores, color_scores, damage_masks, salient_regions, confidence_scores, detector_versions, reference_style, generated_time, and profile_version. Spatial, temporal, and color scores are composed of category, intensity, confidence, statistical range, and evidence citations. For example, the fuzzy term simultaneously stores laplacian_value, frequency_energy, model_score, and the final fused intensity. The damage mask stores the mask location, category, confidence level, cross-frame association, and storage reference, avoiding the direct writing of large-size masks into the policy text.

[0052] Next, the restoration graph generation module is set up. This module is connected to the degradation profile fusion module and has a built-in degradation-processing module mapping table. It is used to determine the primary and secondary degradations based on the degradation profile, select candidate restoration modules from the mapping table, and generate a directed restoration graph (DAG) composed of restoration nodes based on pre-processing, post-processing, mutual exclusion, and parallelizable constraints. The module mapping library is indexed by degradation category, intensity range, content constraints, device resources, and output specifications. It can be configured according to video type or user style preferences, but the version used is recorded each time it is executed.

[0053] Next, the parameter setting and trial processing module is used. This module connects to the restoration image generation module and generates parameters for each node in the restoration image according to a piecewise function. It selects a 0.5 to 5 second preview segment to execute the candidate restoration image and calculates quality gating indicators such as sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker. If the threshold is not met, this module outputs parameter downgrading or model replacement instructions to the restoration image generation module, triggering the regeneration of the restoration image or parameter adjustment, forming a closed loop.

[0054] Finally, a repair execution log module is configured. This module, connected to the parameter and trial processing module, outputs a repair plan (RepairPlan) after the quality threshold is met, driving the execution engine to run the full-screen repair according to the plan. The repair plan's data structure includes: segment identifier, degradation vector, node list, topology relationship, model version, parameters, region mask reference, quality threshold, failure retry strategy, and result storage path. The repair execution log module writes back actual running metrics (processing time, actual quality score, failed nodes, etc.) to the repair graph generation module to update subsequent strategies for similar segments. Video frames, features, masks, strategies, task descriptions, or logs passed between modules are all associated with video stream identifiers, timestamps, and configuration versions to ensure video processing order, traceability, and post-failure recovery capabilities. The system can consist of a processor, memory, video input / output interfaces, and network interfaces. The memory stores program instructions, and the processor executes these instructions to implement the above method steps. Each functional module can be deployed on the same terminal (ARM, x86, GPU, NPU, or AI accelerator) or distributed across terminals, edge nodes, or servers.

[0055] By setting up ten functional modules—video sampling, five parallel detection modules, degradation profile fusion, restoration image generation, parameter and trial processing, and restoration execution record—and clearly defining their hierarchical connections and feedback loops, a complete pipeline from video input to restoration output was realized. The parallel operation of the five detection modules improved detection efficiency, the degradation profile fusion module centrally summarized the system to unify the data format, the feedback loop between restoration image generation and parameter trial processing realized closed-loop quality control, and the write-back mechanism of the restoration execution record module enabled continuous self-optimization of the strategy.

[0056] In the embodiments of this application, in the multidimensional spatial degradation analysis module: the fuzzy score is obtained by fusing Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; the noise score estimates the high-frequency residual variance in the texture-flat region and excludes subtitles and edge regions; the block effect score statistically analyzes the brightness abrupt changes on both sides of the coding block boundary; the ringing score detects the alternating overshoot and undershoot patterns on both sides of strong edges; the temporal degradation analysis module estimates the transformation of adjacent frames by feature point matching or optical flow, removes foreground motion to obtain a jitter score, and statistically analyzes the brightness difference or color difference of the corresponding region after motion compensation to obtain a flicker score; the structural damage mask module generates a scratch mask using linear morphology and cross-frame continuity, generates a blemish mask using small-area isolated anomalies and temporal persistence, and generates a damage mask containing pixel position, category, confidence level, and cross-frame association identifier.

[0057] In another possible embodiment, the specific implementation of each detector in the multidimensional spatial degradation analysis module is as follows: Blur detection subunit: First, the image is divided into non-overlapping blocks. After excluding textureless sky and solid-color backgrounds, the Laplacian variance, Sobel gradient magnitude, and Fourier frequency domain high-frequency energy of each block are calculated. Then, the three traditional indicators are weighted and fused with the output score of the learning-based fuzzy classifier (CNN) to obtain a blur score between 0 and 1. Local blurred regions are represented and saved as block-level heatmaps. Noise detection subunit: In texture-flat regions (regions where the local gradient is less than a threshold), the variance of the difference between the image and the low-pass filter result is estimated as the noise intensity, excluding subtitle regions and strong edge regions. Robustness is further improved by combining wavelet high-frequency subband coefficients and chroma channel variance, and temporal stability is used to determine whether it is a style element such as film grain. Block effect detection subunit: Pixels are taken on both sides of the boundary of 8×8, 16×16, or predictive coding blocks to calculate the brightness jump variable. The average jump and maximum jump of the entire frame are statistically analyzed as the block effect score. If the bitstream is accessible, the judgment is enhanced by combining the quantization parameters QP and GOP structure. Ringing detection subunit: Extracts the brightness profile along the normal direction at strong edge locations, detecting alternating overshoot peaks and undershoot valleys on both sides of the edge; the larger the oscillation amplitude, the higher the ringing score. Then, the specific implementation of each detector in the temporal degradation analysis module is as follows: Jitter detection subunit: Estimates the global transformation matrix (translation, rotation, affine) between adjacent frames through ORB / SIFT feature point matching or Farneback optical flow, uses the RANSAC algorithm to remove the outer points of the foreground moving subject, and retains the inner points of the background to estimate global motion. High-pass filtering is applied to the global motion parameters, and high-frequency abnormal displacements without narrative intent are extracted as jitter scores. Image stabilization strategy sets an upper limit for the cropping ratio (e.g., not exceeding 10% of the frame area) to prevent over-cropping. Flicker detection subunit: First, uses optical flow or global motion estimation to perform motion compensation alignment on adjacent frames, then calculates the brightness difference and color difference in the corresponding aligned areas, using median and quantile statistics to exclude abnormal values ​​such as local flashes and screen switching; the obtained statistics are used as flicker scores. Film flicker is estimated separately for low-frequency brightness trends and high-frequency jitter. The specific implementation of each detector in the structural damage masking module is further explained: Scratch Masking Subunit: Performs top-hat transformation and line segment detection on a single frame to extract elongated line-shaped candidates. Utilizing the characteristics of stable cross-frame position and continuous direction of scratches, the mask is propagated through optical flow between adjacent frames and then confidently fused with the detection result of the current frame. In case of conflict, high-confidence regions are retained. The output mask includes the position, category (scratch), confidence level, and cross-frame association identifier for each scratch pixel. Stains Masking Subunit: Detects small, isolated anomalous regions (significantly different from their surroundings but with an area less than a threshold). Temporally verifies persistence (occurring at the same location in multiple frames) to eliminate random noise and generates a stain mask.The damage mask output subunit merges scratch masks, blemish masks, and missing region masks (from occlusion detection, black block detection, or uncertain output of the repair model) into a unified damage mask structure. Each mask element contains four fields: pixel location, category, confidence level, and cross-frame association identifier. These are stored as independent files and recorded in the degraded image using a reference path. Finally, the outputs of the three modules are sent to the degraded image fusion module for fusion.

[0058] By implementing four sub-units—blurring, noise, block effect, and ringing—in the multidimensional spatial degradation analysis module, and implementing jitter detection based on RANSAC foreground culling and flicker detection after motion compensation in the temporal degradation analysis module, and implementing scratch / stain mask generation based on morphology and temporal propagation in the structural damage mask module, refined and multi-evidence detection of each degradation dimension is achieved. The mask structure includes four fields: position, category, confidence, and cross-frame correlation, providing accurate pixel-level guidance for subsequent restoration map generation.

[0059] In embodiments of this application, the parameter and test processing module includes: a parameter generation unit, used to generate denoising intensity according to a piecewise function. ,in The fusion intensity of the noise category in the degraded image. The denoising intensity is limited to 0 to 100, which is a rounding function; the number of scratch mask dilation pixels varies between 1 and 12 depending on the fusion intensity of the scratch category in the degraded image; the super-resolution factor is determined by the input and target resolution, ranging from 1.5 to 4, and the upper limit of the generation intensity of the text region is lower than the upper limit of the background region; the quality gating unit is used to calculate the sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker index on the preview segment, and outputs parameter downgrading or model replacement instructions to the restoration image generation module when any protection index fails to meet the threshold; the manual review unit is connected to the quality gating unit and is used to display the synchronous comparison, local magnification, and mask overlay of the original and restoration results in low confidence / high risk scenarios, receive annotations from reviewers, and generate new policy constraints.

[0060] In another possible embodiment, the parameter and trial processing module first includes a parameter generation unit. This unit is used to generate parameters for each repair node according to a piecewise function based on the fusion intensity in the degraded image, specifically including: denoising intensity parameters: according to the formula... Calculation, where The fusion strength (0 to 1) for the noise category. The rounding function is used, and the calculation result is limited to between 0 and 100. The denoising intensity is 20 when the noise fusion intensity is 0, and 80 when the noise fusion intensity is 1. Scratch mask dilation parameter: The number of pixels for mask dilation varies linearly between 1 and 12 with the scratch fusion intensity; the more severe the scratch, the larger the dilation range, ensuring complete scratch coverage. Super-resolution ratio parameter: Determined by the input and target resolutions, it is selected between 1.5 and 4 times, with an upper limit of 4 to avoid over-generating false details. Region differentiation parameter: The upper limit of the generative restoration intensity for text regions is set lower than that for background regions. For example, the generation intensity factor for background regions can be 0.8, while it is limited to 0.3 for text regions; the generative enhancement intensity for face regions is also limited. During parameter generation, the source degradation dimension, mapping table version, and final truncation value for each parameter are recorded. Then, the parameters are processed by the trial processing module, including a quality gating unit. This unit is used to calculate six types of quality metrics and make threshold judgments after performing candidate restoration images on preview clips from 0.5 to 5 seconds: Objective quality metrics include sharpness improvement (the change in Laplacian variance or high-frequency energy in the frequency domain before and after restoration, which should be positive and exceed the minimum improvement threshold) and noise residue (the variance of high-frequency residuals in flat areas after restoration, which should be lower than the noise threshold). Conservation metrics include OCR confidence change (the difference in OCR confidence in text areas before and after restoration, which should not be negative or decrease beyond the tolerance), facial feature similarity (the cosine similarity of the ArcFace feature vectors of faces before and after restoration, which should be higher than the threshold of 0.9), and color difference (the ΔE color difference in Lab color space before and after restoration, which should be within the range allowed by style constraints). Temporal metrics include temporal flicker metrics (the statistical difference in motion compensation brightness between adjacent frames after restoration, which should not increase compared to before restoration). The gating judgment adopts a "one-vote veto" logic: even if the sharpness improvement is significant, if the OCR confidence decreases, the facial similarity is too low, or the flicker increases, it is not considered as passing. When any metric fails to meet the threshold, the quality gating unit outputs a parameter downgrade instruction (e.g., reducing the denoising intensity by 20% or reducing the text region generation intensity to the lowest level) or a model replacement instruction (e.g., replacing GAN generative super-resolution with conservative interpolation super-resolution), which is sent to the repair image generation module to trigger rearrangement. Then, the process is repeated, forming a closed-loop iteration. For low-confidence / high-risk scenarios (e.g., suspected facial region scratches but insufficient detection confidence), the quality gating unit does not automatically adjust but triggers a manual review branch. A manual review unit is then set up. This unit, connected to the quality gating unit, is used to display an interactive review interface to reviewers when low-confidence / high-risk scenarios or when multiple automatic adjustments fail to meet the threshold. The interface includes: synchronized side-by-side playback of the original video and the restoration results, local magnification comparison (can be magnified to view faces, text, scratches), mask overlay display (displays damage masks and key area protection masks on the screen), degradation scoring panel (displays the fusion strength and confidence of various degradations), and restoration graph node and parameter panel (displays the current restoration graph topology and parameter values ​​of each node).Reviewers can annotate issue types (e.g., text distortion, facial distortion, residual scratches, color shift), timecodes, region coordinates, severity, and suggested actions (e.g., reduce intensity, replace the model, mark as not requiring repair, manually repair), forming new policy constraints or training samples that are fed back to the repair map generation module. Finally, once all gating indicators pass or are confirmed by manual review, the parameter and trial processing module sends the final repair map and parameters to the repair execution record module, which then generates a RepairPlan and performs full-screen repair.

[0061] By internally defining a parameter generation unit, a quality gating unit, and a manual review unit within the parameter and trial processing module, a three-tiered assurance mechanism is implemented, encompassing automatic parameter mapping, preview quality verification, and manual fallback review. The piecewise function in the parameter generation unit ensures a quantitative correspondence between parameters and degradation intensity; the six-category veto logic in the quality gating unit ensures that semantic protection and temporal stability are not sacrificed; and the manual review unit provides an expert intervention channel for low-confidence / high-risk scenarios. The collaboration of these three components achieves a balance between automation and security in the remediation strategy.

[0062] In the embodiments of this application, the data structure of the degraded profile includes: profile identifier, video identifier, segment identifier, start timestamp, end timestamp, sampling plan, spatial score, temporal score, color score, damage mask, salient region, confidence score, detector version, reference style, generation time, and profile version; the data structure of the repair plan includes: segment identifier, degradation vector, node list, topological relationship, model version, parameters, region mask reference, quality threshold, failure retry strategy, and result storage path; the video frames, features, masks, strategies, task descriptions, or logs transmitted between modules are all associated with video stream identifiers, timestamps, and configuration versions to ensure video processing order, traceability, and recovery capability after failure.

[0063] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0064] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0065] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A method for automatically generating video restoration strategies based on multi-dimensional degradation recognition, characterized in that, include: The input video is divided into at least one video segment, and keyframes and consecutive sampled frames are extracted from each segment. Spatial degradation detection, temporal degradation detection, exposure color detection, structural damage detection, and key area detection are performed on the segments respectively to generate a degradation profile containing various degradation intensities and their confidence levels, structural damage masks, and key area protection constraints. Based on the degradation profile, the primary and secondary degradations are determined, candidate repair modules are selected from the degradation-processing module mapping table, and a directed repair graph consisting of repair nodes is generated according to the processing dependency constraints. Parameters are generated for each node in the repair graph. A preview segment is selected to execute the repair graph and calculate the quality gate index. If the threshold is not met, the node parameters of the repair graph are adjusted or the module is replaced until the threshold is met and then the repair plan is output. Perform video repairs according to the repair plan and write back the actual quality metrics to update the strategy.

2. The automatic generation method for video restoration strategies based on multi-dimensional degradation recognition according to claim 1, characterized in that, The spatial degradation detection includes: fuzzy scoring is obtained by fusing Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; noise scoring estimates the high-frequency residual variance in flat texture regions and excludes subtitles and edge regions; block effect scoring statistically analyzes the brightness abrupt changes on both sides of the coded block boundary; and ringing scoring detects alternating overshoot and undershoot patterns on both sides of strong edges. The time degradation detection includes: estimating the transformation of adjacent frames through feature point matching, optical flow, or global motion model; after removing the motion of the foreground subject, using abnormal global translation, rotation, or affine changes as jitter scores; and obtaining flicker scores by statistically analyzing the brightness difference, color difference, or feature difference of the corresponding region after motion compensation. The structural damage detection includes: scratch detection using elongated linear morphology, cross-frame positional stability, and directional continuity to generate scratch masks; blemish detection using small-area isolated anomalies and temporal persistence to generate blemish masks; missing regions are determined by occlusion detection, black block detection, or uncertainty output of the repair model; the mask includes pixel position, category, confidence level, and cross-frame association identifier.

3. The automatic generation method for video restoration strategies based on multi-dimensional degradation recognition according to claim 1, characterized in that, The key area detection includes the detection of face areas, text areas, and logo / watermark areas; for face areas, feature vectors, key points, and skin color statistics are recorded; for text areas, OCR text, character boxes, and confidence scores are recorded; for logo / watermark areas, boundary and transparency estimates are recorded; the key area protection constraints are used to control the subsequent repair module from performing free-generative repairs on the protected content.

4. The automatic generation method for video restoration strategies based on multi-dimensional degradation recognition according to claim 1, characterized in that, The fusion intensity of various degradations in the degraded image is determined according to the following formula: ； in, Let be the fusion intensity of the i-th type of degradation. Let j be the score of the j-th detection method for the i-th type of degradation. For the corresponding confidence level, These are the weighting coefficients.

5. The automatic generation method for video restoration strategies based on multi-dimensional degradation recognition according to claim 1, characterized in that, The primary degradation is defined as the degradation type where the score exceeds the primary threshold and ranks in the top K, where K ranges from 1 to 4. Secondary degradation uses a threshold lower than the primary threshold. The processing dependency constraints include: image stabilization nodes precede temporal fusion nodes, strong noise processing nodes precede super-resolution nodes, and mask repair nodes precede modules that amplify defects. When the text region accounts for a high proportion, the free generation of nodes is restricted and an OCR verification node is added.

6. The automatic generation method for video restoration strategies based on multi-dimensional degradation recognition according to claim 5, characterized in that, The node parameters are generated from the degradation intensity via a piecewise function or table lookup, including: denoising intensity is... ,in The fusion intensity of the noise category in the degraded image. The function is a rounding function, and the denoising intensity is limited to 0 to 100; The number of pixels in the scratch mask dilation varies between 1 and 12 depending on the fusion strength of the scratch category in the degraded image; The super-resolution ratio is determined by the input and target resolutions, ranging from 1.5 to 4; the upper limit of the generation intensity of the text region is lower than that of the background region. The quality gating indicators include sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference, and temporal flicker. When OCR confidence decreases, facial feature similarity falls below the threshold, or flicker increases, the strength of the corresponding node is reduced, the conservative model is replaced, or a manually reviewed node is inserted. The repair plan includes fragment identifiers, degradation vectors, node lists, topological relationships, model versions, parameters, region mask references, quality thresholds, failure retry strategies, and result storage paths. The execution engine runs according to the repair plan and writes back the actual metrics to update subsequent strategies for similar fragments.

7. An automatic video restoration strategy generation system based on multi-dimensional degradation recognition, characterized in that, The video restoration strategy automatic generation method based on multi-dimensional degradation recognition, as described in any one of claims 1-6, includes: The video sampling module is used to divide the input video into at least one video segment and extract keyframes and continuous sampling frames. A multidimensional spatial degradation analysis module, connected to the video sampling module, is used to perform spatial degradation detection on the segment, including at least blur detection, noise detection, block effect detection, and ringing detection; A time degradation analysis module, connected to the video sampling module, is used to perform time degradation detection on the segment, including at least jitter detection and flicker detection; An exposure color analysis module, connected to the video sampling module, is used to perform exposure and color detection on the segment, including at least low light detection, overexposure detection and color cast detection; A structural damage mask module, connected to the video sampling module, is used to perform structural damage detection on the segment, including at least scratch detection, spot detection and missing region detection, and generate a corresponding damage mask. A key area protection module, connected to the video sampling module, is used to detect key areas and generate protection constraints. The key areas include at least a face area, a text area, and a logo / watermark area. The degradation profile fusion module is connected to the multidimensional spatial degradation analysis module, temporal degradation analysis module, exposure color analysis module, structural damage mask module, and key area protection module, respectively, and is used to fuse the outputs of each module to generate a degradation profile containing various degradation intensities and their confidence levels, damage masks, and protection constraints. The restoration graph generation module is connected to the degradation profile fusion module and has a built-in degradation-processing module mapping table. It is used to select candidate modules based on the degradation profile and generate a directed restoration graph according to processing dependency constraints. It is also used to update the degradation-processing module mapping table or node parameter generation strategy. The parameter and trial processing module is connected to the repair graph generation module. It is used to generate parameters for each node in the repair graph, select a preview segment to execute the repair graph and calculate the quality gate index. If the threshold is not met, the parameters are adjusted or the module is replaced. The repair execution record module is connected to the parameter and trial processing module. It is used to output and execute the repair plan after the threshold is met, and write the actual quality indicators back to the repair diagram generation module.

8. The automatic video restoration strategy generation system based on multi-dimensional degradation recognition according to claim 7, characterized in that, In the multidimensional spatial degradation analysis module: the fuzzy score is obtained by fusing the Laplacian variance, gradient magnitude, high-frequency energy in the frequency domain, and a learned fuzzy classifier; the noise score estimates the high-frequency residual variance in the texture flat region and excludes subtitles and edge regions; the block effect score statistically analyzes the brightness abrupt changes on both sides of the coded block boundary; the ringing score detects the alternating overshoot and undershoot patterns on both sides of strong edges. The time degradation analysis module estimates the transformation of adjacent frames by feature point matching or optical flow, removes foreground motion to obtain a jitter score, and calculates the brightness difference or color difference of the corresponding area after motion compensation to obtain a flicker score. The structural damage masking module generates scratch masks using linear morphology and cross-frame continuity, generates blemish masks using small-area isolated anomalies and temporal persistence, and generates damage masks containing pixel location, category, confidence level, and cross-frame association identifiers.

9. The automatic generation system for video restoration strategies based on multi-dimensional degradation recognition according to claim 7, characterized in that, The parameters and trial processing module includes: The parameter generation unit is used to generate the denoising intensity according to the piecewise function. ,in The fusion intensity of the noise category in the degraded image. The denoising intensity is limited to 0 to 100; the number of scratch mask dilation pixels varies between 1 and 12 depending on the fusion intensity of the scratch category in the degraded image; the super-resolution factor is determined by the input and target resolution, ranging from 1.5 to 4, and the upper limit of the generation intensity of the text region is lower than the upper limit of the background region. The quality gating unit is used to calculate the sharpness improvement, noise residue, OCR confidence change, facial feature similarity, color difference and temporal flicker index on the preview segment. When any protection index fails to meet the threshold, it outputs parameter downgrade or model replacement instructions to the repair image generation module. The manual review unit, connected to the quality gating unit, is used to display synchronous comparison, local magnification, and masking of the original and repaired results in low-confidence / high-risk scenarios, and to receive annotations from reviewers and generate new policy constraints.

10. The automatic video restoration strategy generation system based on multi-dimensional degradation recognition according to claim 7, characterized in that, The data structure of the degraded profile includes: profile identifier, video identifier, segment identifier, start timestamp, end timestamp, sampling plan, spatial score, temporal score, color score, damage mask, salient region, confidence score, detector version, reference style, generation time, and profile version. The data structure of the repair plan includes: fragment identifier, degradation vector, node list, topological relationship, model version, parameters, region mask reference, quality threshold, failure retry strategy and result storage path; The video frames, features, masks, policies, task descriptions, or logs transmitted between modules are all associated with video stream identifiers, timestamps, and configuration versions to ensure video processing order, traceability, and recovery capabilities after a failure.

Images