A video quality diagnosis method and system based on feature recognition

Abstract

This application provides a video quality diagnosis method and system based on feature recognition, belonging to the field of video processing technology. The method includes: acquiring a first video frame and a second video frame from the video to be diagnosed; determining element evaluation features and temporal evaluation features based on the first and second video frames; acquiring a third video frame from the video to be diagnosed; determining whether the third video frame is a key data frame based on the element evaluation features; if the third video frame is a key data frame, performing resolution reconstruction and artifact compression processing on the third video frame to obtain a target video frame; evaluating the target video frame according to a preset quality evaluation standard to obtain a quality evaluation result for the target video frame; and determining the quality diagnosis result of the video to be diagnosed based on the quality evaluation results of multiple target video frames and the temporal evaluation features. This application aims to improve the problems of insufficient comprehensiveness and accuracy in video quality diagnosis in existing solutions.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, and in particular relates to a video quality diagnosis method and system based on feature recognition. Background Technology

[0002] The main goal of video quality diagnostics is to assess and ensure key indicators such as the clarity, smoothness, and color reproduction of video content, so as to provide users with the best viewing experience.

[0003] Currently, video quality assessment typically involves extracting video frames using image processing methods, resizing them to a fixed size, and then inputting them into a convolutional neural network (CNN). The CNN scores the quality of each frame, and the overall video quality is determined by averaging the scores of all frames. Summary of the Invention

[0004] This application provides a video quality diagnosis method and system based on feature recognition. The technical solution adopted in this application is as follows:

[0005] Firstly, a video quality diagnosis method based on feature recognition is provided, the method comprising:

[0006] The first and second video frames in the video to be diagnosed are obtained, and the element evaluation features and temporal evaluation features are determined based on the first and second video frames. The second video frame is the video frame after the first video frame.

[0007] The third video frame in the video to be diagnosed is obtained, and based on the feature evaluation characteristics, it is determined whether the third video frame is a key data frame.

[0008] When the third video frame is the key data frame, resolution reconstruction and compression artifact processing are performed on the third video frame to obtain the target video frame. The third video frame is the video frame after the second video frame.

[0009] The target video frame is evaluated according to the preset quality evaluation criteria to obtain the quality evaluation result of the target video frame.

[0010] Based on the quality evaluation results and temporal assessment characteristics of multiple target video frames, the quality diagnosis result of the video to be diagnosed is determined.

[0011] In one embodiment of this application, determining element evaluation features and temporal evaluation features based on a first video frame and a second video frame includes:

[0012] Determine the feature extraction of the first and second video frames to obtain the feature evaluation features;

[0013] The temporal evaluation features are determined based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

[0014] In one embodiment of this application, determining whether a third video frame is a key data frame based on element evaluation features includes:

[0015] The first key data score for the third video frame is determined based on the number of feature evaluation elements.

[0016] Based on the distribution location of the feature evaluation elements, determine the second key data score of the third video frame;

[0017] Based on the first key data score and the second key data score, determine whether the third video frame is a key data frame.

[0018] In one embodiment of this application, performing resolution reconstruction and compression artifact processing on the third video frame to obtain the target video frame includes:

[0019] Obtain the current resolution of the third video frame, and determine the resolution correction parameters based on the comparison results between the current resolution of the third video frame and the resolutions of the first and second video frames;

[0020] Based on the resolution correction parameters, the third video frame is reconstructed and compressed to remove artifacts, in order to obtain the target video frame.

[0021] In one embodiment of this application, a target video frame is evaluated according to a preset quality evaluation standard to obtain a quality evaluation result for the target video frame, including:

[0022] Based on the preset quality evaluation criteria, the target video frame is evaluated in multiple dimensions to obtain quality evaluation scores in multiple dimensions.

[0023] The quality evaluation result of the target video frame is determined by the weighted summation of the quality evaluation scores of each dimension and the corresponding weight coefficients.

[0024] In one embodiment of this application, the quality diagnosis result of the video to be diagnosed is determined based on the quality evaluation results and temporal evaluation features of multiple target video frames, including:

[0025] The temporal evaluation features are dynamically adjusted based on the inter-frame variation relationship of the feature characteristics between each target video frame and the previous video frame.

[0026] Based on the quality evaluation results of multiple target video frames and the dynamically adjusted temporal evaluation characteristics, the quality diagnosis result of the video to be diagnosed is determined.

[0027] In one embodiment of this application, the method further includes:

[0028] If the third video frame is not a critical data frame, continue with the steps of acquiring the third video frame in the video to be diagnosed and determining whether the third video frame is a critical data frame based on the feature evaluation characteristics.

[0029] Secondly, based on the same inventive concept, a feature recognition-based video quality diagnosis system is provided, the system comprising:

[0030] The first acquisition module is used to acquire the first video frame and the second video frame in the video to be diagnosed, and to determine the element evaluation features and temporal evaluation features based on the first video frame and the second video frame. The second video frame is the video frame after the first video frame.

[0031] The second acquisition module is used to acquire the third video frame in the video to be diagnosed, and determine whether the third video frame is a key data frame based on the feature evaluation characteristics.

[0032] The reconstruction module is used to perform resolution reconstruction and compression artifact processing on the third video frame when the third video frame is the key data frame, so as to obtain the target video frame. The third video frame is the video frame after the second video frame.

[0033] The first evaluation module is used to evaluate the target video frame according to the preset quality evaluation standard in order to obtain the quality evaluation result of the target video frame.

[0034] The second evaluation module is used to determine the quality diagnosis result of the video to be diagnosed based on the quality evaluation results and temporal evaluation characteristics of multiple target video frames.

[0035] In one embodiment of this application, the first acquisition module includes:

[0036] The first determining submodule is used to extract determining element features from the first video frame and the second video frame to obtain element evaluation features.

[0037] The second determination submodule is used to determine the temporal evaluation features based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

[0038] In one embodiment of this application, the second acquisition module includes:

[0039] The first scoring submodule is used to determine the first key data score of the third video frame based on the number of feature evaluation features.

[0040] The second scoring submodule is used to determine the second key data score of the third video frame based on the distribution location of the feature evaluation characteristics.

[0041] The third scoring submodule is used to determine whether the third video frame is a key data frame based on the first key data score and the second key data score.

[0042] In one embodiment of this application, the reconstruction module includes:

[0043] The comparison submodule is used to obtain the current resolution of the third video frame and determine the resolution correction parameters based on the comparison results between the current resolution of the third video frame and the resolutions of the first and second video frames.

[0044] The processing submodule is used to reconstruct the resolution and compress artifacts of the third video frame according to the resolution correction parameters in order to obtain the target video frame.

[0045] In one embodiment of this application, the first evaluation module includes:

[0046] The fourth scoring submodule is used to perform multi-dimensional evaluation of the target video frame according to the preset quality evaluation criteria in order to obtain quality evaluation scores in multiple dimensions.

[0047] The fifth scoring submodule is used to determine the quality evaluation result of the target video frame based on the weighted sum of the quality evaluation scores of each dimension and the corresponding weight coefficients.

[0048] In one embodiment of this application, the second evaluation module includes:

[0049] The correction submodule is used to dynamically adjust the temporal evaluation features based on the inter-frame change relationship of the feature features between each target video frame and the previous video frame.

[0050] The diagnostic submodule is used to determine the quality diagnostic result of the video to be diagnosed based on the quality evaluation results of multiple target video frames and the dynamically adjusted temporal evaluation characteristics.

[0051] In one embodiment of this application, the system further includes an execution module:

[0052] The execution module is used to continue executing the steps of acquiring the third video frame in the video to be diagnosed and determining whether the third video frame is a critical data frame based on the feature evaluation characteristics, if the third video frame is not a critical data frame.

[0053] In summary, the above-mentioned video quality diagnosis method and system based on feature recognition have the following technical advantages:

[0054] The feature-based video quality diagnosis method provided in this application effectively captures dynamic changes between frames in a video by extracting element evaluation features from the first and second video frames and combining them with temporal evaluation features. This not only helps identify the static quality of the video but also identifies changes in the temporal dimension, such as scene transitions and motion smoothness, making the video quality assessment more comprehensive and accurate. This application uses element evaluation features to determine whether the third video frame is a key data frame, solving the problem that traditional methods may overlook the quality assessment of certain key frames. If the third video frame is identified as a key data frame, further resolution reconstruction and artifact compression processing will be performed. This key frame-specific processing method can effectively improve the visual quality of the video and prevent some low-quality frames from affecting the overall video experience. In this application, the video is diagnosed not only based on the quality evaluation results of each target video frame but also by combining temporal evaluation features. This combination considers both the static quality of each frame and the dynamic changes on the timeline, accurately reflecting the overall video quality. Through multi-dimensional comprehensive analysis, the system can better identify potential problems in the video, such as inter-frame inconsistencies, dropped frames, or quality fluctuations, thereby improving the accuracy and reliability of video quality diagnosis. This application, through frame-by-frame processing and analysis, dynamically adjusts the quality restoration strategies for temporal features and keyframes based on the specific circumstances of the video. This not only improves the quality of each frame but also ensures the consistency and continuity of the overall video quality. In summary, this application, through frame-by-frame quality assessment of video frames, identification and processing of key data frames, dynamic adjustment of temporal assessment features, and quality diagnosis based on multi-dimensional features, effectively overcomes the shortcomings of existing technologies, providing a more accurate and comprehensive video quality assessment and solving the problems of insufficient dynamic change capture and inter-frame quality inconsistency in video quality assessment. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating the steps of a video quality diagnosis method based on feature recognition provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the functional modules of a feature recognition-based video quality diagnosis system provided in an embodiment of this application. Detailed Implementation

[0057] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0058] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0059] Hereinafter, the terms "first," "second," etc., are used for descriptive convenience only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units refer to two or more processing units.

[0060] Furthermore, in the embodiments of this application, "upper," "lower," "left," and "right" are not limited to the orientation of the components schematically placed in the accompanying drawings. It should be understood that these directional terms can be relative concepts, used for relative description and clarification, and can change accordingly depending on the orientation of the components in the accompanying drawings. In the accompanying drawings, for clarity, the thickness of layers and regions is exaggerated, and the dimensional proportions between the parts in the drawings do not reflect the actual dimensional proportions.

[0061] In the embodiments of this application, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral part; it can be a direct connection or an indirect connection through an intermediate medium. In addition, the term "electrical connection" can be a direct electrical connection or an indirect electrical connection through an intermediate medium.

[0062] In this application, the term "module" typically refers to a logically divided functional structure. A "module" can be implemented purely in hardware, or a combination of hardware and software. In this application, "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or both A and B existing simultaneously.

[0063] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0064] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0065] Currently, methods for identifying video quality involve extracting images from the video, unifying each image to a fixed size, inputting them into a convolutional neural network, and then having the neural network evaluate the quality score of each frame. The average of all scores is then used as the overall quality score of the video. This method has several drawbacks: First, it ignores the temporal dimension of the video, focusing only on the static features of a single frame, thus failing to detect time-related issues such as stuttering, dropped frames, or motion blur. Second, standardizing images to a fixed size may lead to the loss of detail in high-resolution videos or the introduction of additional artifacts, failing to accurately reflect video quality. Third, using a single average value as a video quality score lacks robustness and can easily mask fluctuations in local quality or low-quality segments. Furthermore, this method primarily relies on low-level feature extraction, lacking analysis of semantically relevant issues, such as the inability to identify the impact of accidental occlusion or color deviation on the scene. Additionally, an improperly set frame extraction strategy may result in insufficient representativeness of the overall video quality; extracting too few frames may miss important information, while extracting too many may increase computational redundancy. Finally, this method does not specifically detect compression artifacts (such as blockiness and color banding), and the model's generalization ability may be limited, exhibiting unstable performance under different resolutions, lighting conditions, or dynamic scenes.

[0066] Based on this, the applicant proposed the following technical concept: by extracting the element features and temporal features of video frames, the video quality is dynamically analyzed; key frames are selected and their resolution is reconstructed and compression artifacts are optimized to improve the visual quality of the frames; finally, the overall video quality is comprehensively evaluated based on the key frame quality and temporal features, thereby solving the problems of missing time information and inaccurate quality diagnosis in the prior art.

[0067] Reference Figure 1 The present invention provides a video quality diagnosis method based on feature recognition, which may specifically include the following steps:

[0068] S101: Obtain the first and second video frames from the video to be diagnosed, and determine the element evaluation features and temporal evaluation features based on the first and second video frames.

[0069] In this embodiment, after acquiring the video to be diagnosed, frame extraction is first performed. The first and second video frames are consecutive frames extracted sequentially from the video to be diagnosed in chronological order. These two frames are used to extract the video's element evaluation features and temporal evaluation features, providing basic information for subsequent keyframe selection and video quality assessment. The first and second video frames are chosen as the initial analysis objects because, as consecutive starting frames of the video, they can reflect the video's basic image quality and initial dynamic change characteristics. By extracting element evaluation features and temporal evaluation features from these two frames, the overall style, image quality level, and inter-frame motion trends of the video can be quickly captured, providing a reliable benchmark for subsequent keyframe selection. In addition, the starting frames are usually highly representative, effectively avoiding insufficient or one-sided information due to random frame selection without increasing computational complexity, providing better initial conditions for the comprehensiveness and accuracy of video quality assessment. The specific steps for determining the element evaluation features and temporal evaluation features based on the first and second video frames may include:

[0070] S1011: Extract feature determination elements from the first video frame and the second video frame to obtain feature evaluation features;

[0071] S1012: Determine the temporal evaluation features based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

[0072] In the implementations of S1011 and S1012, element features can refer to key attributes or core indicators used to evaluate video or image quality. These indicators reflect the basic picture quality and content characteristics of the video or image. Element features and temporal features have different roles and significance in video quality evaluation. Element features are mainly used to evaluate whether a single frame is a key image. By analyzing indicators such as sharpness, contrast, texture detail, color saturation, and noise level, it is determined whether the frame contains important information or has high representativeness, providing a basis for key frame selection. Temporal features, on the other hand, are used to analyze the dynamic relationship between frame changes, including motion trajectory, consistency, blur level, and the smoothness of scene transitions, reflecting the dynamic performance of the video in the time dimension. The two types of features complement each other. Element features ensure the accuracy of key frame selection, while temporal features guarantee a comprehensive evaluation of the dynamic playback quality of the video, thereby achieving accurate diagnosis and comprehensive judgment of video quality.

[0073] First, the first and second video frames are input into a feature extraction model or algorithm to extract core features related to video quality. These features can include, but are not limited to, multi-dimensional indicators such as sharpness, contrast, color saturation, noise level, and texture detail. Then, the differences or trends in the feature elements between the first and second frames are analyzed to reflect the dynamic characteristics of the video over time. For example: inter-frame motion: capturing the direction and speed changes of objects in the frame through optical flow algorithms or motion vector calculations. Inter-frame consistency: calculating the stability or consistency of feature changes, such as whether there are abrupt changes in brightness or color shifts. Blur variation: analyzing changes in sharpness between frames to assess whether problems such as motion blur occur. Through these analyses, a set of temporal evaluation features can be formed to quantify the dynamic performance of the video, such as inter-frame continuity, scene transition smoothness, and temporal correlation.

[0074] S102: Obtain the third video frame in the video to be diagnosed, and determine whether the third video frame is a key data frame based on the feature evaluation characteristics.

[0075] In this embodiment, after determining the element features and temporal evaluation features, it can be determined whether the following video frame is a keyframe. Keyframe extraction is necessary because extracting too few frames may miss important information (such as low-quality segments), while extracting too many frames increases computational overhead and leads to information redundancy. Specifically, element evaluation features reflect important indicators such as frame sharpness, contrast, texture detail, and information richness. Based on these indicators, key data frames with high representativeness or containing important information can be effectively selected. The selection of key data frames not only reduces redundant computation in subsequent processing but also allows resources to be concentrated on more in-depth quality optimization and evaluation of frames of significant importance, improving the efficiency and accuracy of video quality diagnosis. The specific steps may include:

[0076] S1021: Determine the first key data score of the third video frame based on the number of feature evaluation elements;

[0077] S1022: Determine the second key data score of the third video frame based on the distribution location of the feature evaluation characteristics;

[0078] S1023: Determine whether the third video frame is a key data frame based on the first key data score and the second key data score.

[0079] In the embodiments S1021 to S1023, by analyzing the number of feature evaluation features in the third video frame, it can be determined whether the frame contains sufficient key information. If the frame has a large number of feature evaluation features and rich information, it usually means that the frame has high representativeness and importance in the video, and therefore can be assigned a high first key data score. In this step, the distribution position of the feature evaluation features in the image is analyzed to determine whether these features are evenly distributed or concentrated in the key areas of the image. Frames with relatively even feature distribution or concentrated in important areas usually mean that they are of high importance in the overall video, and therefore can be assigned a high second key data score. Finally, based on the comprehensive evaluation result of the first and second key data scores, it is determined whether the frame is a key data frame. If both scores are high, it indicates that the frame has strong representativeness and information value, and can be used as a key frame in the video for subsequent processing and analysis. Through the above three steps, it is possible to accurately determine whether a video frame is a key data frame based on the number and distribution position of the feature evaluation features, thereby optimizing the accuracy of video quality diagnosis and analysis.

[0080] S103: When the third video frame is a key data frame, perform resolution reconstruction and compression artifact processing on the third video frame to obtain the target video frame.

[0081] If the third video frame is designated as the key data frame, then adaptive adjustments need to be made to it to obtain the target video frame. The purpose of adaptive adjustments is to optimize for specific characteristics and quality issues of the video frame, thereby improving the overall video performance. Specific steps may include:

[0082] S1031: Obtain the current resolution of the third video frame, and determine the resolution correction parameters based on the comparison results between the current resolution of the third video frame and the resolutions of the first and second video frames;

[0083] S1032: Based on the resolution correction parameters, perform resolution reconstruction and compression artifact processing on the third video frame to obtain the target video frame.

[0084] In the implementations of S1031 to S1033, unifying all images to a fixed size may lead to the loss of detail, especially for high-resolution videos, where scaling can blur key details or introduce artifacts. Determining resolution correction parameters based on the resolution comparison of the first and second video frames is primarily to ensure consistency in quality and visual performance across all frames in the video. If the resolution of the third video frame differs significantly from that of the first and second frames, it may cause inconsistencies between the images, affecting the smoothness and visual effect of the video. By comparing the resolutions of the first and second frames and ensuring that the resolution of the third frame is consistent with the first two frames, a smooth and uniform image effect can be achieved during video playback. Specifically, firstly, the resolution information of the third video frame, i.e., the pixel dimensions (such as width and height) of the frame, is extracted. Then, the resolution of the third video frame is compared with the resolutions of the first and second video frames. If the resolution of the third video frame differs significantly from the first two frames, it may indicate inconsistent image quality, leading to inconsistencies or a decrease in video quality during playback. Based on the comparison results, resolution correction parameters are generated. The correction parameter can be a scaling factor used to adjust the resolution of the third video frame to maintain consistency with other frames. If the resolution of the third frame is too low, it can be increased through interpolation or super-resolution techniques; if the resolution is too high, it can be appropriately reduced to avoid redundant data or computational burden.

[0085] Based on the obtained resolution correction parameters, adjust the resolution of the third video frame. If resolution needs to be increased, interpolation algorithms (such as bilinear interpolation, cubic interpolation, or more advanced super-resolution techniques) can be used to reconstruct details and make the image clearer. If resolution needs to be reduced, methods such as downsampling are used to ensure resolution consistency and no quality loss.

[0086] Compression artifact removal: After resolution processing, artifacts caused by compression (such as blocking and banding) may still exist. In this case, compression artifact removal techniques (such as deblocking filtering and banding removal) are needed to improve image quality. This process effectively removes visual defects caused by compression or transmission, resulting in a more natural and smooth video image.

[0087] After these two processing steps, the third video frame is adjusted to become the target video frame. The quality of this frame will be significantly improved, ensuring that it is consistent with the quality of other video frames, thereby providing high-quality image data for subsequent quality assessment and analysis.

[0088] In one feasible implementation, to obtain higher-quality target video frames, optimization can be achieved through multi-scale feature image fusion technology. Specifically, multi-scale feature image fusion combines image features from different resolutions or scales to integrate information from different scales, thereby improving image detail and visual quality. This method extracts features at multiple scales, capturing information at different levels of the image, such as low-level edge details and high-level semantic information, and then fuses this information into a complete image feature map. This approach can compensate for details that may be missing at a single scale, especially when processing frames with low resolution or insufficient detail, effectively restoring image clarity and detail, and improving the quality of the final target video frame. Multi-scale fusion also enhances image robustness, enabling it to maintain good visual performance under various viewing environments and devices, thus providing a more accurate and high-quality visual experience.

[0089] S104: Evaluate the target video frame according to the preset quality evaluation standard to obtain the quality evaluation result of the target video frame.

[0090] In this embodiment, after obtaining the target video frame, it can be evaluated according to preset quality evaluation criteria to obtain its quality evaluation result. Quality evaluation criteria typically include a series of objective and subjective indicators, such as sharpness, contrast, color accuracy, noise level, and detail recovery. These indicators comprehensively reflect the visual effect and information quality of the video frame. By using these criteria, the quality of the target video frame can be quantitatively analyzed, and its compliance with predetermined quality requirements can be assessed. Specific steps may include:

[0091] S1041: Based on the preset quality evaluation standards, perform multi-dimensional evaluation on the target video frame to obtain quality evaluation scores in multiple dimensions;

[0092] S1042: Determine the quality evaluation result of the target video frame based on the weighted summation of the quality evaluation scores of each dimension and the corresponding weight coefficients.

[0093] In this implementation, the target video frame is first evaluated across multiple dimensions according to preset quality assessment criteria. These dimensions may include sharpness, contrast, color accuracy, noise level, detail retention, and compression artifacts. For each dimension, the system calculates a corresponding quality score, reflecting the target video frame's performance in each aspect. For example, color accuracy can be scored based on color deviation. After obtaining the quality assessment scores for each dimension, a weighted calculation is performed based on the importance of each dimension. This is because different quality dimensions may have different impacts on the overall perception of video quality. For example, sharpness and detail retention may be more important to user experience, while compression artifacts may have a smaller impact. Therefore, the quality score for each dimension is multiplied by a corresponding weighting coefficient, which reflects the relative importance of each dimension in the overall quality assessment. By weighted summing of these quality scores, a comprehensive quality assessment result is obtained, reflecting the overall quality of the target video frame. This comprehensive score can be used for subsequent quality analysis, optimization, or diagnostic steps.

[0094] Through these steps, the system can perform a comprehensive and multi-faceted quality assessment of the target video frames, ensuring the accuracy and comprehensiveness of the video quality assessment and providing a reliable basis for subsequent processing and decision-making.

[0095] S105: Determine the quality diagnosis result of the video to be diagnosed based on the quality evaluation results and temporal evaluation characteristics of multiple target video frames.

[0096] In this embodiment, after obtaining multiple target video frames, the overall quality of the video to be diagnosed can be accurately diagnosed by comprehensively considering the quality evaluation results and temporal evaluation features of the multiple target video frames. Specifically, the quality evaluation result of each target video frame reflects the static image quality of that frame, while the temporal evaluation features describe the dynamic changes and inter-frame consistency of the video. By combining this information, the overall quality performance of the video can be comprehensively evaluated. For example, if some frames have low quality or abnormal temporal features (such as blurring, stuttering, etc.), it may affect the overall quality of the video. Finally, by combining the quality evaluation and temporal features of each frame, the system can derive the video quality diagnosis result, thereby providing a basis for further optimization or adjustment of the video quality. Specific steps may include:

[0097] S1051: Dynamically adjust the temporal evaluation features based on the inter-frame variation relationship of the feature characteristics between each target video frame and the previous video frame.

[0098] S1052: Determine the quality diagnosis result of the video to be diagnosed based on the quality evaluation results of multiple target video frames and the dynamically adjusted temporal evaluation characteristics.

[0099] In the embodiments of S1051 to S1052, the temporal evaluation features in the aforementioned steps are temporal evaluation features of the entire video determined by the first and second video frames. However, this approach does not effectively reflect the dynamic temporal changes of each target video frame. This is because temporal features derived from only the first and second video frames typically only describe the temporal change patterns at the beginning of the video, without considering dynamic changes that may occur in the middle and subsequent frames, such as acceleration, deceleration, stuttering, or other sudden quality issues. To overcome this problem, dynamic adjustments should be made to better reflect the dynamic characteristics of the video. Specifically, by analyzing the inter-frame changes in the feature characteristics between the target video frame and the previous video frame, the system can identify the change patterns between each frame in the video. These changes may include image blurring, motion smoothness, and inter-frame coherence, reflecting the dynamic performance of the video. By comparing the feature characteristics of each frame with the previous frame, the system can evaluate the quality changes between frames and make adjustments.

[0100] The purpose of dynamically adjusting temporal evaluation features is to ensure that video quality diagnosis can accurately reflect changes in inter-frame relationships, especially image distortion, stuttering, or other quality issues that may occur in dynamic scenes. By dynamically adjusting temporal features, the system can give higher weight to the dynamic performance of the video (such as motion blur, stuttering, etc.), thereby more accurately assessing the overall quality of the video.

[0101] The system integrates quality assessment results from multiple target video frames with dynamically adjusted temporal evaluation features. The quality assessment results reflect the static image quality of each frame, while the temporal evaluation features reflect the dynamic quality performance of the video. By combining these two, the system can more comprehensively evaluate the overall quality of the video.

[0102] Through weighted summation, accumulation, or other comprehensive calculation methods, the final quality diagnosis result of the video to be diagnosed is obtained. This result not only reflects the quality of each frame in the video, but also reflects the overall dynamic performance of the video based on the adjustment of temporal features. If the video has obvious dynamic quality problems (such as stuttering, frame drops, latency, etc.), the dynamic adjustment of temporal features will increase the weight of these problems, thereby ensuring that these problems can be identified in a timely manner and included in the final quality diagnosis result.

[0103] In one feasible implementation, the method further includes:

[0104] If the third video frame is not a critical data frame, continue with the steps of acquiring the third video frame in the video to be diagnosed and determining whether the third video frame is a critical data frame based on the feature evaluation characteristics.

[0105] In this embodiment, specifically, if the third video frame is determined to be a non-critical data frame during the aforementioned feature evaluation process (i.e., its image quality or temporal characteristics do not meet the preset standards), this frame may not play a critical role in the overall quality of the video, and therefore further optimization such as resolution correction and compression artifact processing is not required. However, to ensure the completeness and accuracy of video diagnosis, the system will continue to perform feature evaluation on the third video frame to confirm whether it should be marked as a critical data frame.

[0106] If, in subsequent evaluations, the third video frame meets the criteria to become a critical data frame (e.g., improved image quality or changes in temporal characteristics), the system will reassess whether further processing is needed. Through this mechanism, the system can flexibly respond to different situations in the video, ensuring that each frame receives appropriate quality assessment and processing, thereby improving the accuracy and reliability of the overall video quality diagnosis.

[0107] The feature-based video quality diagnosis method provided in this application effectively captures dynamic changes between frames in a video by extracting element evaluation features from the first and second video frames and combining them with temporal evaluation features. This not only helps identify the static quality of the video but also identifies changes in the temporal dimension, such as scene transitions and motion smoothness, making the video quality assessment more comprehensive and accurate. This application uses element evaluation features to determine whether the third video frame is a key data frame, solving the problem that traditional methods may overlook the quality assessment of certain key frames. If the third video frame is identified as a key data frame, further resolution reconstruction and artifact compression processing will be performed. This key frame-specific processing method can effectively improve the visual quality of the video and prevent some low-quality frames from affecting the overall video experience. In this application, the video is diagnosed not only based on the quality evaluation results of each target video frame but also by combining temporal evaluation features. This combination considers both the static quality of each frame and the dynamic changes on the timeline, accurately reflecting the overall video quality. Through multi-dimensional comprehensive analysis, the system can better identify potential problems in the video, such as inter-frame inconsistencies, dropped frames, or quality fluctuations, thereby improving the accuracy and reliability of video quality diagnosis. This application, through frame-by-frame processing and analysis, dynamically adjusts the quality restoration strategies for temporal features and keyframes based on the specific circumstances of the video. This not only improves the quality of each frame but also ensures the consistency and continuity of the overall video quality. In summary, this application, through frame-by-frame quality assessment of video frames, identification and processing of key data frames, dynamic adjustment of temporal assessment features, and quality diagnosis based on multi-dimensional features, effectively overcomes the shortcomings of existing technologies, providing a more accurate and comprehensive video quality assessment and solving the problems of insufficient dynamic change capture and inter-frame quality inconsistency in video quality assessment.

[0108] Secondly, based on the same inventive concept, and referring to... Figure 2 This application illustrates a feature-based video quality diagnostic system 200 according to an embodiment of the present application. The system includes:

[0109] The first acquisition module 201 is used to acquire the first video frame and the second video frame in the video to be diagnosed, and to determine the element evaluation features and temporal evaluation features based on the first video frame and the second video frame. The second video frame is the video frame after the first video frame.

[0110] The second acquisition module 202 is used to acquire the third video frame in the video to be diagnosed, and determine whether the third video frame is a key data frame based on the element evaluation features.

[0111] The reconstruction module 203 is used to perform resolution reconstruction and compression artifact processing on the third video frame when the third video frame is a key data frame, so as to obtain the target video frame. The third video frame is a video frame after the second video frame.

[0112] The first evaluation module 204 is used to evaluate the target video frame according to the preset quality evaluation standard in order to obtain the quality evaluation result of the target video frame.

[0113] The second evaluation module 205 is used to determine the quality diagnosis result of the video to be diagnosed based on the quality evaluation results and temporal evaluation characteristics of multiple target video frames.

[0114] In one embodiment of this application, the first acquisition module includes:

[0115] The first determining submodule is used to extract determining element features from the first video frame and the second video frame to obtain element evaluation features.

[0116] The second determination submodule is used to determine the temporal evaluation features based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

[0117] In one embodiment of this application, the second acquisition module includes:

[0118] The first scoring submodule is used to determine the first key data score of the third video frame based on the number of feature evaluation features.

[0119] The second scoring submodule is used to determine the second key data score of the third video frame based on the distribution location of the feature evaluation characteristics.

[0120] The third scoring submodule is used to determine whether the third video frame is a key data frame based on the first key data score and the second key data score.

[0121] In one embodiment of this application, the reconstruction module includes:

[0122] The comparison submodule is used to obtain the current resolution of the third video frame and determine the resolution correction parameters based on the comparison results between the current resolution of the third video frame and the resolutions of the first and second video frames.

[0123] The processing submodule is used to reconstruct the resolution and compress artifacts of the third video frame according to the resolution correction parameters in order to obtain the target video frame.

[0124] In one embodiment of this application, the first evaluation module includes:

[0125] The fourth scoring submodule is used to perform multi-dimensional evaluation of the target video frame according to the preset quality evaluation criteria in order to obtain quality evaluation scores in multiple dimensions.

[0126] The fifth scoring submodule is used to determine the quality evaluation result of the target video frame based on the weighted sum of the quality evaluation scores of each dimension and the corresponding weight coefficients.

[0127] In one embodiment of this application, the second evaluation module includes:

[0128] The correction submodule is used to dynamically adjust the temporal evaluation features based on the inter-frame change relationship of the feature features between each target video frame and the previous video frame.

[0129] The diagnostic submodule is used to determine the quality diagnostic result of the video to be diagnosed based on the quality evaluation results of multiple target video frames and the dynamically adjusted temporal evaluation characteristics.

[0130] In one embodiment of this application, the system further includes an execution module:

[0131] The execution module is used to continue executing the steps of acquiring the third video frame in the video to be diagnosed and determining whether the third video frame is a critical data frame based on the feature evaluation characteristics, if the third video frame is not a critical data frame.

[0132] It should be noted that the specific implementation of the video quality diagnosis system 200 based on feature recognition in this application refers to the specific implementation of the video quality diagnosis method based on feature recognition proposed in the first aspect of the aforementioned application, and will not be repeated here.

[0133] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0134] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0135] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0136] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0137] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, units, and processes described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0139] In the embodiments provided in this application, it should be understood that the disclosed systems, methods, and approaches can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video quality diagnosis method based on feature recognition, characterized in that, include: The first and second video frames in the video to be diagnosed are acquired, and element evaluation features and temporal evaluation features are determined based on the first and second video frames. The second video frame is a video frame after the first video frame. The temporal evaluation features include inter-frame motion features, inter-frame consistency features, and blurring change features. Obtain the third video frame from the video to be diagnosed, and determine whether the third video frame is a key data frame based on the number and distribution of the features evaluated by the aforementioned elements, including: Based on the number of the aforementioned feature evaluation elements, the first key data score of the third video frame is determined; Based on the distribution location of the evaluation features of the elements, the second key data score of the third video frame is determined; Based on the first key data score and the second key data score, determine whether the third video frame is a key data frame; In the case that the third video frame is a key data frame, resolution reconstruction and compression artifact processing are performed on the third video frame to obtain the target video frame, wherein the third video frame is a video frame after the second video frame. The target video frame is evaluated according to a preset quality evaluation standard to obtain the quality evaluation result of the target video frame. Based on the quality evaluation results of multiple target video frames and the temporal evaluation features, the quality diagnosis result of the video to be diagnosed is determined, including: The temporal evaluation features are dynamically adjusted based on the inter-frame variation relationship of the feature characteristics between each target video frame and the previous video frame. The quality diagnosis result of the video to be diagnosed is determined based on the quality evaluation results of multiple target video frames and the dynamically adjusted temporal evaluation characteristics.

2. The video quality diagnosis method according to claim 1, characterized in that, The step of determining the element evaluation features and temporal evaluation features based on the first video frame and the second video frame includes: The first video frame and the second video frame are subjected to feature extraction to obtain the feature evaluation features. The temporal evaluation features are determined based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

3. The video quality diagnosis method according to claim 1, characterized in that, The step of performing resolution reconstruction and compression artifact processing on the third video frame to obtain the target video frame includes: Obtain the current resolution of the third video frame, and determine the resolution correction parameters based on the comparison results between the current resolution of the third video frame and the resolutions of the first and second video frames; Based on the resolution correction parameters, the third video frame is subjected to resolution reconstruction and compression artifact processing to obtain the target video frame.

4. The video quality diagnosis method according to claim 1, characterized in that, The target video frame is evaluated according to a preset quality evaluation standard to obtain the quality evaluation result of the target video frame, including: The target video frame is evaluated in multiple dimensions according to the preset quality evaluation criteria to obtain quality evaluation scores in multiple dimensions. The quality evaluation result of the target video frame is determined by weighted summation of the quality evaluation scores of each dimension and their corresponding weight coefficients.

5. The video quality diagnosis method according to claim 1, characterized in that, The method further includes: If the third video frame is not a critical data frame, the process continues to obtain the third video frame from the video to be diagnosed and determine whether the third video frame is a critical data frame based on the element evaluation features.

6. A video quality diagnostic system based on feature recognition, characterized in that, The system for implementing the method of claim 1 includes: The first acquisition module is used to acquire the first video frame and the second video frame in the video to be diagnosed, and to determine the element evaluation features and the temporal evaluation features based on the first video frame and the second video frame, wherein the second video frame is the video frame after the first video frame. The second acquisition module is used to acquire the third video frame in the video to be diagnosed, and determine whether the third video frame is a key data frame based on the element evaluation features. The reconstruction module is used to perform resolution reconstruction and compression artifact processing on the third video frame when the third video frame is the key data frame, so as to obtain the target video frame, wherein the third video frame is a video frame after the second video frame. The first evaluation module is used to evaluate the target video frame according to a preset quality evaluation standard to obtain the quality evaluation result of the target video frame. The second evaluation module is used to determine the quality diagnosis result of the video to be diagnosed based on the quality evaluation results of multiple target video frames and the temporal evaluation features.

7. The video quality diagnostic system according to claim 6, characterized in that, The first acquisition module includes: The first determining submodule is used to extract determining element features from the first video frame and the second video frame to obtain the element evaluation features. The second determining submodule is used to determine the temporal evaluation features based on the inter-frame variation relationship between the feature features of the first video frame and the feature features of the second video frame.

8. The video quality diagnostic system according to claim 6, characterized in that, The second acquisition module includes: The first scoring submodule is used to determine the first key data score of the third video frame based on the number of evaluation features of the elements. The second scoring submodule is used to determine the second key data score of the third video frame based on the distribution location of the evaluation features of the elements. The third scoring submodule is used to determine whether the third video frame is a key data frame based on the first key data score and the second key data score.

Images