An accelerated method, system, and application for video quality assessment based on image group extraction.

Abstract

This invention discloses an accelerated method for video quality assessment based on image group extraction. The method includes the following steps: Step 1: A user uploads the original video and a video to be evaluated (re-encoded from the original video) to an object storage server via a client, and initiates a video quality assessment request; Step 2: The two types of videos uploaded in Step 1 are preprocessed and segmented to obtain multiple video slices; Step 3: The multiple video slices obtained by segmentation are respectively subjected to quality assessment, and the assessment results are stored in the object storage server; Step 4: The quality assessment results of the multiple video slices are anomaly handled and merged to obtain the quality score of the video to be evaluated. This invention also discloses an acceleration system for implementing the above acceleration method, and the application of the above acceleration method or system in accelerating video quality assessment.

Description

Technical Field

[0001] This invention belongs to the field of video transcoding and reference video quality assessment technology, and relates to an acceleration method, acceleration system and application for video quality assessment based on image group extraction. Background Technology

[0002] Over the past few decades, the ways and scenarios in which people watch videos have undergone tremendous changes, gradually shifting from traditional television systems to streaming video on desktop computers, laptops, and smartphones via the internet. According to a report by Tencent, streaming traffic is estimated to account for approximately 74% of all internet traffic by 2024. Streaming media can be broadly categorized into on-demand and live streaming. Both on-demand and live streaming require conversion based on the characteristics of the client's device; that is, the original video must be converted according to the video format, resolution, and network bandwidth supported by the client's device. This process is called video transcoding.

[0003] During video transcoding, video quality metrics, including video content clarity, color accuracy, and distortion levels, will inevitably change. Therefore, video quality assessment is essential, crucial for ensuring a high-quality viewing experience for users and driving streaming service providers to continuously improve video quality. First, it helps ensure that the videos requested by users remain at a high quality, allowing users to watch smoothly based on their real-time network and device conditions, reducing issues such as buffering, freezing, and blurring. This is vital for streaming service providers, as users always expect the best possible viewing experience. Second, video quality assessment helps identify technical problems or errors that negatively impact the overall video quality, such as encoding errors, bitrate anomalies, or resolution discrepancies. Finally, video quality assessment helps content creators and service providers identify areas for service improvement and obtain feedback to enhance the overall user viewing experience.

[0004] Video quality estimation is divided into three categories: full-reference video quality estimation, half-reference video quality estimation, and no-reference video quality estimation. Their meanings, advantages and disadvantages, and examples are shown in Table 1 below.

[0005] Table 1 Comparison of Video Quality Estimation Methods

[0006]

[0007] Full-reference video quality estimation and semi-reference video quality estimation are both methods based on a reference video. They estimate the quality of the test video by comparing the similarity between the reference video and the test video. Many semi-reference video quality estimation methods are derived from full-reference video quality estimation methods. The main difference is that semi-reference video quality estimation only compares local information of the video and is often used in video transmission scenarios. No-reference video quality estimation assesses quality by analyzing the video's own features. [1] The advantage of having a reference video quality estimate is that it can relatively accurately assess the quality of compressed video. SSIM (Structural Similarity) and PSNR (Peak Signal-to-Noise Ratio) are two common methods for calculating video quality assessment metrics. Their advantages are ease of understanding, simple calculation, and fast processing speed; their disadvantage is that they only focus on pixel values ​​and do not consider factors such as perceived color difference, sharpness, clarity, and dynamic range, failing to objectively reflect human perception of video quality. In contrast, VMAF (Video Multimethod Assessment Fusion), an emerging video quality assessment method combining neural network models, employs multiple algorithms for quality assessment and merges the results. Compared to traditional algorithms like SSIM and PSNR, it can more accurately reflect the viewer's actual perception of video quality. However, VMAF requires higher computational power, resulting in higher computational and time costs. Therefore, it is necessary to find a method that can guarantee the accuracy of video quality measurement with lower computational power and time costs.

[0008] Existing scene detection algorithms can be broadly categorized into two types: traditional pixel-based scene recognition algorithms and deep learning-based scene detection algorithms. Pixel-based scene recognition algorithms include OpenCV's built-in `scenedetect`. [4] Algorithm and ffmpeg's built-in selectgt [5] The algorithm calculates the numerical values ​​of each frame (composed of pixel saturation, brightness, and hue values) and compares the numerical differences between adjacent frames. Whenever the difference exceeds a certain threshold, the current frame is considered a scene change frame (the current frame's image has changed sufficiently compared to the previous frame). Deep learning-based scene detection algorithms include Fast R-CNN and YOLO. These algorithms perform object detection on each frame of the video, and when they identify the presence of specific objects or the occurrence of specific behaviors, they consider the current frame a scene change frame. Extracting video slices from each scene change frame and then evaluating the video quality can effectively reduce the number of computations.

[0009] In addition to the above, most video quality assessment services are currently deployed directly on virtual machines or containers. This deployment method offers high flexibility, allowing users to customize various configurations. However, to ensure stable operation under different needs, users also require significant manual intervention and resource planning. Considering the varying QoS requirements of different application scenarios, besides deploying services directly on virtual machines or containers, alternative deployment methods such as serverless computing offer the potential to reduce service costs while maintaining QoS. In serverless computing, server management and maintenance are handled by the cloud provider, allowing users to focus more on their business logic and improve development efficiency. The serverless architecture provides a relatively complete pay-as-you-go model, where users only pay for the resources they actually use. The disadvantage of serverless is its poor performance in scenarios with high real-time requirements. In summary, in scenarios with less stringent real-time requirements, serverless can effectively help users reduce operational and labor costs while ensuring service quality. Summary of the Invention

[0010] To address the shortcomings of existing technologies, the present invention aims to provide an accelerated method, system, and application for video quality assessment based on image group extraction. The accelerated method in this invention belongs to the field of full-reference and semi-reference video quality estimation (with reference video quality estimation) and relies on an effective scene detection method to select frames that can represent image groups, using these frames as extraction points to generate video slices.

[0011] The main technical problem addressed by this invention is how to effectively shorten the time cost of video quality assessment while ensuring that the video quality assessment results are highly consistent with those obtained by the human eye. A Group of Pictures (GOP) is a series of consecutive frames in a video. A GOP can contain one or more video slices as needed. By extracting video slices from the GOP, video quality assessment is performed on the video slices within the entire GOP to evaluate the overall quality of the GOP. Finally, statistical methods are used to calculate the overall video quality to be measured based on the quality of each GOP. Detailed explanations of GOPs and video slices are as follows:

[0012] A group of pictures (GOPs) refers to a series of consecutive frames in a video. It consists of one I-frame (a frame that can be decoded independently, called a keyframe) and several B / P frames (frames that cannot be decoded independently; the decoding of B and P frames depends on the I-frame). A GOP is the basic unit accessed by the video image encoder and decoder. Each video can be considered to be composed of multiple GOPs.

[0013] The video slice refers to a continuous video frame in a group of pictures (GOPs). This invention does not distinguish whether the video slice has the original keyframes of the image group, because when generating video slices, the video cutting tool will automatically create keyframes for the video slices.

[0014] Furthermore, this invention also addresses the differences in Service Level Agreements (SLAs) across various video application scenarios. Different application scenarios have varying quality of service requirements for video services. For example, cloud gaming and live streaming, which have high latency, require faster response times and concurrent processing capabilities, while storing and processing pre-recorded videos do not require high real-time performance. Considering these practical application scenarios, the second core problem addressed by this invention is to select appropriate cloud deployment methods to provide services for different application scenarios, saving on the economic costs of the service itself while ensuring QoS. In summary, the method proposed in this invention can reduce the computational load of video quality assessment while effectively ensuring the video quality assessment results. For different application scenarios, it selectively uses containerized server mode and serverless deployment services to reduce the economic costs of the service itself while ensuring QoS. For example, containerized server mode can be used for use cases and needs that are more sensitive to latency, while serverless deployment services can be used for use cases and needs that have no latency requirements or are not sensitive to latency.

[0015] This invention proposes an accelerated method and system for video quality assessment based on image group extraction to meet users' needs for efficient and accurate video quality assessment in various video application scenarios. When facing latency-sensitive requirements, deploying the service directly on a container can complete the video quality assessment within a limited time to ensure high QoS. When handling latency-independent requirements, the serverless design allows developers to focus on developing specific business functions without spending time on cloud server maintenance. Furthermore, the architecture, supporting elastic scaling and automatic expansion / contraction, can dynamically and elastically provide users with easy-to-use video quality assessment services while meeting application SLAs. When a user submits a video object access link (which provides access to the source video and the video to be assessed, re-encoded from the source video) as a parameter to the client, the client sends the request to the server. The server allocates appropriate storage and computing resources based on the video characteristics and calls pre-deployed and well-developed video segmentation, video quality assessment, and video quality analysis modules to process the video. These modules align the video, detect image groups, and use frame selection algorithms, including YOLO and Scenedetect, which can achieve the purpose of this invention, to select a portion of frames (i.e., scene change frames) based on the range of the image group. Then, the image group is cut into video slices using the scene change frames as cutoff points. Various video quality estimation algorithms, including SSIM and VMAF, are applied to the video slices. The video quality evaluation results of the video slices are weighted and summed to obtain the final video quality to be evaluated. The quality weight of each video slice is the ratio of the number of frames in the image group to which the slice belongs to the total number of video frames.

[0016] The architecture diagram of this invention is as follows: Figure 2As shown. The client first uploads the source video and the video to be re-encoded based on the source video for video quality assessment to the object storage server and obtains a response (access link for the video object). Then, it sends the access link along with the video quality assessment request. When the system is deployed in a serverless manner, the function controller (FaaSController) in the serverless system captures the request, encapsulates it as an event, and distributes the event to the container where the corresponding cloud function is deployed. When the system is deployed directly in a container, the container directly handles the access request. The container's video segmentation module includes functions such as reading the source video and re-encoded video based on the object access link of the event, aligning video timestamps, detecting image groups, using a frame selection algorithm to segment the video into corresponding video slices based on the content of the image groups, and storing them in the corresponding object storage server. The most important part of the video segmentation module is using a frame selection algorithm to select suitable frames for segmentation. Frame selection algorithms can be implemented in various ways, generally including algorithms based on deep learning to identify key frame features and traditional methods such as pixel-based scene recognition or naive isometric frame extraction algorithms. The core idea of ​​this algorithm is to select a small number of frames that can represent the complete image group for quality assessment, thereby obtaining a quality assessment result that can represent the entire video. During algorithm execution, the algorithm reads the video and finds each scene change frame. Each scene change frame represents all frames from the current scene change frame to the next scene change frame. The quality assessment result of all scene change frames in the image group is sufficient to represent the quality assessment result of the entire image group. The video segmentation module uses the scene change frames as cut points to generate video slices. The video quality assessment module's function includes selecting an appropriate video quality assessment algorithm to assess the quality of each video slice according to actual needs. After obtaining the video quality assessment result, the video quality assessment module transmits the result to the video quality analysis module. A schematic diagram illustrating the usage of the video quality segmentation module and the video quality assessment module is shown below. Figure 3 and Figure 4 As shown, these two figures illustrate the application of equidistant frame extraction and the application of deep learning algorithms for frame extraction. The right side of both figures depicts the actual video quality assessment process and results, that is, the quality assessment of all frames in the image group. Figure 2 The left side describes how frames 3 and 8 were selected for quality assessment based on an equidistant sampling algorithm. Figure 3The left side depicts a frame extraction algorithm based on a deep learning model, extracting frames 4 and 7 for quality assessment (the algorithm considers these two frames more representative of all frames in the two image groups). The middle of each image shows the score (x+y) / 2 obtained from assessing the quality of a partial frame and the score (X+Y) / 2 obtained from assessing the quality of the complete frame. The video quality analysis module waits for all video quality assessment modules to return results, then performs a weighted average of all quality assessment results to obtain the final video quality result and returns it; the weight of the video slice is the proportion of the number of frames in the image group to the total number of video frames.

[0017] The process of this invention is as follows Figure 5 As shown, the specific steps for system implementation are as follows:

[0018] Step 1: The user uploads the video to the object storage server and initiates a video quality assessment request.

[0019] The client uploads the source video and the video (re-encoded from the source video) whose quality needs to be evaluated to the object storage server and obtains the corresponding return result (access link to the object storage). The client initiates a video quality evaluation request, attaching the access link to the video in object storage to the request. The video quality evaluation service can be deployed in serverless mode and / or directly in a container and / or directly on-premises. When the video quality evaluation service is deployed in serverless mode, the FaaS Controller captures the request and encapsulates it as an event, and initializes the corresponding container based on the request. When the video quality evaluation service is deployed directly in a container, the container directly captures the request. The video quality evaluation service of this invention can also be deployed directly on-premises.

[0020] Step 2: The video cutting module preprocesses and cuts the video.

[0021] The video segmentation module retrieves the source video and the video awaiting quality evaluation from the object storage server based on the video's link in object storage. This module aligns the two video segments by timestamp and then crops out the common portion. Finally, it uses video analysis tools (such as ffmpeg)... [2] mediainfo [3] The system uses open-source tools to locate the display timestamps and frame sequence numbers of keyframes in the source video, and determines the corresponding image groups based on the timestamps and sequence numbers. Then, video processing tools select one or more frames from the image groups of both the source video and the video to be quality evaluated, and cut them into several video slices. The video cutting module records the number of video frames corresponding to each video slice and uploads all of this information to an object storage server.

[0022] Step 3: The video quality assessment module performs quality assessments on multiple video slices in parallel.

[0023] The container allocates an appropriate number of video quality assessment modules based on the number of corresponding video segments. Each video quality assessment module retrieves the corresponding video slice from the object storage server and uses a video quality assessment tool to evaluate the quality of the video slice. The specific quality assessment dimensions depend on the actual application method, which may include PSNR, SSIM, VIF, VMA, etc., with different assessment methods targeting different metrics. PSNR evaluates the difference in grayscale values ​​for each frame of a video; a higher PSNR indicates higher video quality. SSIM evaluates the structural features of each frame of a video scene. This algorithm compares the brightness, contrast, and structural similarity of corresponding images; a higher SSIM indicates higher video quality. VIF evaluates the ratio of reference image information (common information between the source and recoded video images in the HVS channel) to distortion image information (common information between the original and recoded video images in the distortion channel); a higher VIF indicates higher video quality. VMAF is also a commonly used video quality assessment method. VMAF includes three main metrics: VIF, DLM, and TI. DLM evaluates the loss of detail, i.e., the loss of useful visual information affecting content visibility; TI measures temporal motion metrics. In general, a higher VMAF indicates higher video quality. In actual video quality assessment, an appropriate video assessment algorithm is selected according to the requirements. The algorithm evaluates the temporal and spatial information of the video. After obtaining the quality assessment results for each video slice, the corresponding video quality score for each video slice is stored in an object storage server.

[0024] Step four: The video quality analysis module performs anomaly handling and merging on the quality results of the video segments.

[0025] The video quality analysis module retrieves the video quality score and the number of video frames for each video slice from the object storage server. First, it removes any anomalies in the video quality results. These anomalies stem from noise generated during the image group segmentation process. Current video segmentation tools cannot accurately cut video slices, resulting in more frames remaining in the original or re-encoded video slices. Existing video quality assessment algorithms perform a one-to-one quality evaluation on all frames; if there are extra frames that cannot be evaluated one-to-one, their quality result will be 0, leading to these anomalies. The anomaly handling process involves extracting the quality results of the first n frames (typically n = 2) for each video slice and calculating the average. Then, the proportion of image group frames to the total number of video frames is used as the quality weight for the video slice's quality score. A weighted score is calculated as the quality score of the video to be evaluated. After returning the quality score to the user, the system releases all allocated resources.

[0026] This invention also proposes an acceleration system for implementing the above-mentioned accelerated video quality assessment method, the acceleration system comprising:

[0027] Clients, containers, and object storage servers;

[0028] The client is used to upload videos and initiate video quality assessment requests;

[0029] The container is equipped with a processing module that includes a video cutting module, a video quality assessment module, and a video quality analysis module, which are used to cut, assess, and analyze the uploaded videos.

[0030] The video cutting module is used to cut the uploaded video into video slices for subsequent module processing.

[0031] The video quality assessment module is used to assess the quality of video content in video slices using a video quality estimation algorithm to obtain a quality assessment score.

[0032] The video quality analysis module is used to remove abnormal data and perform weighted scoring to obtain the final quality score of the video to be evaluated.

[0033] The object storage server is used to store the original video and the video to be evaluated, which is re-encoded based on the original video, as well as their respective video information and quality evaluation scores.

[0034] Furthermore, when the system is deployed in a serverless manner, the acceleration system also includes a function controller (FaaS Controller) that can capture video quality assessment requests, encapsulate the requests as events, and then distribute the events to containers.

[0035] This invention also proposes the application of the above-mentioned video quality assessment acceleration method or acceleration system in accelerating video quality assessment.

[0036] The beneficial effects of this invention include: This invention proposes a video quality assessment method and system based on image group extraction. Existing technologies cannot guarantee the accuracy of video quality measurement with low computing power and time costs, and are not flexible enough in deploying quality assessment services. This invention, through methods such as frame selection using non-deep learning methods and serverless or direct container deployment, can achieve high-accuracy video quality assessment with lower time and economic costs. Compared to existing technologies, the assessment time of this invention can be reduced to less than 50% of the original time, and the relative error of the assessment is less than 1%. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is an illustration of the image group and video slice of the present invention.

[0039] Figure 2 This is the system architecture diagram of the present invention.

[0040] Figure 3 This is a schematic diagram of the module functions under the equidistant frame extraction method of the present invention.

[0041] Figure 4 This is a schematic diagram of the module functions under the deep learning algorithm frame extraction of this invention.

[0042] Figure 5 This is a swimlane diagram of the system of the present invention.

[0043] Figure 6 This is a reference figure for Embodiment 1 of the present invention.

[0044] Figure 7 This is a reference figure for Embodiment 2 of the present invention.

[0045] Figure 8 This is a reference figure for Embodiment 3 of the present invention.

[0046] Figure 9 This is a reference figure for Embodiment 4 of the present invention.

[0047] Figure 10 This is a reference figure for Embodiment 5 of the present invention.

[0048] Figure 11 This is a reference figure for Embodiment 6 of the present invention. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the present invention are all common knowledge and general knowledge in the art, and the present invention does not have any particular limitations.

[0050] This invention provides an accelerated method for video quality assessment based on image group extraction, the method comprising the following steps:

[0051] Step 1: The user uploads the original video and the video to be evaluated, which is re-encoded based on the original video, to the object storage server through the client, and initiates a video quality evaluation request;

[0052] Step 2: Preprocess and cut the two types of videos uploaded in Step 1 to obtain multiple video slices;

[0053] Step 3: Perform quality assessment on each of the multiple video slices obtained from the cutting process, and store the assessment results in an object storage server;

[0054] Step 4: Perform anomaly processing and merging on the quality assessment results of multiple video slices to obtain the quality score of the video to be evaluated.

[0055] The present invention also provides an acceleration system for implementing the above-described acceleration method, and the application of the above-described acceleration method or acceleration system in accelerated video quality assessment.

[0056] Specifically, this invention designs an accelerated video quality assessment method and system based on image group extraction. Compared with traditional video quality assessment methods, this invention can achieve a quality assessment result with a relative error of less than 1% with less than 50% of the time and economic cost. During video encoding, a new keyframe is only encoded when there is a significant scene change or when there are a sufficient number of non-keyframes since the previous keyframe. Therefore, by identifying keyframes, dividing the video into image groups, and further identifying scene-changing frames within these image groups, it can be ensured that whenever the scene depicted by a frame changes significantly, it will be divided into different image groups, thus guaranteeing that the video slices obtained by extracting image groups are sufficiently representative in content. Since video frames are continuous on the screen, the quality assessment scores of adjacent frames can be considered approximately continuous. By extracting frames from the middle of each video slice, or by using machine learning algorithms to select a small number of suitable representative frames, the number of calculations can be reduced, thereby accelerating video quality assessment.

[0057] The method proposed in this invention belongs to the category of reference-based video quality assessment methods. In general, compared to previous reference-based video quality estimation algorithms, and relative to PSNR, SSIM, or MS-SSIM algorithms, the video quality assessment method trained by an accelerated neural network model using image group extraction encompasses more visually relevant factors, such as chromatic aberration, sharpness, clarity, and dynamic range. Furthermore, this algorithm also incorporates temporal information, measuring video quality from a temporal perspective, which is more consistent with human perception; that is, the video quality it calculates better represents the true perception of the human eye. When using the video quality assessment method trained by the neural network in the system to evaluate videos, the method of extracting consecutive frames to create video slices ensures that temporal information is not lost, thus achieving an evaluation result that is more consistent with the human eye. Compared to video quality assessment algorithms trained by neural network models without image group extraction acceleration, the image group extraction acceleration method ensures that the relative error of the video quality assessment result is less than 1%.

[0058] Due to the complexity of neural networks, deep learning-based frame selection methods often incur additional overhead (ranging from a few seconds to tens of seconds depending on the actual video length). Therefore, non-deep learning frame selection methods can be used to achieve stable and negligible overhead (less than 1 second). In summary, this method significantly reduces runtime while maintaining essentially the same accuracy. Furthermore, the image group extraction acceleration method drastically reduces the algorithm's computational load. Based on an average extraction ratio of 1 frame per 10 frames, its computational load is approximately 10% of the original. Compared to ordinary video quality assessment methods with acceleration enabled, the actual runtime after image group extraction acceleration is reduced to less than half. Even on machines with average configurations, video quality assessment can be completed in a relatively short time.

[0059] Deploying video quality assessment services using different methods in various video application scenarios can save costs while ensuring system service quality. In latency-independent scenarios, serverless deployment offers greater scalability compared to on-demand deployment, thus saving costs. In latency-sensitive scenarios, deploying services directly in containers meets high QoS requirements. This is because, besides significant differences in video duration and content, different video application scenarios have varying service quality requirements. Choosing the optimal deployment method based on the service type, request volume, and specific video content is more cost-effective. In specific application scenarios, image groups identified through scene change frame recognition may generate multiple video slices. Systems using both deployment methods can allocate corresponding threads for video quality assessment based on the actual number of video slices. Furthermore, since there is no coupling between different image groups, the video quality assessment module can achieve maximum parallelism, enabling the majority of videos to complete quality assessments quickly.

[0060] The video quality assessment methods used in the following embodiments include traditional pixel-based and structure-based quality assessment methods, as well as novel deep learning-based video quality assessment methods. The former includes common and computationally inexpensive methods such as PSNR and SSIM, while VMAF is a leading example of the latter. The video frame selection methods in the embodiments include selection methods based on video frame position and selection methods based on deep learning. The deployment of these embodiments can be achieved through direct container deployment, serverless deployment, or local deployment.

[0061] Due to the numerous combinations of deployment methods, quality assessment methods, and frame selection methods, six representative deployment schemes will be described in detail below. While other schemes are not described in detail, their deployment methods, video quality assessment methods, and frame selection methods are all combinations of the above. Examples 1 to 4 illustrate the specific implementation of different video quality assessment methods and different frame selection methods in a serverless system scenario; Example 5 illustrates the specific implementation of direct container deployment. Example 6 illustrates the specific implementation in a local scenario. Although Examples 1 to 5 only mention one client and one serverless system container in the descriptions and corresponding diagrams, the actual number of clients and serverless system containers can be arbitrary in actual deployment.

[0062] Example 1: The video quality assessment method uses a pixel-based assessment method, the frame selection method uses a non-deep learning selection method, and the deployment is in serverless computing mode.

[0063] like Figure 6As shown, the client has the source video source.mp4 and the target.mp4, which is generated by re-encoding the source video and is waiting to have its video quality checked. The specific implementation steps are as follows:

[0064] 1. The client uploads source.mp4 and target.mp4 to the object storage server, obtains the corresponding video access links, and sends the access links along with the video quality assessment request. FaaSController captures the request, encapsulates it as an event, and initializes the corresponding container based on the request.

[0065] 2. The video splitting module accesses the object storage server via the access link to read the source video and obtains several image groups for keyframe segmentation. Within each image group, a non-deep learning video frame selection method is used to select several video frames that can represent that image group, and the corresponding frames are then segmented into video slices.

[0066] 3. The video quality assessment module (evaluatemodule) uses a traditional pixel-based video quality assessment method to evaluate the video quality of the corresponding video slices and obtain the video quality score for each video slice.

[0067] 4. The video quality analysis module (analyzemodule) processes and analyzes the video quality scores of each video slice, finally obtaining the video quality score of target.mp4 relative to source.mp4, and returns the result to the client.

[0068] 5. Program execution ends. Experiments show that the actual runtime of the accelerated system in this embodiment is less than half that of the traditional method, and the relative error of the quality assessment results is less than 1%. These optimizations stem from the parallelism inherent in the accelerated system itself.

[0069] Example 2: The video quality assessment method uses a pixel-based method, the frame selection method uses a deep learning-based method, and the deployment is in serverless computing mode.

[0070] like Figure 7 As shown, the client has the source video source.mp4 and the target.mp4, which is generated by re-encoding the source video and is waiting to have its video quality checked. The specific implementation steps are as follows:

[0071] 1. The client uploads source.mp4 and target.mp4 to the object storage server, obtains the corresponding video access links, and sends the access links along with the video quality assessment request. FaaSController captures the request, encapsulates it as an event, and initializes the corresponding container based on the request.

[0072] 2. The video segmentation module accesses the object storage server via the access link to read the source video and obtains several image groups for keyframe segmentation. Within each image group, a specific deep learning method is used to select several video frames that can represent that image group, and the corresponding frames are segmented into video slices.

[0073] 3. The video quality assessment module (evaluatemodule) uses a traditional pixel-based video quality assessment method to evaluate the video quality of the corresponding video slices and obtain the video quality score for each video slice.

[0074] 4. The video quality analysis module (analyzemodule) processes and analyzes the video quality scores of each video slice, finally obtaining the video quality score of target.mp4 relative to source.mp4, and returns the result to the client.

[0075] 5. Program execution ends. Experiments show that the actual runtime of the accelerated system in this embodiment is mostly below 80% of that of the traditional method, with a small portion exceeding it; the relative error of the quality assessment results is less than 1%. This is because deep learning-based selection methods incur significant overhead when processing long videos.

[0076] Example 3: The video quality assessment method uses a deep learning-based assessment method, while the frame selection method uses a non-deep learning selection method, deployed in a serverless computing mode.

[0077] like Figure 8 As shown, the client has the source video source.mp4 and the target.mp4, which is generated by re-encoding the source video and is waiting to have its video quality checked. The specific implementation steps are as follows:

[0078] 1. The client uploads source.mp4 and target.mp4 to the object storage server, obtains the corresponding video access links, and sends the access links along with the video quality assessment request. FaaSController captures the request, encapsulates it as an event, and initializes the corresponding container based on the request.

[0079] 2. The video splitting module accesses the object storage server via the access link to read the source video and obtains several image groups for keyframe segmentation. Within each image group, a non-deep learning video frame selection method is used to select several video frames that can represent that image group, and the corresponding frames are then segmented into video slices.

[0080] 3. The video quality assessment module (evaluatemodule) uses a deep learning model-based video quality assessment method to evaluate the video quality of the corresponding video slices and obtain the video quality score of the video slice.

[0081] 4. The video quality analysis module (analyzemodule) analyzes and aggregates the video quality scores of each video slice, finally obtaining the video quality score of target.mp4 relative to source.mp4, and returns the result to the client.

[0082] 5. Program execution ends. Experiments show that the actual runtime of the accelerated system in this embodiment is less than half that of the traditional method, and the relative error of the quality assessment results is less than 1%. These optimizations stem from the parallelism inherent in the accelerated system itself.

[0083] Example 4: The video quality assessment method uses a deep learning-based assessment method, the frame selection method uses a deep learning-based selection method, and the deployment is in serverless computing mode.

[0084] like Figure 9 As shown, the client has the source video source.mp4 and the target.mp4, which is generated by re-encoding the source video and is waiting to have its video quality checked. The specific implementation steps are as follows:

[0085] 1. The client uploads source.mp4 and target.mp4 to the object storage server, obtains the corresponding video access links, and sends the access links along with the video quality assessment request. FaaSController captures the request, encapsulates it as an event, and initializes the corresponding container based on the request.

[0086] 2. The video segmentation module accesses the object storage server via the access link to read the source video and obtains several image groups for keyframe segmentation. Within each image group, a specific deep learning method is used to select several video frames that can represent that image group, and the corresponding frames are segmented into video slices.

[0087] 3. The video quality assessment module (evaluatemodule) uses a deep learning-based video quality assessment model to evaluate the video quality of the corresponding video slices and obtain the video quality score for each video slice.

[0088] 4. The video quality analysis module (analyzemodule) processes and analyzes the video quality scores of each video slice, finally obtaining the video quality score of target.mp4 relative to source.mp4, and returns the result to the client.

[0089] 5. Program execution ends. Experiments show that the actual runtime of the accelerated system in this embodiment is mostly below 80% of that of the traditional method, with a small portion exceeding it; the relative error of the quality assessment results is less than 1%. This is because deep learning-based selection methods incur significant overhead when processing long videos.

[0090] Example 5: Direct Container Deployment

[0091] To meet the high QoS requirements of latency-sensitive video applications, this example describes an implementation scenario using direct container deployment of services. Figure 10 As shown, the client has the source video source.mp4 and the target.mp4, which is generated by re-encoding the source video and is waiting to have its video quality checked. The specific implementation steps are as follows:

[0092] 1. The client uploads source.mp4 and target.mp4 to the object storage server, obtains the corresponding video access links, and sends the access links along with the video quality assessment request. The container directly captures the request.

[0093] 2. The video split module accesses the object storage server via the access link to read the source video and obtains several image groups for keyframe segmentation. Within each image group, a frame selection method is used to select frames and cut them into video slices.

[0094] 3. The video quality assessment module (evaluatemodule) uses video quality assessment methods to evaluate the video quality of the corresponding video slices and obtain the video quality score for each video slice.

[0095] 4. The video quality analysis module (analyzemodule) processes and analyzes the video quality scores of each video slice, finally obtaining the video quality score of target.mp4 relative to source.mp4, and returns the result to the client.

[0096] 5. Program execution complete. Direct container deployment accelerates the actual runtime of the system compared to serverless system instances. This is because, in serverless deployments, the container's cold start takes several hundred milliseconds. However, for longer videos, this time overhead is negligible. The relative error is independent of the deployment method and remains below 1%.

[0097] Example 6: Specific Implementation Scenarios for Local Deployment

[0098] While the method described in this invention is well-suited for serverless scenarios, it can also be deployed locally. For example... Figure 11 As shown, the local machine contains the relevant code for the video cutting module, quality assessment module, and quality processing module, as well as the source video source.mp4 and the target.mp4 generated from the re-encoded source video and awaiting quality verification. Specific implementation steps are described below:

[0099] 1. Since source.mp4 and target.mp4 are stored locally, and the relevant code modules are also deployed locally, the video cutting module will directly read the local source.mp4 and target.mp4 to obtain the image group, and use the frame selection method to select the frames in the image group to cut the frames into video slices.

[0100] 2. The video quality assessment module (evaluatemodule) uses video quality assessment methods to evaluate the video quality of the corresponding video slices and obtain the video quality score for each video slice.

[0101] 3. The video quality analysis module (analyzemodule) processes and analyzes the video quality scores of each video slice, and finally obtains the video quality score of target.mp4 relative to source.mp4.

[0102] 4. Program execution terminates. In a local deployment environment, the actual runtime of the accelerated system is also faster than that of the serverless system, with the relative error consistently remaining below 1%.

[0103] References

[0104] [1] Chu Jiang, Chen Qiang, Yang Xichen. A review of full-reference image quality assessment [J]. Computer Applications Research, 2014, 31(01):13-22.

[0105] [2]AboutFFmpeg.https: / / ffmpeg.org / about.html.

[0106] [3]MediaInfo. https: / / mediaarea.net / en / MediaInfo.

[0107] [4]SceneDect. https: / / www.scenedetect.com /

[0108] [5] FFmpeg-Filter. https: / / ffmpeg.org / ffmpeg-filters.html#select_002c- aselect

[0109] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of this invention are included in this invention and are protected by the appended claims.

Claims

1. An accelerated method for video quality assessment based on image group extraction, characterized in that, The method includes the following steps: Step 1: The user uploads the original video and the video to be evaluated, which is re-encoded based on the original video, to the object storage server through the client, and initiates a video quality evaluation request; In step one, after the video is uploaded to the object storage server, an access link to the object storage can be obtained; the deployment of the video quality assessment service includes serverless deployment and / or direct container deployment and / or direct local deployment; when the video quality assessment service is deployed in serverless mode, the function controller captures the request and encapsulates it into an event, and initializes the corresponding container according to the request; when the video quality assessment service is deployed directly in the container, the container directly captures the request. Step 2: Preprocess and cut the two types of videos uploaded in Step 1 to obtain multiple video slices; Step 3: Perform quality assessment on each of the multiple video slices obtained from the cutting process, and store the assessment results in an object storage server; Step 4: Perform anomaly processing and merging on the quality assessment results of multiple video slices to obtain the quality score of the video to be evaluated.

2. The acceleration method as described in claim 1, characterized in that, In step two, the preprocessing refers to aligning the original video and the video to be evaluated, which is re-encoded based on the original video, according to the timestamps, cropping out the common parts, using video analysis tools to locate the display timestamps of key frames in the source video and the sequence numbers of the frames in the video, and determining the corresponding image groups based on the timestamps and sequence numbers; the video analysis tools are open source tools, including ffmpeg and mediainfo.

3. The acceleration method as described in claim 1, characterized in that, In step two, a frame selection algorithm is used to select scene change frames based on the range of the image group. Using the scene change frames as truncation points, the original video and the video to be evaluated, which is re-encoded based on the original video, are cut into one or more video slices with a length of one or more frames. The video slices and the frame count data contained in the image group are then uploaded to the object storage server. The frame selection algorithm includes YOLO, Scenedetect, pixel-based scene recognition frame extraction algorithm, and equidistant frame extraction algorithm.

4. The acceleration method as described in claim 1, characterized in that, In step three, a video quality assessment algorithm is used to evaluate each video slice. The video quality assessment process evaluates the temporal and spatial information of the video to obtain the quality assessment result of each video slice, and stores the video quality score corresponding to each video slice in the object storage server.

5. The acceleration method as described in claim 1, characterized in that, In step four, the video quality assessment results stored in the object storage server are obtained, and the abnormal parts in the video quality assessment results are removed. The proportion of the number of frames in the image group to the total number of video frames is used as the quality weight of the video slice quality score, and the weighted score of the video slice quality score is calculated as the quality score of the video to be evaluated. The abnormal parts refer to the abnormal results that occur when the original video and the video frames of the re-encoded video cannot be matched one by one due to noise generated during the image group cutting process.

6. An acceleration system for implementing the acceleration method as described in any one of claims 1-5, characterized in that, The acceleration system includes: Clients, containers, object storage servers; The client is used to upload videos and initiate video quality assessment requests; The container is equipped with a processing module that includes a video cutting module, a video quality assessment module, and a video quality analysis module, which are used to cut, assess, and analyze the uploaded videos. The video cutting module is used to cut the uploaded video into video slices for subsequent module processing. The video quality assessment module is used to assess the quality of video content in video slices using a video quality estimation algorithm to obtain a quality assessment score. The video quality analysis module is used to remove abnormal data and perform weighted scoring to obtain the final quality score of the video to be evaluated. The object storage server is used to store the original video and the video to be evaluated, which is re-encoded based on the original video, as well as their respective video information and quality evaluation scores.

7. The acceleration system as described in claim 6, characterized in that, When the system is deployed in a serverless manner, the acceleration system also includes a function controller that can capture video quality assessment requests, encapsulate the requests as events, and then distribute the events to containers.

8. The application of the acceleration system as described in claim 6 or 7 in accelerating video quality assessment.

Images