Method for video processing effect comparison and evaluation

Abstract

This invention discloses a method for comparing and evaluating video processing effects, characterized by comprising: S10, pre-setting the system: setting up a video processing effect comparison and evaluation system; S20, configuring and loading: the user starts the set video processing effect comparison and evaluation system, configures the layout and number of playback areas, and loads the video files to be compared in each area; S30, adaptive and initial synchronization: automatically scaling each video to fit its playback area, reading the frame rate of each video, and automatically calculating the initial time-frame mapping relationship so that the multiple screens are roughly synchronized at the start of playback; S40, performing synchronous comparison operation; S50, recording the evaluation process: during or after the comparison, the following operations are performed: at key frames, the current state of the multiple screens is saved using a screenshot unit, and a screen recording unit is started to record a video containing the complete operation and playback process as evaluation evidence for archiving.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of multimedia testing and evaluation, and particularly relates to a method for video processing effect comparison and evaluation. BACKGROUND

[0002] With the development of mobile Internet, users watching videos on mobile phones and other vertical screen devices has become mainstream. Therefore, a large number of traditional horizontal screen shooting video contents need to be converted into vertical screen format through algorithms such as object tracking and cutting, audio tracking and cutting, and center cutting. However, after horizontal-to-vertical processing, the resolution, frame ratio, and even frame rate of the video may change, and it is difficult to accurately evaluate the content retention, visual comfort, and overall effect of the video using objective peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators.

[0003] At present, the evaluation of horizontal-to-vertical algorithm effect mainly relies on artificial subjective viewing. The existing video playing and comparison methods have the following significant shortcomings: (1) synchronization difficulty: the resolution and frame rate of the source video and the processed video are different, and it is difficult to achieve accurate frame-level synchronous playing and comparison in ordinary players, and manual alignment is time-consuming and laborious; (2) comparison inconvenience: there is a lack of efficient tools to simultaneously display the source video and the output videos of multiple different algorithms (such as A / B testing), and it is not possible to perform intuitive and convenient parallel comparison; (3) low efficiency: the evaluation process relies on real-time operation of testers, and it is not possible to process test sets in batches. Each evaluation needs to repeat the tedious steps of file loading, manual synchronization, and playing control; (4) process is not reproducible: the observation points and comparison moments during subjective evaluation are difficult to record and save, resulting in a lack of traceability of the evaluation results, and the evaluation conclusions of different personnel or at different times may not be consistent. SUMMARY

[0004] In view of the above problems, the present application provides a method for video processing effect comparison and evaluation, which is used to support multi-video synchronous playing, convenient comparison, and record the evaluation process, so as to improve the efficiency and reliability of horizontal-to-vertical algorithm evaluation.

[0005] To solve the above technical problems, the present application adopts the following technical solution: A method for video processing effect comparison and evaluation, comprising the following steps: S10, system pre-setting: setting a video processing effect comparison and evaluation system; S20, configuration and loading: a user starts the set video processing effect comparison and evaluation system, configures the layout and number of playing areas, and loads video files to be compared in each area; S30, adaptive and initial synchronization: automatically scale each video to fit its playing area, read the frame rate of each video, automatically calculate the initial time-frame mapping relationship, and make the multi-picture roughly synchronized at the beginning of playing; S40, synchronization comparison operation: user subjective evaluation comparison, including at least one of the following: using play / pause, fast forward / fast reverse for macro browsing; enabling linked playing mode to synchronize observation of other videos based on one video; using manual synchronization unit for frame-level fine adjustment to align specific key actions or scene transition frames; in separate playing mode, independently control a certain area and focus on the details of a specific video; S50, record evaluation process: during or after the comparison process, the following operations are performed: at the key frame, save the current multi-picture state using the screenshot unit and start the screen recording unit to record a video containing the complete operation and playing process as evaluation evidence.

[0006] Preferably, in S10, the preset video processing effect comparison and evaluation system includes a picture layout and file loading module, a video adaptive and scaling module, an intelligent synchronization control module, a playing linkage and control module, and an evaluation recording module. The intelligent synchronization unit further includes an automatic synchronization unit and a manual synchronization unit. The evaluation recording module further includes a screenshot unit and a screen recording unit.

[0007] Preferably, the picture layout and file loading module is used to divide the playing window into N playing areas that can be freely adjusted in size and position, N≥2, and each playing area can independently load a video file.

[0008] Preferably, the video adaptive and scaling module is used to perform frame scaling on input videos of different resolutions and different aspect ratios to adapt to their respective assigned playing areas while maintaining the original proportion without distortion or according to user settings for cropping / filling.

[0009] Preferably, the automatic synchronization unit is used to automatically calculate and establish the time mapping relationship between different video frame numbers according to the metadata of each loaded video. When the user performs playing, pausing, or jumping operations, the automatic synchronization unit automatically calculates and displays the video frames corresponding to the same time point for each area.

[0010] Preferably, the manual synchronization unit is used to provide a frame-level precise control interface, and the user can independently input the target frame number for each area or use the frame-in and frame-out buttons for fine adjustment to achieve fine alignment of multi-picture content.

[0011] Preferably, the playing linkage and control module is used to provide a global control panel to allow the user to set all areas to play in linkage or separately.

[0012] Preferably, the screenshot unit is used to capture the screen of the entire current playback window or a specified area and save it as an image file.

[0013] Preferably, the screenshot unit is used to record the dynamic images and audio of the entire playback window to generate a new video file.

[0014] Preferably, the preset video processing effect comparison and evaluation system further includes a batch processing and automation module, which is used for the user to specify the source file directory and one or more output file directories.

[0015] Preferably, it further includes, S60, batch automated evaluation: the user switches to batch processing mode, sets the source directory, output directory and matching rules, the system automatically traverses the files, and repeats S20 to S50 for each group of videos to generate a series of recorded video files containing the complete comparison process.

[0016] The present invention has the following beneficial effects: (1) Solved the problem of video synchronization from different sources: Through the dual modes of automatic frame rate mapping and manual frame level adjustment, the problem of video comparison asynchrony caused by different resolutions and frame rates is effectively solved, so that subjective evaluation focuses on the content itself.

[0017] (2) Improved comparison efficiency and experience: Flexible multi-screen layout and linkage control make A / B testing or multi-algorithm parallel comparison extremely intuitive and convenient, greatly improving the work efficiency of evaluators.

[0018] (3) The evaluation process has been visualized and archived: the screenshot and screen recording functions transform subjective and instantaneous evaluation judgments into objective multimedia records that can be saved and played back for a long time, ensuring the traceability and reproducibility of the evaluation results, and facilitating team review and problem retrospection.

[0019] (4) Significantly improves the level of test automation: Batch processing mode frees testers from repetitive mechanical operations. Only one configuration is needed to complete the automated comparison and record generation of the entire test set, which is particularly suitable for regression testing in algorithm iteration and testing of a large number of samples. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of a method for comparing and evaluating video processing effects according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See Figure 1 The diagram illustrates the steps of a method for comparing and evaluating video processing effects according to an embodiment of the present invention, including the following steps: S10, Perform system pre-setting: Set up a video processing effect comparison and evaluation system; S20, Configure and load: The user starts the video processing effect comparison and evaluation system, configures the layout and number of playback areas, and loads the video files to be compared in each area, such as the original video, the algorithm-processed video A, and the algorithm-processed video B. S30 performs adaptive and initial synchronization: automatically scales each video to fit its playback area, reads the frame rate of each video, and automatically calculates the initial time-frame mapping relationship so that multiple screens are roughly synchronized when playback begins. S40, perform synchronous comparison operation: the user performs subjective evaluation and comparison, including at least any of the following: use play / pause, fast forward / rewind for macro browsing; enable linked playback mode to observe other videos synchronously with one video as a reference; use manual synchronization unit for frame-level fine adjustment to align specific key action or scene transition frames; in separate playback mode, independently control a certain area to focus on the details of a specific video; S50, record the evaluation process: During or after the comparison, perform the following operations: At key frames, use the screenshot unit to save the current multi-screen state and start the screen recording unit to record a video containing the complete operation and playback process as evaluation evidence for archiving.

[0023] In a specific application example, S10 includes a preset video processing effect comparison and evaluation system, which includes a screen layout and file loading module, a video adaptive scaling module, an intelligent synchronization control module, a playback linkage and control module, and an evaluation recording module. The intelligent synchronization unit further includes an automatic synchronization unit and a manual synchronization unit; the evaluation recording module further includes a screenshot unit and a screen recording unit.

[0024] Furthermore, the screen layout and file loading module divides the playback window into N freely resizable and repositionable playback areas, where N≥2. Each playback area can independently load a video file; for example, area 1 loads the source landscape video, area 2 loads the portrait output of algorithm A, and area 3 loads the portrait output of algorithm B. The video adaptation and scaling module scales input videos of different resolutions and aspect ratios to fit their assigned playback areas, while maintaining the original aspect ratio without distortion or cropping / filling according to user settings. The automatic synchronization unit automatically calculates and establishes a time mapping relationship between different video frame numbers based on the metadata (frame rate, FPS) of each loaded video. When the user performs play, pause, or jump operations, the automatic synchronization unit automatically calculates and displays the corresponding video frames at the same time point for each area. The manual synchronization unit provides a frame-level precise control interface, allowing users to independently input the target frame number for each area or use the frame forward and frame backward buttons for fine-tuning to achieve precise alignment of multi-screen content. The playback linkage and control module provides a global control panel, allowing users to set all areas to play in sync (with a main control area controlling the progress of all areas) or play individually. It supports setting different playback rates (e.g., 0.5x, 1x, 2x). The screenshot unit captures the entire playback window or a specified area and saves it as an image file, which can be appended with timestamps and frame numbers. The screen recording unit records the dynamic images and audio of the entire playback window, generating a new video file that fully records the entire process of synchronized multi-screen playback and user operations (such as pointers and annotations).

[0025] In a specific application example, the pre-defined video processing effect comparison and evaluation system further includes a batch processing and automation module, allowing users to specify a source file directory and one or more output file directories. The system automatically pairs the source video with the processed video based on pre-defined matching rules (such as filename keywords, regular expressions, and sequential matching). Subsequently, the system automatically and sequentially executes the process of loading files -> synchronous playback -> recording the evaluation process -> saving the recorded files, without requiring manual intervention.

[0026] In a specific application example, another embodiment of the present invention provides a method for comparing and evaluating video processing effects. Based on S10 to S50, it further includes S60, batch automated evaluation: The user switches to batch processing mode, sets the source directory, output directory and matching rules, and the system automatically traverses the files, repeating S20 to S50 for each group of videos to generate a series of recorded video files containing the complete comparison process, which can be subsequently viewed and evaluated by testers.

[0027] To more intuitively demonstrate the implementation effect of the method of the present invention, the following is a specific implementation process of a particular application embodiment: S801, the evaluator opens the preset system and divides the main window into three equally sized vertical areas.

[0028] S802, load the original landscape video source_01.mp4 (1920x1080, 30fps) in region 1.

[0029] S803 loads the vertical video output_A_01.mp4 (1080x1920, 30fps) processed using an object tracking algorithm in region 2.

[0030] S804 loads the vertical video output_B_01.mp4 (1080x1920, 29.97fps) processed using the center cropping algorithm in region 3.

[0031] S805: The system automatically scales the video to fit the region. Due to slight differences in frame rate, the system automatically calculates the mapping threshold based on 30fps and 29.97fps.

[0032] S806, the evaluator clicked "Linked Playback" and played the video with Region 1 (source video) as the main controller. At frame 250 (approximately 8.3 seconds), a fast panning shot was detected.

[0033] S807, the evaluator paused playback and used the manual synchronization unit to precisely adjust the images of the three areas to the same instant when the object began to move (e.g., all positioned around frame 255).

[0034] S808, then use the "Frame Forward" button to compare frame by frame between the two algorithms in this motion scene to see if the objects remain in the center of the image and whether there is any loss of image quality.

[0035] S809, the evaluators believed that this was a weakness of Algorithm B, so they used the screenshot function to save the current state of the three screens side by side and named it "Quick pan_Algorithm B lost the main subject.jpg".

[0036] S810. Subsequently, the evaluators turned on the screen recording function and played the video in sync from frame 250 to frame 300, recording a complete comparison process of about 1.7 seconds, which was saved as compare_segment_01.mkv.

[0037] S811, after completing the current video evaluation, the evaluator enters batch processing mode. The source directory is set to . / sources / , and the output directories are . / algo_A_outputs / and . / algo_B_outputs / , with the matching rule being "files with the same name".

[0038] S812, the system runs automatically, generating 20 comparison recording files from batch_compare_001.mkv to batch_compare_020.mkv for the 20 sets of test videos in the directory.

[0039] S813 allows evaluators or other reviewers to gain a comprehensive understanding of the performance of algorithms A and B across the entire test set simply by viewing these 20 recordings. This eliminates the need for repeated operations, ensuring consistent and reliable evaluation conclusions.

[0040] The methods described above for comparing and evaluating video processing effects adaptively handle input videos of different specifications, achieving precise frame synchronization between multiple screens automatically or manually, reducing the complexity of manual alignment. They provide flexible layout and linkage control, facilitating multi-angle subjective comparisons. The entire comparison process (including the timing of multi-screen content and user interactions) is completely saved via screenshots or screen recordings, forming reusable evaluation evidence. Finally, by introducing a batch processing mode, the evaluation process is automated, automatically matching and sequentially playing test sets, and recording the comparison process for subsequent offline evaluation, greatly improving testing and evaluation efficiency.

[0041] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for comparing and evaluating video processing effects, characterized in that, Includes the following steps: S10, Perform system pre-setting: Set up a video processing effect comparison and evaluation system; S20, Configure and load: The user starts the video processing effect comparison and evaluation system, configures the layout and number of playback areas, and loads the video files to be compared in each area. S30 performs adaptive and initial synchronization: automatically scales each video to fit its playback area, reads the frame rate of each video, and automatically calculates the initial time-frame mapping relationship so that multiple screens are roughly synchronized when playback begins. S40, perform synchronous comparison operation: the user performs subjective evaluation and comparison, including at least any of the following: use play / pause, fast forward / rewind for macro browsing; enable linked playback mode to observe other videos synchronously with one video as a reference; use manual synchronization unit for frame-level fine adjustment to align specific key action or scene transition frames; in separate playback mode, independently control a certain area to focus on the details of a specific video; S50, record the evaluation process: During or after the comparison, perform the following operations: At key frames, use the screenshot unit to save the current multi-screen state and start the screen recording unit to record a video containing the complete operation and playback process as evaluation evidence for archiving.

2. The method for comparing and evaluating video processing effects as described in claim 1, characterized in that, In S10, the preset video processing effect comparison and evaluation system includes a screen layout and file loading module, a video adaptive scaling module, an intelligent synchronization control module, a playback linkage and control module, and an evaluation recording module. The intelligent synchronization unit further includes an automatic synchronization unit and a manual synchronization unit; the evaluation recording module further includes a screenshot unit and a screen recording unit.

3. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The screen layout and file loading module is used to divide the playback window into N playback areas whose size and position can be freely adjusted, where N≥2, and each playback area can independently load a video file.

4. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The video adaptation and scaling module is used to scale input videos of different resolutions and aspect ratios to fit their respective assigned playback areas, while maintaining the original aspect ratio without distortion or cropping / filling according to user settings.

5. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The automatic synchronization unit is used to automatically calculate and establish the time mapping relationship between different video frame numbers based on the metadata of each loaded video. When the user performs playback, pause, or jump operation, the automatic synchronization unit automatically calculates and displays the video frame corresponding to the same time point for each region.

6. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The manual synchronization unit provides a frame-level precise control interface, allowing users to independently input the target frame number for each region or use the frame forward and frame backward buttons for fine-tuning to achieve precise alignment of multi-screen content.

7. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The playback linkage and control module provides a global control panel, allowing users to set all areas to play in a linked manner or play individually.

8. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The screenshot unit is used to capture the screen of the entire current playback window or a specified area and save it as an image file.

9. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The screenshot unit is used to record the dynamic images and audio of the entire playback window and generate a new video file.

10. The method for comparing and evaluating video processing effects as described in claim 2, characterized in that, The preset video processing effect comparison and evaluation system further includes a batch processing and automation module, which is used for users to specify source file directories and one or more output file directories.

11. The method for comparing and evaluating video processing effects as described in claim 10, characterized in that, Further, S60, batch automated evaluation: The user switches to batch processing mode, sets the source directory, output directory and matching rules, and the system automatically traverses the files, repeating S20 to S50 for each group of videos to generate a series of recorded video files containing the complete comparison process.

Images