Video repeated frame detection method and device

Abstract

The invention provides a video repeated frame detection method and device. The method comprises the following steps: storing a video signal which is output by a video source and follows an HDMI protocol specification to obtain a target video; each video frame of the target video is read, for each video frame, the difference feature value of the video frame is calculated, and the difference feature value represents the difference degree between the video frame and the adjacent previous video frame; and for every two different characteristic values with adjacent sizes, calculating the difference value between the larger value and the smaller value in the two different characteristic values with adjacent sizes, determining the maximum value in all the difference values, if the maximum value is greater than a preset threshold value, determining a target threshold value according to the maximum value, and if the target threshold value is greater than the preset threshold value, determining the target threshold value according to the target threshold value. And determining the video frame with the difference characteristic value smaller than the target threshold value in each video frame as the repeated frame. According to the scheme, a real picture seen by human eyes can be tested, the threshold value does not need to be manually set, multi-step verification is not needed, the detection is accurate, and the processing is rapid.

Description

Technical Field

[0001] This invention relates to the field of video processing, and in particular to a method and apparatus for detecting duplicate frames in video. Background Technology

[0002] During video recording, acquisition, and transmission, duplicate frames frequently occur due to insufficient hardware performance, driver issues, encoding delays, and other reasons. This means that multiple adjacent frames display the same or similar content, causing video stuttering and buffering. Accurate detection of duplicate frames is crucial for assessing video quality and diagnosing system performance problems.

[0003] Existing duplicate frame detection technologies mainly suffer from the following problems: 1. Inability to test the actual image seen by the human eye: For devices with video recording or encoding capabilities, traditional methods obtain recorded or encoded videos and use them for duplicate frame detection. However, this video is submitted to the display layer by the device, not the video actually displayed on the screen, meaning it is not the video actually seen by the human eye. This leads to deviations in the raw data used for testing. Furthermore, video recording functions use a fixed low bitrate and cannot specify the bitrate, resulting in incomplete information saved in the recorded video, ultimately leading to inaccurate or even erroneous test results. For devices without video recording or encoding capabilities, traditional methods cannot perform duplicate frame detection because they cannot obtain the raw data.

[0004] 2. Requires manual threshold setting: Traditional methods are usually based on comparing the inter-frame difference value with a fixed threshold. However, the optimal threshold is different for different video content, different scenes, and even videos in the same scene. It requires repeated manual debugging and lacks universality.

[0005] 3. Requires multi-step verification: Some methods first screen through a simple difference calculation process, and then use complex algorithms such as SSIM (structural similarity) and PSNR (peak signal-to-noise ratio) for secondary verification, which has high computational cost and slow processing speed.

[0006] In view of this, how to provide an automatic video duplicate frame detection solution that can test the real image seen by the human eye, without the need for manual threshold setting, without the need for multi-step verification, and with accurate detection and fast processing has become an urgent technical problem to be solved. Summary of the Invention

[0007] To address the technical problems existing in the prior art, this application provides a method and apparatus for detecting duplicate video frames.

[0008] In a first aspect, embodiments of this application provide a video repeating frame detection method, including: The target video is obtained by saving the video signal output from the video source that conforms to the HDMI protocol specification; Read each video frame of the target video. For each video frame, calculate the difference feature value of that video frame, where the difference feature value represents the degree of difference between that video frame and the adjacent previous video frame. For every two adjacent difference feature values, calculate the difference between the larger and smaller values, determine the maximum value among all differences, and if the maximum value is greater than a preset threshold, determine the target threshold based on the maximum value. Video frames with difference feature values ​​less than the target threshold are identified as duplicate frames, where the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

[0009] Secondly, embodiments of this application also provide a video repeating frame detection device, comprising: The storage unit is used to store the target video obtained by the video signal output from the video source conforming to the HDMI protocol specification; The first calculation unit is used to read each video frame of the target video. For each video frame, it calculates the difference feature value of the video frame, wherein the difference feature value represents the degree of difference between the video frame and the adjacent previous video frame. The second calculation unit is used to calculate the difference between the larger and smaller values ​​of any two adjacent difference feature values, determine the maximum value among all differences, and if the maximum value is greater than a preset threshold, determine the target threshold based on the maximum value. Video frames in each video frame whose difference feature value is less than the target threshold are determined as duplicate frames, where the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

[0010] The video duplicate frame detection method and apparatus provided in this application are based on a video source that outputs a video signal conforming to the HDMI protocol specification. This video is the actual video displayed on screen and seen by the user's eyes. A high bitrate can be specified when saving the video signal, ensuring the target video retains complete video information and guaranteeing the accuracy of the test results. Furthermore, duplicate frame detection can be performed as long as the video source can output an HDMI video signal, without requiring the video source to have video recording or encoding capabilities, making this solution highly versatile. The target threshold is automatically determined based on the difference feature values ​​of each video frame of the target video, eliminating the need for manual threshold setting. After determining the target threshold, video frames with difference feature values ​​less than the target threshold are identified as duplicate frames. Detection can be completed in a single step, without the need for complex algorithms such as SSIM or PSNR, or multi-step verification, resulting in accurate detection and rapid processing. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of a video duplicate frame detection method provided in this application. Figure 2 This is a schematic diagram of an embodiment of a video duplicate frame detection device provided in this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0013] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0015] Reference Figure 1 The diagram shown is a flowchart illustrating a video repeating frame detection method provided in an embodiment of this application. The method includes: S10. Save the video signal output from the video source that conforms to the HDMI protocol specification to obtain the target video; In this embodiment, the video source can be a hardware device capable of outputting HDMI (High Definition Multimedia Interface) video signals, such as a cloud box, PC, or mobile phone. The video source can transmit video signals to the video capture card via the HDMI channel, and the target video can be obtained by saving the video signal captured by the video capture card. For example, the video source is a cloud box, which is connected to a video capture card. The video capture card is plugged into a computer, which runs the software FFmpeg. The cloud server sends cloud gaming video streams to the cloud box, and the cloud box transmits video signals to the video capture card via the HDMI channel. FFmpeg records the video stream received by the video capture card as a video file (i.e., the target video).

[0016] S11. Read each video frame of the target video. For each video frame, calculate the difference feature value of the video frame. The difference feature value represents the degree of difference between the video frame and the adjacent previous video frame. In this embodiment, the cv2.VideoCapture library of OpenCV can be used to read video frames. Before calculating the difference feature value, each video frame can be preprocessed. Preprocessing can include image conversion and cropping. Image conversion can be to convert the video frame to a grayscale image, and cropping can be to crop specific areas of the video frame, such as cropping 10% of the top and bottom of the grayscale image to remove possible interference factors such as black borders and watermarks. Except for the first video frame, each video frame in the target video has a preceding adjacent video frame (i.e., the previous video frame among two adjacent video frames). Therefore, a difference feature value can be calculated for each of the other video frames. For the first video frame, its difference feature value can be set to a certain value, such as 0.

[0017] S12. For every two adjacent difference feature values, calculate the difference between the larger and smaller values ​​of the two adjacent difference feature values, determine the maximum value among all differences, and if the maximum value is greater than the preset threshold, determine the target threshold based on the maximum value. Determine the video frames in each video frame whose difference feature value is less than the target threshold as duplicate frames, where the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

[0018] In this embodiment, the differential feature values ​​can be sorted in ascending order, and the bottom 10% of the data (i.e., the largest 10%) are deleted. For each pair of remaining adjacent data, the difference between the latter and former data is calculated, and the maximum value among all differences is determined. The maximum value is then checked against a preset threshold: if the maximum value is greater than the preset threshold, it is determined that duplicate frames exist in the target video; if the maximum value is less than or equal to the preset threshold, it is determined that no duplicate frames exist in the target video. The preset threshold is typically set to 5 based on experience. A maximum value greater than 5 indicates a significant bimodal distribution of differential feature values, meaning duplicate frames exist. Assuming the maximum value is greater than the preset threshold, and the maximum value is the difference between the differential feature value W1 (the second differential feature value) of video frame P1 and the differential feature value W2 (the first differential feature value) of video frame P2, then the target threshold η ∈ (W2, W1). Video frames in the target video with differential feature values ​​less than the target threshold η are duplicate frames, and video frames with differential feature values ​​greater than or equal to the target threshold η are non-duplicate frames.

[0019] The video duplicate frame detection method provided in this application embodiment targets a video signal that conforms to the HDMI protocol specification and is actually displayed on screen and seen by the user's eyes. A high bitrate can be specified when saving the video signal, ensuring the target video retains complete video information and thus guaranteeing the accuracy of the test results. Furthermore, as long as the video source can output an HDMI video signal, duplicate frame detection can be performed, without requiring the video source to have video recording or encoding capabilities, making this solution highly versatile. The target threshold is automatically determined based on the difference feature values ​​of each video frame of the target video, eliminating the need for manual threshold setting. After determining the target threshold, video frames with difference feature values ​​less than the target threshold are identified as duplicate frames. Detection can be completed in a single judgment, without the need for complex algorithms such as SSIM or PSNR, or multi-step verification, resulting in accurate detection and rapid processing.

[0020] In addition to the problems listed in the background section, some existing methods use the average difference of pixels across the entire image as a feature, which is easily diluted by static background areas, thus making them insufficiently sensitive to changes in key local areas. To address this issue, when calculating the difference feature value, only specific pixels can be considered. Specifically, let video frame P2 be the next video frame after video frame P1. The difference feature value of video frame P2 can be the average of the N1 largest absolute values ​​of the differences between the preset color channel values ​​of corresponding pixels in video frame P2 and video frame P1, where N1 is a positive integer. The preset color channel value can be the Y channel value, and N1 can be set as needed, preferably 100.

[0021] In addition, traditional methods typically obtain the video frame interval by capturing system logs, and then calculate the frame rate and stuttering rate based on the frame interval. This method has two problems: First, it requires intruding into the system installation program (such as a program that captures system logs); second, the obtained frame interval is the frame interval generated when the system submits the frame to the display layer, not the actual frame interval during display. This leads to inaccurate measured frame rate and stuttering rate.

[0022] To address this issue, based on the foregoing method embodiments, the method may further include: Obtain the duration and frame count of the target video, and calculate the actual frame interval of the target video based on the duration and frame count; The display duration of non-repeating frames in each video frame is calculated based on the frame interval. The display duration of non-repeating frame P3 is calculated as frame interval × (1 + N2), where N2 is the number of consecutive repeating frames after non-repeating frame P3 in each video frame. The entire group of consecutive repeating frames is adjacent to non-repeating frame P3.

[0023] In this embodiment, the ffprobe tool can be used to obtain the precise duration of the target video (in seconds). The actual frame interval is the ratio of duration to frame count (in milliseconds). When calculating the display duration, the display duration of repeating frames is 0 (i.e., not displayed), and the formula for calculating the display duration of non-repeating frames is: Display duration of non-repeating frame P3 = Frame interval × (1 + N²). The display duration of non-repeating frames is the display duration of the image shown on the monitor when the video signal from the video source is output to the monitor (i.e., the display duration of the image directly seen by the human eye or the display frame interval). After calculating the display duration of each non-repeating frame, the actual frame rate and stuttering rate of the real video seen by the human eye can be calculated. Obviously, the entire solution only requires the video source to output an HDMI video signal to test the actual frame rate and stuttering rate, without requiring system intrusion.

[0024] This solution utilizes an innovative detection algorithm to achieve automatic, accurate, and rapid detection of duplicate frames in videos. It can test the real image seen by the human eye, without the need for manually setting thresholds or complex similarity calculations. Furthermore, it can accurately synchronize video duration, solving the problems of strong threshold dependence, slow processing speed, and low processing accuracy in existing technologies. Moreover, this solution does not require system intrusion and can be tested without any developer mode or debugging mode, making it of significant practical value and application prospects.

[0025] Reference Figure 2 The diagram shown is a structural schematic of a video duplicate frame detection device provided in an embodiment of this application. The device includes: The storage unit 20 is used to store the target video obtained by the video signal output from the video source conforming to the HDMI protocol specification; The first calculation unit 21 is used to read each video frame of the target video. For each video frame, it calculates the difference feature value of the video frame, wherein the difference feature value represents the degree of difference between the video frame and the adjacent previous video frame. The second calculation unit 22 is used to calculate the difference between the larger and smaller values ​​of any two adjacent difference feature values, determine the maximum value among all differences, and if the maximum value is greater than a preset threshold, determine the target threshold based on the maximum value. Video frames in each video frame whose difference feature value is less than the target threshold are determined as duplicate frames, wherein the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

[0026] The video duplicate frame detection device provided in this application embodiment targets a video signal that conforms to the HDMI protocol specification and is saved from the video source output. This video is the one that is actually displayed on the screen and seen by the user's eyes. A high bitrate can be specified when saving the video signal, ensuring that the target video retains complete video information, thereby guaranteeing the accuracy of the test results. Furthermore, as long as the video source can output an HDMI video signal, duplicate frame detection can be performed, without requiring the video source to have video recording or encoding capabilities, making this solution highly versatile. The target threshold is automatically determined based on the difference feature values ​​of each video frame of the target video, eliminating the need for manual threshold setting. After determining the target threshold, video frames with difference feature values ​​less than the target threshold are identified as duplicate frames. Detection can be completed in a single judgment, without the need for complex algorithms such as SSIM or PSNR, or multi-step verification, resulting in accurate detection and fast processing.

[0027] Based on the aforementioned device embodiments, the storage unit can be used for: The target video is obtained by saving the video signal captured by the video capture card. The video source transmits the video signal to the video capture card through the HDMI channel.

[0028] Based on the aforementioned device embodiments, the second computing unit can also be used for: Obtain the duration and frame count of the target video, and calculate the actual frame interval of the target video based on the duration and frame count; The display duration of non-repeating frames in each video frame is calculated based on the frame interval. The display duration of non-repeating frame P3 is calculated as frame interval × (1 + N2), where N2 is the number of consecutive repeating frames after non-repeating frame P3 in each video frame. The entire group of consecutive repeating frames is adjacent to non-repeating frame P3.

[0029] The video repeating frame detection device provided in this application embodiment is implemented in the same way as the video repeating frame detection method provided in this application embodiment, and the effect it can achieve is also the same as the video repeating frame detection method provided in this application embodiment, so it will not be described again here.

[0030] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting duplicate frames in a video, characterized in that, include: The target video is obtained by saving the video signal output from the video source that conforms to the HDMI protocol specification; Read each video frame of the target video. For each video frame, calculate the difference feature value of that video frame, where the difference feature value represents the degree of difference between that video frame and the adjacent previous video frame. For every two adjacent difference feature values, calculate the difference between the larger and smaller values, determine the maximum value among all differences, and if the maximum value is greater than a preset threshold, determine the target threshold based on the maximum value. Video frames with difference feature values ​​less than the target threshold are identified as duplicate frames, where the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

2. The method as described in claim 1, characterized in that, The process of obtaining the target video from the video source output, which conforms to the HDMI protocol specification, includes: The target video is obtained by saving the video signal captured by the video capture card. The video source transmits the video signal to the video capture card through the HDMI channel.

3. The method as described in claim 1 or 2, characterized in that, Let the next video frame after video frame P1 be video frame P2. The difference feature value of video frame P2 is the average of the N1 largest absolute values ​​of the differences between the preset color channel values ​​of each corresponding pixel of video frame P2 and video frame P1, where N1 is a positive integer.

4. The method as described in claim 3, characterized in that, The default color channel value is the Y channel value, and N1 is 100.

5. The method as described in claim 1, characterized in that, Also includes: Obtain the duration and frame count of the target video, and calculate the actual frame interval of the target video based on the duration and frame count; The display duration of non-repeating frames in each video frame is calculated based on the frame interval. The display duration of non-repeating frame P3 is calculated as frame interval × (1 + N2), where N2 is the number of consecutive repeating frames after non-repeating frame P3 in each video frame. The entire group of consecutive repeating frames is adjacent to non-repeating frame P3.

6. A video duplicate frame detection device, characterized in that, include: The storage unit is used to store the target video obtained by the video signal output from the video source conforming to the HDMI protocol specification; The first calculation unit is used to read each video frame of the target video. For each video frame, it calculates the difference feature value of the video frame, wherein the difference feature value represents the degree of difference between the video frame and the adjacent previous video frame. The second calculation unit is used to calculate the difference between the larger and smaller values ​​of any two adjacent difference feature values, determine the maximum value among all differences, and if the maximum value is greater than a preset threshold, determine the target threshold based on the maximum value. Video frames in each video frame whose difference feature value is less than the target threshold are determined as duplicate frames, where the target threshold ∈ (first difference feature value, second difference feature value), and the difference between the second difference feature value and the first difference feature value is the maximum value.

7. The apparatus as claimed in claim 6, characterized in that, The storage unit is used for: The target video is obtained by saving the video signal captured by the video capture card. The video source transmits the video signal to the video capture card through the HDMI channel.

8. The apparatus as claimed in claim 6 or 7, characterized in that, Let the next video frame after video frame P1 be video frame P2. The difference feature value of video frame P2 is the average of the N1 largest absolute values ​​of the differences between the preset color channel values ​​of each corresponding pixel of video frame P2 and video frame P1, where N1 is a positive integer.

9. The apparatus as claimed in claim 8, characterized in that, The default color channel value is the Y channel value, and N1 is 100.

10. The apparatus as claimed in claim 6, characterized in that, The second computing unit is also used for: Obtain the duration and frame count of the target video, and calculate the actual frame interval of the target video based on the duration and frame count; The display duration of non-repeating frames in each video frame is calculated based on the frame interval. The display duration of non-repeating frame P3 is calculated as frame interval × (1 + N2), where N2 is the number of consecutive repeating frames after non-repeating frame P3 in each video frame. The entire group of consecutive repeating frames is adjacent to non-repeating frame P3.

Images