A method for detecting video frame freezing

Abstract

The application relates to a video detection technology and discloses a video frame freezing detection method, which comprises the following steps: segmenting a video scene through scene detection; acquiring image frames for the segmented video scene; acquiring abnormal image frames by comparing the pixels, colors and brightness of adjacent frame images; acquiring similar images for the abnormal image frames through a scene detection algorithm; and acquiring frame freezing by filtering the similar images in the first abnormal frame. The application can accurately detect frame freezing in a video and further reduce the false detection rate and the missed detection rate of video detection by using a scene detection method and a front and back image similarity analysis method and giving full play to the advantages of an HSV space.

Description

Technical Field

[0001] This invention relates to video detection technology, and more particularly to a method for detecting video frame clipping. Background Technology

[0002] During video editing, production, and playback, due to human error, improper operation, or other reasons, one or a few frames that are completely unrelated to the semantics of the currently playing scene may be interspersed between frames of the video scene. For example, an advertisement or nationally prohibited content may be interspersed within a plot scene. During playback, these interspersed frames will flash by, affecting the playback effect and audience experience, and may even cause playback accidents.

[0003] There is currently no unified definition for frame sandwiching. Most existing technologies use image similarity comparison or scene analysis to find "alternative" image frames between or within scenes, which are considered frame sandwiching. For example, existing patent 201110173830.0; however, this method of identifying frame sandwiching has a high false detection rate and a high false negative rate. Summary of the Invention

[0004] This invention addresses the problem in existing technologies that determine video frame intervals through scene switching, which results in high false detection and false negative rates during the video frame interval confirmation process. It provides a method for detecting video frame intervals.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for detecting video frame clips, the method comprising:

[0007] Video scene segmentation is performed through scene detection.

[0008] Image frame acquisition: For segmented video scenes, image frames are acquired.

[0009] Abnormal image frames are obtained by comparing the pixels, colors, and brightness of adjacent frames.

[0010] For abnormal image frames, similar graphics are identified using scene detection algorithms.

[0011] The sandwich frame is obtained by filtering similar images in the first abnormal frame.

[0012] As a preferred option, the video scene is segmented as follows:

[0013] Video initialization involves initializing the video scene queue and video scene buffer queue for the detected video.

[0014] For video scene segmentation, if the video scene buffer queue is greater than 1 and the video scene queue length is greater than the video scene length threshold, output the current video scene, which is the segmented video scene.

[0015] Preferably, for image frame acquisition, the segmented video scene is segmented, and if the scene length is not less than the image frame length threshold, then the segmented image frame is output.

[0016] Preferably, the acquisition of abnormal image frames includes:

[0017] S1: Obtain the difference between the current frame of the image frame in the video scene and the first frame of the first scene in the video scene. If the difference is less than the set pixel threshold, it is marked as a normal scene; otherwise, execute S2.

[0018] S3. For video scenes not labeled in S2, obtain the difference between the first frame of the video scene and the first frame of the last scene in the video scene. If the difference is less than the set color threshold, it is labeled as a normal scene; otherwise, proceed to S4.

[0019] S4. For video scenes not labeled in S3, obtain the brightness of the video scene and compare it with a brightness threshold. If the brightness is greater than the set brightness threshold, it is labeled as a normal video scene; otherwise, it is labeled as an abnormal image frame.

[0020] Preferred methods for determining similar graphics using scene detection algorithms include:

[0021] Step 1: Read RGB format image frames from the video stream;

[0022] Step 2: Iterate through the RGB pixels in the RGB format image frame from Step 1 and map the RGB pixels to the HSV space;

[0023] Step 3: Traverse the HSV data of all pixels in the image frame in the HSV space in Step 2, convert the HSV data to XYZ coordinates, and cache the XYZ data.

[0024] Step 4: Calculate the difference between the XYZ data of the current frame image and the XYZ data of the previous frame image;

[0025] Step 5: Scene switching determination. By comparing the difference calculated in Step 4 with the set threshold, if the difference is greater than the set threshold, then it is a scene switching; otherwise, the scene continues and returns to Step 1 to continue reading image frames.

[0026] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of a video frame detection method.

[0027] To address the aforementioned technical problems, the present invention also provides an electronic device comprising: a memory for storing a computer program; and a processor for executing the computer program to implement a method for detecting video frame clipping.

[0028] This invention, by adopting the above technical solutions, has significant technical effects:

[0029] This invention fully leverages the advantages of the HSV space through scene detection and before-and-after image similarity analysis methods, enabling accurate detection of frame gaps in videos and further reducing the false detection and false negative rates in video detection. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention;

[0031] Figure 2 This is a flowchart of Embodiment 1 of the present invention;

[0032] Figure 3 This is a flowchart of the scene detection algorithm of the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0034] Example 1

[0035] A method for detecting video frame clips, the method comprising:

[0036] Video scene segmentation is performed through scene detection.

[0037] Image frame acquisition: For segmented video scenes, image frames are acquired.

[0038] Abnormal image frames are obtained by comparing the pixels, colors, and brightness of adjacent frames.

[0039] For abnormal image frames, similar graphics are identified using scene detection algorithms.

[0040] The sandwich frame is obtained by filtering similar images in the first abnormal frame.

[0041] The video scene is segmented as follows:

[0042] Video initialization involves initializing the video scene queue and video scene buffer queue for the detected video.

[0043] For video scene segmentation, if the video scene buffer queue is greater than 1 and the video scene queue length is greater than the video scene length threshold, output the current video scene, which is the segmented video scene.

[0044] Image frame acquisition: The segmented video scene is obtained. If the scene length is not less than the image frame length threshold, the segmented image frame is output.

[0045] The acquisition of abnormal image frames includes:

[0046] S1: Obtain the difference between the current frame of the image frame in the video scene and the first frame of the first scene in the video scene. If the difference is less than the set pixel threshold, it is marked as a normal scene; otherwise, execute S2.

[0047] S3. For video scenes not labeled in S2, obtain the difference between the first frame of the video scene and the first frame of the last scene in the video scene. If the difference is less than the set color threshold, it is labeled as a normal scene; otherwise, proceed to S4.

[0048] S4. For video scenes not labeled in S3, obtain the brightness of the video scene and compare it with the brightness threshold. If the brightness is greater than the set brightness threshold, it is labeled as a normal video scene; otherwise, it is an abnormal image frame.

[0049] Methods for identifying similar graphics using scene detection algorithms include:

[0050] Step 1: Read RGB format image frames from the video stream;

[0051] Step 2: Iterate through the RGB pixels in the RGB format image frame from Step 1 and map the RGB pixels to the HSV space;

[0052] Step 3: Traverse the HSV data of all pixels in the image frame in the HSV space in Step 2, convert the HSV data to XYZ coordinates, and cache the XYZ data.

[0053] Step 4: Calculate the difference between the XYZ data of the current frame image and the XYZ data of the previous frame image;

[0054] Step 5: Scene switching determination. By comparing the difference calculated in Step 4 with the set threshold, if the difference is greater than the set threshold, then it is a scene switching; otherwise, the scene continues and returns to Step 1 to continue reading image frames.

[0055] Example 2

[0056] Based on Embodiment 1, this embodiment is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a video frame detection method.

[0057] Example 3

[0058] Based on Embodiment 1, this embodiment is an electronic device, which includes: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of a video frame detection method.

[0059] Example 4

[0060] Based on Example 1, this example defines a minimum effective scene length (number of frames in the image sequence), such as 3 frames, as a threshold. The length of each scene is counted; if the length of a certain "scene" is greater than the set threshold (e.g., 3 frames), it is called a normal scene; otherwise, it is a suspicious scene or suspicious frame. Suspicious scenes between two normal scenes are then examined.

[0061] Referring to the method for calculating the difference between adjacent frames in scene detection, the correlation between the suspect frame and the preceding and following scenes is calculated. If the suspect frame belongs to the preceding or following normal scene, it is a transition effect and not considered a frame sandwich. If the length of the suspect scene exceeds the set maximum frame sandwich length threshold, it is considered garbled text and also not considered a frame sandwich. Suspect frames with significant brightness changes, being too bright (or too dark), such as 8-bit grayscale greater than 200 (or less than 30), often indicate sudden changes in flash or stage lighting. In such cases, the image outline is faded and distorted, rendering similarity measurement and scene judgment meaningless, thus requiring special consideration. They cannot be simply classified as frame sandwiches; the specific project requirements must be considered.

[0062] After excluding these special cases, the suspected frame can be identified as a sandwich frame. Examples are as follows (different letters represent different scenes, and numbers represent the frame numbers within those scenes):

[0063] Example 1: There is an adjacent scene A / B / C sequence of frames A1 A2 A3 A4 A5 A6 A7B1C1C2C3C4C5C6, and B1 has no brightness change. It can be determined that B1 is a sandwich frame.

[0064] Example 2: Scene frame sequence A1 A2 A3 A4 A5 A6 B1 A7B2A8B3 B4. B1 and B2 are very different from the frames before and after them, indicating scene switching. However, this situation of scene frames interleaving is a video effect and cannot be regarded as frame clipping.

[0065] Example 3: A1 A2 A3 A4 A5 A6 A7 B1 B2 A8 A9 A10. B1 and B2 differ greatly from the preceding and following scenes, suggesting frame clipping. However, if the brightness of frames B1 and B2 is much higher than the previous frame A7 and the following frame A8, it is a special case with flash and requires special handling; otherwise, it is judged as frame clipping.

[0066] Example 4: A1 A2 A3 A4 A5 A6 A7B1C1 D1 E1 F1 G1 H1 H2 H3 H4..., if the scene changes in five consecutive frames B1C1 D1E1 F1 G1, and excluding the aforementioned brightness anomalies, this situation cannot be considered as frame skipping, but rather as special effects or garbled text.

[0067] Example 5: Given a scene frame sequence A1 A2 A3 A4 A5 A6B1C1D1D2D3D4, and B1 C1 without any brightness abrupt change, it can be determined that B1C1 is a sandwiched frame.

Claims

1. A method for detecting video frame clipping, the method comprising: Video scene segmentation is performed through scene detection. Image frame acquisition: For segmented video scenes, image frames are acquired. Abnormal image frames are obtained by comparing the pixels, colors, and brightness of adjacent frames. The acquisition of abnormal image frames includes: S1: Obtain the difference between the current frame of the image frame in the video scene and the first frame of the first scene in the video scene. If the difference is less than the set pixel threshold, it is marked as a normal scene; otherwise, execute S2. S2, for video scenes not labeled in S1, obtain the difference between the first frame of the video in the video scene and the first frame of the first scene in the video scene. If the difference is less than the set color threshold, it is labeled as a normal scene; otherwise, proceed to S3. S3. For video scenes not labeled in S2, obtain the difference between the first frame of the video scene and the first frame of the last scene in the video scene. If the difference is less than the set color threshold, it is labeled as a normal scene; otherwise, proceed to S4. S4. For video scenes not labeled in S3, obtain the brightness of the video scene and compare it with the brightness threshold. If the brightness is greater than the set brightness threshold, it is labeled as a normal video scene; otherwise, it is an abnormal image frame. For abnormal image frames, similar graphics are identified using scene detection algorithms. Methods for identifying similar graphics using scene detection algorithms include: Step 1: Read RGB format image frames from the video stream; Step 2: Iterate through the RGB pixels in the RGB format image frame from Step 1 and map the RGB pixels to the HSV space; Step 3: Traverse the HSV data of all pixels in the image frame in the HSV space in Step 2, convert the HSV data to XYZ coordinates, and cache the XYZ data. Step 4: Calculate the difference between the XYZ data of the current frame image and the XYZ data of the previous frame image; Step 5, scene switching determination: By comparing the difference calculated in step 4 with the set threshold, if the difference is greater than the set threshold, then scene switching occurs; otherwise, the scene continues and returns to step 1 to continue reading image frames. The sandwich frame is obtained by filtering similar images in the first abnormal frame.

2. The video frame insertion detection method according to claim 1, characterized in that, The video scene is segmented as follows: Video initialization involves initializing the video scene queue and video scene buffer queue for the detected video. For video scene segmentation, if the video scene buffer queue is greater than 1 and the video scene queue length is greater than the video scene length threshold, output the current video scene, which is the segmented video scene.

3. The video frame insertion detection method according to claim 1, characterized in that, Image frame acquisition: The segmented video scene is obtained. If the scene length is not less than the image frame length threshold, the segmented image frame is output.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a video frame detection method as described in any one of claims 1 to 3.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of a video frame detection method as described in any one of claims 1 to 3 when executing the computer program.

Images