Video transition detection method, device and electronic equipment

Abstract

The present disclosure relates to a method, device, and electronic device for detecting video transitions, and relates to the field of video detection technology. The method comprises: after acquiring video frame images of a video stream, calculating the dominant color ratio of the video frame images frame by frame; determining the starting and ending transition video frame images based on the dominant color ratio of each video frame image; determining the transition start image based on the number of frames between the starting and ending transition video frame images, and the dominant color ratio of the video frame image following the ending transition video frame image; and determining the transition end image based on the dominant color ratio of the video frame images following the transition start image. This improves the accuracy of video transition scene detection.

Description

Technical Field

[0001] The present disclosure relates to the field of video detection technology, and in particular to a method, device and electronic equipment for detecting video transitions. Background Art

[0002] With the surge in video creation, more and more creators are pursuing high-quality video productions. This has led to the addition of novel animation effects to videos. While these effects are aesthetically pleasing, they disrupt the inherent characteristics of normal video. In the past, video transitions were mostly solid colors, transitioning from one video content to another. With the addition of special effects, transitions have become more diverse, breaking the monotony of solid colors.

[0003] The video transition detection method in the related art cannot detect the transition scene with added special effects. Therefore, there is an urgent need for a video transition detection method for detecting video transition scenes under different special effects. Summary of the Invention

[0004] The present disclosure provides a method, device, and electronic device for detecting video transitions, which at least solve the problem in related technologies that transition scenes with added animation effects in videos cannot be detected. The technical solutions of the present disclosure are as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a method for detecting a video transition is provided, comprising:

[0006] Get the video frame image of the video stream;

[0007] Calculate the main color ratio of each video frame image frame by frame;

[0008] Determine the starting jump video frame image and the ending jump video frame image according to the main color ratio of each video frame image;

[0009] Determine a transition start image according to the number of frames between the start jump video frame image and the end jump video frame image, and the main color tone ratio of the next video frame image after the end jump video frame image;

[0010] The transition end image is determined according to the main color tone ratio of the video frame image subsequent to the transition start image.

[0011] Optionally, determining a start jump video frame image and an end jump video frame image according to a main color ratio of each video frame image includes:

[0012] If the main color ratio of a video frame image of the video stream is greater than or equal to a first threshold, and the main color ratio of a video frame image subsequent to the video frame image is less than the first threshold, determining the video frame image subsequent to the video frame image as a starting jump video frame image;

[0013] If the main color ratio of a video frame image of the video stream is less than a first threshold, and the main color ratio of at least one video frame image subsequent to the video frame image is greater than or equal to the first threshold, the video frame image is determined to be an end jump video frame image.

[0014] Optionally, determining the transition start image according to the number of frames between the start jump video frame image and the end jump video frame image, and the main color tone ratio of the video frame image following the end jump video frame image, includes:

[0015] If the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color tone ratio of the video frame image after the ending jump video frame image is greater than or equal to the first threshold, then the video frame image before the starting jump video frame image is determined to be the transition start image.

[0016] Optionally, determining the transition end image according to the main color tone ratio of the video frame image subsequent to the transition start image includes:

[0017] If the main color tone ratio of a frame of video frame image following the transition start image is less than a first threshold, and the number of frames between the transition start image and the frame of video frame image following the transition start image is greater than or equal to a third threshold, then the frame of video frame image following the transition start image is determined to be the transition end image.

[0018] Optionally, the main tone ratio is a black pixel ratio, and the main tone ratio of each video frame image is calculated frame by frame, including:

[0019] Traversing each pixel of each video frame image, if the pixel value of the pixel is less than a fourth threshold, determining the pixel as a black pixel;

[0020] The ratio of the number of black pixels in each video frame image to the total number of pixels in the corresponding video frame image is calculated frame by frame to determine the black pixel ratio of each video frame image.

[0021] Optionally, calculating the main color ratio of each video frame image frame by frame includes:

[0022] The pixel variance of each video frame image is calculated frame by frame; the main color ratio of each video frame image is determined according to the pixel variance of each video frame image; the pixel variance is negatively correlated with the main color ratio.

[0023] Optionally, the pixel variance of each video frame image is calculated frame by frame, including:

[0024] Traversing each pixel of each video frame image, determining the number of pixels in each video frame image and the pixel value of each pixel;

[0025] Determine the pixel mean of each video frame image according to the number of pixels in each video frame image and the pixel value of each pixel;

[0026] The pixel variance of each video frame image is determined according to the pixel value and the pixel mean of each pixel of each video frame image.

[0027] Optionally, before calculating the main color ratio of each video frame image frame by frame, the following steps are included:

[0028] Convert each video frame image of the video stream to a video frame image of the HSV color model space.

[0029] Optionally, the video transition detection method further includes:

[0030] If the number of frames between the starting jump video frame image and the ending jump video frame image is greater than or equal to the second threshold, the transition start image is re-determined in the subsequent video frame images of the video stream; and / or, if the main color tone ratio of a frame of video frame image subsequent to the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame image subsequent to the transition start image is less than the third threshold, the transition end image is re-determined in the subsequent video frame images of the video stream.

[0031] According to a second aspect of an embodiment of the present disclosure, a video transition detection device is provided, comprising:

[0032] An acquisition module is configured to acquire a video frame image of a video stream;

[0033] A calculation module is configured to calculate the main color ratio of each video frame image frame by frame;

[0034] The first determining module is configured to determine a starting jump video frame image and an ending jump video frame image according to a main color ratio of each video frame image;

[0035] The second determining module is configured to determine the transition start image according to the number of frames between the start jump video frame image and the end jump video frame image, and the main color ratio of the video frame image after the end jump video frame image;

[0036] The third determining module is configured to determine a transition ending image according to a main tone ratio of a video frame image subsequent to the transition starting image.

[0037] Optionally, the first determining module is specifically configured to execute:

[0038] If the main color ratio of a video frame image of the video stream is greater than or equal to a first threshold, and the main color ratio of a video frame image subsequent to the video frame image is less than the first threshold, determining the video frame image subsequent to the video frame image as a starting jump video frame image;

[0039] If the main color ratio of a video frame image of the video stream is less than a first threshold, and the main color ratio of at least one video frame image subsequent to the video frame image is greater than or equal to the first threshold, the video frame image is determined to be an end jump video frame image.

[0040] Optionally, the second determining module is specifically configured to execute:

[0041] If the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color tone ratio of the video frame image after the ending jump video frame image is greater than or equal to the first threshold, then the video frame image before the starting jump video frame image is determined to be the transition start image.

[0042] Optionally, the third determining module is specifically configured to execute:

[0043] If the main color tone ratio of a frame of video frame image following the transition start image is less than a first threshold, and the number of frames between the transition start image and the frame of video frame image following the transition start image is greater than or equal to a third threshold, then the frame of video frame image following the transition start image is determined to be the transition end image.

[0044] Optionally, the main color ratio is a black pixel ratio, and the calculation module is specifically configured to execute:

[0045] Traversing each pixel of each video frame image, if the pixel value of the pixel is less than a fourth threshold, determining the pixel as a black pixel;

[0046] The ratio of the number of black pixels in each video frame image to the total number of pixels in the corresponding video frame image is calculated frame by frame to determine the black pixel ratio of each video frame image.

[0047] Optionally, the computing module is specifically configured to execute:

[0048] The pixel variance of each video frame image is calculated frame by frame; the main color ratio of each video frame image is determined according to the pixel variance of each video frame image; the pixel variance is negatively correlated with the main color ratio.

[0049] Optionally, the computing module is specifically configured to execute:

[0050] Traversing each pixel of each video frame image, determining the number of pixels in each video frame image and the pixel value of each pixel;

[0051] Determine the pixel mean of each video frame image according to the number of pixels in each video frame image and the pixel value of each pixel;

[0052] The pixel variance of each video frame image is determined according to the pixel value and the pixel mean of each pixel of each video frame image.

[0053] Optionally, the video transition detection device further includes:

[0054] The conversion module is configured to convert each video frame image of the video stream into a video frame image of the color model HSV space.

[0055] Optionally, the video transition detection device further includes:

[0056] The fourth determination module is configured to execute if the number of frames between the starting jump video frame image and the ending jump video frame image is greater than or equal to the second threshold, then redetermine the transition start image in the subsequent video frame images of the video stream; and / or, if the main color tone ratio of a frame of video frame image subsequent to the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame of video frame image subsequent to the transition start is less than the third threshold, then redetermine the transition end image in the subsequent video frame images of the video stream.

[0057] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the above-mentioned video transition detection method.

[0058] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned method for detecting video transitions.

[0059] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the above-mentioned video transition detection method when executed by a processor.

[0060] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: Since the restrictions on transition judgment are relaxed and the constraints on jump video frame images are added, after acquiring the video frame images of the video stream, the electronic device can determine the starting jump video frame image and the ending jump video frame image from the video frame images based on the main color ratio of each video frame image. As a result, the electronic device can determine the transition start image based on the starting jump video frame image and the ending jump video frame image, and determine the transition end image based on the video frame images of the subsequent video stream, thereby improving the accuracy of video transition scene recognition and increasing the hit rate of special effect transition scenes.

[0061] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0063] Figure 1 is a flow chart showing a method for detecting a video transition according to an exemplary embodiment;

[0064] Figure 2 This is a scene of a method for detecting a video transition according to an exemplary embodiment. Figure 1 ;

[0065] Figure 3 This is a scene of a method for detecting a video transition according to an exemplary embodiment. Figure 2 ;

[0066] Figure 4 This is a scene of a method for detecting a video transition according to an exemplary embodiment. Figure 3 ;

[0067] Figure 5 is a flow chart showing a method for determining a main color ratio according to an exemplary embodiment;

[0068] Figure 6 is a flow chart showing another method for determining a main color ratio according to an exemplary embodiment;

[0069] Figure 7 is a block diagram of a device for detecting video transition according to an exemplary embodiment;

[0070] Figure 8 The figure is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0072] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0073] In the related art, the video frame images of the transition scenes in the video are mostly based on pure colors (for example, black), forming a switch from one video content to another. Most of the existing transition detection algorithms follow a pattern. For example, the video is first disassembled into multiple frames of video frame images frame by frame, and then the statistical characteristics of each frame (for example, the black pixel ratio) are calculated. According to different statistical characteristics, different statistical thresholds are pre-set. If the statistical characteristics of a certain frame of video frame image meet the corresponding statistical threshold, the frame of video frame image is determined to be the transition start image. If there are continuous video frame images that meet the statistical threshold, then this continuous time is recorded. When a certain duration is met, it can be determined to be a transition.

[0074] However, with the addition of animation special effects, the types of transitions have become very diverse, breaking the rules of pure colors (for example, slowly disappearing and reappearing, jumping transitions, or animations in the picture, etc.). The existing transition detection algorithm determines that once a transition occurs in a video, all the video frames therein must meet statistical characteristics. This is not reasonable for transitions with special effects, because the several frames of video frames with special effects will destroy the statistical characteristics. Once the existing video transition detection method finds that a certain frame of video frame does not meet the statistical characteristics, it will reset the parameters of the transition, resulting in the problem of being unable to accurately detect the transition scene.

[0075] To this end, the disclosed embodiments provide a method for detecting video transitions. The method uses an electronic device to obtain video frame images of a video stream and calculate the dominant color ratio of each video frame image frame by frame. Based on the dominant color ratio of each video frame image, the method determines the starting and ending transition video frame images. The method also determines the transition start image based on the number of frames between the starting and ending transition video frame images and the dominant color ratio of the video frame image following the ending transition video frame image. Finally, the method determines the transition end image based on the dominant color ratio of the video frame image following the transition start image. This method improves the accuracy of video transition scene recognition and increases the hit rate of special effect transition scene detection.

[0076] In some embodiments, the above-mentioned video transition detection method can be used in electronic devices. When the electronic device is a terminal, the terminal can be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device and other devices. The present disclosure does not impose any special restrictions on the specific form of the terminal.

[0077] Figure 1 FIG. 1 is a flow chart of a method for detecting a video transition according to an exemplary embodiment. Figure 1 As shown, the video transition detection method is used in an electronic device and includes the following steps.

[0078] In step S101 , a video frame image of a video stream is obtained.

[0079] In the embodiments of the present disclosure, the video stream may be a video stream collected by the electronic device, a video stream downloaded from a server, a video stream stored locally on the electronic device, etc. The embodiments of the present disclosure do not limit the method for obtaining the video stream.

[0080] It can be understood that the video stream is composed of video frame images spliced together, so the electronic device can extract each video frame image from the video stream.

[0081] In step S102 , the main tone ratio of each video frame is calculated frame by frame.

[0082] Hue refers to the overall color tendency of an image and is the overall color effect. The dominant hue is the hue that accounts for the largest proportion of the hues in the entire video frame. The dominant hue ratio refers to the proportion of the primary hue in a video frame. For example, the dominant hue ratio can be the black pixel ratio, which is the ratio of the number of black pixels in a video frame to the total number of pixels in the entire video frame.

[0083] In an embodiment of the present disclosure, after the electronic device obtains the video frame image of the video stream, it can cluster the various tones in each video frame image. Then, based on the clustering results of the various tones of the video frame image, the electronic device calculates the proportion of the main tones in each video frame image to the total tones, and thus obtains the main tones ratio.

[0084] In one possible embodiment of the present disclosure, after acquiring a video frame image of a video stream, the electronic device may convert each video frame image of the video stream into a video frame image in a hue saturation value (HSV) color model space. The electronic device then calculates the dominant hue ratio of each video frame image in the HSV space frame by frame.

[0085] HSV is a color space created by AR Smith in 1978 based on the intuitive characteristics of color. It is also called the hexcone model. The color parameters in this model are: hue (H), saturation (S), and brightness (V). Saturation indicates how close the color is to the light spectrum; brightness indicates how bright the color is.

[0086] It should be noted that the video frames of a video stream captured by an electronic device are typically images in a red-green-blue (RGB) color mode. Since images in the RGB color mode differ significantly from human visual perception, and HSV is a commonly used color space, in the disclosed embodiments, after the electronic device captures the video frames of a video stream, it can convert each video frame of the video stream into an HSV space so that each video frame can be processed in the HSV space.

[0087] In step S103 , a starting transition video frame image and an ending transition video frame image are determined according to the main color ratio of each video frame image.

[0088] Among them, the starting jump video frame image refers to a video frame image that appears for the first time in subsequent video frame images whose video frame images in the video stream meet the set threshold, in which the main color ratio of the frame image does not meet the set threshold, and the main color ratio of the frame image before the frame image meets the set threshold.

[0089] The ending jump video frame image refers to a video frame image in a video stream whose main color ratio does not meet a set threshold, and a video frame image whose main color ratio meets the set threshold is at least one subsequent video frame image.

[0090] If the threshold is set to a first threshold, and the first threshold can be a pre-set maximum value of the main color ratio that can determine that the video frame image is a pure color image. In the embodiment of the present disclosure, if the main color ratio of the video frame image is greater than or equal to the first threshold, then the video frame image can be determined to be a pure color image. For example, assuming that the main color ratio is a black pixel ratio, if the black pixel ratio of a certain video frame image is greater than or equal to the first threshold, then the video frame image can be determined to be a black image.

[0091] In an embodiment of the present disclosure, when an animation special effect is added to a transition segment in a video stream, after determining that the main color ratio of a video frame image in the video stream is greater than or equal to a first threshold, and the main color ratio of a video frame image subsequent to the video frame image is less than the first threshold, the video frame image subsequent to the video frame image is determined to be a starting jump video frame image. That is, a jump video frame image refers to a video frame image whose main color ratio is less than the first threshold between consecutive video frame images (pure color images) whose main color ratio is greater than or equal to the first threshold.

[0092] As an example, assuming that the main color ratio of the 5th frame video frame image of the video stream is greater than the first threshold, and the main color ratio of the 6th frame video frame image is less than the first threshold, the electronic device can determine that the 6th frame video frame image is the starting jump video frame image.

[0093] In an embodiment of the present disclosure, if the main color tone ratio of a frame of video frame image in a video stream is less than a first threshold, and the main color tone ratio of at least one frame of video frame image after the frame of video frame image is greater than or equal to the first threshold, then the frame of video frame image is determined to be an end jump video frame image.

[0094] As an example, assuming that the main color tone ratio of the 10th frame video frame image of the video stream is less than the first threshold, and the main color tone ratios of the subsequent frames (for example, 3 frames) of the 10th frame video frame image are greater than or equal to the first threshold, the electronic device can determine that the 10th frame video frame image is the end jump video frame image.

[0095] In step S104 , a transition start image is determined according to the number of frames between the start jump video frame image and the end jump video frame image and the main tone ratio of the video frame image following the end jump video frame image.

[0096] A transition is the transition or change between scenes in a video. A transition with special effects involves editing two frames together using electronic stunts or post-production software to create a scene change. The transition start image is the first video frame that begins the transition after a scene ends.

[0097] In one possible case of an embodiment of the present disclosure, after the electronic device determines the starting jump video frame image and the ending jump video frame image from the video stream image, if the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color ratio of the video frame image after the ending jump video frame image is greater than or equal to the first threshold, then the video frame image before the starting jump video frame image is determined to be the transition start image. That is, the main color ratio of the transition start image is greater than or equal to the first threshold, the main color ratio of the video frame image after the transition start image is less than the first threshold, and the number of jump video frame images with a main color ratio less than the first threshold is less than the second threshold. In this way, the situation in which a video frame image with a main color ratio greater than the first threshold appears in the positive film in the video stream and is mistakenly judged as a transition start image is avoided.

[0098] It is understood that when an animation effect is added to a transition segment in a video stream, the electronic device may detect a jump video frame image with a main color ratio less than the first threshold after detecting a solid color image with a main color ratio greater than or equal to a first threshold. If the number of consecutive jump video frame images is less than a second threshold, the electronic device may determine that the consecutive jump video frame images are video frame images corresponding to the special effect animation in the transition segment.

[0099] As an example, Figures 2 to 4 As shown, assuming that the transition segment of the video stream is increased Figure 2 The animation effect of the gradual transition in, or, adding Figure 3 Animation effects in, or, added Figure 4 After the electronic device detects a video frame image whose main color ratio is greater than or equal to a first threshold (for example, a pure black video frame image), it may detect Figures 2 to 4 If the number of consecutive jump video frame images is less than the second threshold, the electronic device can determine that the consecutive jump video frame images are video frame images corresponding to the special effect animation in the transition segment.

[0100] In another possible embodiment of the present disclosure, if the number of frames between the start transition video frame image and the end transition video frame image is greater than or equal to a second threshold, the electronic device re-determines the transition start image from subsequent video frames in the video stream. That is, the electronic device determines, based on the main color ratio of the subsequent video frames in the video stream, a video frame image whose main color ratio is greater than or equal to the first threshold.

[0101] For example, the video frame image whose main tone ratio is greater than or equal to the first threshold is the 7th video frame image. If the main tone ratio of the 8th video frame image is less than the first threshold, the 8th video frame image is determined to be the starting jump video frame image. If the number of frames from the starting jump video frame image to the ending jump video frame image is greater than or equal to the second threshold, the transition start image is re-determined in the subsequent video frame images of the video stream. Thus, when the number of frames of the continuous jump video frame images is large and does not meet the condition of being less than the second threshold, the electronic device re-determines the transition start image in the subsequent video frame images of the video stream.

[0102] In step S105 , a transition end image is determined according to the main tone ratio of the video frame image subsequent to the transition start image.

[0103] In one possible case of an embodiment of the present disclosure, based on the main color tone ratio of the video frame image subsequent to the transition start image, it is determined that the main color tone ratio of a frame of video frame image subsequent to the transition start image is less than a first threshold, and the transition duration reaches a threshold, that is, the number of frames from the transition start image to the frame of video frame image subsequent to the transition start image is greater than or equal to a third threshold, then the frame of video frame image subsequent to the transition start image is determined to be the transition end image.

[0104] As an example, assuming that the third threshold is 10 frames, if the main color tone ratio of the 20th frame video frame image of the video stream is less than the first threshold, and the number of frames between the transition start image (the 5th frame video frame image) and the 20th frame video frame image is greater than 10 frames, then the 20th frame video frame image is determined to be the transition end image.

[0105] In another possible case of the embodiment of the present disclosure, if the main color tone ratio of a video frame image subsequent to the transition start image is less than a first threshold, and the number of frames between the transition start image and the video frame image subsequent to the transition start image is less than a third threshold, the transition end image is re-determined in the subsequent video frame images of the video stream.

[0106] It is understood that even if a video frame with a main tone ratio less than the first threshold appears after the transition start image, if the number of frames between the transition start image and the transition end image is less than the third threshold, that is, the transition duration has not reached the time threshold, the electronic device can re-determine the transition end image from the subsequent video frames in the video stream, thereby improving the accuracy of video transition detection.

[0107] The technical solution provided by the above embodiment brings at least the following beneficial effects: As can be seen from S101-S105, since the restrictions on transition judgment are relaxed and the constraints on jump video frame images are added, after acquiring the video frame images of the video stream, the electronic device can determine the starting jump video frame image and the ending jump video frame image from the video frame images based on the main color ratio of each video frame image. As a result, the electronic device can determine the transition start image based on the starting jump video frame image and the ending jump video frame image, and determine the transition end image based on the video frame images of the subsequent video stream, thereby improving the accuracy of video transition scene recognition and increasing the hit rate of special effect transition scenes.

[0108] In some embodiments, if the main tone ratio is the black pixel ratio, such as Figure 5 As shown, when calculating the main tone ratio of each video frame image frame by frame in the above step S102, it can specifically include steps S501 and S502.

[0109] In step S501 , each pixel of each video frame image is traversed, and if the pixel value of the pixel is less than a fourth threshold, the pixel is determined to be a black pixel.

[0110] The fourth threshold value can be a preset maximum pixel value of black pixels. The pixel value is the value assigned by the computer when the original image is digitized. It represents the average brightness information of a small square in the original, or the average reflection (transmission) density information of the small square.

[0111] In one possible scenario of an embodiment of the present disclosure, if the video frame image is an image in RGB color mode, the electronic device may pre-set a fourth threshold value, traverse each pixel of each video frame image according to the length and width of the image, and determine that the pixel is a black pixel if the pixel value is less than the fourth threshold value. If the pixel value is greater than or equal to the fourth threshold value, the pixel is determined not to be a black pixel.

[0112] As an example, assuming that the fourth threshold is RGB (0.5, 0.5, 0.5), the electronic device traverses each pixel of each video frame image according to the length and width of the viewing image. If the pixel value of the pixel in the RGB color mode is less than RGB (0.5, 0.5, 0.5), the pixel is determined to be a black pixel. For example, if the pixel value of a certain pixel of a certain video frame image is RGB (0.1, 0, 0.2), the pixel is determined to be a black pixel. If the pixel value of a certain pixel of a certain video frame image is RGB (0.5, 0.6, 0.6), the pixel is determined not to be a black pixel.

[0113] In another possible embodiment of the present disclosure, after acquiring a video frame image of a video stream, the electronic device converts each video frame image of the video stream into a video frame image in the HSV space. The electronic device traverses each pixel of each video frame image in the HSV space and determines that the pixel is a black pixel if the pixel value of the pixel is less than a fourth threshold.

[0114] As an example, assuming that the fourth threshold is HSV (0.1, 0.1, 0.1), the electronic device traverses each pixel of each video frame image according to the length and width of the viewing image. If the pixel value of the pixel in the HSV space is less than RGB (0.1, 0.1, 0.1), the pixel is determined to be a black pixel. For example, if the pixel value of a certain pixel of a certain video frame image is RGB (0.01, 0, 0.02), the pixel is determined to be a black pixel. If the pixel value of a certain pixel of a certain video frame image is RGB (0.5, 0.6, 0.6), the pixel is determined not to be a black pixel.

[0115] In step S502 , the ratio of the number of black pixels in each video frame image to the total number of pixels in the corresponding video frame image is calculated frame by frame to determine the black pixel ratio of each video frame image.

[0116] In an embodiment of the present disclosure, the electronic device traverses each pixel of each video frame image, determines the black pixels in each video frame image, and then calculates the ratio of the number of black pixels in each video frame image to the total number of pixels of the corresponding video frame image frame by frame to determine the black pixel ratio of each video frame image.

[0117] As an example, assuming that the total number of pixels in a certain video frame image is 5000 and the number of black pixels is 4500, the ratio of black pixels in the video frame image is 4500 / 5000=9 / 10.

[0118] The technical solution provided by the above embodiment provides at least the following beneficial effects: As can be seen from S501-S502, the electronic device can traverse each pixel in each video frame image and determine that the pixel is a black pixel if the pixel value is less than the fourth threshold. The electronic device can then determine the black pixel ratio of the corresponding video frame image based on the number of black pixels and the total number of pixels in each video frame image. Therefore, when detecting video transitions, the electronic device can use the black pixel ratio as a statistical characteristic to determine the transition start image and transition end image.

[0119] In some embodiments, the electronic device may also use the pixel variance of the video frame image to represent the color change of the image. Figure 6 As shown, when calculating the main tone ratio of each video frame image frame by frame in the above step S102, step S601 may be further specifically included.

[0120] In step S601 , the pixel variance of each video frame image is calculated frame by frame.

[0121] In step S602 , the main tone ratio of each video frame image is determined according to the pixel variance of each video frame image.

[0122] The pixel variance is negatively correlated with the main color ratio. That is, the smaller the pixel variance, the less color variation in the video frame image, and the more pure the image.

[0123] In one possible case of an embodiment of the present disclosure, if the video frame image of the video stream obtained by the electronic device is an image in the RGB color space, the electronic device can traverse the pixel values of each video frame image in the RGB color mode, and then calculate the pixel variance of each video frame image frame by frame based on the pixel value of each video frame image in the RGB color mode.

[0124] In another possible case of the embodiments of the present disclosure, if the electronic device obtains the video frame image of the video stream, it can convert each video frame image of the video stream into a video frame image in the HSV space. The electronic device can traverse the pixel values of each video frame image in the HSV space, and then calculate the pixel variance of each video frame image frame by frame based on the pixel value of each video frame image in the HSV space.

[0125] In the embodiment of the present disclosure, the pixel variance of each video frame image of the video stream is negatively correlated with the corresponding main color ratio. Therefore, after the electronic device determines the pixel variance of each video frame image frame by frame, it can determine the main color ratio of each video frame image based on the pixel variance of each video frame image.

[0126] In an embodiment of the present disclosure, after the electronic device obtains the video frame image of the video stream, it traverses each pixel of each video frame image to determine the number of pixels of each video frame image and the pixel value of each pixel. Then, the electronic device determines the pixel mean of each video frame image based on the number of pixels of each video frame image and the pixel value of each pixel. Finally, the electronic device determines the pixel variance of each video frame image based on the pixel value and the pixel mean of each pixel of each video frame image, and determines the main color ratio of each video frame image based on the pixel variance of each video frame image. Thus, the color change of each video frame image can be determined based on the pixel variance of the video frame image.

[0127] As an example, suppose that after an electronic device determines the number of pixels and the pixel value of each pixel in a video frame image, it can add the pixel values of each pixel together to obtain the total pixel value of the video frame image. Furthermore, the ratio of the total pixel value to the number of pixels in the video frame image is calculated to obtain the pixel mean of the video frame image. Furthermore, the average of the squares of the differences between the pixel value of each pixel in the video frame image and the pixel mean is calculated to obtain the pixel variance of the video frame image.

[0128] The technical solution provided by the above embodiment brings at least the following beneficial effects: As can be seen from S601-S602, the electronic device can calculate the pixel variance of each video frame of the video stream frame by frame, determine the main color ratio of each video frame based on the pixel variance of each video frame, and further determine the transition start image and transition end image based on the main color ratio of each video frame. As a result, during the video software post-processing process, the electronic device can accurately detect the transition start image and transition end image in videos with special effects animation, thereby improving the accuracy of video transition detection.

[0129] It is understandable that, in actual implementation, the electronic device described in the embodiment of the present disclosure may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding data processing methods, and these execution hardware structures and / or software modules may constitute an electronic device. It should be easily appreciated by those skilled in the art that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0130] Figure 7 FIG. 1 is a block diagram of a video transition detection device according to an exemplary embodiment. Figure 7The device includes an acquisition module 710, a calculation module 720, a first determination module 730, a second determination module 740 and a third determination module 750.

[0131] The acquisition module 710 may be configured to acquire a video frame image of a video stream.

[0132] The calculation module 720 may be configured to calculate the main tone ratio of each video frame image frame by frame.

[0133] The first determining module 730 may be configured to determine a start transition video frame image and an end transition video frame image according to a main color ratio of each video frame image.

[0134] The second determination module 740 can be configured to determine the transition start image based on the number of frames between the start jump video frame image and the end jump video frame image, and the main color ratio of the next video frame image after the end jump video frame image.

[0135] The third determining module 750 may be configured to determine a transition end image according to a main tone ratio of a video frame image subsequent to the transition start image.

[0136] Optionally, the first determining module 730 may be further configured to execute:

[0137] If the main color ratio of a video frame image of the video stream is greater than or equal to a first threshold, and the main color ratio of a video frame image subsequent to the video frame image is less than the first threshold, determining the video frame image subsequent to the video frame image as a starting jump video frame image;

[0138] If the main color ratio of a video frame image of the video stream is less than a first threshold, and the main color ratio of at least one video frame image subsequent to the video frame image is greater than or equal to the first threshold, the video frame image is determined to be an end jump video frame image.

[0139] Optionally, the second determining module 740 may also be configured to execute:

[0140] If the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color tone ratio of the video frame image after the ending jump video frame image is greater than or equal to the first threshold, then the video frame image before the starting jump video frame image is determined to be the transition start image.

[0141] Optionally, the third determining module 750 may also be configured to execute:

[0142] If the main color tone ratio of a frame of video frame image following the transition start image is less than a first threshold, and the number of frames between the transition start image and the frame of video frame image following the transition start image is greater than or equal to a third threshold, then the frame of video frame image following the transition start image is determined to be the transition end image.

[0143] Optionally, the main color ratio is a black pixel ratio, and the calculation module 720 may be further configured to execute:

[0144] Traverse each pixel of each video frame image, and if the pixel value of the pixel is less than a fourth threshold, determine that the pixel is a black pixel; calculate the ratio of the number of black pixels in each video frame image to the total number of pixels of the corresponding video frame image frame by frame to determine the black pixel ratio of each video frame image.

[0145] Optionally, the calculation module 720 may be further configured to execute:

[0146] The pixel variance of each video frame image is calculated frame by frame; the main tone ratio of each video frame image is determined according to the pixel variance of each video frame image; the pixel variance and the main tone ratio are negatively correlated.

[0147] Optionally, the calculation module 720 may be further configured to execute:

[0148] Traverse each pixel of each video frame image to determine the number of pixels and the pixel value of each pixel in each video frame image; determine the pixel mean of each video frame image based on the number of pixels and the pixel value of each pixel in each video frame image; determine the pixel variance of each video frame image based on the pixel value and the pixel mean of each pixel in each video frame image.

[0149] Optionally, the device may further include:

[0150] The fourth determination module is configured to execute if the number of frames between the starting jump video frame image and the ending jump video frame image is greater than or equal to the second threshold, then redetermine the transition start image in the subsequent video frame images of the video stream; and / or if the main color tone ratio of a frame of video frame image subsequent to the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame image subsequent to the transition start image is less than the third threshold, then redetermine the transition end image in the subsequent video frame images of the video stream.

[0151] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0152] The video transition detection device of the disclosed embodiment, after acquiring the video frame images of the video stream, the electronic device can determine the starting jump video frame image and the ending jump video frame image according to the main color ratio of each video frame image, determine the transition start image according to the number of frames between the starting jump video frame image and the ending jump video frame image, and the main color ratio of the video frame image after the ending jump video frame image, and determine the transition end image according to the main color ratio of the video frame image subsequent to the transition start image. Thus, the electronic device can determine the transition start image according to the starting jump video frame image and the ending jump video frame image, and determine the transition end image according to the video frame images of the subsequent video stream, thereby improving the accuracy of video transition scene recognition and increasing the hit rate of special effect transition scenes.

[0153] As described above, the embodiments of the present disclosure can divide the electronic device into functional modules according to the above method examples. Among them, the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. In addition, it should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module.

[0154] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] In order to implement the above embodiments, the present disclosure also provides an electronic device.

[0156] The electronic device includes: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the video transition detection method of the above embodiment.

[0157] As an example, the electronic device may be a user terminal such as a mobile phone or a computer. Figure 8 FIG1 is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment. The electronic device may include at least one processor 81 , a communication bus 82 , a memory 83 and at least one communication interface 84 .

[0158] The processor 81 may be a central processing unit (CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0159] The communication bus 82 may include a pathway for transmitting information between the aforementioned components.

[0160] The communication interface 84 uses any transceiver or other device for communicating with other devices or communication networks, such as servers, Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc.

[0161] The memory 83 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processing unit via a bus. The memory may also be integrated with the processing unit.

[0162] The memory 83 is used to store application code for executing the solution of the present disclosure, and the execution is controlled by the processor 81. The processor 81 is used to execute the application code stored in the memory 83, thereby realizing the functions of the method of the present disclosure.

[0163] In a specific implementation, as an embodiment, the processor 81 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 in.

[0164] In a specific implementation, as an embodiment, the electronic device may include multiple processors, such as Figure 8 81 and processor 85 in FIG. Each of these processors can be a single-CPU processor or a multi-CPU processor. A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0165] In a specific implementation, as an embodiment, the electronic device may further include an input device 86 and an output device 87. The input device 86 communicates with the output device 87 and can accept user input in various ways. For example, the input device 86 can be a mouse, keyboard, touch screen device, or sensor device. The output device 87 communicates with the processor 81 and can display information in various ways. For example, the output device 81 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, etc.

[0166] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0167] In an exemplary embodiment, the present disclosure also provides a computer-readable storage medium including instructions, on which instructions are stored. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer is enabled to execute the video transition detection method provided in the above-mentioned embodiment.

[0168] In an exemplary embodiment, the present disclosure further provides a computer program product, including a computer program / instruction, which implements the video transition detection method of the above embodiment when the computer program / instruction is executed by a processor.

[0169] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting video transitions, characterized in that: include: Get the video frame image of the video stream; Calculate the main color ratio of each video frame image frame by frame; Determining a starting jump video frame image and an ending jump video frame image according to a main color ratio of each video frame image; Determine a transition start image according to the number of frames between the start jump video frame image and the end jump video frame image, and the main color tone ratio of a video frame image following the end jump video frame image; If the main color tone ratio of the frame of video frame image following the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame of video frame image following the transition start image is greater than or equal to the third threshold, then the frame of video frame image following the transition start image is determined to be the transition end image.

2. The method according to claim 1, characterized in that The determining, according to the main color ratio of each video frame image, a starting jump video frame image and an ending jump video frame image, includes: If the main tone ratio of a video frame image of the video stream is greater than or equal to the first threshold, and the main tone ratio of a video frame image subsequent to the video frame image is less than the first threshold, determining the video frame image subsequent to the video frame image as the starting jump video frame image; If the main tone ratio of a frame of video frame image in the video stream is less than the first threshold, and the main tone ratio of at least one frame of video frame image subsequent to the frame of video frame image is greater than or equal to the first threshold, then the frame of video frame image is determined to be the end jump video frame image.

3. The method according to claim 1, characterized in that The step of determining a transition start image according to the number of frames between the start transition video frame image and the end transition video frame image, and the main color tone ratio of a subsequent video frame image of the end transition video frame image, includes: If the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color tone ratio of the subsequent video frame image of the ending jump video frame image is greater than or equal to the first threshold, then the previous video frame image of the starting jump video frame image is determined to be the transition start image.

4. The method according to claim 1, wherein The main tone ratio is the black pixel ratio, and the main tone ratio of each video frame image is calculated frame by frame, including: Traversing each pixel of each video frame image, and if the pixel value of the pixel is less than a fourth threshold, determining that the pixel is a black pixel; The ratio of the number of black pixels in each video frame image to the total number of pixels in the corresponding video frame image is calculated frame by frame to determine the black pixel ratio of each video frame image.

5. The method according to claim 1, wherein The frame-by-frame calculation of the main tone ratio of each video frame image includes: Calculate the pixel variance of each video frame image frame by frame; The main tone ratio of each video frame image is determined according to the pixel variance of each video frame image; the pixel variance is negatively correlated with the main tone ratio.

6. The method according to claim 5, characterized in that The pixel variance of each video frame image is calculated frame by frame, including: Traversing each pixel of each video frame image to determine the number of pixels in each video frame image and the pixel value of each pixel; Determining a pixel mean value of each video frame image according to the number of pixels in each video frame image and the pixel value of each pixel; The pixel variance of each video frame image is determined according to the pixel value of each pixel of each video frame image and the pixel mean.

7. The method according to any one of claims 4 to 6, characterized in that: Before calculating the main tone ratio of each video frame image frame by frame, the method includes: Each video frame image of the video stream is converted into a video frame image in an HSV color model space.

8. The method according to claim 3, characterized in that The method further comprises: If the number of frames between the start jump video frame image and the end jump video frame image is greater than or equal to the second threshold, then re-determining the transition start image in subsequent video frame images of the video stream; and / or, If the main color tone ratio of a video frame image subsequent to the transition start image is less than the first threshold, and the number of frames between the transition start image and the video frame image subsequent to the transition start image is less than the third threshold, the transition end image is re-determined in the subsequent video frame images of the video stream.

9. A video transition detection device, characterized in that: include: An acquisition module is configured to acquire a video frame image of a video stream; A calculation module is configured to calculate the main color ratio of each video frame image frame by frame; A first determining module is configured to determine a starting jump video frame image and an ending jump video frame image according to a main color ratio of each video frame image; The second determining module is configured to determine the transition start image according to the number of frames between the start jump video frame image and the end jump video frame image, and the main color ratio of the video frame image after the end jump video frame image; The third determination module is configured to execute if the main color tone ratio of the frame of video frame image following the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame of video frame image following the transition start image is greater than or equal to the third threshold, then determine that the frame of video frame image following the transition start image is the transition end image.

10. The device according to claim 9, characterized in that The first determining module is further configured to execute: If the main tone ratio of a video frame image of the video stream is greater than or equal to the first threshold, and the main tone ratio of a video frame image subsequent to the video frame image is less than the first threshold, determining the video frame image subsequent to the video frame image as the starting jump video frame image; If the main tone ratio of a frame of video frame image in the video stream is less than the first threshold, and the main tone ratio of at least one frame of video frame image subsequent to the frame of video frame image is greater than or equal to the first threshold, then the frame of video frame image is determined to be the end jump video frame image.

11. The device according to claim 9, characterized in that The second determining module is further configured to execute: If the number of frames between the starting jump video frame image and the ending jump video frame image is less than the second threshold, and the main color tone ratio of the subsequent video frame image of the ending jump video frame image is greater than or equal to the first threshold, then the previous video frame image of the starting jump video frame image is determined to be the transition start image.

12. The device according to claim 9, characterized in that The main tone ratio is a black pixel ratio, and the calculation module is further configured to execute: Traversing each pixel of each video frame image, and if the pixel value of the pixel is less than a fourth threshold, determining that the pixel is a black pixel; The ratio of the number of black pixels in each video frame image to the total number of pixels in the corresponding video frame image is calculated frame by frame to determine the black pixel ratio of each video frame image.

13. The device according to claim 9, characterized in that The computing module is further configured to execute: Calculate the pixel variance of each video frame image frame by frame; The main tone ratio of each video frame image is determined according to the pixel variance of each video frame image; the pixel variance is negatively correlated with the main tone ratio.

14. The device according to claim 13, characterized in that The computing module is further configured to execute: Traversing each pixel of each video frame image to determine the number of pixels in each video frame image and the pixel value of each pixel; Determining a pixel mean value of each video frame image according to the number of pixels in each video frame image and the pixel value of each pixel; The pixel variance of each video frame image is determined according to the pixel value of each pixel of each video frame image and the pixel mean.

15. The device according to any one of claims 12 to 14, characterized in that The device further comprises: The conversion module is configured to convert each video frame image of the video stream into a video frame image of a color model HSV space.

16. The device according to claim 11, characterized in that The device further comprises: The fourth determination module is configured to execute if the number of frames between the starting jump video frame image and the ending jump video frame image is greater than or equal to the second threshold, then redetermine the transition start image in the subsequent video frame images of the video stream; and / or, if the main color tone ratio of a frame of video frame image subsequent to the transition start image is less than the first threshold, and the number of frames between the transition start image and the frame of video frame image subsequent to the transition start image is less than the third threshold, then redetermine the transition end image in the subsequent video frame images of the video stream.

17. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video transition detection method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method for detecting video transitions according to any one of claims 1 to 8.

19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for detecting video transitions according to any one of claims 1 to 8 is implemented.

Images