Video Title and End Detection Method and Device

Abstract

The present application provides a method and device for detecting video opening and closing credits. When performing detection, this solution uses the SIFT algorithm to find key points and their feature information in video frame images, can extract local features in the images, and avoids the influence of image rotation, scale scaling, brightness change, etc. on the features. Therefore, the feature information of key points in a single video can be used to accurately identify video frame images that meet the preset conditions for opening / closing credits during video playback, without being affected by the fact that the opening / closing credits of different video series do not have a fixed form, and has better accuracy and versatility.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to an unsupervised method and device for detecting video headers and endings based on image features. Background Art

[0002] In recent years, the number of video programs available on video platforms, such as video websites, IPTV (Interactive Personality TV), and Internet TV, has increased significantly, and each video platform has become increasingly aware of its user service and user competition. TV series typically have fixed opening and closing credits. While watching a TV series, many users would like to skip these opening and closing credits and proceed directly to the main content. To address this need, various video platforms currently use the following solutions:

[0003] Manual labeling: Manually label the opening and ending credits of each episode of a TV series. This method requires the least technical investment but the most manpower.

[0004] Leverage user viewing behavior patterns. This method collects big data on user viewing behavior and annotates the locations where users frequently jump to at the beginning and end of a TV series. This approach doesn't primarily analyze the relevant characteristics of the TV series itself; it requires collecting viewing data before annotating the series, making it an indirect approach. Furthermore, because user jump locations may vary due to different operating habits, the accuracy of the resulting processing results is limited.

[0005] Modes that utilize video features. Currently, there are many publicly available methods for utilizing video features, such as searching for specific images and finding shot cuts. These methods require using similar videos as training samples and then processing them with artificial intelligence to find the opening and ending credits. However, due to the wide variety of opening and ending credits in current videos, ranging from previews of the next episode to behind-the-scenes footage, and even the lack of a fixed format for different episodes of the same TV series, detecting them using existing methods is difficult and lacks accuracy. Summary of the invention

[0006] One purpose of the present application is to provide a method and device for detecting video headers and end credits.

[0007] To achieve the above objectives, the present application provides a method for detecting the end of a video, the method comprising:

[0008] Cut out the video clip of preset length at the end of the video file;

[0009] Sample the video clip at a preset sampling frequency and extract a preset number of video frame images;

[0010] Use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, and obtain a key point list for each video frame image, where the key point list includes the key points of the video frame image and the feature information corresponding to the key points;

[0011] Traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtain an image similarity point array, where the image similarity point array includes the number of common key points between each video frame image and its corresponding associated video frame image;

[0012] Find the target position where the number of common key points meets the preset conditions from the image similarity point array, and determine the sampling position corresponding to the target position as the end-credit position.

[0013] Further, sampling the video clip at a preset sampling frequency and extracting a preset number of video frame images includes:

[0014] Sample the video clip at a sampling frequency of 1 time per second and extract a preset number of video frame images.

[0015] Further, traversing and calculating the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtaining an image similarity point array includes:

[0016] Perform the following traversal processing on each video frame image: obtain a video frame image and an associated video frame image with a sampling interval of N from the video frame image, compare each key point in the key point list of the video frame image and the key point list of the associated video frame image, and determine the same key points in the two video frame images as the common key points;

[0017] Count the number of common key points between each video frame image and its corresponding associated video frame image, and obtain an image similarity point array.

[0018] Further, finding the target position where the number of common key points meets the preset conditions from the image similarity point array includes:

[0019] Find the maximum value position from the image similarity point array, where the maximum value position is the array position with the largest number of common key points;

[0020] Search forward from the maximum value position. When the position of the array where the number of common key points found for the first time is lower than the first threshold is found, determine this array position as the candidate array position;

[0021] Search forward from the candidate array position. If no array position with the number of common key points higher than the second threshold is found, determine the candidate array position as the target position;

[0022] If an array position with the number of common key points higher than the second threshold is found, continue to search forward. When a new array position where the number of common key points is lower than the first threshold is found, update this array position as the candidate array position and continue to search forward from the candidate array position until traversing to the first array position of the image similarity point array;

[0023] Determine the candidate array position as the target position.

[0024] Further, the first threshold and the second threshold are respectively determined based on a preset ratio of the maximum value of the number of common key points.

[0025] Further, finding the maximum value position from the image similarity point array includes:

[0026] Generate a histogram according to the image similarity point array, where the horizontal axis of the histogram is the array position and the vertical axis is the number of common key points;

[0027] Select the highest point from the histogram and determine the horizontal axis position corresponding to the highest point as the maximum value position.

[0028] The embodiment of the present application also provides a video title detection method, and the method includes:

[0029] Intercept a video segment with a preset duration at the head of the video file;

[0030] Sample the video segment at a preset sampling frequency and extract a preset number of video frame images;

[0031] Use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, and obtain a key point list for each video frame image, where the key point list includes the key points of the video frame image and the feature information corresponding to the key points;

[0032] Traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtain an image similarity point array, where the image similarity point array includes the number of common key points between each video frame image and its corresponding associated video frame image;

[0033] From the image similarity point array, a target position whose number of common key points meets a preset condition is found, and a sampling position corresponding to the target position is determined as a header position.

[0034] Furthermore, finding a target position whose number of common key points meets a preset condition from the image similarity point array includes:

[0035] Finding a maximum value position from the image similarity point array, wherein the maximum value position is the array position with the largest number of shared key points;

[0036] Searching backward from the maximum value position, when the first array position where the number of common key points is lower than a first threshold is found, determining the array position as a candidate array position;

[0037] Searching backward from the candidate array position, if no array position with the number of common key points higher than a second threshold is found, determining the candidate array position as the target position;

[0038] If an array position is found where the number of shared key points is higher than the second threshold, continue searching backward. When a new array position is found where the number of shared key points is lower than the first threshold, update the array position to a candidate array position, and continue searching backward from the candidate array position until the last array position of the image similarity point array is traversed.

[0039] An embodiment of the present application also provides a computing device, which includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the video header and end detection method.

[0040] An embodiment of the present application further provides a computer-readable medium having computer program instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the video header and end detection method.

[0041] Compared with the prior art, after intercepting a video clip of a preset duration at the end of a video file, a detection scheme provided by an embodiment of the present application can sample the video clip at a preset sampling frequency, extract a preset number of video frame images, and then use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, obtain a key point list of each video frame image, traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, obtain an image similarity point array, find a target position where the number of common key points meets the preset conditions from the image similarity point array, and determine the sampling position corresponding to the target position as the end position of the video. Since the SIFT algorithm is used in this scheme to find the key points and their feature information in the video frame image, local features in the image can be extracted, and the influence of image rotation, scale scaling, brightness change, etc. on the features can be avoided. Therefore, the feature information of the key points in a single video can be used to accurately identify the video frame images that meet the preset conditions for the end of the video during the video playback, without being affected by the non-fixed forms of the end credits of different video episodes, and has better accuracy and generality.

[0042] In addition, the detection scheme provided by an embodiment of the present application can also be used for the detection of the video start. When detecting the start, a video clip of a preset duration at the head of the video file can be intercepted, sampled at a preset sampling frequency, extract a preset number of video frame images, and then use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, obtain a key point list of each video frame image, traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, obtain an image similarity point array, find a target position where the number of common key points meets the preset conditions from the image similarity point array, and determine the sampling position corresponding to the target position as the start position, also without being affected by the non-fixed forms of the starts of different video episodes, and has better accuracy and generality. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objectives, and advantages of the present application will become more obvious:

[0044] Figure 1 It is a processing flowchart of a video end detection method provided by an embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the comparison mode of the key point list in an embodiment of the present application;

[0046] Figure 3The processing flow chart for determining the target position in the embodiments of this application;

[0047] Figure 4 A histogram of the image similarity point array in the embodiments of this application;

[0048] Figure 5 Another histogram of the image similarity point array in the embodiments of this application;

[0049] Figure 6 Another histogram of the image similarity point array in the embodiments of this application;

[0050] Figure 7 The processing flow chart of a video title detection method provided by the embodiments of this application;

[0051] The same or similar reference numerals in the drawings represent the same or similar components. Detailed implementation manners

[0052] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation to the present invention.

[0053] Those skilled in the art of this technology can understand that unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0054] A video end credit detection method provided by an embodiment of the present application uses the SIFT (Scale-invariant feature transform) algorithm to calculate the key points between different sampled video frame images respectively, and counts the number of common key points that are the same in two video frame images, and uses the number of common key points to judge the starting position of the end credits. Since the SIFT algorithm can find the key points and their feature information in the video frame images, can extract the local features in the images, and avoids the influence of image rotation, scale scaling, brightness change, etc. on the features, the feature information of the key points in a single video can be used to accurately identify the video frame images that meet the preset conditions of the end credits during the video playback, without being affected by the fact that the end credits of different video episodes do not have a fixed form, and has better accuracy and versatility.

[0055] In an actual scenario, the execution subject of this method can be a user device, a network device, or a device formed by integrating the user device and the network device through a network, or can also be an application program running on the above devices. The user device includes but is not limited to various terminal devices such as computers, mobile phones, and tablet computers; the network device includes but is not limited to network hosts, single network servers, servers with multiple network servers concentrated, or servers in a distributed cloud network. Here, the distributed cloud network is composed of a large number of hosts or network servers based on Cloud Computing.

[0056] Figure 1 Shows the processing flow of a video end credit detection method provided by an embodiment of the present application,

[0057] at least including the following processing steps:

[0058] Step S101, intercept a video segment with a preset duration at the end of the video file. In an actual scenario, different types of video files will have end credits of different lengths. For example, a TV drama of about 40 minutes generally has end credits of about 3 - 4 minutes, while a short drama of about 20 minutes may have end credits of about 2 - 3 minutes, and for a movie with a longer overall duration, its end credits may be 5 - 6 minutes or even longer. Therefore, when intercepting the video segment, the appropriate preset duration can be set according to the type of the video file to be processed, and on the premise of ensuring that the complete end credits can be intercepted, the intercepted video segment is shortened as much as possible to reduce the computational amount of subsequent processing.

[0059] In this embodiment, taking the processing of TV dramas as an example, the preset duration can be set to 5 minutes. First, a 5-minute video clip is intercepted from the end of the TV drama video. Setting it to 5 minutes mainly considers that the current normal length of the end credits of TV dramas is about 3-4 minutes, and there are also relevant regulations on the length of the opening and ending credits of TV dramas in the film and television industry, that is, the longest opening and ending credits of TV dramas do not exceed 4 minutes. Therefore, the intercepted range can be set to 5 minutes at the end of the video file.

[0060] Step S102: Sample the video clip at a preset sampling frequency and extract a preset number of video frame images. Among them, the preset sampling frequency refers to the frequency of extracting video frame images from the video file according to the time sequence of video playback, and its specific value can be set according to the needs of the actual scenario. The higher the sampling frequency, the more video frame images are extracted per unit time.

[0061] In this embodiment, considering that if the sampling frequency is too high, the interval between the sampled video frame images will be too close, which may cause the similarity between adjacent video frame images during normal playback to be too high, bringing problems to subsequent similarity calculation. At the same time, considering that in the end credits of the video, although the form may have many changes, there will still be certain rules in the layout. For example, there will be regularly scrolling subtitles at the end of the video file, and there will be certain layout rules between the scrolling subtitles and between the scrolling subtitles and other image contents. Therefore, the interval between the sampled video frame images cannot be too small to avoid the inability to reflect the changes in video content between similar video frame images.

[0062] Based on the above considerations, in this embodiment, the sampling frequency can be set to 1 time per second, that is, sample the video clip at a sampling frequency of 1 time per second and extract a preset number of video frame images. Thus, for the aforementioned 5-minute end-credit video clip, sample once every 1 second. The number of video frame images that can be extracted from a video clip is 5 minutes × 60 seconds × 1 image / second = 300 images. These video frame images can be recorded as P1, P2,..., P300, and according to the sampling order of these video frame images, these video frame images can be located at the corresponding time points in the video file.

[0063] Step S103: Use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, and obtain the key point list of each video frame image.

[0064] The SIFT algorithm is a machine vision algorithm used to detect and describe local features in images. It finds extreme points in the spatial scale and extracts their positions, scales, and rotation invariants. The description and detection of local features can help identify objects. The SIFT algorithm or the extracted feature information is based on some local appearance interest points on the object and is independent of the size and rotation of the image. It also has a relatively high tolerance for light, noise, and slight perspective changes. Based on these characteristics, they are highly prominent and relatively easy to extract. In a large feature database, it is easy to identify objects and there are few misidentifications. The feature information description using the SIFT algorithm also has a relatively high detection rate for partial object occlusion. Even with more than 3 SIFT object features, it is sufficient to calculate the position and orientation. Under the current computer hardware speed and a small feature database condition, the identification speed can be close to real-time operation, suitable for fast and accurate matching in a large database. Taking the end credits of a TV drama in this embodiment as an example, the local features extracted based on the SIFT algorithm mainly come from the rolling subtitles at the end credits. When the rolling subtitles are played, there is a certain similarity in the local features of the rolling subtitles among the video frame images. Therefore, based on this similarity, it can be determined whether the end credits are being played currently. In addition, some end credit video styles can also be considered, such as the layout of the video image, with the video in the middle and a fixed display bar below, etc.

[0065] In the solution of this embodiment, based on the key points and their corresponding feature information found in the video frame images by the SIFT algorithm, it is possible to be independent of rotation, scale scaling, and brightness changes, and to shield the features of non-key points and the image edges. Therefore, through the above steps, a key point list for each video frame image can be obtained. The key point list includes each key point and feature information of the video frame object. In an actual scenario, for each key point and its corresponding feature information, it can be represented in the form of a group of vectors, specifically including the position, scale, and direction of the key point, etc. Taking the previous scenario as an example, since the number of video frame images sampled from the video segments of each video file is 300, the 300 key point lists corresponding to the number of video frame images generated in this step are respectively recorded as KeyPoint_Of_P1, KeyPoint_Of_P2, ……, KeyPoint_Of_300.

[0066] Step S104, traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtain an image similarity point array.

[0067] Among them, the associated video frame image refers to a video frame image with a sampling interval relatively close to that of the video frame image being compared. The sampling interval refers to the time sequence interval between two video frame images during sampling. Taking the video frame images P1 to P300 sampled at a fixed time sequence as an example, the video frame image P1 and the video frame image P2 are adjacent video frame images successively sampled, and there is no other video frame image between them, so the sampling interval between them is defined as 0. While there is the video frame image P2 between the video frame image P1 and the video frame image P3 during sampling, so the sampling interval between them is defined as 1. To ensure a certain correlation between the two video frame images for calculating the common key points, it is necessary to ensure that the sampling interval between the two video frame images is not too large. For example, in this embodiment, it can be set to 1, 2, or 3, etc.

[0068] Furthermore, considering that if the sampling interval is too close, the video content contained in the video frame image may not change much, and at the same time, the two video frame images being compared cannot be too far apart, resulting in the failure to capture the changes in the features to be grabbed. Therefore, in some embodiments of the present application, the associated video frame image is set as the video frame image with a sampling interval of 1. That is, the associated video frame image of the video frame image P1 is P3. The corresponding key point list KeyPoint_Of_P1 is compared with KeyPoint_Of_P3 to calculate the number of common key points existing between the two video frame images P1 and P3. Other video frame images are also processed in the same way. The comparison mode is as Figure 2 shown. The video frame image P2 is compared with its associated video frame image P4 according to the corresponding key point list, and the video frame image P3 is compared with its associated video frame image P5 according to the corresponding key point list, and so on until the comparison between the video frame image P298 and its associated video frame image P300 is completed to determine the number of common key points. Based on the above traversal calculation, a total of 298 data on the number of common key points can be obtained, from which an array with a length of 298 can be generated. This array is the image similarity point array SamepointNumber, and the image similarity point array includes the number of common key points between each video frame image and its corresponding associated video frame image.

[0069] The common key points refer to the same two key points in two video frame images. Taking the two video frame images P1 and P3 being compared as an example, each key point in the key point list KeyPoint_Of_P1 can be compared with that in KeyPoint_Of_P3 to find the same key points in the two video frame images and determine them as the common key points.

[0070] Thus, in some embodiments of the present application, when obtaining the array of image similarity points, the following traversal processing can be first performed on each video frame image: First, obtain a video frame image and an associated video frame image with a sampling interval of N from the video frame image, where N can be set to a non-negative integer according to the actual requirements of the detection scheme. For example, in this embodiment, in order to prevent the time interval between adjacent video frame images from being too small and the image content from being too close, a method of comparing every other video frame image is adopted, and N is set to 1. In this way, by such an interval method, local features of the end credits that are superimposed can be found as much as possible, such as local features across images like subtitles or credits, reducing the interference of the underlying video content on the determination of the end credits.

[0071] Then, compare each key point in the list of key points of the video frame image and the list of key points of the associated video frame image, and determine the same key points in the two video frame images as the common key points. In an actual scenario, since the list of key points of the video frame image contains the feature information of each key point, by comparing the feature information of each key point in the two video frame images, it can be determined whether the two key points are the same key points. For example, a part of the same local image content will appear in the video frame image at the same time. Since the content of these local images is the same, the feature information extracted by the SIFT algorithm should also be the same or highly similar. By comparing the feature information, the common key points corresponding to these same local image contents can be found from the two video frame images.

[0072] After completing the traversal processing of each video frame image, the number of common key points between each video frame image and its corresponding associated video frame image can be counted, thereby obtaining the array of image similarity points.

[0073] Step S105, from the array of image similarity points, find the target position where the number of common key points meets the preset condition, and determine the sampling position corresponding to the target position as the end credits position.

[0074] Since each array element in the array of image similarity points represents the similarity degree of local features between video frame images, the larger the value of the array element, that is, the more common key points, the more similar the local features between the two video frame images. Therefore, according to the type of the detected video file, an appropriate preset condition can be set, and based on the specific situation of the number of common key points in the array of image similarity points, the video frame image when the end credits start playing can be determined, so as to accurately detect the position of the end credits in the video file.

[0075] In this solution, the SIFT algorithm is used to find the key points and their feature information in the video frame image, which can extract the local features in the image and avoid the influence of image rotation, scale scaling, brightness change, etc. on the features. Therefore, the feature information of the key points in a single video can be used to accurately identify the video frame images that meet the preset conditions at the end of the video during the video playback process, without being affected by the non-fixed form of the end credits of different video episodes, and has better accuracy and versatility. It can be applied not only to TV dramas but also to the end recognition of single-episode videos such as individual movies.

[0076] In some embodiments of the present application, when finding the target position where the number of common key points in the image similarity point array meets the preset conditions, the following processing flow can be adopted as Figure 3 shown, including the following steps:

[0077] Step S301, find the maximum value position from the image similarity point array. Among them, the maximum value position is the array position with the largest number of common key points. Taking the aforementioned image similarity point array SamepointNumber as an example, it contains 298 array elements, which correspond to the number of common key points between video frame images P1 and P3, P2 and P2,..., P298 and P300 in sequence. Find the maximum value among these array elements, and the array position where the maximum value is located is the maximum value position.

[0078] In this embodiment, when finding the maximum value position from the image similarity point array, the histogram method can be adopted. First, generate a histogram according to the image similarity point array. In this histogram, the horizontal axis (x-axis) represents the array position, and the vertical axis (y-axis) represents the number of common key points. For example, the height of the histogram line segment at the abscissa 20 in the histogram is 33, and the vertex coordinates of this line segment are (20, 33), which means that there are 33 common key points between video frame images P20 and P22. After generating the histogram, the highest point can be selected from the histogram, and the horizontal axis position corresponding to the highest point is determined as the maximum value position. Taking Figure 4 the shown histogram as an example, it can be determined that its highest point is the vertex of the histogram line segment at the abscissa 150. From this, the horizontal axis position here can be determined as the maximum value position, that is, the 150th array element in the image similarity point array, corresponding to video frame images P150 and P152. This array position represents the time point when the local features in the end credits of the video highly overlap, such as the time points corresponding to two video frame images when the cast and crew credits fill the entire screen.

[0079] Step S302, search forward from the maximum value position. When finding the first array position where the number of common key points is lower than the first threshold, determine this array position as the candidate array position.

[0080] Among them, the first threshold can be determined based on a preset ratio of the maximum value of the number of the common key points. For example, in this embodiment, the preset ratio is set to 30%. If the maximum value of the number of key points is 160, then the first threshold is 48. Thus, when searching forward from the maximum value position, when finding the array position where the number of the first common key points is lower than 48, this array position is determined as the candidate array position. Taking the previous scenario as an example, searching forward from the maximum value position means searching forward from the abscissa 150. The histogram line segment corresponding to the number of key points 48 is the line segment at the abscissa 36 position. The array position with the abscissa 36 is determined as the candidate array position, that is, the 36th array element in the image similarity point array, corresponding to the video frame images P36 and P38. This point array position represents the time point when the overlapping local features start to appear in the end of the video. For example, the time points corresponding to the two video frame images when the cast and crew list starts to appear on the screen. This time point has a high probability of being the start time point of the end of the video.

[0081] If the histogram corresponding to the image similarity point array is as Figure 4 shown, then at this time, the candidate array position can be directly determined as the target position. However, in the actual scenario, generally speaking, the content change situation in the intercepted video segment will be more complex and changeable. Therefore, the histogram corresponding to the image similarity point array will also present other forms. Taking the Figure 5 shown histogram as an example, it can be known from this histogram that during the playback of the video segment, the change situation of the similarity degree will be more complex, and there are multiple related video frame images with a higher similarity degree. For example, the cast and crew lists with different names may appear multiple times and cover the entire screen multiple times. Thus, it will generate the Figure 5 shown histogram.

[0082] Therefore, step S303 can be continued to search forward from the candidate array position to search for the array position where the number of the common key points is higher than the second threshold.

[0083] Among them, the second threshold can also be determined based on a preset ratio of the maximum value of the number of the common key points. For example, in this embodiment, the preset ratio is set to 80%. If the maximum value of the number of key points is 145, then the second threshold is 116. Taking the Figure 5Taking the scene shown as an example, the abscissa of the maximum value position found from the array of image similarity points is 118. Searching forward from the maximum value position, the abscissa of the array position where the number of first common key points found is lower than the first threshold is 70. At this time, the array position with abscissa 70 can be temporarily determined as the candidate array position. Then search forward from the candidate array position with abscissa 70 to search for the array position where the number of common key points is higher than the second threshold. In this solution, this array position can be defined as the sub-peak of the histogram. In theory, there can be multiple sub-peaks in a histogram.

[0084] Step S304, if the array position of the first number in the array of image similarity points is traversed and the array position where the number of common key points is higher than the second threshold is still not found, then determine the candidate array position as the target position.

[0085] Step S305, if the array position where the number of common key points is higher than the second threshold is found, continue to search forward to search for the array position where the number of new common key points is lower than the first threshold. Taking the detection of the end credits of a TV drama as an example, the scene represented by this situation can be considered that there may be some parts that are not completely the same in the middle of the end credits, but still regard these parts as part of the end credits. For example, the cast lists with different names may appear multiple times, and in the scene where they cover the entire screen multiple times, the array position where the number of common key points is higher than the second threshold may represent the video frame image when other cast lists cover the entire screen. This part also belongs to the end credits, and it is necessary to search for the time point when this part of the cast list starts to appear on the screen. In this embodiment, it can be achieved by searching for the array position where the number of new common key points is lower than the first threshold.

[0086] Step S306, when the array position where the number of new common key points is lower than the first threshold is found, update the array position to the candidate array position, and continue to search forward from the candidate array position, and start to execute from step S303 again until the array position of the first number in the array of image similarity points is traversed. Taking Figure 6 the histogram shown as an example, if the maximum value of the number of key points is 110, the abscissa of the array position where the number of first common key points is lower than the first threshold is 70, and if the preset ratio of the second threshold is 80%, then the second threshold is 88. After executing step S303, the array position where the number of common key points is higher than the second threshold can be found. At this time, continuing to search forward can find the array position where the number of new common key points is lower than the first threshold, and the abscissa of this array position is 28. Thus, the array position with abscissa 28 can be updated to the candidate array position.

[0087] Step S306, after traversing the first array position of the image similarity point array, the candidate array position can be determined as the target position, and the sampling position corresponding to the target position is the end-credit position. For example, for Figure 4 the shown scenario, the abscissa of the target position is 36, corresponding to the number of common key points between video frame images P36 and P38. Then, the time point of video frame image P36 can be used as the start time point of the end credits. Also, for Figure 6 the shown scenario, the abscissa of the target position is 28, corresponding to the number of common key points between video frame images P28 and P30. Then, the time point of video frame image P28 can be used as the start time point of the end credits.

[0088] Based on a similar technical principle, an embodiment of the present application also provides a method for detecting the start of a video, which can be applied to the detection of the start position of a video file in a video. The overall principle of this method is to calculate the key points between different sampled video frame images by using the SIFT algorithm, and count the number of identical common key points in two video frame images, and use the number of common key points to determine the start position of the video start. This method includes the processing steps as Figure 7 shown:

[0089] Step S701, intercept a video segment with a preset duration at the head of the video file.

[0090] Step S702, sample the video segment at a preset sampling frequency, and extract a preset number of video frame images.

[0091] Step S703, use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, and obtain the key point list of each video frame image, where the key point list includes the key points of the video frame image and the feature information corresponding to the key points.

[0092] Step S704, traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtain an image similarity point array. The image similarity point array includes the number of common key points between each video frame image and its corresponding associated video frame image.

[0093] Step S705, find the target position where the number of common key points meets the preset conditions from the image similarity point array, and determine the sampling position corresponding to the target position as the start position of the video start.

[0094] This solution is similar to the technical principle of the aforementioned video end detection method, and the difference lies in that: when intercepting a video segment for the video start detection method, content with a preset duration is intercepted from the head of the video file, so as to ensure that the video start to be detected can be included in this part of the video segment. After obtaining the required video segment, the video segment can be processed in a similar manner to determine the start position of the video file.

[0095] When detecting the video start, in the process of finding the target position where the number of common key points in the image similarity point array meets the preset conditions, the maximum value position can be found from the image similarity point array first, where the maximum value position is the array position with the largest number of common key points. Then, search backward from the maximum value position. When the first array position where the number of common key points is lower than the first threshold is found, this array position is determined as the candidate array position. Then, search backward from the candidate array position. If no array position where the number of common key points is higher than the second threshold is found, the candidate array position is determined as the target position; if an array position where the number of common key points is higher than the second threshold is found, continue to search backward. When a new array position where the number of common key points is lower than the first threshold is found, update this array position to the candidate array position and continue to search backward from the candidate array position until the last array position of the image similarity point array is traversed.

[0096] The embodiment of the present application also provides a computing device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions. Among them, when the computer program instructions are executed by the processor, the device is triggered to execute the aforementioned video start or end detection method.

[0097] In particular, the method and / or embodiment in the embodiment of the present application can be implemented as a computer software program. For example, the embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. When the computer program is executed by the processing unit, the above functions defined in the method of the present application are executed.

[0098] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer-readable medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0099] In this application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination of the above.

[0100] The computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0101] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0102] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist separately without being assembled into the device. The above computer-readable medium carries one or more computer program instructions, and the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of the foregoing multiple embodiments of the present application.

[0103] It should be noted that the present application can be implemented in software and / or a combination of software and hardware. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. Additionally, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with the processor to perform each step or function.

[0104] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present application, the present application can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present application. Any reference signs in the claims should not be construed as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of elements or devices recited in the apparatus claims can also be implemented by one element or device through software or hardware. The words such as "first" and "second" are used to denote names and do not denote any particular order.

Claims

1. A method for detecting the end of a video, characterized in that The method is applied to the end credits detection of a video series, and includes: Cut out the video clip of preset length at the end of the video file; Sampling the video clip according to a preset sampling frequency to extract a preset number of video frame images; Calculate the key points of each video frame image and the feature information corresponding to the key points using the SIFT algorithm, and obtain a key point list for each video frame image, wherein the key point list includes the key points of the video frame image and the feature information corresponding to the key points; traversing and calculating the number of common key points between each video frame image and its corresponding associated video frame image according to the key point list, and obtaining an image similarity array, wherein the image similarity array includes the number of common key points between each video frame image and its corresponding associated video frame image; Finding a target position whose number of shared key points meets a preset condition from the image similarity point array, and determining a sampling position corresponding to the target position as the end position; The method of finding a target position whose number of common key points meets a preset condition from the image similarity point array includes: Finding a maximum value position from the image similarity point array, wherein the maximum value position is the array position with the largest number of shared key points; Searching forward from the maximum value position, when the first array position where the number of common key points is lower than a first threshold is found, determining the array position as a candidate array position; Searching forward from the candidate array position, if no array position with the number of common key points higher than a second threshold is found, determining the candidate array position as the target position; If an array position is found where the number of shared key points is higher than the second threshold, the search continues forward. When a new array position is found where the number of shared key points is lower than the first threshold, the array position is updated as a candidate array position, and the search continues forward from the candidate array position until the first array position of the image similarity point array is traversed. The candidate array position is determined as the target position.

2. The method according to claim 1, wherein Sampling the video clip according to a preset sampling frequency to extract a preset number of video frame images includes: The video clip is sampled at a sampling frequency of once per second to extract a preset number of video frame images.

3. The method according to claim 1, wherein The number of common key points between each video frame image and its corresponding associated video frame image is calculated according to the key point list to obtain an array of image similarity points, including: Perform the following traversal processing on each video frame image: obtain a video frame image and an associated video frame image with a sampling interval N from the video frame image, compare the key point list of the video frame image with the key point list of the associated video frame image, and determine the same key points in the two video frame images as shared key points; Count the number of common key points between each video frame image and its corresponding associated video frame image to obtain an array of image similarity points.

4. The method according to claim 1, wherein The first threshold and the second threshold are respectively determined based on preset ratios of the maximum number of the common key points.

5. The method according to claim 1, wherein Finding the maximum value position from the image similarity point array includes: Generate a histogram based on the array of image similarity points, where the horizontal axis of the histogram is the array position and the vertical axis is the number of common key points; Select the highest point from the histogram and determine the horizontal axis position corresponding to the highest point as the maximum value position.

6. A video title detection method, characterized in that, The method is applied to the detection of the opening titles of video series and includes: Intercept a video segment of a preset duration at the head of the video file; Sample the video segment at a preset sampling frequency and extract a preset number of video frame images; Use the SIFT algorithm to calculate the key points of each video frame image and the feature information corresponding to the key points, and obtain a list of key points for each video frame image, where the list of key points includes the key points of the video frame image and the feature information corresponding to the key points; Traverse and calculate the number of common key points between each video frame image and its corresponding associated video frame image according to the list of key points, and obtain an array of image similarity points, where the array of image similarity points includes the number of common key points between each video frame image and its corresponding associated video frame image; From the array of image similarity points, find a target position where the number of common key points meets a preset condition, and determine the sampling position corresponding to the target position as the opening title position; Among them, finding a target position where the number of common key points meets a preset condition from the array of image similarity points includes: Find the maximum value position from the array of image similarity points, where the maximum value position is the array position with the largest number of common key points; Search backward from the maximum value position, and when the first array position where the number of common key points is lower than the first threshold is found, determine this array position as the candidate array position; Search backward from the candidate array position. If no array position where the number of common key points is higher than the second threshold is found, determine the candidate array position as the target position; If an array position where the number of common key points is higher than the second threshold is found, continue to search backward. When a new array position where the number of common key points is lower than the first threshold is found, update this array position as the candidate array position and continue to search backward from the candidate array position until the last array position of the array of image similarity points is traversed.

7. A computing device, wherein, The device includes a memory for storing computer program instructions and a processor for executing the computer program instructions. When the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 6.

8. A computer-readable medium, on which computer program instructions are stored, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 6.

Images