Television episode identification method and apparatus

Abstract

The application provides a television series episode opening and closing identification method and device based on audio features. In the scheme, audio features are used, and the processing of the features is realized in the form of a string. Compared with the processing of video features, the technical threshold and computing power requirements are greatly reduced. For some television series without fixed opening and closing, or using a frame freeze of the end of the main film as the closing, the same opening and closing music is often used, so the processing effect is also good.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a method and device for identifying a television series episode head and tail. BACKGROUND

[0002] In recent years, there are more and more video programs in video websites, IPTV (Interactive Personality TV), Internet TV and other video platforms, and the awareness of serving users and competing for users of various video platforms is also increasing. A television series in a video program usually has a fixed head and tail. In the process of watching a television series, many users want to skip the head and tail and directly watch the main content. In view of the demand of video users to quickly skip the head and tail, the current schemes adopted by various video platforms mainly include the following:

[0003] 1. Artificial annotation mode. The head and tail positions of each episode of a television series are annotated by artificial means. This mode has the least technical investment and the largest manpower investment.

[0004] 2. Video feature mode. The video features of the head and tail of a television series are analyzed, and the head and tail of each episode of the television series are automatically identified and annotated based on the video features. This mode is the current mainstream mode and basically meets the demand, but it requires a large amount of image processing, and the technical investment and computing power required in the analysis and identification process are relatively large. At the same time, the analysis by using video features has poor processing effect for the head and tail without fixed video content or the television series using the frame freezing of the end of the main content as the tail.

[0005] 3. User viewing behavior mode. The user viewing behavior is collected by big data, and the positions frequently jumped by users at the beginning and end of a television series are annotated. This mode does not analyze the relevant features of the television series itself, but needs to collect the viewing data of the audience before annotating the television series, which is an indirect way. Moreover, the positions jumped by users may differ due to different operation habits, and the accuracy of the processing result obtained is also insufficient. SUMMARY

[0006] An object of the present application is to provide a method and device for identifying the head and tail of a television series episode based on audio features.

[0007] To achieve the above object, the present application provides a method for identifying the head of a television series episode, which comprises:

[0008] obtaining video files of the same television series as a sample set;

[0009] audio sample the content of a first time interval of a video file in the sample set, obtain sample audio data, and convert the sample audio data into a string form, wherein the first time interval is a preset time interval from a video start time in the video file;

[0010] select sample audio data of two video files from the sample set, and determine a maximum common similar substring in the two sample audio data;

[0011] based on the maximum common similar substring, match the sample audio data of the video files in the sample set, and determine a target substring in the sample audio data that meets a preset condition in terms of similarity;

[0012] determine a position of a trailer of the video file according to a position of the target substring in the sample audio data.

[0013] According to another aspect of the present application, a method for identifying a trailer of a television series is also provided, and the method comprises:

[0014] obtain video files of the same television series as a sample set;

[0015] audio sample content of a second time interval of a video file in the sample set, obtain sample audio data, and convert the sample audio data into a string form, wherein the second time interval is a preset time interval ending at a video end time in the video file;

[0016] select sample audio data of two video files from the sample set, and determine a maximum common similar substring in the two sample audio data;

[0017] based on the maximum common similar substring, match the sample audio data of the video files in the sample set, and determine a target substring in the sample audio data that meets a preset condition in terms of similarity;

[0018] determine a position of a trailer of the video file according to a position of the target substring in the sample audio data.

[0019] Further, determining the maximum common similar substring in the two sample audio data comprises:

[0020] comparing the two sample audio data by traversing the strings corresponding to the two sample audio data in a sliding window manner;

[0021] record the number of identical characters of the identical substring under each window;

[0022] determine the window with the largest number of identical characters as the window in which the maximum common similar substring is located according to the number of identical characters;

[0023] In the window where the maximum common similar substring is located, the maximum common similar substring in the two sampled audio data is determined by searching.

[0024] Further, in the window where the maximum common similar substring is located, the maximum common similar substring in the two sampled audio data is determined by searching, including:

[0025] A part of the string in the window where the maximum common similar substring is located is selected as a seed string.

[0026] In the string corresponding to the two sampled audio data, an extended string corresponding to the two sampled audio data is obtained by extending the seed string by a preset number of characters forward and / or backward.

[0027] The two extended strings are compared.

[0028] If the similarity between the two extended strings is lower than a preset value, the maximum common similar substring is determined according to the extended strings.

[0029] If the similarity between the two extended strings is not lower than a preset value, the process of extending the extended string by a preset number of characters forward and / or backward in the string corresponding to the two sampled audio data is continued, and the process of comparing the extended strings is continued until the similarity between the two extended strings is lower than a preset value.

[0030] Further, the comparison of the two extended strings includes:

[0031] The Hamming distance of the two extended strings is calculated, and if the Hamming distance is higher than a first threshold value, it is determined that the similarity between the two extended strings is lower than a preset value, otherwise, it is determined that the similarity between the two extended strings is not lower than a preset value; or

[0032] The number of identical characters between the two extended strings is compared, and if the number of identical characters is less than a second threshold value, it is determined that the similarity between the two extended strings is lower than a preset value, otherwise, it is determined that the similarity between the two extended strings is not lower than a preset value.

[0033] Further, the determination of the maximum common similar substring according to the extended string includes:

[0034] The comparison results of the characters at both ends of the two extended strings are checked, and if the characters at both ends are inconsistent, the inconsistent characters are deleted until the characters at both ends of the two extended strings are consistent, and the current extended string is determined as the maximum common similar substring.

[0035] Further, when traversing and comparing the strings corresponding to the two sampled audio data in a sliding window manner, the sliding distance of the sliding window is a plurality of characters.

[0036] Further, based on the maximum common similarity substring, the sampled audio data of video files in the sample set are matched to determine the target substrings in the sampled audio data whose similarity meets preset conditions, including:

[0037] Using the maximum common similar substring, the sampled audio data of each video file in the sample set are traversed and compared one by one;

[0038] For each video file's sampled audio data, when a substring is found where the ratio of the Hamming distance between it and the longest common similar substring is less than or equal to a third threshold, or the ratio of the number of identical characters to the longest common similar substring is greater than or equal to a fourth threshold, the substring is identified as the target substring.

[0039] Furthermore, this application also provides a method for identifying duplicate content in TV drama series, the method comprising:

[0040] Obtain video files from the same TV series as a sample set;

[0041] Audio sampling is performed on the content of the third time interval of the video files in the sample set to obtain sampled audio data, which is then converted into a string. The third time interval is the time interval in the video file that contains the repeated content.

[0042] Select sampled audio data from two video files in the sample set and determine the maximum common similar substring between the two sampled audio data.

[0043] Based on the maximum common similar substring, the sampled audio data of video files in the sample set are matched to determine the target substring in the sampled audio data that meets the preset conditions for similarity.

[0044] The location of repeated content in the video file is determined based on the position of the target substring in the sampled audio data.

[0045] This application embodiment also provides a computing device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the identification method.

[0046] This application also provides a computer-readable medium storing computer program instructions thereon, which can be executed by a processor to implement the identification method.

[0047] Compared to existing technologies, the solution provided in this application utilizes audio features from TV drama series to achieve automatic identification of the beginning and end of the series. This solution first obtains video files from the same TV drama series as a sample set, then samples the audio from the first or second time interval of the video files in the sample set, obtaining sampled audio data, and converting it into a string format. Specifically, the sampled audio data corresponding to the first time interval is used for identifying the beginning of the series, and the sampled audio data corresponding to the second time interval is used for identifying the end of the series.

[0048] During identification, sampled audio data from two video files can be selected from the sample set, and the maximum common similarity substring between the two sampled audio data can be determined. This maximum common similarity substring is the sample for the opening or closing credits. Based on this maximum common similarity substring, the sampled audio data of the video files in the sample set can be matched to determine the target substring in the video file that meets the preset similarity criteria. Then, based on the position of the target substring in the sampled audio data, the opening or closing credits position of the video file can be determined. This allows for the identification and marking of the opening or closing credits in a TV series. Because this solution utilizes audio features and processes them in string form, it significantly reduces the technical threshold and computational requirements compared to video feature processing. Furthermore, for some opening and closing credits without fixed video content, or for TV series that use a freeze-frame at the end of the main feature as the closing credits, the same opening and closing music is often used, thus achieving similar good processing results. Attached Figure Description

[0049] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0050] Figure 1 A flowchart illustrating a method for identifying the opening sequence of a TV series, provided in an embodiment of this application;

[0051] Figure 2 This is a flowchart illustrating the process of determining the maximum common similar substring between two sampled audio data in this embodiment of the application.

[0052] Figure 3 This is a schematic diagram illustrating the logic of sliding comparison between two strings in an embodiment of this application;

[0053] Figure 4 This is a schematic diagram illustrating the comparison of similarity between two strings in an embodiment of this application;

[0054] Figure 5 This is a schematic diagram illustrating the extraction of the maximum common similarity substring between strings str3 and str4 based on an embodiment of this application.

[0055] Figure 6 This is a schematic diagram of a comparison scenario when determining the target substring in an embodiment of this application;

[0056] Figure 7 A flowchart illustrating a method for identifying the end credits of a TV series, provided as an embodiment of this application;

[0057] Figure 8 A flowchart illustrating a method for identifying duplicate content in a TV series, provided as an embodiment of this application.

[0058] Figure 9 This is a schematic diagram illustrating the process of adding opening and closing credits to a TV series using the technical solution provided in the embodiments of this application.

[0059] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0060] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0061] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0062] This application provides a method for recognizing the opening or closing credits of a TV series. This method utilizes audio features and processes the features in the form of strings. Compared with video feature processing, it greatly reduces the technical threshold and computing power requirements for implementation. For some opening and closing credits without fixed video content, or for TV series that use a still frame at the end of the main feature as the closing credits, the same opening and closing music is often still used, so it also has a good processing effect.

[0063] In practical scenarios, the execution subject of this method can be a user device, a network device, or a device composed of a user device and a network device integrated through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, and tablets; the network device includes, but is not limited to, network hosts, single network servers, servers aggregated from multiple network servers, or servers in a distributed cloud network. Here, the distributed cloud network consists of a large number of hosts or network servers based on cloud computing.

[0064] Figure 1 This application provides a processing flow for a method to identify the opening sequence of a TV series, which includes at least the following processing steps:

[0065] Step S101: Obtain video files from the same TV series as a sample set.

[0066] This step is the video file selection step, mainly used to determine the range of video files for the TV series to be identified. Specifically, this can be done by creating a corresponding file list for the same TV series in the same file directory or memory storage space, thereby determining the range of video files to be identified subsequently. The video files refer to the video files corresponding to each episode of the TV series. For example, for a 10-episode TV series drama1, it can contain 10 video files file1 to 10. If it is necessary to identify the opening sequence of drama1, then video files file1 to 10 can be obtained as the sample set S1.

[0067] Step S102: Audio sampling is performed on the content of the first time interval of the video files in the sample set to obtain sampled audio data, which is then converted into a string.

[0068] This step samples the audio content at the beginning of the video file to narrow down the recognition range, reduce data processing, and thus improve processing efficiency. To ensure that audio data including the beginning is sampled, the first time interval can be set to a preset time interval from the start time of the video file. The length of this time interval can be set according to the length of the beginning of the video in the actual scenario. For example, when the first time interval is set to N minutes, this step will collect audio data within the first N minutes of the video as sampled audio data. In practical scenarios, N can generally be set to 4, that is, audio sampling is performed on the first 4 minutes of each episode of a TV series.

[0069] The sampled audio data obtained after sampling can be converted into a string. In practical scenarios, audio sampling can be performed at a frequency of 100 samples per second. Each sample is a 2-byte integer value, which is then encoded in hexadecimal and converted into a 4-byte string. Taking the aforementioned 4 minutes of sampled audio data as an example, after converting it into a string, a string of length 96,000 bytes (4 minutes × 60 seconds / minute × 100 samples / second × 4 bytes) can be obtained. This string represents the sampled audio data after audio sampling for each video file.

[0070] Step S103: Select sampled audio data from two video files in the sample set, and determine the maximum common similar substring between the two sampled audio data.

[0071] Since the sample set typically includes audio data from two or more video files, random selection can be used. For example, taking sample set S1 in the aforementioned scenario as an example, which contains 10 video files file1 to 10, after sampling the audio content of its first time interval, 10 strings of sampled audio data related to the opening sequence, headstr1 to 10, will be obtained. In this step, two of them can be randomly selected, such as headstr1 and headstr3.

[0072] A common similar substring is a substring of two strings that is more than a certain degree of similarity, while the maximum common similar substring is the common similar substring with the most characters among all common similar substrings.

[0073] In this embodiment of the application, when determining the maximum common similar substring in two sampled audio data, the following method can be used: Figure 2 The processing steps shown are as follows:

[0074] Step S201: A sliding window is used to traverse and compare the strings corresponding to the two sampled audio data. For ease of explanation, this embodiment uses two strings of 5 characters each to illustrate the traversal and comparison logic, where the first string str1 is 35EFA and the second string str2 is C5EFB. The sliding comparison logic is as follows: Figure 3 As shown, the traversal process includes the following cases:

[0075] A) Compare strings with a length of 1 character in the forward direction;

[0076] B) Compare strings with a length of 2 characters in the forward direction;

[0077] C) Compare strings with a length of 3 characters in the forward direction;

[0078] D) Compare strings with a length of 4 characters in the forward direction;

[0079] E) Compare strings with a length of 5 characters in the forward direction;

[0080] F) Compare strings with a length of 4 characters in reverse order;

[0081] G) Compare strings with a length of 3 characters in reverse order;

[0082] H) Compare strings with a length of 2 characters in reverse order;

[0083] I) Compare strings with a length of 1 character in reverse order;

[0084] During this traversal and comparison process, the sliding window moves at a distance of one character. However, in real-world scenarios, due to the longer length of the strings, the sliding distance can be set to multiple characters to improve processing efficiency. For example, in this embodiment, since the string converted from the sampled audio data is 96,000 bytes long, and each hexadecimal value occupies 4 characters, the sliding window can be set to move at a distance of 4 characters.

[0085] Step S202: Record the number of identical characters in the same substring under each window. Taking the aforementioned scenario as an example, the number of identical characters in the same substring of the 9 windows A to I is shown in Table 1 below:

[0086] Serial number Number of identical characters A 0 B 0 C 0 D 0 E 3 F 0 G 0 H 0 I 0

[0087] Table 1

[0088] Step S203: Based on the number of identical characters, the window with the highest number of identical characters is determined as the window containing the maximum common similar substring. Taking the aforementioned scenario as an example, the window with the highest number of identical characters is the window corresponding to index E, where the identical characters are "5EF", and the number of identical characters is 3.

[0089] Step S204: In the window containing the longest common similar substring, find the longest common similar substring between the two sampled audio data. The longest common similar substring does not require identical characters, but rather a similarity exceeding a certain value. In real-world scenarios, the strings corresponding to the sampled audio data are often quite long, requiring further searching within the window containing the longest common similar substring to obtain it.

[0090] The following are further methods for obtaining the maximum common similar substring:

[0091] First, a portion of the strings within the window containing the maximum common similar substring is selected as the seed string. Once the window containing the maximum common similar substring is determined, a substring of a preset length can be extracted from the strings contained within that window as the seed string. For example, taking a 96,000-byte string as an example, its corresponding seed string could be a string obtained from sampling for one second, i.e., a 400-byte string. In practical scenarios, the length of the seed string can be set according to the needs of the actual scenario; no limitation is imposed here.

[0092] In this embodiment, the processing procedure is described using sampled audio data represented by two strings, str3 and str4. String str3 is “1234546EF25AB0CB0AC0898007651234567887980642C85B9966”, and string str4 is “0AC61234567887982642C8855ABC784567EEFF3355E56832377C”. Based on the aforementioned steps S201-S203, the window containing the maximum common similar substring can be determined as the window offset 24 characters to the right of str3, which contains the maximum common similar substring.

[0093] Within the window containing the largest common similar substring, select a portion of the strings as a seed string.

[0094] Then, based on the seed string, a preset number of characters can be extended forward and / or backward in the strings corresponding to the two sampled audio data to obtain the extended strings corresponding to the two sampled audio data. Taking the aforementioned seed string "12345678" as an example, in the strings str3 and str4 corresponding to the two sampled audio data, a preset number of characters can be extended forward and / or backward based on the seed string. Taking str3 as an example, if it is extended forward by 4 characters, the resulting extended string is "12345678"; if it is extended backward by 4 characters, the resulting extended string is "56788798". Similarly, the same extension method can be applied to string str4. When the string length is long, to ensure processing efficiency, the number of characters added each time can be increased, for example, by 8 characters forward or 12 characters backward each time.

[0095] The method of expanding the string can be preset according to the needs of the actual scenario. For example, it can expand forward first and then backward; or it can expand backward first and then forward; or it can expand forward and backward at the same time; or it can expand in both directions alternately; in addition, it can also expand according to other preset order.

[0096] After each synchronous expansion, a comparison can be performed based on the two new expanded strings. If the similarity between the expanded strings is not lower than a preset value, it means that the two expanded strings are still similar enough, and further expansion can be attempted to obtain a longer similar common substring. At this point, a new expanded string can be obtained by expanding the strings corresponding to the two sampled audio data by a preset number of characters forward and / or backward based on the expanded string, and the comparison of the expanded strings can continue until the similarity between the expanded strings is lower than the preset value.

[0097] The similarity between extended strings can be determined based on the number of identical characters or the Hamming distance. When based on the number of identical characters, the number of identical characters between the two extended strings is compared. If the number of identical characters is less than a second threshold, the similarity between the extended strings is determined to be lower than a preset value; otherwise, the similarity is determined to be no less than a preset value. When based on the Hamming distance, the Hamming distance between the two extended strings is calculated. If the Hamming distance is higher than a first threshold, the similarity between the extended strings is determined to be lower than a preset value; otherwise, the similarity is determined to be no less than a preset value.

[0098] Figure 4 The diagram illustrates the comparison of similarity between two strings. When the similarity is set to 90%, the first threshold can be set to 10% of the extended string length, and the second threshold can be set to 90% of the extended string length. Taking a 20-character extended string as an example, if the Hamming distance is greater than 2, or the number of identical characters is less than 18, it indicates that the similarity between the extended strings is lower than the preset value; otherwise, it is determined that the similarity between the extended strings is not lower than the preset value.

[0099] If the comparison result shows that the similarity between the extended strings is lower than a preset value, it means that the similarity between the two extended strings is insufficient and should not be extended further. In this case, the maximum common similar substring can be determined based on the current extended strings. In practical scenarios, either of the two extended strings can be arbitrarily selected as the maximum common similar substring. For example, in the aforementioned scenario, when the similarity is insufficient, the extended string str3' obtained based on string str3 is "1234567887980642C85B", while the extended string str4' obtained based on string str4 is "1234567887982642C855". In this case, either extended string str3' or extended string str4' can be arbitrarily selected as the maximum common similar substring.

[0100] In some embodiments of this application, to improve the accuracy of subsequent processing, it can be ensured that the characters at both ends are at least the same when obtaining the maximum common similar substring. Therefore, when determining the maximum common similar substring based on the current extended string, the character comparison results at both ends of the two extended strings are first checked. If the characters at both ends are inconsistent, the inconsistent characters are deleted until the characters at both ends of the two extended strings are consistent, and the current extended string is determined as the maximum common similar substring. Taking the aforementioned two extended strings str3' and str4' as examples, since the first four characters are both "1234", no deletion is needed. However, the last four characters are "C85B" and "C855" respectively, so the last four characters need to be deleted. Then, any string after deleting the different characters at the last end, i.e., "1234567887980642" or "1234567887982642", is selected as the maximum common similar substring. Figure 5 This is a diagram illustrating the extraction of the longest common similar substring between strings str3 and str4.

[0101] Step S104: Based on the maximum common similar substring, match the sampled audio data of the video files in the sample set to determine the target substring in the sampled audio data that meets the preset conditions for similarity.

[0102] Step S105: Determine the beginning position of the video file based on the position of the target substring in the sampled audio data.

[0103] The two steps described above implement the process of identifying the title sequence in the sampled audio data of each video file based on the maximum common similarity substring. Since the maximum common similarity substring can be considered a template for the audio content corresponding to the title sequence, it can be used to match the sampled audio data of the video files in the sample set. Taking the sampled audio data headstr1~10 from the aforementioned 10 video files as an example, the maximum common similarity substring commonheadstr corresponding to the title sequence can be used to match headstr1~10 respectively, finding the target substring whose similarity meets the preset conditions in these 10 sampled audio data.

[0104] In some embodiments of this application, the target substring can be determined in the following manner:

[0105] First, for the sampled audio data of each video file in the sample set, the maximum common similarity substring is used to iterate and compare the sampled audio data of each video file in the sample set one by one.

[0106] Since it's necessary to identify the opening sequence of each episode of the TV series, each video file in the sample set can be processed separately. Furthermore, to improve processing efficiency, we can directly start from the i-th character of the sampled audio data and select strings with the same length as the longest common similar substring for traversal and comparison. Here, i can be set to any positive integer as needed; for example, setting it to 1 indicates traversing and comparing from the first character, while setting it to 200 indicates traversing and comparing from the 200th character. Similarly, the sliding distance during traversal can be set to multiple characters to further improve processing efficiency.

[0107] During the traversal and comparison of each sampled audio data, the traversal status of each compared substring in the sampled audio data and the similarity between them and the maximum common similar substring are recorded. Then, it is determined whether the similarity between the current substring in the sampled audio data being compared with the maximum common similar substring meets a preset condition. In practical scenarios, the preset condition can be based on the number of identical characters between the two or the Hamming distance, etc. Specifically, for the sampled audio data of each video file, when a substring is found where the ratio of the Hamming distance to the length of the maximum common similar substring is less than or equal to a third threshold, or the ratio of the number of identical characters to the length of the maximum common similar substring is greater than or equal to a fourth threshold, the substring is determined as the target substring. For example, in some embodiments of this application, the third threshold can be set to 10%, and the fourth threshold can be set to 90%.

[0108] Taking the longest common similarity substring "1234567887982642" as an example, if one of the audio sample data strings is "1234546EF25AB0CB0AC0898007651234567887980642C85B9966", then the target substring can be determined to be a substring with a length of 16 characters, starting from the 29th character. Figure 6 As shown.

[0109] After determining the target substring, the position of the video file's intro can be determined based on the target substring's location in the sampled audio data. Taking the aforementioned scenario as an example, the position of the target substring in the sampled audio data is the position indicated by the underscore in the following string: "1234546EF25AB0CB0AC0898007651234567887980642C85B9966". Since these strings are converted from audio data in chronological order, the corresponding time points in the video file can be determined based on the correspondence, thereby identifying the intro position of the video file and completing the annotation of the TV series' intro.

[0110] Other embodiments of this application also provide a method for identifying the end credits of a TV series, the processing flow of which is as follows: Figure 7 As shown, the processing steps include the following:

[0111] Step S701: Obtain video files from the same TV series as a sample set.

[0112] Step S702: Audio sampling is performed on the content of the second time interval of the video file in the sample set to obtain sampled audio data and convert it into a string. The second time interval is a preset time interval in the video file that ends at the end time of the video.

[0113] Step S703: Select sampled audio data from two video files in the sample set, and determine the maximum common similar substring in the two sampled audio data.

[0114] Step S704: Based on the maximum common similar substring, match the sampled audio data of the video files in the sample set to determine the target substring in the sampled audio data that meets the preset conditions for similarity.

[0115] Step S705: Determine the end position of the video file based on the position of the target substring in the sampled audio data.

[0116] This scheme is similar in principle to the intro recognition method, but the difference lies in the fact that the outro recognition scheme samples the audio data from a second time interval—that is, the content corresponding to the preset time interval in the video file that ends at the video's end time. This allows it to obtain sampled audio data containing the outro content. Furthermore, by processing and recognizing this sampled audio data, the outro position of the video file can be identified.

[0117] Furthermore, based on similar technical principles, embodiments of this application can also provide a method for identifying duplicate content in a TV series. This method can be applied to the identification of any duplicate content in a TV series, including duplicate content in the opening credits, ending credits, or other positions. For example, this scheme can be used to identify advertisements with identical content inserted in the main program, so as to mark the advertisements and automatically skip them during playback. The processing flow of this method is as follows: Figure 8 As shown, the processing steps include the following:

[0118] Step S801: Obtain video files from the same TV series as a sample set;

[0119] Step S802: Audio sampling is performed on the content of the third time interval of the video files in the sample set to obtain sampled audio data and convert it into a string. The third time interval is the time interval in the video file that contains the repeated content.

[0120] Step S803: Select sampled audio data from two video files in the sample set, and determine the maximum common similar substring in the two sampled audio data.

[0121] Step S804: Based on the maximum common similar substring, match the sampled audio data of the video files in the sample set to determine the target substring in the sampled audio data that meets the preset conditions for similarity.

[0122] Step S805: Determine the position of the repeated content in the video file based on the position of the target substring in the sampled audio data.

[0123] Figure 9 This paper illustrates the process of adding opening and closing credits to a 10-episode TV series titled "Like a Flowing River" using the technical solution provided in the embodiments of this application. The process includes the following steps:

[0124] Step S1: Collect video files from 10 episodes of the TV series "Like a Flowing River" as a sample set.

[0125] Step S2: Extract a 4-minute intro and a 4-minute outro from the video file of each episode.

[0126] Step S3: Audio is captured from the opening and closing sections separately and encoded to form the opening and closing strings. This results in 20 strings: 10 opening strings (HeadStr1-10) and 10 closing strings (TailStr1-10). Each string is 96,000 bytes long. The 96,000 bytes are derived from the following: each sampled audio data is 4 minutes long, and at 100 samples per second, a total of 4 × 60 × 100 = 24,000 2-byte integer data samples are collected. Each integer is encoded as a 4-byte hexadecimal character, resulting in a 96,000-byte string.

[0127] Step S4: Use a sliding window method to find the window containing the longest common similar substring.

[0128] Take two strings, such as HeadStr1 and HeadStr2 from the beginning of episode 1 and episode 2, and find the number of identical characters between them.

[0129] The following is the pseudocode for this part of the processing:

[0130] #defineCompareNumber 96000

[0131] int I = 0;

[0132] While(I<2·CompareNumber)

[0133] {

[0134] Compare the similarity between the first I characters of HeadStr1 and the last I characters of HeadStr2;

[0135] / / Here, identical characters indicate similarity, so increment the counter by 1.

[0136] / / If the characters are different, they are considered different, and the counter remains unchanged;

[0137] The similarity count is recorded in the array Count[i] at this moment;

[0138] I = I + 4; / / Here, we skip 4 characters directly, meaning the sliding distance of the sliding window is 4 characters, because one audio sample data corresponds to four bytes of characters;

[0139] }

[0140] Find the largest number in the Count array, which means finding the condition in the sliding window where the maximum number of identical characters is found. The window with the maximum number of identical characters is the window containing the maximum common similar substring;

[0141] Considering the practical situation, the above algorithm can be further simplified by directly comparing the audio from 1 second later, with the corresponding data starting from string 400. Similarly, the end of the string can be ended early, thus reducing the number of loops and further improving efficiency.

[0142] The same processing can be applied to the string at the end of the video.

[0143] Step S5: Find the longest common similar substring within a specific window of the sliding window.

[0144] When obtaining the window containing the longest common similar substring in step S4 above, the longest common similar substring can be found further within that window.

[0145] The method for finding the longest common similar substring is to first find a suitable seed string of a specific length in HeadStr1 and HeadStr2 based on the window where the longest common similar substring is located, and then continuously add four characters before and after it to form an extended string until the similarity between the extended strings is lower than a certain threshold.

[0146] The pseudocode for finding common similar substrings is as follows:

[0147] Find a common string of length 4000 bytes in the window (corresponding to 10 seconds of completely identical content);

[0148] Enter the front-end string loop:

[0149] While(1)

[0150] {

[0151] Compare the first four characters of the common string. If they are the same, include them directly in the common string. If they are different, check if the number of different characters exceeds a threshold. If not, include them in the common string.

[0152] The comparison terminates when the beginning of the string is reached.

[0153] The process terminates when the number of distinct items exceeds the threshold.

[0154] }

[0155] Determine if the first four characters of a common string are the same; if not, delete consecutive different first four characters.

[0156] Enter the backend string loop:

[0157] While(1)

[0158] {

[0159] Compare the last four characters of the common string. If they are the same, include them directly in the common string. If they are different, check if the number of different characters exceeds a threshold. If not, include them in the common string.

[0160] The comparison terminates when the end of the string is reached.

[0161] The process terminates when the number of distinct items exceeds the threshold.

[0162] }

[0163] Determine if the last four characters of a common substring are the same; if not, delete consecutive different last-character strings.

[0164] Obtain the longest common similar substring CommonHeadStr related to the beginning of the sequence.

[0165] During the processing, the common string is either the seed string or the extended string obtained after expansion.

[0166] Similarly, the longest common similar substring CommonTailStr for the end credits can be obtained in the same way. Using the found CommonHeadStr and CommonTailStr, the beginning and end credits of all video files can be identified.

[0167] Step S6: Based on the found CommonHeadStr and CommonTailStr, match them in all the sampled header strings HeadStr1-10 and 10 tail strings TailStr1-10 to determine the position of each header and tail.

[0168] The pseudocode for the opening sequence is as follows:

[0169] I = 0;

[0170] While(1)

[0171] {

[0172] Starting from the i-th position of HeadStr, select a string of length CommonHeadStr and compare it with CommonHeadStr, then record the position and the number of identical characters at that position;

[0173] I = I + 4;

[0174] The loop exits when a substring is found in which the ratio of the number of identical characters to the length of CommonHeadStr exceeds the fourth threshold.

[0175] }

[0176] The starting position of the title sequence is obtained from the starting position of the substring;

[0177] The end position of the intro can be calculated by adding the length of the intro (i.e., the length of CommonHeadStr) to the starting position of the intro.

[0178] Mark the position of the opening sequence in the video file;

[0179] Based on the same principle, the end credits can be marked. The pseudocode used for the end credits is as follows:

[0180] I = 0;

[0181] While(1)

[0182] {

[0183] Starting from the i-th position of TailStr, select a string of length CommonTailStr and compare it with CommonTailStr, then record the position and the number of identical characters at that position;

[0184] I = I + 4;

[0185] Exit the loop when a substring is found in which the ratio of the number of identical characters to the length of CommonTailStr exceeds the fourth threshold;

[0186] }

[0187] The starting position of the end of the sequence is obtained from the starting position of the substring;

[0188] The end position of the end credits can be calculated by adding the length of the end credits (i.e., the length of CommonTailStr) to the starting position of the end credits.

[0189] Mark the end of the video file.

[0190] Repeat the above process for the opening and closing title strings HeadStr1-10 and HeadStr1-10 to complete the opening and closing title tags for each episode of the TV series.

[0191] This application also provides a computing device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the aforementioned identification method.

[0192] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a processing unit, it performs the functions defined in the methods of this application.

[0193] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0194] In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0195] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0196] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0197] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The aforementioned computer-readable medium carries one or more computer program instructions, which may be executed by a processor to implement the methods and / or technical solutions of the various embodiments of this application.

[0198] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0199] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A method for recognizing the opening sequence of a TV series, characterized in that, The method includes Obtain video files from the same TV series as a sample set; Audio sampling is performed on the content of the first time interval of the video files in the sample set to obtain sampled audio data, which is then converted into a string. The first time interval is a preset time interval in the video file starting from the video start time. Select sampled audio data from two video files in the sample set and determine the maximum common similar substring between the two sampled audio data. Based on the maximum common similar substring, the sampled audio data of video files in the sample set are matched to determine the target substring in the sampled audio data that meets the preset conditions for similarity. The position of the video file's intro is determined based on the position of the target substring in the sampled audio data; Among them, determining the maximum common similar substring in two sampled audio data includes: A sliding window approach is used to iterate and compare the strings corresponding to the two sampled audio data. Record the number of identical characters in the same substring under each window; Based on the number of identical characters, the window with the most identical characters is determined as the window containing the maximum common similar substring; Select a portion of the strings within the window containing the longest common similar substring as the seed string; In the strings corresponding to the two sampled audio data, based on the seed string, a preset number of characters are extended forward and / or backward to obtain the extended string corresponding to the two sampled audio data; Compare two extended strings; If the similarity between the extended strings is lower than a preset value, then the maximum common similar substring is determined based on the extended strings; If the similarity between the extended strings is not lower than a preset value, then in the strings corresponding to the two sampled audio data, a new extended string is obtained by extending the extended string forward and / or backward by a preset number of characters, and the extended strings are compared until the similarity between the extended strings is lower than the preset value.

2. A method for identifying the end credits of a TV series, characterized in that, The method includes Obtain video files from the same TV series as a sample set; Audio sampling is performed on the content of the second time interval of the video files in the sample set to obtain sampled audio data, which is then converted into a string. The second time interval is a preset time interval in the video file that ends at the end time of the video. Select sampled audio data from two video files in the sample set and determine the maximum common similar substring between the two sampled audio data. Based on the maximum common similar substring, the sampled audio data of video files in the sample set are matched to determine the target substring in the sampled audio data that meets the preset conditions for similarity. The end position of the video file is determined based on the position of the target substring in the sampled audio data; Among them, determining the maximum common similar substring in two sampled audio data includes: A sliding window approach is used to iterate and compare the strings corresponding to the two sampled audio data. Record the number of identical characters in the same substring under each window; Based on the number of identical characters, the window with the most identical characters is determined as the window containing the maximum common similar substring; Select a portion of the strings within the window containing the longest common similar substring as the seed string; In the strings corresponding to the two sampled audio data, based on the seed string, a preset number of characters are extended forward and / or backward to obtain the extended string corresponding to the two sampled audio data; Compare two extended strings; If the similarity between the extended strings is lower than a preset value, then the maximum common similar substring is determined based on the extended strings; If the similarity between the extended strings is not lower than a preset value, then in the strings corresponding to the two sampled audio data, a new extended string is obtained by extending the extended string forward and / or backward by a preset number of characters, and the extended strings are compared until the similarity between the extended strings is lower than the preset value.

3. The method according to claim 1 or 2, characterized in that, Compare two extended strings, including Calculate the Hamming distance between two extended strings. If the Hamming distance is higher than a first threshold, determine that the similarity between the extended strings is lower than a preset value; otherwise, determine that the similarity between the extended strings is not lower than the preset value. Compare the number of identical characters between two extended strings. If the number of identical characters is less than a second threshold, then the similarity between the extended strings is determined to be lower than a preset value; otherwise, the similarity between the extended strings is determined to be not lower than a preset value.

4. The method according to claim 1 or 2, characterized in that, The longest common similar substring is determined based on the extended string, including... Check the character comparison results at both ends of the two extended strings. If the characters at both ends are inconsistent, delete the inconsistent characters until the characters at both ends of the two extended strings are consistent, and determine the current extended string as the maximum common similar substring.

5. The method according to claim 1 or 2, characterized in that, When using a sliding window to traverse and compare the strings corresponding to two sampled audio data, the sliding distance of the sliding window is multiple characters.

6. The method according to claim 1 or 2, characterized in that, Based on the maximum common similarity substring, the sampled audio data of video files in the sample set are matched to determine the target substrings in the sampled audio data that meet the preset similarity conditions, including... Using the maximum common similar substring, the sampled audio data of each video file in the sample set are traversed and compared one by one; For each video file's sampled audio data, when a substring is found where the ratio of the Hamming distance between it and the longest common similar substring is less than or equal to a third threshold, or the ratio of the number of identical characters to the longest common similar substring is greater than or equal to a fourth threshold, the substring is identified as the target substring.

7. A method for identifying duplicate content in a TV series, characterized in that, The method includes Obtain video files from the same TV series as a sample set; Audio sampling is performed on the content of the third time interval of the video files in the sample set to obtain sampled audio data, which is then converted into a string. The third time interval is the time interval in the video file that contains the repeated content. Select sampled audio data from two video files in the sample set and determine the maximum common similar substring between the two sampled audio data. Based on the maximum common similar substring, the sampled audio data of video files in the sample set are matched to determine the target substring in the sampled audio data that meets the preset conditions for similarity. The position of the repeated content in the video file is determined based on the position of the target substring in the sampled audio data; Among them, determining the maximum common similar substring in two sampled audio data includes: A sliding window approach is used to iterate and compare the strings corresponding to the two sampled audio data. Record the number of identical characters in the same substring under each window; Based on the number of identical characters, the window with the most identical characters is determined as the window containing the maximum common similar substring; Select a portion of the strings within the window containing the longest common similar substring as the seed string; In the strings corresponding to the two sampled audio data, based on the seed string, a preset number of characters are extended forward and / or backward to obtain the extended string corresponding to the two sampled audio data; Compare two extended strings; If the similarity between the extended strings is lower than a preset value, then the maximum common similar substring is determined based on the extended strings; If the similarity between the extended strings is not lower than a preset value, then in the strings corresponding to the two sampled audio data, a new extended string is obtained by extending the extended string forward and / or backward by a preset number of characters, and the extended strings are compared until the similarity between the extended strings is lower than the preset value.

8. A computing device, wherein, The device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to perform the method of any one of claims 1 to 7.

9. A computer-readable medium having stored thereon computer program instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 7.

Images