Video data screening method, device, computer equipment and storage medium

Abstract

A video data screening method, apparatus, computer device, and storage medium, the method comprising: obtaining a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video; calculating first similarities between each pair of to-be-processed video features and reference video features; clustering each video feature based on the first similarity to obtain a video feature set; the video features may include to-be-processed video features or reference video features; calculating second similarities between each video feature in the video feature set; and screening the to-be-processed video based on the second similarity. The above method utilizes the first similarity between video features for preliminary clustering, and calculates the second similarity between video features within the obtained video feature set. This method can screen out identical or similar videos while reducing the number of video feature comparisons, and can effectively reduce the processing workload when screening massive amounts of video.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video data screening method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the development of multimedia technology, more and more videos are being created. Identical or highly similar videos may appear in various applications. For example, in short video scenarios, popular videos within a certain period may be widely forwarded, with the same audio and visuals recreated, or repeatedly released within the same period. For these types of videos, identification and screening are usually required to avoid duplicates.

[0003] In related technologies, MD5 (a cryptographic hash function) is usually used to perform operations on videos. Videos with the same MD5 value are determined to be the same. However, this method can only filter out completely identical videos and cannot filter out highly similar videos. Summary of the Invention

[0004] Based on this, it is necessary to provide a video data screening method, device, computer equipment and storage medium that can screen out identical or similar videos in response to the above technical problems.

[0005] A method for screening video data, comprising:

[0006] Obtaining a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video;

[0007] Calculating first similarities between each of the to-be-processed video features and the reference video features;

[0008] Clustering each video feature based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features;

[0009] respectively calculating a second similarity between each of the video features in the video feature set;

[0010] The videos to be processed are screened based on the second similarity.

[0011] A video data screening device, comprising:

[0012] An acquisition module, configured to acquire a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video;

[0013] A first similarity calculation module is used to calculate the first similarity between each of the to-be-processed video features and the reference video features;

[0014] A clustering module, configured to cluster each video feature based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features;

[0015] A second similarity calculation module, configured to respectively calculate a second similarity between each of the video features in the video feature set;

[0016] A screening module is used to screen the videos to be processed based on the second similarity.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0018] Obtaining a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video;

[0019] Calculating first similarities between each of the to-be-processed video features and the reference video features;

[0020] Clustering each video feature based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features;

[0021] respectively calculating a second similarity between each of the video features in the video feature set;

[0022] The videos to be processed are screened based on the second similarity.

[0023] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0024] Obtaining a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video;

[0025] Calculating first similarities between each of the to-be-processed video features and the reference video features;

[0026] Clustering each video feature based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features;

[0027] respectively calculating a second similarity between each of the video features in the video feature set;

[0028] The videos to be processed are screened based on the second similarity.

[0029] The above-mentioned video data screening method, device, computer equipment and storage medium obtain a to-be-processed video and a reference video, as well as the video features corresponding to the to-be-processed video and the reference video respectively; then calculate the first similarity between each to-be-processed video feature and the reference video feature, perform preliminary clustering on the to-be-processed video feature and the reference video feature based on the first similarity to obtain a video feature set; and calculate the second similarity of each video feature within the video feature set, and finally screen the to-be-processed video based on the second similarity between the video features. The above-mentioned method uses the first similarity between the video features corresponding to the videos to perform preliminary clustering, and then calculates the second similarity between the video features within the video feature set obtained by clustering, thereby screening the to-be-processed video that is identical or similar to the reference video. The preliminary clustering followed by a small-scale comparison of video features can reduce the number of video feature comparisons, and can effectively reduce the processing level when screening massive amounts of videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A diagram showing an application environment of a video data screening method according to an embodiment;

[0031] Figure 2 1 is a flow chart of a method for screening video data in one embodiment;

[0032] Figure 3 1 is a flow chart of clustering video features based on a first similarity to obtain a video feature set in one embodiment;

[0033] Figure 4 1 is a flow chart of screening videos to be processed based on a second similarity in one embodiment;

[0034] Figure 5 A schematic diagram of the overall process of video data screening in one embodiment;

[0035] Figure 6 is a schematic diagram of calculating the second similarity in a specific embodiment;

[0036] Figure 7 1 is a flow chart of a method for screening video data in a specific embodiment;

[0037] Figure 8 is a structural block diagram of a video data screening device in one embodiment;

[0038] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0040] In some embodiments, the video data screening method provided by the present application can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The terminal 102 obtains the video to be processed and the reference video, as well as the video features corresponding to the video to be processed and the reference video respectively; then the first similarity between each feature of the video to be processed and the reference video feature is calculated respectively, and the features of the video to be processed and the reference video feature are preliminarily clustered based on the first similarity to obtain a video feature set; and the second similarity of each video feature is calculated within the video feature set, and finally the video to be processed is screened based on the second similarity between the video features. The reference video can be obtained from the server 104, and the video obtained after deduplication of the video to be processed can be sent to the server 104 for storage. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, portable wearable devices and vehicle-mounted terminals, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0041] Furthermore, in some embodiments, the server can be a service node in a blockchain. Blockchain is a novel application model that integrates computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0042] In one embodiment, Figure 2 As shown, a video data screening method is provided, which is applied to Figure 1 The terminal in is taken as an example to illustrate, including steps S210 to S250.

[0043] Step S210: Obtain a video to be processed, a reference video, features of the video to be processed corresponding to the video to be processed, and features of the reference video corresponding to the reference video.

[0044] The video to be processed is a video that needs to be screened; the video to be processed can be obtained in any way. In one embodiment, the video to be processed can be obtained from a database of a short video application, the Internet, etc. using a crawler tool.

[0045] In the embodiments of the present application, the reference video represents the video used for deduplication; the video to be processed is compared with the reference video, and if they are identical or similar, the video to be processed can be considered duplicate; the reference video can be obtained in any manner. In one embodiment, the reference data can be obtained from a database.

[0046] The features corresponding to the video data represent a portion of the features from the video data. In one embodiment, the speech content extracted from the video data is selected as the video feature; in another embodiment, the text content corresponding to the speech content extracted from the video data is selected as the video feature; in another embodiment, an image extracted from the video data is selected as the video feature; or a dynamic video segment extracted from the video data can be selected as the video feature, and so on.

[0047] Furthermore, the video features corresponding to the video data can be obtained in any way. For example, if the video feature is the voice content in the video data, it can be obtained directly or extracted from the video data. For example, if the video feature is an image, the cover image of the video data (the image displayed before the video is played) can be obtained, or the video feature containing one or more frames of images can be arbitrarily selected from the video data.

[0048] In this embodiment, the video features corresponding to the to-be-processed video are recorded as to-be-processed video features, and the video features corresponding to the reference video are recorded as reference video features.

[0049] In another embodiment, after obtaining the video to be processed and the reference video, feature extraction can be performed on the video to be processed and the reference video to obtain the features of the video to be processed and the features of the reference video.

[0050] Step S220 , respectively calculating the first similarity between the to-be-processed video features and the reference video features.

[0051] It should be noted that the terms "first" and "second" used in the various embodiments of this application are merely for naming distinctions and do not represent any practical meaning. Calculating the first similarity between video features can be achieved in any manner. In one embodiment, the similarity calculated using the first similarity calculation method for the acquired to-be-processed video features and the reference video features is recorded as the first similarity.

[0052] Among them, the similarity algorithm represents a method for calculating the similarity between video features. In one embodiment, taking the video features corresponding to the to-be-processed video and the reference video as speech text content as an example, the first similarity can be to calculate the text similarity between the speech text content corresponding to the to-be-processed video and the reference video. Among them, the text similarity can be calculated in any way; for example, the similarity between the speech text content is calculated based on pinyin, and the more text content with the same consecutive pinyin, the greater the text similarity; for example, the similarity between the speech text content can be calculated based on a related model, and so on. Taking the video features corresponding to the to-be-processed video and the reference video as pictures as an example, the first similarity can be to calculate the similarity between the speech and pictures corresponding to the to-be-processed video and the reference video. It can be understood that in other embodiments, calculating the first similarity between the to-be-processed video features and the reference video features can also be achieved in other ways.

[0053] In one embodiment, calculating first similarities between the processing video feature and the reference video feature includes selecting any one video feature (which may be the processing video feature or the reference video feature) as the first video feature and calculating first similarities between the other video features and the first video feature. In this embodiment, by selecting any one video feature as the first video feature and calculating first similarities with the remaining video features, first similarities between any two video features can be obtained.

[0054] In another embodiment, the first similarities between the features of the video to be processed and the reference video features are calculated separately, including: using each reference video feature as the second video feature, and calculating the first similarities between each video feature to be processed and each second video feature. In this embodiment, using the reference video feature as the second video feature, the first similarities between each video feature to be processed and the second video feature are calculated, and then sorting is performed based on the first similarities to obtain a preliminary sorting result corresponding to each reference video feature. During subsequent screening, the videos to be processed with a high similarity to any reference video feature are filtered out based on the preliminary sorting results.

[0055] In other embodiments, calculating the first similarity and performing sorting based on the first similarity to obtain a preliminary sorting result may be achieved in other ways.

[0056] Step S230 , clustering the video features based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features.

[0057] Clustering is the process of grouping a collection of physical or abstract objects into multiple clusters of similar objects. A cluster generated by clustering is a set of data objects that are similar to objects in the same cluster. In this embodiment, clustering is performed based on the calculated first similarity between the basic video features and the video features to be processed. The resulting set is referred to as the video feature set.

[0058] Furthermore, in one embodiment, the to-be-processed video features and the reference video features with a first similarity greater than a threshold may be clustered into the same cluster, ie, a video feature set.

[0059] Step S240 , respectively calculating the second similarity between each video feature in the video feature set.

[0060] In one embodiment, a second similarity calculation method is used to calculate the similarity of each video feature in the video feature set, which is recorded as the second similarity. The second similarity algorithm can be any algorithm. For example, in one specific embodiment, the video features extracted from the processing video and the reference video are speech text content. The second similarity algorithm can calculate the Levenshtein distance between the speech text content corresponding to the processing video and the reference video as the second similarity. The Levenshtein distance, also known as the edit distance, refers to the minimum number of editing operations required to convert two strings from one to the other. Editing operations include replacing one character with another, inserting a character, and deleting a character. Generally speaking, the smaller the edit distance, the greater the similarity between the two strings.

[0061] In another specific embodiment, the second similarity algorithm may also be to calculate the Levenstein ratio between the speech text content corresponding to the processed video and the reference video. The Levenstein ratio calculation formula is r = (sum - ldist) / sum. Wherein, sum represents the sum of the lengths of the str1 and str2 strings; for example, if str1 = 'abc' and str2 = 'cde', sum = 3 + 3 = 6. ldist represents the class edit distance, which describes the minimum number of operations required to convert one string into another, where the operations include insertion, deletion, and replacement. Deletion and insertion operations are +1, and replacement is +2. In other embodiments, the second similarity may also be calculated by other methods.

[0062] In this embodiment, each video feature is initially clustered based on the first similarity between them, and relatively similar video features are clustered together. When calculating feature similarity, the second similarity is calculated only for the relatively similar video features obtained through the initial clustering. Feature similarity is no longer calculated for video features with low similarity determined during the initial clustering process, thereby reducing the number of second similarity calculations between video features and effectively improving processing efficiency in application scenarios involving a large number of videos.

[0063] Step S250: screening the videos to be processed based on the second similarity.

[0064] After respectively calculating the second similarity between each video feature in each video feature set, the second similarity of the video features can be used to represent the similarity of the videos, that is, based on the second similarity, it can be determined whether the video to be processed is identical or similar to the reference video, so the video to be processed can be screened and deduplicated based on the second similarity.

[0065] In one embodiment, screening the videos to be processed based on the second similarity includes eliminating videos to be processed whose second similarity is greater than a feature similarity threshold. In another embodiment, screening the videos to be processed based on similarity includes sorting the videos to be processed based on their second similarity to any reference video, and eliminating the first preset number of videos to be processed with the largest second similarity. For example, for video feature set A, assume that it contains video features corresponding to reference videos A, B, C, D, and E, as well as video features corresponding to reference videos 1-20. Select any reference video C from reference videos A, B, C, D, and E, sort the videos to be processed 1-20 based on their second similarity to reference video C, and eliminate the videos to be processed corresponding to the first preset number of video features with the largest second similarity values ​​from reference videos 1-20. The feature similarity threshold and the preset number can be set according to actual circumstances and can be set to fixed or dynamic values. In other embodiments, screening the videos to be processed based on the second similarity can also be achieved through other methods.

[0066] The above-mentioned video data screening method obtains a to-be-processed video and a reference video, as well as the video features corresponding to the to-be-processed video and the reference video respectively; then calculates the first similarity between each to-be-processed video feature and the reference video feature, performs preliminary clustering on the to-be-processed video feature and the reference video feature based on the first similarity to obtain a video feature set; and calculates the second similarity of each video feature within the video feature set, and finally screens the to-be-processed video based on the second similarity between the video features. The above-mentioned method uses the first similarity between the video features corresponding to the videos to perform preliminary clustering, and then calculates the second similarity between the video features within the clustered video feature set, thereby screening out to-be-processed videos that are identical or similar to the reference video. The preliminary clustering followed by a small-scale comparison of video features can reduce the number of video feature comparisons, and can effectively reduce the processing level when screening massive amounts of videos.

[0067] In one embodiment, Figure 3 As shown, clustering each video feature based on the first similarity to obtain a video feature set includes steps S231 to S233.

[0068] Step S231 : sorting the video features in order of size according to the first similarity to obtain a preliminary sorting result.

[0069] By sorting according to the size of the first similarity, video features with large first similarity values ​​can be sorted together in the preliminary sorting results. When screening subsequently, video features with large first similarity values ​​are compared first, which can improve the screening efficiency of video data.

[0070] In one embodiment, the video feature ranked first in the preliminary sorting result can be set according to the sorting order of the preliminary sorting result; for example, when sorting from large to small according to the first similarity in the preliminary sorting result, the first place in the sorting is selected from the beginning; and when sorting from small to large according to the first similarity in the preliminary sorting result, the first place in the sorting is selected from the end.

[0071] Step S232 : for each video feature, determine adjacent video features of the video feature within a preset range in the preliminary sorting results.

[0072] In one embodiment, for each video feature, adjacent video features of the video feature within a preset range are determined in the preliminary sorting results, including: selecting video features in the preliminary sorting results in sequence according to the sorting order as initial video features; selecting video features within a preset range that are sorted after the initial video features in the preliminary sorting results, and determining them as adjacent video features of the video feature.

[0073] The specific size of the preset range can be set according to actual conditions. In this embodiment, since the preliminary sorting result is obtained by sorting based on the first similarity value, the video features that are within the preset range can be determined based on the sorting position difference between the video features. For example, when the video feature with the first sorting position is used as the initial video feature A, the video features with sorting positions between 2nd and 501st (the preset range is set as the sorting position difference within 500 as an example) are selected as the adjacent video features of the initial video feature A. After the adjacent video features of the video feature with the first sorting position are determined, the video feature with the second sorting position is selected as the initial video feature B, and the video features with sorting positions between 3rd and 502nd (the preset range is set as the sorting position difference within 500 as an example) are selected as the adjacent video features of the initial video feature B, and so on.

[0074] Furthermore, when selecting adjacent video features of a video feature, video features within a preset range can be selected in only one direction; for example, for a video feature with a ranking position of 2, video features with ranking positions of 3 to 502 (the preset range is set as the ranking position difference within 500, for example) are selected as adjacent video features; for another example, for a video feature with a ranking position of 10, video features with ranking positions of 11 to 510 are selected as adjacent video features; and so on.

[0075] Step S233: The video feature and the adjacent video features are taken as a video feature set corresponding to the video feature.

[0076] For each video feature, a corresponding adjacent video feature is determined respectively, and the video feature and the adjacent video features are determined as a video feature set corresponding to the video feature.

[0077] In this embodiment, when clustering the video features, they are sorted in order of magnitude based on the first similarity value. Each of these video features is recorded as an initial video feature. Adjacent video features that are within a preset range from the initial video feature are selected, and the adjacent video features are combined with the initial video feature to form a video feature set for the initial video feature. A video feature set corresponding to each video feature is obtained, and each video feature set contains the same number of video features.

[0078] Furthermore, in one embodiment, respectively calculating the second similarity between each video feature in the video feature set includes: in the video feature set corresponding to the initial video feature, respectively calculating the second similarity between the initial video feature and each non-initial video feature.

[0079] Among them, non-initial video features refer to video features other than the initial video features in the video feature set. In this embodiment, in the video feature set corresponding to the initial video feature, only the second similarity between the initial video feature and the non-initial video feature is calculated, which can avoid repeated calculations. For example, in the video feature set corresponding to the initial video feature A, the second similarity between the initial video feature A and the non-initial video features B, C, D, and E is calculated respectively; and the second similarity is not calculated between the combinations of non-video features such as video features B and C, video features B and D, and video features B and E in the video feature set. When video feature B is a video feature set of the initial video feature, the second similarity can be calculated for the above-mentioned video feature combinations, thereby avoiding repeated calculations.

[0080] In another embodiment, each video feature is clustered based on the first similarity to obtain a video feature set, including: video features whose first similarity values ​​are greater than a preset threshold are grouped as a video feature set. In this embodiment, no sorting is performed, and the video feature set is directly selected based on the first similarity value. For example, if the first similarity value between video feature A and video feature B is greater than the preset threshold, the first similarity value between video feature B and video feature C is greater than the preset threshold, and the first similarity value between video feature A and video feature D is greater than the preset threshold, video features A, B, C, and D are grouped into one video feature set.

[0081] In one embodiment, Figure 4 As shown, screening the videos to be processed based on the second similarity includes steps S241 to S243.

[0082] Step S241 : When a target video feature is detected in non-initial video features, a video identifier corresponding to the target video feature and the initial video feature is detected; the target video feature is a video feature whose second similarity with the initial video feature is greater than a similarity threshold.

[0083] In this embodiment, a non-initial video feature whose second similarity with the initial video feature is greater than a similarity threshold is recorded as a target video feature. When the target video feature is detected in the video feature set of the initial video feature, the video identifiers corresponding to the target video feature and the initial video feature are checked to determine the type of video data corresponding to each of the target video feature and the initial video feature.

[0084] Among them, the video identifier is used to distinguish the characteristics of the video. In one embodiment, the video identifier includes an identifier corresponding to the baseline video and an identifier corresponding to the video to be processed. In one embodiment, the video identifier can be marked for each video after obtaining the video to be processed and the baseline video; further, when the video features of each video data are obtained, the video identifier is marked in the corresponding video feature. Since the preliminary sorting result includes the baseline video features and the video features to be processed, in this embodiment, if it is detected that the second similarity between the target video features and the initial video features is greater than the similarity threshold, the video identifier of the target video features is read for detection. Among them, the similarity threshold can be set according to the actual situation, for example, it can be set to 80%, 90%, etc.

[0085] Step S242: If the video identifier corresponding to the target video feature and the initial video feature does not include a reference video identifier, a repeat identifier is set for the target video feature or the initial video feature; if the video identifier corresponding to the target video feature and the initial video feature only includes one reference video identifier, a repeat identifier is set for the video feature corresponding to the video identifier to be processed.

[0086] In this embodiment, the duplicate flag is used to identify the to-be-processed video that duplicates the reference video. If the second similarity between the initial video feature and the non-initial video feature is greater than the similarity threshold, indicating that the two videos are highly similar, the video flags corresponding to the two videos are determined. The non-initial video feature (target video feature) whose second similarity is greater than the similarity threshold is determined to be the reference video or the to-be-processed video.

[0087] Because the reference video is used to filter out duplicate videos to be processed, when setting a duplicate flag, the duplicate flag is only set for the videos to be processed corresponding to the features of the video to be processed, and no duplicate flag is set for the reference video corresponding to the reference video features. Furthermore, in this embodiment, if the video flags corresponding to the target video features and the initial video features do not include the reference video flag, indicating that both video features correspond to the videos to be processed, a duplicate flag is set for the videos to be processed corresponding to either the target video features or the initial video features.

[0088] In another embodiment, if the video identification package corresponding to the target video feature and the initial video feature contains only one reference video identification, it means that the videos corresponding to the target video feature and the initial video feature include a to-be-processed video and a reference video. At this time, a duplicate identification is set for the video corresponding to the to-be-processed video identification in the two.

[0089] Furthermore, in one embodiment, the same repeat flag is set for the to-be-processed videos corresponding to the target video features whose second similarity with the same initial video feature is greater than the similarity threshold. For example, in the video feature set corresponding to the initial video feature a, the initial video feature a corresponds to the reference video A, and the to-be-processed videos whose second similarity with the initial video feature a is greater than the similarity threshold include X1, X2, ..Xn, and repeat flag 1 is set for X1, X2, ..Xn. For another example, in the video feature set corresponding to the initial video feature b, the initial video feature b corresponds to the reference video B, and the to-be-processed videos whose second similarity with the initial video feature b is greater than the similarity threshold include Y1, Y2, ..Yn, and repeat flag 2 is set for Y1, Y2, ..Yn, and so on.

[0090] In this embodiment, by setting the same repeat flag for the videos to be processed that have a higher second similarity to the same reference video, and setting different repeat flags for the videos to be processed that have a higher second similarity to different reference videos, it can help to filter out the videos to be processed that are more similar and belong to the same category from the videos to be processed.

[0091] Step S243: Filter the videos to be processed that contain duplicate identifiers.

[0092] After calculating the second similarity between each video feature in the video feature set using the above method, the videos to be processed that have a higher similarity with the benchmark video can be screened out according to the set duplicate identifier, and these videos to be processed that carry the duplicate identifier are filtered, thus completing the screening and filtering of the video data.

[0093] In this embodiment, different video identifiers are set for the reference video and the video to be processed. During screening, only the video features of the video to be processed with the video identifier as the video to be processed are set with a repeated identifier, which can prevent the reference video from being filtered out during screening.

[0094] In one embodiment, for a to-be-processed video containing repeated identifiers, the second similarity between the corresponding video feature and other video features is no longer calculated, which can reduce the number of similarity calculations and improve the efficiency of video data screening.

[0095] In one embodiment, the features of the video to be processed include the speech and text content to be processed, and the features of the reference video include the reference speech and text content. In one embodiment, speech recognition processing is performed on the video to be processed and the reference video to obtain the corresponding speech and text content. In one embodiment, the speech and text content represents the text content corresponding to the audio in the video. Furthermore, speech recognition can be implemented in any manner.

[0096] Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction. Speech recognition technology, also known as automatic speech recognition (ASR), aims to convert the lexical content of human speech into computer-readable input, such as keystrokes, binary codes, or character sequences. This differs from speaker recognition and speaker verification, which attempt to identify or verify the speaker of a speech rather than the lexical content.

[0097] Furthermore, in this embodiment, speech recognition technology is used to extract features from the video to be processed and the reference video respectively, and the video to be processed and the reference video are converted into corresponding speech text content, which are recorded as the speech text content to be processed and the reference speech text content respectively.

[0098] In one embodiment, first similarities between each pair of to-be-processed video features and reference video features are calculated respectively.

[0099] Furthermore, in one embodiment, the first similarities between the video features to be processed and the reference video features are calculated respectively, including: starting with the first word, the text similarities of the corresponding text positions in the speech text content to be processed and the reference speech text content are calculated in sequence, and the first similarity includes the text similarity.

[0100] The first word of the speech text content represents the first word in the speech text. In this embodiment, starting from the first word, the similarity of each subsequent word in the same position in the speech text content to be processed and the reference speech text content is calculated in sequence. In one embodiment, the similarity of the texts in the same position can be calculated by the pronunciation, interpretation, etc. of the texts. In a specific embodiment, taking the pronunciation as an example, if the pronunciation is the same, the text similarity is 100%, and if the pronunciation is different, the text similarity is 0; in other embodiments, other similarity determination methods can also be set.

[0101] Furthermore, after determining the similarity of the text corresponding to each position, the overall text similarity of the speech text content can be obtained by integrating the similarities of the text corresponding to each position; for example, in a specific embodiment, by performing weighted summation on the similarities of the text corresponding to each position, the closer the position is, the greater the corresponding weight, which can be set according to actual conditions.

[0102] In this embodiment, the to-be-processed video features and the reference video features are sorted in order of magnitude according to the first similarity to obtain a preliminary sorting result, including: sorting the text similarity in order of magnitude to obtain a preliminary sorting result of the speech text content.

[0103] After obtaining the text similarity of the speech text content, the speech text content can be sorted according to the text similarity to obtain the preliminary sorting result of the speech text content in this embodiment.

[0104] In this embodiment, a first similarity is specifically described as being calculated based on the speech text content of the video. The similarity is determined by using the similarity of text at various positions in the speech text content. In this way, the features of the processed video and the reference video are first sorted, and video features with higher first similarities can be clustered in similar positions. Subsequently, when performing the second similarity calculation, the second similarity between the video features with higher first similarities can be preferentially calculated. For example, when the second similarity is calculated in descending order based on the first similarity, when the second similarity is less than a certain value, it means that the second similarity between other video features following in the order and the initial video feature will also be lower. Under the condition of taking into account a certain degree of accuracy and efficiency, the second similarity between the subsequent video features and the initial video feature can be no longer calculated, thereby reducing the number of times the second similarity is calculated and improving the efficiency of video screening.

[0105] Furthermore, in one embodiment, respectively calculating the second similarity between each video feature in the video feature set includes: respectively calculating the Levenshtein distance between the initial video feature and the to-be-matched video feature as the second similarity.

[0106] The Levenstein distance refers to the minimum number of editing operations required to convert one string into another. Editing operations include replacing one character with another, inserting a character, or deleting a character, etc. Generally speaking, the smaller the edit distance, the greater the similarity between the two strings. In another embodiment, the Levenstein ratio between the initial video features and the video features to be matched can also be calculated as the second similarity. Furthermore, calculating the Levenstein distance or Levenstein ratio between the initial video features and the video features to be matched can be achieved using existing methods.

[0107] In this embodiment, by calculating the Levenstein distance or the Levenstein ratio as the second similarity, a higher second similarity can be obtained for speech text content with similar sentence meanings but not completely consistent word order. When screening videos based on the second similarity, the problem of incomplete screening of such similar videos can be avoided.

[0108] In one embodiment, the features of the video to be processed include the images to be processed, and the features of the reference video include the reference images. In one embodiment, image extraction is performed on the video to be processed and the reference video to obtain corresponding images as video features. Furthermore, in this embodiment, the image features corresponding to the video to be processed are recorded as the images to be processed, and the image features corresponding to the reference video are recorded as the reference images. Extracting images from the video can be achieved in any manner.

[0109] In one embodiment, when extracting pictures from a video, any picture can be selected as a video feature. For example, the video cover can be selected as a video feature, or the first frame of the video can be selected as a video feature, or the video picture corresponding to a specified time can be selected as a video feature, and so on.

[0110] Furthermore, similar to the embodiment of using speech text content as video features, in the embodiment of extracting pictures as video features, the first similarity between the picture to be processed and the reference picture can also be calculated first, and the pictures to be processed and the reference pictures can be sorted based on the first similarity to obtain a preliminary sorting result; then, according to the order of the preliminary sorting results, the initial picture and the adjacent pictures in the preliminary sorting results are determined as the video feature set corresponding to the initial picture, and for any video feature set of the initial picture, the second similarity between each initial picture and the non-initial picture is calculated; finally, the video to be processed is filtered based on the second similarity.

[0111] Calculating the first similarity between the image to be processed and the reference image can be achieved through any method. In one embodiment, the first similarity between the image to be processed and the reference image is calculated using the k-means algorithm; the k-means algorithm (K-means clustering algorithm) is an iterative clustering analysis algorithm. In another embodiment, the first similarity between the image to be processed and the reference image is calculated using the phash algorithm (perceptual hashing algorithm). In other embodiments, the first similarity between the image to be processed and the reference image can also be calculated through other methods.

[0112] Furthermore, when screening the videos to be processed based on the second similarity, if it is detected that the second similarity between the non-initial image and the initial image is greater than the similarity threshold, if the video identifier corresponding to the initial image is the reference video and the video identifier corresponding to the target image is the video to be processed, a duplicate flag is set for the video to be processed corresponding to the target image; if the initial image corresponds to the video to be processed, a duplicate flag is set for the video to be processed corresponding to the initial image. Finally, the videos to be processed containing the duplicate flag are filtered.

[0113] In this embodiment, pictures in the video data are selected as video features to calculate feature similarity, and preliminary clustering is first performed to obtain a preliminary sorting result when calculating the first similarity. Then, the second similarity is calculated based on the video feature set obtained from the preliminary sorting result. This can reduce the number of times the second similarity is calculated and improve the efficiency of video screening.

[0114] Furthermore, in one embodiment, the video data screening method further includes: updating the videos to be processed retained after screening to a database. Furthermore, the video data screening method can be applied to scenarios such as building a video database and video recommendation.

[0115] This application also provides an application scenario, which applies the above-mentioned video data screening method. Specifically, the application of the video data screening method in this application scenario is as follows:

[0116] The above method is applied to the voice annotation scenario to build a historical database. The historical database stores a large number of videos. When the historical database is updated, videos similar to existing videos in the historical database are no longer updated to the database. Therefore, it is necessary to use the above method to screen and filter the videos to be stored. The video data in the historical database can be extracted for voice annotation later. In this embodiment, the voice recognition of the video is described as an example, and the voice text content corresponding to the video data is used as the video feature:

[0117] like Figure 5 As shown, first, short video-related data (regardless of the scenario) is obtained through a crawler tool. At the same time, the audio is converted into text information using ASR technology. Text similarity deduplication is used to eliminate data with high similarity, and data with similarity <80% (the threshold can be adjusted according to needs) is retained to build a historical database. After the historical database is successfully built, subsequent audio data information is matched with the historical database before flowing into the annotation. Data with ASR text similarity ≥80% is eliminated, while data with text similarity <80% (the threshold can be adjusted according to needs) is retained for data production and continuously iterated and accumulated into the historical database. The specific process is as follows:

[0118] 1. Obtain the video to be processed, import it into the ASR audio-to-text service, and export its video link and corresponding audio-to-text content.

[0119] Second, import the video to be processed and its ASR text content into the designed screening service, output the video that does not overlap with the benchmark video in the historical database, and update the historical database.

[0120] 1. The screening service includes the following steps: sorting the speech and text content of the video to be processed and the reference videos in the historical database, specifically based on the textual similarity of the Chinese speech and text content (the aforementioned first similarity): sorting is performed based on the pinyin of the first character of the Chinese speech and text content, with the first character being the same and the next character being sorted backwards. Sentences with the most similar beginnings in the speech and text content are grouped together to obtain a preliminary ranking result. Then, within the preliminary ranking result, each piece of speech and text content (the initial video feature, which can be the speech and text content of the reference video or the speech and text content of the video to be processed) is combined with the next 500 pieces of content (non-initial video features) in order of precedence to form the video feature set corresponding to each speech and text content. A second similarity is calculated for each speech and text content in the video feature set. Based on the second similarity, non-duplicate videos to be processed are screened and output, and the non-duplicate videos to be processed are updated to the historical database.

[0121] 2. Calculating a second similarity between the video to be processed and the reference video, and screening the video to be processed based on the similarity includes the following steps:

[0122] 1) Merge the ASR content (speech and text content) corresponding to the video to be processed with the ASR content of the reference video. Assign the video identifier "new" to the video to be processed (indicating it as the video to be processed) and the video identifier "old" to the reference video (indicating it as the reference video). All ASR content is assigned a non-duplicate identifier. Then, sort the speech and text content, initially grouping similar content together to obtain a preliminary ranking result.

[0123] 2) In the preliminary ranking results, traverse the speech text content data with non-repeated identifiers in the entire ASR content, and perform a second similarity comparison (calculate the Levenstein ratio or Levenstein distance) on each ASR text content with the 500 contents (video feature sets) ranked after it, such as Figure 6 The figure shows a specific example of calculating the second similarity. In this embodiment, speech and text content data with a second similarity greater than a similarity threshold (e.g., 80%) is determined to be a video duplicate. If a duplicate occurs, the video identifier is used to determine whether the currently traversed ASR content is the reference video. If it is the reference video, a duplicate identifier is set for the following 500 non-initial speech and text content that are duplicates of the reference video. If the currently traversed ASR content is the video to be processed, a duplicate identifier is set for the current ASR content.

[0124] 3) After the traversal is completed, the video data with the video mark of "new" and non-duplicate mark is output as the video data after filtering and deduplication. The data to be processed with the video mark of "new" and non-duplicate mark is updated to the historical database.

[0125] In a specific embodiment, the flow chart of the above method can be referred to Figure 7 The steps shown in the figure are as follows. In the figure, if_new = 1 indicates the identifier of the video to be processed, and if_new = 0 indicates the identifier of the reference video; if_drop = 1 indicates a duplicate identifier, and if_drop = 0 indicates a non-duplicate identifier. In another embodiment, the steps of the above method can also be performed by extracting images from the video as video features.

[0126] The above method uses ASR services to convert video data into text information and first performs preliminary clustering using a first similarity (text similarity in this embodiment) to obtain a set of video features corresponding to each piece of spoken text content. The second similarity between each video feature in the video feature set is calculated using the Levenshtein ratio. Traditional MD5 deduplication methods cannot address the problem of secondary creation in the short video field, which involves large-scale reposting or minor modifications. However, the above method, by reducing the video dimension to text and then using text sorting and then small-scale comparison, can solve the problem of filtering out large numbers of duplicate videos, ensuring accuracy while improving processing efficiency.

[0127] It should be understood that, although each step in each flow chart involved in the above-described embodiment is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless clearly stated herein, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each flow chart involved in the above-described embodiment can include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0128] In one embodiment, Figure 8 As shown, a video data screening device is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: an acquisition module 810, a first similarity calculation module 820, a clustering module 830, a second similarity calculation module 840 and a screening module 850, wherein:

[0129] An acquisition module 810 is configured to acquire a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video;

[0130] A first similarity calculation module 820 is used to calculate first similarities between each pair of to-be-processed video features and reference video features;

[0131] A clustering module 830 is configured to cluster each video feature based on the first similarity to obtain a video feature set; the video features include to-be-processed video features or reference video features;

[0132] A second similarity calculation module 840 is used to respectively calculate the second similarity between each video feature in the video feature set;

[0133] The screening module 850 is used to screen the videos to be processed based on similarity.

[0134] The above-mentioned video data screening device obtains a to-be-processed video and a reference video, as well as the video features corresponding to the to-be-processed video and the reference video respectively; then calculates the first similarity between each to-be-processed video feature and the reference video feature, performs preliminary clustering on the to-be-processed video feature and the reference video feature based on the first similarity to obtain a video feature set; and calculates the second similarity of each video feature within the video feature set, and finally screens the to-be-processed video based on the second similarity between the video features. The above-mentioned device performs preliminary clustering using the first similarity between the video features corresponding to the videos, and then calculates the second similarity between the video features within the video feature set obtained by clustering, thereby screening out to-be-processed videos that are identical or similar to the reference video. The preliminary clustering followed by a small-scale comparison of video features can reduce the number of video feature comparisons, and can effectively reduce the processing level, especially when screening massive videos.

[0135] In one embodiment, the first similarity calculation module 820 of the above-mentioned device includes: a preliminary sorting submodule, which is used to sort each video feature in order of size according to the first similarity to obtain a preliminary sorting result; an adjacent video feature determination submodule, which is used to determine, for each video feature, the adjacent video features of the video feature within a preset range in the preliminary sorting result; and a video feature set determination submodule, which is used to use the video feature and the adjacent video features as a video feature set corresponding to the video feature.

[0136] In one embodiment, the adjacent video feature determination submodule of the above-mentioned device includes: an initial video feature selection unit, which is used to select video features in the preliminary sorting results in sequence according to the sorting order as the initial video features; an adjacent video feature selection unit, which is used to select video features within a preset range after the initial video features in the preliminary sorting results, and determine them as adjacent video features of the video features.

[0137] In one embodiment, the second similarity calculation module 840 of the above apparatus is further configured to: calculate the second similarity between the initial video feature and each non-initial video feature in the video feature set corresponding to the initial video feature.

[0138] In one embodiment, the screening module 850 of the above-mentioned device includes: a video identifier reading submodule, which is used to detect the video identifier corresponding to the target video feature and the initial video feature when the target video feature is detected in the non-initial video feature; the target video feature is a video feature whose second similarity with the initial video feature is greater than the similarity threshold; a repeat identifier setting submodule, which is used to set a repeat identifier for the target video feature or the initial video feature if the video identifier corresponding to the target video feature and the initial video feature does not contain a baseline video identifier; if the video identifier corresponding to the target video feature and the initial video feature only contains one baseline video identifier, set a repeat identifier for the video feature corresponding to the video identifier to be processed; a filtering submodule, which is used to filter the video to be processed containing a repeat identifier.

[0139] In one embodiment, the features of the video to be processed include the voice text content to be processed, and the reference video features include the reference voice text content; in this embodiment, the first similarity calculation module of the above-mentioned device is also used to: starting from the first word, calculate the text similarity of each corresponding text position in the voice text content to be processed and the reference voice text content in sequence, and the first similarity includes the text similarity.

[0140] In one embodiment, the to-be-processed video feature includes a to-be-processed picture, and the reference video feature includes a reference picture.

[0141] For specific embodiments of the video data screening device, please refer to the embodiments of the video data screening method described above and will not be repeated here. Each module in the aforementioned video data screening device may be implemented in whole or in part via software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0142] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown. The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication method can be achieved through Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a video data screening method. The display screen of the computer device can be a liquid crystal display or an electronic ink display. The input device of the computer device can be a touch layer covering the display screen, or keys, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0143] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0144] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0146] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0147] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0148] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A video data screening method, characterized in that: The method comprises: Obtaining a to-be-processed video, a reference video, to-be-processed video features corresponding to the to-be-processed video, and reference video features corresponding to the reference video; Calculating first similarities between each of the to-be-processed video features and the reference video features; Sorting the video features in order of magnitude according to the first similarity to obtain a preliminary sorting result; each video feature is a to-be-processed video feature or a reference video feature; For each of the video features, determining adjacent video features of the video feature within a preset range in the preliminary sorting results; Taking the video feature and the adjacent video feature as the video feature set corresponding to the video feature; respectively calculating a second similarity between each of the video features in the video feature set; The videos to be processed are screened based on the second similarity.

2. The video data screening method according to claim 1, characterized in that: The step of determining, for each of the video features, adjacent video features of the video feature within a preset range in the preliminary sorting results includes: Selecting the video features in the preliminary sorting results in order as initial video features; A video feature that is sorted within a preset range after the initial video feature in the preliminary sorting result is selected and determined as an adjacent video feature of the video feature.

3. The video data screening method according to claim 2, wherein: The respectively calculating the second similarity between each of the video features in the video feature set includes: In the video feature set corresponding to the initial video feature, a second similarity between the initial video feature and each non-initial video feature is calculated respectively.

4. The video data screening method according to claim 3, wherein: The screening of the to-be-processed videos based on the second similarity includes: When a target video feature is detected in the non-initial video feature, detecting a video identifier corresponding to the target video feature and the initial video feature; the target video feature is the video feature whose second similarity with the initial video feature is greater than a similarity threshold; If the video identifiers corresponding to the target video feature and the initial video feature do not include the reference video identifier, setting a repeat identifier for the target video feature or the initial video feature; If the video identifiers corresponding to the target video feature and the initial video feature include only one reference video identifier, setting a repeat identifier for the video feature corresponding to the to-be-processed video identifier; The to-be-processed video containing the repeated identifier is filtered.

5. The video data screening method according to any one of claims 1 to 4, characterized in that: The to-be-processed video features include to-be-processed speech and text content, and the reference video features include reference speech and text content; The first similarity between the video features to be processed and the reference video features is calculated separately, including: starting with the first word, sequentially calculating the text similarity of each corresponding text position in the speech text content to be processed and the reference speech text content, wherein the first similarity includes the text similarity.

6. The video data screening method according to any one of claims 1 to 4, characterized in that: The to-be-processed video features include to-be-processed pictures, and the reference video features include reference pictures.

7. A video data screening device, characterized in that: The device comprises: An acquisition module, configured to acquire a video to be processed, a reference video, features of the video to be processed corresponding to the video to be processed, and features of the reference video corresponding to the reference video; A first similarity calculation module is used to calculate the first similarity between each of the to-be-processed video features and the reference video features; A clustering module, configured to cluster the video features based on a first similarity to obtain a video feature set; each video feature is a to-be-processed video feature or a reference video feature; the clustering module comprises a preliminary sorting submodule, an adjacent video feature determination submodule, and a video feature set determination submodule; The preliminary sorting submodule is used to sort the video features in order of size according to the first similarity to obtain a preliminary sorting result; The adjacent video feature determination submodule is configured to determine, for each of the video features, adjacent video features of the video feature within a preset range in the preliminary sorting results; The video feature set determination submodule is configured to use the video feature and the adjacent video feature as the video feature set corresponding to the video feature; A second similarity calculation module, configured to respectively calculate a second similarity between each of the video features in the video feature set; A screening module is used to screen the videos to be processed based on the second similarity.

8. The video data screening device according to claim 7, wherein: The adjacent video feature determination submodule includes: an initial video feature selection unit, configured to sequentially select video features from the preliminary sorting results in a sorting order as initial video features; The adjacent video feature selection unit is configured to select video features that are sorted within a preset range after the initial video feature in the preliminary sorting result and determine them as adjacent video features of the video feature.

9. The video data screening device according to claim 8, characterized in that: The second similarity calculation module is further configured to respectively calculate a second similarity between the initial video feature and each non-initial video feature in the video feature set corresponding to the initial video feature.

10. The video data screening device according to claim 9, characterized in that: The screening module includes: a video identification reading submodule, configured to, when a target video feature is detected in the non-initial video feature, detect the video identifications corresponding to the target video feature and the initial video feature; the target video feature being the video feature having a second similarity with the initial video feature greater than a similarity threshold; a repeat flag setting submodule, configured to set a repeat flag for the target video feature or the initial video feature if the video flags corresponding to the target video feature and the initial video feature do not include the reference video flag; and set a repeat flag for the video feature corresponding to the to-be-processed video flag if the video flags corresponding to the target video feature and the initial video feature only include one reference video flag; The filtering submodule is used to filter the video to be processed that contains the repeated identifier.

11. The video data screening device according to any one of claims 7 to 10, characterized in that: The video features to be processed include the speech text content to be processed, and the benchmark video features include the benchmark speech text content; the first similarity calculation module is also used to: starting with the first word, calculate the text similarity of each corresponding text position in the speech text content to be processed and the benchmark speech text content in sequence, and the first similarity includes the text similarity.

12. The video data screening device according to any one of claims 7 to 10, characterized in that: The to-be-processed video features include to-be-processed pictures, and the reference video features include reference pictures.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images