Video deduplication method, device and equipment, readable storage medium and program product

By acquiring multi-dimensional feature information of the target video and comparing it with the video library index, the video to be deduplicated is determined, which solves the problem of low video deduplication efficiency and achieves more efficient video deduplication processing.

CN118964670APending Publication Date: 2024-11-15MIGU CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411088374.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The low efficiency of video deduplication in existing technologies is mainly due to the excessively large similarity comparison range.

Method used

By acquiring feature information (category, tags, and keywords) from multiple dimensions of the target video, comparing it with the corresponding video indexes in the video library, obtaining a set of candidate videos, calculating the similarity of the intersecting videos, and determining the videos to be deduplicated.

Benefits of technology

It narrows the scope of video similarity comparison, improves video deduplication efficiency, reduces unnecessary similarity comparisons, and increases processing speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964670A_ABST
    Figure CN118964670A_ABST
Patent Text Reader

Abstract

The invention discloses a video deduplication method and device, equipment, a readable storage medium and a program product, and relates to the technical field of video processing, the method comprises the steps that feature information of multiple dimensions of a target video is acquired, and the multiple dimensions comprise at least two of categories, tags and keywords; respectively comparing the feature information of each dimension in the plurality of dimensions with a video index of a corresponding dimension of a video library in sequence to obtain a candidate video set respectively corresponding to each dimension of the target video in the video library; the video index is used for representing an index relationship between the feature information of the corresponding dimension in the video library and the candidate video in the video library; acquiring an intersection video of the plurality of candidate video sets; and when the similarity between the target video and any video in the intersection videos is greater than a similarity threshold, determining that the target video is a video to be subjected to duplicate removal. According to the scheme, the similarity comparison range is reduced, and the video deduplication efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video deduplication method, device, equipment, readable storage medium and program product. Background Art

[0002] With the rise of short videos on the Internet, the number of short videos produced every day has skyrocketed exponentially. A large number of high-quality content creators have emerged, and various cases of plagiarism and copying have also occurred. In particular, the rise of artificial intelligence (AI) technology has further fueled this "adaptation" trend. AI technology is used to rewrite original titles, modify short video sounds, and even tamper with short video images to avoid platform censorship.

[0003] In order to avoid the above situation, it is necessary to perform deduplication processing on short videos. In the relevant deduplication processing, it is necessary to compare the similarity of short videos with each short video in the video library, and perform deduplication processing based on the comparison results. However, the similarity comparison range of this method is relatively large, resulting in low video deduplication efficiency. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video deduplication method, apparatus, device, readable storage medium and program product, so as to solve the problem of low video deduplication efficiency due to a large similarity comparison range.

[0005] In a first aspect, in order to achieve the above-mentioned object, an embodiment of the present application provides a video deduplication method, comprising:

[0006] Acquire feature information of multiple dimensions of a target video, where the multiple dimensions include at least two of categories, tags, and keywords;

[0007] Comparing the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library in turn, to obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library;

[0008] Obtaining intersection videos of multiple candidate video sets;

[0009] When the similarity between the target video and any video in the intersection videos is greater than a similarity threshold, the target video is determined to be a video to be deduplicated.

[0010] Optionally, obtaining feature information of multiple dimensions of a target video includes at least the following two steps:

[0011] Segmenting the video title of the target video and calculating word weights to obtain feature information of the label dimension;

[0012] Classifying the feature information of the label dimension to obtain feature information of the category dimension;

[0013] The audio data of the target video is converted into text data, and feature information of a keyword dimension is extracted from the text data.

[0014] Optionally, the method further comprises:

[0015] For the videos that pass the deduplication detection, based on the feature information of each dimension of the videos that pass the deduplication detection, a video index of the corresponding dimension of the video library is created and / or updated.

[0016] Optionally, when the similarity between the target video and any video in the intersection videos is greater than a similarity threshold, before determining that the target video is a video to be deduplicated, the method further includes:

[0017] Extracting key frames related to the feature information of the keyword dimension in the target video according to the feature information of the keyword dimension;

[0018] The similarity between the target video and each video of the intersection video is determined according to the key frames of the target video and the key frames of each video of the intersection video, wherein the key frames of each video of the intersection video correspond to the feature information of the keyword dimension of each video of the intersection video.

[0019] Optionally, according to the feature information of the keyword dimension, extracting key frames related to the feature information of the keyword dimension in the target video includes:

[0020] In the target video, identifying a video segment corresponding to the feature information of the keyword dimension;

[0021] According to the position of the feature information of the keyword dimension in the text data, the key frames of each of the identified video clips are extracted, and the text data is converted from the audio data of the target video.

[0022] Optionally, determining the similarity between the target video and each video of the intersection video according to the key frame of the target video and the key frame of each video of the intersection video includes:

[0023] Performing image processing on the key frame of the target video to obtain a feature vector of the key frame of the target video;

[0024] Generate a feature vector of the target video according to the feature vectors of each key frame of the target video;

[0025] sequentially calculating the vector products of the feature vector of the target video and the feature vectors of each video of the intersection video, wherein the feature vectors of each video of the intersection video are generated by the feature vectors of the key frames of each video of the intersection video;

[0026] According to each of the vector products, the similarity between the target video and the video corresponding to the vector product in the intersection video is determined.

[0027] In a second aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application provides a video deduplication device, comprising:

[0028] A first acquisition module is used to acquire feature information of multiple dimensions of a target video, where the multiple dimensions include at least two of categories, tags, and keywords;

[0029] A second acquisition module is used to sequentially compare the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library, and obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library;

[0030] A third acquisition module is used to obtain an intersection video of a plurality of the candidate video sets;

[0031] The first determination module is used to determine that the target video is a video to be deduplicated when the similarity between the target video and any video in the intersection videos is greater than a similarity threshold.

[0032] In the third aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application provides a video deduplication device, including: a transceiver, a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein when the computer program is executed by the processor, the video deduplication method described in the first aspect is implemented.

[0033] In a fourth aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the video deduplication method as described in the first aspect is implemented.

[0034] In a fifth aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the video deduplication method as described in the first aspect.

[0035] The above technical solution of the present application has at least the following beneficial effects:

[0036] The video deduplication method of the embodiment of the present application first obtains feature information of multiple dimensions of a target video, wherein the multiple dimensions include at least two of categories, tags and keywords; secondly, the feature information of each dimension of the multiple dimensions is sequentially compared with the video index of the corresponding dimension of the video library, so as to obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library; thirdly, an intersection video of multiple candidate video sets is obtained; finally, when the similarity between the target video and any video in the intersection video is greater than a similarity threshold, the target video is determined to be a video to be deduplicated, wherein the intersection video is one or more videos used for similarity comparison with the target video, and relative to all videos in the video library, the number of intersection videos here is significantly reduced, thereby narrowing the scope of video similarity comparison and improving the efficiency of video deduplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is one of the flowcharts of the video deduplication method according to an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a video index in a category dimension in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of a video index in the label dimension in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a video index based on a keyword dimension in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the relationship between keywords and key frames in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the process of constructing a video index in an embodiment of the present application;

[0043] Figure 7 This is a second flow chart of the video deduplication method according to an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of a video deduplication device according to an embodiment of the present application;

[0045] Fig. 9 A schematic diagram of the structure of a video deduplication device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0048] The video deduplication method, apparatus, device, readable storage medium and program product provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0049] The embodiment of the present application provides a video deduplication method, such as Figure 1 As shown, the method includes:

[0050] Step 101, obtaining feature information of multiple dimensions of a target video, wherein the multiple dimensions include at least two items of categories, tags, and keywords; that is, this step can obtain feature information of the category dimension and feature information of the tag dimension of the target video, or obtain feature information of the category dimension and feature information of the keyword dimension of the target video, or obtain feature information of the tag dimension and feature information of the keyword dimension of the target video, or obtain feature information of the category dimension, feature information of the tag dimension, and feature information of the keyword dimension of the target video; specifically, the feature information of the category dimension is the category to which the target video belongs, where the category may include one or more levels of categories, for example, the feature information of the category dimension of the target video includes a first-level category (sports) and a second-level category (football / basketball), or a first-level category (literary category) and a second-level category (TV series / movies), etc.; the feature information of the tag dimension of the target video includes the video tag of the target video; the feature information of the keyword dimension of the target video includes the video keyword of the target video;

[0051] Here, it should also be noted that in the above step 101, it can be specifically to obtain feature information of different dimensions from different parts of the target video, wherein the different parts can include the title part, the video main part, etc. of the target video, wherein the video main part can include the audio part and the video clip part; for example, one situation is: obtaining feature information of category dimension and / or feature information of label dimension from the title part, which can be specifically: obtaining feature information of label dimension based on keywords in the title part, and obtaining feature information of category dimension based on the classification of these keywords; and obtaining feature information of keyword dimension from the video main part, which can be specifically: converting the audio part into text, extracting keywords from the text, and / or performing unrecognized recognition on the video clip part to obtain keywords corresponding to the image recognition results, and then obtaining feature information of keyword dimension according to the obtained keywords. Another situation is: obtaining feature information of the label dimension and feature information of the category dimension based on the title part and the video clip part, specifically, obtaining keywords from the label part, and obtaining corresponding keywords based on the image recognition result of the video clip part, and then obtaining feature information of the label dimension based on these keywords, and obtaining feature information of the category dimension based on the classification result of these keywords; obtaining feature information of the keyword dimension from the audio part and the video clip part, specifically, converting the audio part into text, extracting keywords from the text, performing image recognition on the video clip part, and obtaining keywords corresponding to the recognition result, and further obtaining feature information of the keyword dimension based on the obtained keywords;

[0052] Step 102, sequentially comparing the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library, and obtaining a candidate video set corresponding to each dimension of the target video in the video library; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library; here, taking the category dimension as an example, if the feature information of the category dimension of the target video includes sports and football, then this step is: first, finding the video index of the category dimension in the video library, then finding the video index with the attribute value of sports in the video index of the category dimension, and then further finding the video index with the attribute value of football in the video index of sports, and finally, determining the videos recorded in the video index of football as the candidate video set corresponding to the category dimension of the target video, that is, the category to which the videos in the candidate video set belong is the same as the category described in the target video, or the feature information of the category dimension of the videos in the candidate video set is the same as the feature information of the category dimension of the target video; the method for obtaining the candidate video set corresponding to other dimensions of the target video is the same as the method for obtaining the candidate video set corresponding to the category dimension, and the example will not be repeated here;

[0053] Step 103, obtaining an intersection video of the plurality of candidate video sets; for example, if feature information of two dimensions of the target video is obtained in step 101, then in step 102, candidate video sets corresponding to each dimension of the two dimensions will be obtained respectively, that is, two candidate videos will be obtained in step 102, and thus this step will take the intersection of the two candidate video sets (that is, extract the same video from the two candidate video sets) to obtain the intersection video, wherein the intersection video in this step may include one or more videos, and the obtained intersection video is a video with a high correlation with the target video;

[0054] Based on the operations of step 101 to step 103 above, videos (intersection videos) with high relevance to the target video can be obtained from the video library based on different dimensions, so as to determine whether the target video is a video to be deduplicated based on these intersection videos. In this way, the number of videos to be compared with the target video for similarity is reduced, the scope of similarity comparison is reduced, and the efficiency of video deduplication is improved to a certain extent.

[0055] Step 104, when the similarity between the target video and any video in the intersection videos is greater than the similarity threshold, the target video is determined to be a video to be deduplicated; here, if the similarity between the target video and at least one video in the intersection videos is greater than the similarity threshold, it indicates that the similarity between the two is high, and the target video is a video adapted based on the at least one video, that is, a repeatedly uploaded video, and the target video needs to be deduplicated, such as reminding the uploader of the target video that the video is a repeatedly uploaded video, or preventing the upload of the target video, so as to protect the rights and interests of the original video owner.

[0056] In the video deduplication method of the embodiment of the present application, first, feature information of multiple dimensions of a target video is obtained, wherein the multiple dimensions include at least two of categories, tags, and keywords; secondly, the feature information of each dimension of the multiple dimensions is sequentially compared with the video index of the corresponding dimension of the video library, to obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library; in this way, the feature information of the corresponding dimension of each candidate video in the obtained candidate video set is consistent with the feature information of the corresponding dimension of the target video. The same, so that the correlation between these candidate videos and the target video is relatively high; again, the intersection video of multiple candidate video sets is obtained; in this way, the correlation between the intersection video finally obtained and the target video is higher than the correlation between other candidate videos in each candidate video set and the target video; in this way, a relatively small number of videos with relatively high correlation with the target video are finally obtained, and finally, when the similarity between the target video and any video in the intersection video is greater than the similarity threshold, the target video is determined to be the video to be deduplicated, in this way, the target video is compared with a relatively small number of videos for similarity, the scope of similarity comparison is narrowed, and the efficiency of video deduplication is improved.

[0057] As an optional implementation, step 101 includes at least the following two steps:

[0058] The video title of the target video is segmented and the word weight is calculated to obtain the feature information of the label dimension; here, specifically, the title of the target video is segmented according to the hot search word library, the label word library, and the short video title library, the word weight is obtained by the term frequency-inverse document frequency (TF / IDF) algorithm, and the corresponding keywords are extracted as the feature information of the label category, for example: N words with larger word weights or words with word weights greater than the word weight threshold are determined as the feature information of the label dimension (the video label of the target video); for example, the video title of the target video is: Manchester City swept Copenhagen to advance to the quarterfinals of the Champions League, showing the dominance for seven consecutive seasons. The video labels (feature information of the label dimension) extracted according to this step include: Manchester City, Copenhagen, Champions League;

[0059] The feature information of the label dimension is classified to obtain feature information of the category dimension; here, the feature information of the category dimension may include only a primary category or multiple categories. In the embodiment of the present application, the feature information of the category dimension preferably includes multiple categories, so that the number of videos in the candidate video set corresponding to the category dimension can be reduced and the relevance of the videos in the candidate video set to the target video can be improved; in addition, this step can classify the feature information of the label dimension based on a classification algorithm (such as a regression algorithm, etc.) or a pre-trained classification model to obtain feature information of the category dimension (the category to which the target video belongs); taking the aforementioned target video having video labels including Manchester City, Copenhagen and the Champions League as an example, the feature information of the category dimension of the target video (the category to which the target video belongs) includes: sports (primary classification) and football (secondary classification);

[0060] The audio data of the target video is converted into text data, and feature information of the keyword dimension is extracted from the text data. Specifically, this step can use semantic-to-text technology to extract the voice data in the target video into text data, so as to extract the feature information of the keyword dimension (keywords in the text data) from the text data. Specifically, the feature information of the keyword dimension in the text data can refer to the implementation process of obtaining the feature information of the label dimension, that is, based on a hot search word library or a public word library on the Internet, the text data is segmented, and the word weight of each segmented word is obtained by the TF / IDF algorithm, so as to extract video keywords from the text data according to the word weight, and determine the video keywords as the feature information of the keyword dimension of the target video. For example, the feature information of the keyword dimension includes specific contents corresponding to each item such as person, time, entity, scene, background, event and score. For example, the keywords in a UEFA Champions League game highlights include: September 6, UEFA Champions League, 1 / 8 finals, Manchester City, De Bruyne, header, long pass, 1-0, supplementary shot, 2-0, free kick and 3-0, etc.

[0061] The above-mentioned optional implementation method is adopted to obtain at least two items of the feature information of the label dimension, the feature information of the category dimension and the feature information of the keyword dimension of the target video, so as to facilitate the subsequent acquisition of a set of candidate videos corresponding to each piece of feature information in the video library based on the obtained feature information. In this way, the scope of subsequent similarity comparison can be narrowed, the efficiency of similarity comparison can be improved, and the efficiency of video deduplication can be further improved.

[0062] Furthermore, as an optional implementation, the method further includes:

[0063] For the videos that pass the deduplication detection, based on the feature information of each dimension of the videos that pass the deduplication detection, a video index of the corresponding dimension of the video library is created and / or updated.

[0064] Here, the videos that have passed the deduplication detection are specifically videos that have been determined to be non-duplicate uploads, that is, original videos, wherein all videos in the video library are videos that have passed the deduplication detection. That is to say, after determining that a video is an original video, the video can be added to the video library, and at the same time, based on the feature information of each dimension of the video, the video index of the corresponding dimension of the video library is created / updated, so as to facilitate the subsequent deduplication detection of other videos based on the created / updated video index.

[0065] Here, examples of video indexes of different dimensions of the video library are as follows Figure 2 , Figure 3 and Figure 4 As shown, Figure 2is a video index in the category dimension, wherein the category dimension includes: a primary category (sports / TV series), a secondary category under each primary category (such as football / basketball under the sports category, or urban drama / martial arts drama under the TV series category); when determining whether a video is a video to be deduplicated, the feature information in the category dimension of the video can be compared with the attribute value in the video index in the category dimension (such as Figure 2 sports / TV series in the video, and further football / basketball or urban drama / martial arts drama) to obtain a corresponding candidate video set; Figure 3 is the video index of the tag dimension, where the attribute value of the video index of the tag dimension is the video tag (such as Figure 3 Champions League / NBA in the Figure 4 is a video index in the keyword dimension, wherein the attribute value of the video index in the keyword dimension is the video keyword (such as Figure 4 De Bruyne, free kick, header in). Among them, the video corresponding to each attribute value ( Figures 2 to 4 In the example, the video connected to the attribute value by the arrow line is the video corresponding to the attribute value. In this way, when performing deduplication detection on the video, the corresponding candidate video set can be obtained based on the matching of the feature information of each dimension of the video with the attribute value in the video index of the corresponding dimension.

[0066] Here, it should be noted that Figure 2 , Figure 3 and Figure 4 It can be seen that the video index of the embodiment of the present application is an inverted index, so it is easy to quickly retrieve videos corresponding to attribute values ​​matching the feature information of the corresponding dimension to obtain candidate videos.

[0067] Further, as another optional implementation, before step 104, the method further includes:

[0068] According to the feature information of the keyword dimension, in the target video, key frames related to the feature information of the keyword dimension are extracted; here, in combination with the description of the feature information of the keyword dimension, it can be known that the feature information of the keyword dimension is the video keyword extracted from the audio data of the target video, and therefore, the key frames in this step should be the key frames in the video clip where the video keyword appears; compared with obtaining all the key frames of the target video in the related art, the number of key frames obtained in this step is relatively small;

[0069] Based on the key frames of the target video and the key frames of each video of the intersection video, the similarity between the target video and each video of the intersection video is determined, wherein the key frames of each video of the intersection video correspond to the feature information of the keyword dimension of each video of the intersection video. Here, as an example, the similarity between the target video and each video in the intersection video can be obtained by calculating based on the feature vectors corresponding to the key frames of the videos.

[0070] In the above optional implementation method, the number of key frames corresponding to the feature information of the acquired keyword dimension is less than the number of all key frames of the video. Based on this, the amount of calculation for subsequently determining the similarity between the target video and each video of the intersection video based on the acquired key frames and the key frames of each video of the intersection video is reduced, which can improve the speed of similarity judgment to a certain extent, thereby improving the efficiency of video deduplication.

[0071] As a specific implementation method, according to the feature information of the keyword dimension, in the target video, key frames related to the feature information of the keyword dimension are extracted, including:

[0072] In the target video, a video segment corresponding to the feature information of the keyword dimension is identified, wherein the voice data corresponding to the identified video segment should include the feature information of the keyword dimension, wherein the length of the video segment can be preset according to a rule; specifically, for example, this step can use image recognition technology, with the help of the feature information of the keyword dimension, to identify the video segment corresponding to the feature information of the keyword dimension in the target video;

[0073] According to the position of the feature information of the keyword dimension in the text data, the key frames of each identified video segment are extracted, and the text data is converted from the audio data of the target video. For example, the audio data can be converted into text data through speech recognition technology, so as to determine the key frame according to the position of the feature information of the keyword dimension in the text data. For example, based on the position of the feature information of the keyword dimension in the text data, the distance / time length between the feature information of the keyword dimension and the first text in the text data can be determined, so as to extract the key frame in the video segment according to the distance / time length.

[0074] As another specific implementation, determining the similarity between the target video and each video of the intersection video according to the key frame of the target video and the key frame of each video of the intersection video includes:

[0075] Performing image processing on the key frame of the target video to obtain a feature vector of the key frame of the target video;

[0076] Generate a feature vector of the target video according to the feature vectors of each key frame of the target video; wherein the feature vector of the target video is arranged in the order in which the feature information of the keyword dimension appears (or the order in which each key frame appears in the target video), specifically as follows: Figure 5 As shown, the feature information of the keyword dimension obtained is arranged in the order of appearance as: keyword A1, keyword A2, keyword A3, ..., keyword An, and the key frames corresponding to each keyword are arranged in the order of appearance in the target video as: key frame A1, key frame A2, key frame A3, ..., key frame An; on this basis, the feature vectors corresponding to each key frame can be expressed as: A1, A2, A3, ..., An, so the feature vector of the target video in this step is expressed as {A1, A2, A3, ..., An}, that is: the feature vector of the target video is generated by arranging the feature vectors of each key frame in the order of appearance of the feature information of the keyword dimension; correspondingly, the ordering of the feature vectors of the key frames of each video in the intersection video is the same as the ordering of the feature vectors of the key frames of the target video, that is: the keywords corresponding to the feature vectors of the Nth key frames of the two videos are the same; that is to say, the feature vectors of the intersection video and the feature vectors of the target video are both sorted in the order of the feature information of the same keyword dimension;

[0077] The vector products of the feature vector of the target video and the feature vectors of each video of the intersection video are calculated in sequence, wherein the feature vectors of each video of the intersection video are generated by the feature vectors of the key frames of each video of the intersection video; here, the feature vectors of each video of the intersection video are generated in the same manner as the feature vector generation manner of the aforementioned implementation manner, and specifically may be generated in the process of performing deduplication detection on each video in the intersection video;

[0078] According to each of the vector products, the similarity between the target video and the video corresponding to the vector product in the intersection video is determined. For example, the vector product is proportional to the similarity. The larger the vector product, the higher the similarity between the two videos.

[0079] In the above specific implementation method, in the feature vectors of the two videos (the target video and one of the intersection videos) whose vector products need to be calculated, the keywords corresponding to the Nth feature vectors are the same. In this way, the process of vector product calculation does not need to consider the order in which the key frames appear in the video, thereby avoiding the problem that the key frames of the video tampered with by AI technology may be disrupted compared to the original video, resulting in a large difference in the feature vectors of the two videos, leading to deduplication failure.

[0080] Next, combine Figure 6, the process of creating / updating video index based on qualified videos in deduplication detection is described:

[0081] Step 601, extracting keyword tags and classifying them according to the short video title; this step may specifically be: first, segmenting the short video according to the hot search word library, the tag word library and the short video title library, obtaining the word weight through the TF / IDF algorithm, extracting the corresponding keywords as tags (keyword tags / video tags / feature information of tag dimensions), and secondly, classifying the keyword tags; that is, this step is similar to the above: segmenting the video title of the target video and calculating the word weight to obtain the feature information of the tag dimension; and classifying the feature information of the tag dimension to obtain the feature information of the category dimension;

[0082] Step 602, converting the short video speech into text, and extracting keywords from the speech text; this step may specifically be: using semantic-to-text technology to extract the short video speech into text. Extracting keywords from the speech (feature information of the keyword dimension); that is, this step is similar to the aforementioned step of converting the audio data of the target video into text data, and extracting the feature information of the keyword dimension from the text data;

[0083] Step 603, performing semantic analysis on the video clips to extract key clips and their keywords; this step specifically includes: using image recognition and other technologies, with the help of the keywords extracted from the voice in step 602, identifying the key clips of the short video; similar to the above, in the target video, identifying the video clip corresponding to the feature information of the keyword dimension, the text data is converted from the audio data of the target video;

[0084] Step 604, extracting corresponding video key frames according to the keywords; this step may specifically be: extracting the key frames of the video according to the positions of the keywords in the semantic text and the segments of the video corresponding to the keywords; similar to the above, extracting the key frames of each identified video segment according to the positions of the feature information of the keyword dimension in the text data, the text data is converted from the audio data of the target video;

[0085] Step 605, constructing an inverted index according to the classification and label; this step is specifically: constructing an inverted index of the classification dimension according to the result of the short video title classification recognition in step 601; constructing an inverted index of the label dimension according to the result of the label extracted from the short video title in step 601;

[0086] Step 606, constructing an inverted index according to the keywords of the video; this step specifically includes: constructing an inverted index of the keyword dimension according to the keywords of the video extracted in step 602;

[0087] The above steps 605 and 606 correspond to the above-mentioned creation and / or updating of the video index of the corresponding dimension of the video library based on the feature information of each dimension of the video that has passed the deduplication detection, wherein the videos in the video library are the videos that have passed the deduplication detection;

[0088] The order of implementing the above steps 605 and 606 is not limited to executing step 605 first and then executing step 606. Step 605 and step 606 may also be executed simultaneously, or step 606 may be executed first and then step 605. After the feature information of the corresponding dimension is obtained, the construction / update of the video index of the corresponding dimension may be executed.

[0089] Step 606, generating a video feature vector according to the video keywords and key frames; this step may specifically be: performing image processing on the key frames to generate a video feature vector {A1, A2, A3, ..., An}, wherein A1, A2, A3, ..., An are image feature vectors corresponding to the order of keywords; this step is similar to the aforementioned: performing image processing on the key frames of the target video to obtain the feature vectors of the key frames of the target video; and generating the feature vector of the target video according to the feature vectors of each key frame of the target video.

[0090] In the embodiment of the present application, by constructing a video index, it is convenient to quickly retrieve candidate videos that meet the feature information of multiple dimensions of the video when performing deduplication detection on the video, thereby improving the retrieval speed and further improving the efficiency of video deduplication.

[0091] Next, combine Figure 7 , the specific implementation process of the video deduplication method of the embodiment of the present application is described:

[0092] Step 701, searching from the multi-level classification of the video to obtain a candidate set 1 that meets the classification of the video;

[0093] Step 702, searching from multiple tags of the video to obtain a candidate set 2 that matches the video tag;

[0094] Step 703, searching from the keyword set of the video to obtain a candidate set 3 that matches the keyword of the video;

[0095] Here, it should be noted that in this embodiment, the execution order of the above steps 701, 702 and 703 is not limited, and they can be executed sequentially or simultaneously. When they are executed sequentially, the execution order of the above steps is not limited;

[0096] In addition, the above steps 701, 702 and 703 correspond to the aforementioned step of "comparing the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension of the video library in sequence, to obtain a set of candidate videos in the video library corresponding to each dimension of the target video". Specifically:

[0097] The above step 701 is specifically as follows: based on the feature information of the category dimension of the video, searching in the video index of the category dimension in the video library / matching the feature information of the category dimension of the video with the video index of the category dimension of the video library, to obtain a candidate set 1 corresponding to the feature information of the category dimension of the video;

[0098] The above step 702 is specifically as follows: based on the feature information of the label dimension of the video, searching in the video index of the label dimension in the video library / matching the feature information of the label dimension of the video with the video index of the label dimension of the video library, to obtain a candidate set 2 corresponding to the feature information of the label dimension of the video;

[0099] The above step 703 is specifically as follows: based on the feature information of the keyword dimension of the video, searching in the video index of the keyword dimension in the video library / matching the feature information of the keyword dimension of the video with the video index of the keyword dimension of the video library, to obtain a candidate set 3 corresponding to the feature information of the keyword dimension of the video;

[0100] Step 704, finding the intersection of the above result sets to obtain a smaller short video result set; this step corresponds to the aforementioned step of "obtaining the intersection video of the plurality of candidate video sets";

[0101] Step 705, searching in a smaller set, and calculating the video vector similarity using a vector product; this step corresponds to the aforementioned steps of "calculating the vector products of the feature vector of the target video and the feature vectors of each video of the intersection video in sequence" and "determining the similarity between the target video and the video corresponding to the vector product in the intersection video according to each of the vector products";

[0102] Step 706, determining whether the two videos are similar based on the threshold set by the test.

[0103] Taking the aforementioned UEFA Champions League match as an example, the above steps 701 to 706 are specifically illustrated as follows:

[0104] First, based on the first-level classification of sports and the second-level classification of football, search in the video index of the category dimension to obtain candidate set 1, that is, short video set 1 of the sports football category, recorded as C1;

[0105] Secondly, based on the search of “UEFA Champions League”, “Manchester City”, and “Copenhagen” in the video index of the label dimension, a short video set 2 of the UEFA Champions League match between Manchester City and Copenhagen is obtained, which is recorded as C2;

[0106] Again, based on “Champions League”, “Manchester City”, “Copenhagen”, “Group Stage”, “2023 / 2024 Season”, “3:0”, “Free Kick”, “De Bruyne”, “xx Stadium”, “Refill Shot”, search in the video index of the keyword dimension, and obtain a short video set 3 that meets the 23 / 24 season Manchester City and Copenhagen 3:0, recorded as C3;

[0107] After that, the intersection of the short video sets obtained above is taken to obtain the short video set C4, that is: C4 = C1∩C2∩C3; here, after finding the intersection of sets C1, C2, and C3, we can get the short video set of Manchester City 3:0 in the 2023 / 2024 Champions League group match between Manchester City and Copenhagen, and the set must also have keywords such as "free kick", "De Bruyne", "xx stadium", "supplementary shot", and "3:0". Generally speaking, the number of this set is small, and the relative performance is very high when performing vector retrieval;

[0108] After that, search in the smaller set C4, compare the video feature vector and the feature vector of the video to be deduplicated, and use the vector product to calculate the vector similarity; specifically, the vector inner product formula E=B1·B2 is used to calculate; where B1 is the feature vector of the video to be deduplicated, and B2 is the video feature vector of each video in the candidate set C4. The vector product E can determine the similarity of the two videos. The larger the E value, the higher the similarity, and the smaller the E value, the lower the similarity;

[0109] Finally, a threshold is set based on experience to determine whether two videos are similar. Specifically, based on experimental experience, a vector product similarity threshold E1 is set. When the vector product E is greater than E1, the videos are judged to be similar and it can be considered that the video is uploaded repeatedly. Otherwise, it is considered to be an original video.

[0110] In the video deduplication method of the embodiment of the present application, a short video multi-level classification index, a multi-level label index and a video keyword index are pre-constructed, and a feature vector of each short video is generated. Afterwards, in the process of deduplication of the video, first, the short video title is subjected to keyword extraction, and then the short video is classified and labeled. The voice in the short video is extracted and converted into text. The keywords in the voice text and the key frames in the corresponding video clips are extracted, and the features of these key frames are extracted to generate feature vectors. Secondly, during retrieval, firstly, candidate set 1 is obtained from the multidimensional classification retrieval, and then candidate set 2 is obtained from the multidimensional label index retrieval, and then candidate set 3 is retrieved from the multiple keyword indexes contained in the video, and then the intersection of the above candidate sets 1, 2, and 3 is taken to obtain a smaller candidate set 4, and finally the video feature vector degree is calculated from the smaller video candidate set 4 using the vector product algorithm, so as to determine whether the input video is a similar video already in the library. In this way, on the one hand, by combining the classification, features, and keyword indexes of different feature dimensions to determine a smaller relevant range, the vector search range is reduced, greatly improving the retrieval speed; on the other hand, similarity judgment is performed based on the feature vectors of the key frames corresponding to the keywords, which solves the problem that all the key frames in the current short video are collected, and the generated short video feature vectors are large, resulting in slow and inefficient video vector retrieval; on the other hand, when making similarity judgments based on feature vectors, there is no need to consider the order of the key frames corresponding to the feature vectors, avoiding the problem that the key frames of the video tampered with by AI technology may be disrupted compared to the original video, resulting in large differences in the feature vectors of the video and causing deduplication failure.

[0111] The embodiment of the present application also provides a video deduplication device, such as Figure 8 As shown, including:

[0112] A first acquisition module 801 is used to acquire feature information of multiple dimensions of a target video, where the multiple dimensions include at least two of categories, tags, and keywords;

[0113] The second acquisition module 802 is used to sequentially compare the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library, and obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library;

[0114] The third acquisition module 803 is used to obtain an intersection video of a plurality of the candidate video sets;

[0115] The first determination module 804 is configured to determine that the target video is a video to be deduplicated when the similarity between the target video and any video in the intersection videos is greater than a similarity threshold.

[0116] Optionally, the first acquisition module 801 is used to perform at least the following two steps:

[0117] Segmenting the video title of the target video and calculating word weights to obtain feature information of the label dimension;

[0118] Classifying the feature information of the label dimension to obtain feature information of the category dimension;

[0119] The audio data of the target video is converted into text data, and feature information of a keyword dimension is extracted from the text data.

[0120] Furthermore, the device also includes:

[0121] A processing module is used to create and / or update the video index of the corresponding dimension of the video library for the video that has passed the deduplication detection based on the feature information of each dimension of the video that has passed the deduplication detection, wherein the videos in the video library are the videos that have passed the deduplication detection.

[0122] Furthermore, the device also includes:

[0123] An extraction module, configured to extract key frames related to the feature information of the keyword dimension in the target video according to the feature information of the keyword dimension;

[0124] The second determination module is used to determine the similarity between the target video and each video of the intersection video based on the key frames of the target video and the key frames of each video of the intersection video, wherein the key frames of each video of the intersection video correspond to the feature information of the keyword dimension of each video of the intersection video.

[0125] Optionally, the extraction module includes:

[0126] An identification submodule, used to identify, in the target video, a video segment corresponding to the feature information of the keyword dimension;

[0127] The extraction submodule is used to extract the key frames of each of the identified video clips according to the position of the feature information of the keyword dimension in the text data, and the text data is converted from the audio data of the target video.

[0128] Optionally, the second determining module includes:

[0129] An acquisition submodule, used for performing image processing on the key frame of the target video to obtain a feature vector of the key frame of the target video;

[0130] A generating submodule, used for generating a feature vector of the target video according to the feature vectors of each key frame of the target video;

[0131] A calculation submodule, used for sequentially calculating the vector products of the feature vector of the target video and the feature vectors of each video of the intersection video, wherein the feature vectors of each video of the intersection video are generated by the feature vectors of the key frames of each video of the intersection video;

[0132] A determination submodule is used to determine the similarity between the target video and the video corresponding to the vector product in the intersection video according to each of the vector products.

[0133] It should be noted here that the above-mentioned video deduplication device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned video deduplication method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0134] An embodiment of the present application also provides a video deduplication device, including a transceiver 910, a processor 900, a memory 920, and a program stored in the memory 920 and executable on the processor 900; wherein, when the processor 900 executes the program, the video deduplication method as described above is implemented.

[0135] The transceiver 910 is used to receive and send data under the control of the processor 900.

[0136] Among them, Fig. 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 900 and various circuits of memory represented by memory 920 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 910 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different devices, the user interface 930 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0137] The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 can store data used by the processor 900 when performing operations.

[0138] The embodiment of the present application also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the video deduplication method embodiment described above is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for executing the methods described in each embodiment of the present application.

[0140] Therefore, an embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the video deduplication method as described above and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0141] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0142] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video deduplication method, characterized in that: include: Acquire feature information of multiple dimensions of a target video, where the multiple dimensions include at least two of categories, tags, and keywords; Comparing the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library in turn, to obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library; Obtaining intersection videos of multiple candidate video sets; When the similarity between the target video and any video in the intersection videos is greater than a similarity threshold, the target video is determined to be a video to be deduplicated.

2. The video deduplication method according to claim 1, characterized in that: Acquiring feature information of multiple dimensions of a target video includes at least the following two steps: Segmenting the video title of the target video and calculating word weights to obtain feature information of the label dimension; Classifying the feature information of the label dimension to obtain feature information of the category dimension; The audio data of the target video is converted into text data, and feature information of a keyword dimension is extracted from the text data.

3. The video deduplication method according to claim 1, characterized in that: The method further comprises: For the videos that pass the deduplication detection, based on the feature information of each dimension of the videos that pass the deduplication detection, a video index of the corresponding dimension of the video library is created and / or updated.

4. The video deduplication method according to claim 1, characterized in that: When the similarity between the target video and any video in the intersection videos is greater than a similarity threshold, before determining that the target video is a video to be deduplicated, the method further includes: Extracting key frames related to the feature information of the keyword dimension in the target video according to the feature information of the keyword dimension; The similarity between the target video and each video of the intersection video is determined according to the key frames of the target video and the key frames of each video of the intersection video, wherein the key frames of each video of the intersection video correspond to the feature information of the keyword dimension of each video of the intersection video.

5. The video deduplication method according to claim 4, characterized in that: According to the feature information of the keyword dimension, in the target video, key frames related to the feature information of the keyword dimension are extracted, including: In the target video, identifying a video segment corresponding to the feature information of the keyword dimension; According to the position of the feature information of the keyword dimension in the text data, the key frames of each of the identified video clips are extracted, and the text data is converted from the audio data of the target video.

6. The video deduplication method according to claim 4, characterized in that: Determining the similarity between the target video and each video of the intersection video according to the key frame of the target video and the key frame of each video of the intersection video includes: Performing image processing on the key frame of the target video to obtain a feature vector of the key frame of the target video; Generate a feature vector of the target video according to the feature vectors of each key frame of the target video; sequentially calculating the vector products of the feature vector of the target video and the feature vectors of each video of the intersection video, wherein the feature vectors of each video of the intersection video are generated by the feature vectors of the key frames of each video of the intersection video; According to each of the vector products, the similarity between the target video and the video corresponding to the vector product in the intersection video is determined.

7. A video deduplication device, characterized in that: include: A first acquisition module, configured to acquire feature information of multiple dimensions of a target video, wherein the multiple dimensions include at least two of categories, tags, and keywords; A second acquisition module is used to sequentially compare the feature information of each dimension in the multiple dimensions with the video index of the corresponding dimension in the video library, and obtain a set of candidate videos in the video library corresponding to each dimension of the target video; wherein the video index is used to represent the index relationship between the feature information of the corresponding dimension in the video library and the candidate videos in the video library; A third acquisition module is used to obtain an intersection video of a plurality of the candidate video sets; The first determination module is used to determine that the target video is a video to be deduplicated when the similarity between the target video and any video in the intersection videos is greater than a similarity threshold.

8. A video deduplication device, characterized in that: include: A transceiver, a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the video deduplication method according to any one of claims 1 to 6 when executed by the processor.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the video deduplication method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the video deduplication method according to any one of claims 1 to 6.