Duplicate video verification method and device based on frame features

Abstract

The present disclosure relates to a method, device, equipment, computer program product and computer-readable storage medium for duplicate video verification based on frame features, which can be applied to various scenarios such as cloud technology, artificial intelligence, blockchain, smart transportation, and assisted driving. The method provided by the embodiments of the present disclosure verifies the video in a matching manner, and determines at least one optimal matching frame of each video frame in the original video in each video frame in the video to be verified based on the feature similarity of the frame features between the videos. This method of determining the relatively optimal matching frame within a limited range does not rely on the absolute feature similarity, and thus does not rely on the limited threshold, which weakens the importance of the absolute feature similarity of the image in duplicate video verification and improves the effectiveness and accuracy of the verification method. In addition, the method greatly improves the accuracy of duplicate video verification by identifying the timing characteristics of the optimal video frame.

Description

Technical Field

[0001] The present disclosure relates to the field of duplicate video verification, and more specifically, to a duplicate video verification method, apparatus, device, computer program product and storage medium. Background Art

[0002] In the Internet era, multimedia video, as an important data carrier, plays a huge role in information dissemination. Duplicate video detection can not only reduce the spread of redundant information, but also play a role in protecting copyright and preventing the spread of bad videos. Therefore, duplicate video detection is widely used in business scenarios such as low-quality transfer identification and copyright detection on various video platforms. However, information expansion and the typical adversarial relationship between video detectors and video transfer parties have brought challenges to duplicate video detection.

[0003] From the attached picture Figure 1A , 1B , 1C, on the one hand, the video carrier edits the video in various ways to prevent detection, and the video detector needs to constantly revise the detection model and supplement the detection algorithm. For example, in the verification scheme that compares the image similarity of the video frame with a fixed threshold to determine whether the video is repeated, it is easy to have the problem of low verification effectiveness due to the deliberate reduction of image similarity by the video carrier's editing. On the other hand, different videos themselves may also have the problem of high similarity in picture composition. Therefore, most of the duplicate video verification schemes in the prior art require very complex feature extraction and feature matching logic. Summary of the invention

[0004] In order to solve the above problems, the present disclosure performs verification based on frame features with relatively optimal feature similarity during repeated video verification.

[0005] Embodiments of the present disclosure provide a method, apparatus, device, computer program product, and computer-readable storage medium for repeated video verification based on frame features.

[0006] An embodiment of the present disclosure provides a method for duplicate video verification based on frame features, the method comprising: obtaining a first video and a second video to be verified, wherein the first video may be an original video and the second video may be a video to be verified, and vice versa; extracting frames of the first video and the second video at a predetermined time interval to obtain multiple video frames of the first video and multiple video frames of the second video; extracting frame features of each video frame in the multiple video frames of the first video and the multiple video frames of the second video; determining feature similarities between each video frame of the first video and each video frame of the second video based on the frame features of each video frame of the first video and the frame features of each video frame of the second video; determining, for each video frame in the first video, at least one optimal matching frame of the video frame in the multiple video frames of the second video based on the feature similarities between each video frame of the first video and each video frame of the second video; and performing duplicate video verification on the first video and the second video based on the positions of the optimal matching frames corresponding to each video frame in the first video in the second video.

[0007] According to an embodiment of the present disclosure, the duplicate video check of the first video and the second video based on the position of the best matching frame corresponding to each video frame in the first video in the second video includes: sorting the best matching frames corresponding to multiple video frames in the first video according to the timing of the multiple video frames in the first video; determining whether there is a linear timing relationship between the sorted best matching frames and the second video based on the position of each best matching frame in the second video frame; if there is a linear timing relationship, determining the first video and the second video as duplicate videos, and if there is no linear timing relationship, determining the first video and the second video as non-duplicate.

[0008] According to an embodiment of the present disclosure, the feature similarity between each frame of the first video and each frame of the second video is determined by calculating the cosine similarity between each frame of the first video and each frame of the second video.

[0009] According to an embodiment of the present disclosure, for each video frame in the first video, determining at least one optimal matching frame of the video frame in multiple video frames of the second video includes: determining at least one video frame having the highest cosine similarity with the video frame in the first video frame among the multiple video frames of the second video as the at least one optimal matching frame. The "highest cosine similarity" includes the cosine similarity of the maximum value, the cosine similarities of the top three numerical values, or the cosine similarities of a number set by the user.

[0010] According to an embodiment of the present disclosure, based on frame features of each video frame of the first video and frame features of each video frame of the second video, feature similarities between each video frame of the first video and each video frame of the second video are respectively determined, including: constructing a video feature matrix of the first video using frame features of each video frame of the first video; constructing a video feature matrix of the second video using frame features of each video frame of the second video; and based on the video feature matrix of the first video and the video feature matrix of the second video, determining the feature similarities between each video frame of the first video and each video frame of the second video by matrix multiplication to obtain a similarity matrix, and determining a similarity matrix between the first video and the second video based on the similarity matrix.

[0011] According to an embodiment of the present disclosure, in the case where each video frame in the first video corresponds to a column of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding column of the similarity matrix; or in the case where each video frame in the first video corresponds to a row of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding row of the similarity matrix, wherein determining whether the sorted best matching frames have a linear temporal relationship based on the position of each best matching frame in the second video frame includes: determining whether the highest similarities of each column or row in the similarity matrix have a linear temporal relationship based on a regression algorithm.

[0012] According to an embodiment of the present disclosure, obtaining the first video and the second video to be verified includes: obtaining the first video and the second video to be verified from a primary duplicate video detection result, wherein the primary duplicate video detection result is a preliminary detection result of duplicate video detection on a massive video database.

[0013] According to an embodiment of the present disclosure, extracting frame features of each video frame of the multiple video frames of the first video and the multiple video frames of the second video includes any one of the following: extracting frame features of the video frames using a first neural network, wherein, in the massive video database, repeated video detection is performed using the frame features of each video frame extracted by the first neural network; extracting frame features of the video frames using a second neural network, the scale of the second neural network is smaller than the scale of the first neural network; or extracting frame features of the video frames from a frame feature pool related to the massive video database.

[0014] An embodiment of the present disclosure provides a duplicate video verification device, the device comprising: an acquisition module, used to acquire a first video and a second video to be verified; a frame extraction module, used to extract frames of the first video and the second video at a predetermined time interval to obtain multiple video frames of the first video and multiple video frames of the second video; an extraction module, used to extract frame features of the video frames for each of the multiple video frames of the first video and the multiple video frames of the second video; a feature similarity determination module, used to determine the feature similarities between each video frame of the first video and each video frame of the second video based on the frame features of each video frame of the first video and the frame features of each video frame of the second video; an optimal matching frame determination module, used to determine, for each video frame in the first video, at least one optimal matching frame of the video frame in the multiple video frames of the second video based on the feature similarities between each video frame of the first video and each video frame of the second video; and a verification module, used to perform duplicate video verification on the first video and the second video based on the position of the optimal matching frame corresponding to each video frame in the first video in the second video.

[0015] An embodiment of the present disclosure provides a repeated video verification device, which includes: one or more processors; and one or more memories, in which a computer executable program is stored. When the computer executable program is executed by the processor, a method according to an embodiment of the present disclosure is executed.

[0016] An embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon. When the instructions are executed by a processor, the methods according to the embodiments of the present disclosure are implemented.

[0017] The embodiments of the present disclosure provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes a method according to an embodiment of the present disclosure.

[0018] The method provided by the embodiment of the present disclosure verifies the video in a matching manner, and determines at least one optimal matching frame of each video frame in the original video (or the video to be verified) in each video frame in the video to be verified (or the original video) based on the feature similarity of the frame features between the videos. This method of determining the relatively optimal matching frame within a limited range does not rely on the absolute feature similarity, and thus does not rely on the limited threshold, which weakens the importance of the absolute feature similarity of the image in the duplicate video verification, and improves the effectiveness and accuracy of the verification method. In addition, since it does not rely on the absolute feature similarity, the features to be extracted do not have to represent a large number of videos in the massive video library, but only need to have good representation capabilities in a small number of videos in the original video and the video to be verified. Therefore, a smaller-scale neural network can be used to realize feature extraction, thereby greatly reducing the feature calculation cost and optimization cost. In addition, the method provided by the embodiment of the present disclosure greatly improves the accuracy of duplicate video verification by identifying the temporal features of the optimal video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0020] Figure 1A is an exemplary schematic diagram showing an edited repeat video; Figure 1B is an exemplary schematic diagram showing different videos with high similarity in picture composition; Figure 1C is another exemplary schematic diagram showing different videos with high similarity in picture composition;

[0021] Figure 2 is a schematic flow chart showing a method for checking duplicate videos based on frame features according to an embodiment of the present disclosure;

[0022] Figure 3 is an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix according to an embodiment of the present disclosure;

[0023] Figure 4 It shows that according to Figure 3 An exemplary schematic diagram of the optimal matching frame of the heat map matrix;

[0024] Figure 5 It shows that according to Figure 4 An exemplary schematic diagram of the optimal matching frame and the fitted ideal linear relationship;

[0025] Fig. 6Ais an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix of non-repetitive videos according to an embodiment of the present disclosure; Figure 6B is an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix of repeated videos according to an embodiment of the present disclosure

[0026] Figure 7 is an exemplary schematic diagram showing another heat map matrix generated based on a similarity matrix of repeated videos according to an embodiment of the present disclosure;

[0027] Figure 8 is an exemplary schematic diagram showing an apparatus for checking duplicate videos according to an embodiment of the present disclosure;

[0028] Fig. 9 is an exemplary schematic diagram showing a repeated video verification device according to an embodiment of the present disclosure;

[0029] Fig.10 is an architecture showing a computing device according to an embodiment of the present disclosure; and

[0030] Fig.11 is an exemplary schematic diagram illustrating a storage medium according to the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0032] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance or ranking.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0034] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] Machine learning is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of AI and is the basis for making computing devices intelligent. The so-called machine learning is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computing devices simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Deep learning is a technology that uses deep neural network systems to perform machine learning. Machine learning / deep learning can usually include artificial neural networks, reinforcement learning, supervised learning, unsupervised learning and other technologies.

[0037] Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of the information (anti-counterfeiting) and generate the next block.

[0038] The solution provided by the embodiments of the present disclosure involves technologies such as image processing and repeated video verification of artificial intelligence, which can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, blockchain, smart transportation, assisted driving, etc. In addition, the solution provided by the embodiments of the present disclosure can be executed by a server or by a user terminal, which includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., or can also be executed through a distributed system based on blockchain.

[0039] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0040] Features: In machine learning, a feature is a measurable property or characteristic of an object being observed. Features are usually numerical, but structural features (such as strings) can be used in pattern recognition such as grammar. The concept of "feature" is related to the explanatory variables used in statistical techniques such as linear regression.

[0041] Embedding: Also known as embedding, a method of converting discrete variables into continuous vectors. In recent years, neural networks and deep learning have been used more and more widely, from computer vision to natural language processing to time series prediction. Among them, all inputs must be converted into numbers before they can be processed and calculated by the model (for example, to obtain feature similarity), so a digital expression is needed to replace the original sample input form such as text or pictures. Therefore, in the application process of deep learning, the method of embedding has greatly expanded the application of neural networks in various aspects. It can not only reduce the spatial dimension of discrete variables, but also meaningfully represent the variables.

[0042] Frame features: Perform image embedding of a specific dimension on the image frames of the video or obtain the frame features of the video through other methods.

[0043] Feature similarity: A measure of the similarity between two feature vectors (matrices).

[0044] In summary, the solutions provided by the embodiments of the present disclosure involve technologies such as neural networks, image recognition, and feature similarity measurement. The embodiments of the present disclosure will be further described below in conjunction with the accompanying drawings.

[0045] Figure 1Ais an exemplary schematic diagram showing an edited duplicate video, wherein the left image (a) and the right image (b) involve the same video, i.e., a duplicate video, and different edited borders. It can be seen that the right image (b) involves the same video image as the left image (a), but the right image (b) changes the position and size of the video content, changes the name of the video, and adds other borders that account for a large proportion of the image, resulting in that the left and right images do not have many duplicate parts as a whole. Currently, most common duplicate video detection technologies are based on matching the overall image features of the video frame, obtaining feature similarity, and setting a fixed threshold. If the obtained feature similarity is less than the threshold, the image is determined to be duplicated, and if the feature similarity is greater than the threshold, the image is determined to be non-duplicate. In this case, if the video is edited to a certain extent by means of, for example, changing the resolution, adding / reducing a logo, changing subtitles, partially blurring, adjusting borders, adding shadows, etc., the matching degree of the overall image features of the frame can be reduced and the feature similarity can be increased, that is, the image of the actually repeated video is determined to be non-duplicate. Therefore, in the actual confrontation between video protectors and video movers, the feasibility and effectiveness of this method of determining whether two images are duplicated by setting a fixed threshold will be greatly reduced due to editing behavior.

[0046] Figure 1B is an exemplary schematic diagram showing different videos with high similarity in screen composition, where the left image (a) and the right image (b) are different videos of the same game, i.e., non-repeated videos. As can be seen from the figure, most of the background in the game screen is the same game map, while different image features such as game characters account for a small proportion, resulting in a high degree of similarity in overall image features, which is easy to be judged as repeated when a fixed threshold is used to determine whether a video is repeated.

[0047] Figure 1C is another exemplary schematic diagram showing different videos with high similarity in picture composition, in which the left image (a) and the right image (b) respectively relate to different performance videos of two groups of different crosstalk actors, that is, non-repeated videos. As can be seen from the figure, most of the image consists of the layout of the crosstalk performance and the actors' costumes, and the layout and costumes in the crosstalk field have a high degree of overlap, resulting in a high similarity of overall image features. In addition to game videos and crosstalk videos, other videos with small image changes or a small proportion of image changes are also easily misjudged.

[0048] Depend on Figure 1A , Figure 1B , Figure 1CIt can be seen that the solution of using a fixed threshold to check duplicate videos is very easy to fail, not only in the case of various artificial editing by the video transfer party to counter the video detection party, but also in the case of high similarity of the video images themselves. In order to ensure the effectiveness of checking duplicate videos, it is usually necessary to: continuously correct and supplement the verification model to reduce the interference of various edits on the determination of feature similarity; continuously verify the online effect, so as to correct the threshold used to determine duplication. Therefore, the threshold-based method for checking duplicate videos usually requires high expert optimization costs and operation and maintenance costs.

[0049] Figure 2 It is a schematic flow chart showing a method for checking duplicate videos based on frame features according to an embodiment of the present disclosure.

[0050] In step S1, a first video and a second video to be verified are obtained, wherein the first video may be the original video and the second video may be the video to be verified, and vice versa. The first video and the second video may be primary videos that directly implement the verification method according to the present disclosure, or may be secondary videos that have undergone primary repeated video detection, that is, the first video and the second video to be verified are obtained from the primary repeated video detection results that have performed repeated video detection and narrowed the scope to be verified, wherein the primary repeated video detection results are preliminary detection results after repeated video detection is performed on a massive video database.

[0051] Optionally, in the massive video database, the frame features of each video frame extracted by the first neural network can be used, and primary repeated video detection can be performed based on the extracted frame features. For example, repeated video detection can be performed based on the proportion of the number of identical frames in the video. In the primary repeated video detection, in order to take into account both detection efficiency and detection accuracy, the frame features of the video frames extracted by the first neural network have better global characterization capabilities.

[0052] In step S2, the first video and the second video are frame extracted at predetermined time intervals, for example, 1 frame is extracted per second or 1 frame is extracted every 2 seconds, so as to obtain multiple video frames of the first video and multiple video frames of the second video.

[0053] In step S3, for each of the multiple video frames of the first video and the multiple video frames of the second video, frame features of the video frame are extracted.

[0054] Optionally, the frame features of the video frame may be extracted from a frame feature pool associated with the massive video database. Alternatively, the frame features of the video frame may be extracted using a first neural network used in primary repeated video detection in the massive video database.

[0055] Optionally, a second neural network is used to extract frame features of the video frame, and the scale of the second neural network is smaller than that of the first neural network. For the second neural network, not only can a public pre-trained model, such as MobileNet, MobileNetV2, MobileNetV3, etc., be used, but it can also be designed or trained by itself. In the repeated video verification according to the present disclosure, the video is verified in a matching manner, that is, one-to-one, and one-to-many when necessary, and since the number of video frames is limited, the video frame features to be extracted only need to have good representation capabilities in the local video to be verified, and do not need to be able to measure all videos in the video library, that is, the global representation capabilities of the video frame features to be extracted are not important. Therefore, there is the feasibility of using a smaller neural network. A smaller neural network usually has a smaller amount of computation, so that in the feature extraction process, not only the millisecond level of feature extraction efficiency can be guaranteed, but also the extraction quality, that is, relatively effective frame granularity information, can be guaranteed.

[0056] According to the embodiments of the present disclosure, with the help of a neural network, more robust features can be extracted. Compared with traditional feature extraction methods, more robust verification results can be obtained, further improving the effectiveness of the verification method.

[0057] In step S4, based on the frame features of each video frame of the first video and the frame features of each video frame of the second video, the feature similarities between each video frame of the first video and each video frame of the second video are determined respectively.

[0058] Similarity metrics and distance metrics are usually used to measure the feature similarity between video frames. Similarity metrics, that is, calculate the similarity between objects. The larger the value of the similarity metric, the greater the similarity between the objects and the smaller the difference. Distance metrics are used to measure the distance between objects in space. In contrast to similarity metrics, the greater the distance, the greater the difference between the objects. In the present disclosure, both similarity metrics and distance metrics are used as a measure of the similarity between two feature vectors (matrices), and are not regarded as limitations on the method steps, that is, obtaining the feature similarity between each frame also includes obtaining the feature distance between each frame.

[0059] Common similarity measurement methods include Euclidean distance, cosine similarity, Jaccard distance, etc. Euclidean distance measures the absolute straight-line distance between points in space, which is directly related to the position coordinates of each point, so it can reflect the absolute difference in individual numerical features. Cosine similarity measures the angular relationship between space vectors, which better reflects the difference in direction, so it is insensitive to absolute values, and corrects the problem of inconsistent measurement standards that may exist between objects. In the image and video fields involved in the present disclosure, the feature dimensions of the objects studied are often high. Cosine similarity still maintains the property of "1 when the same, 0 when orthogonal, and -1 when opposite" in high-dimensional situations, while the value of Euclidean distance is affected by the dimension, and the range is (0, +∞].

[0060] Optionally, the feature similarity between each frame of the first video and each frame of the second video is determined by calculating the cosine similarity between each frame of the first video and each frame of the second video.

[0061] In step S5, based on the feature similarity between each video frame of the first video and each video frame of the second video, for each video frame in the first video, at least one best matching frame of the video frame among multiple video frames in the second video is determined.

[0062] Optionally, in an embodiment where cosine similarity is used to characterize feature similarity, at least one video frame in the second video having the highest cosine similarity with a video frame in the first video frame is determined as the at least one optimal matching frame. The "highest cosine similarity" includes the cosine similarity of the maximum value, and the first three (or a number set by the user) cosine similarities arranged from large to small similarity values.

[0063] Optionally, in an embodiment where the Euclidean distance or the Jaccard distance is used to characterize the feature similarity, at least one video frame in the second video having the shortest distance to the video frame in the first video frame is determined as the at least one optimal matching frame, wherein the "shortest distance" includes the minimum distance value and the first three (or a number set by the user) distance values ​​arranged from small to large.

[0064] In step S6, based on the position of the best matching frame corresponding to each video frame in the first video in the second video, a duplicate video check is performed on the first video and the second video.

[0065] Optionally, step S6 includes: sorting the best matching frames corresponding to the multiple video frames in the first video according to the timing of the multiple video frames in the first video; determining whether there is a linear timing relationship between the sorted best matching frames based on the position of each best matching frame in the second video frame; if there is a linear timing relationship, determining the first video and the second video as repeated videos; if there is no linear timing relationship, determining the first video and the second video as non-repeating.

[0066] As described above, after step S4, the feature similarity between each video frame of the first video and each video frame of the second video can be obtained. Since both the first video and the second video contain multiple video frames, the feature similarity between their frames constitutes a feature similarity matrix. Below, an example implementation of constituting the feature similarity distance in step S4 will be specifically described.

[0067] Among them, taking cosine similarity as an example, the formation of feature similarity matrix includes but is not limited to the following two methods:

[0068] In the first method, the video feature matrix of the first video is constructed using the frame features of each video frame of the first video. Constructing a video feature matrix of the second video using the frame features of each video frame of the second video Then, based on the video feature matrix of the first video and the video feature matrix of the second video, the feature cosine similarity between each video frame of the first video and each video frame of the second video is determined by matrix multiplication to obtain a cosine similarity matrix

[0069]

[0070] Wherein, x and y represent the first video and the second video respectively, k and e represent the number of frame features of the first video and the second video respectively, and n represents the dimension of the video frame feature. Optionally, the frame features are normalized, thereby not only reducing the amount of calculation, but also directly calculating the feature cosine similarity between each video frame of the first video and each video frame of the second video through matrix multiplication.

[0071] In the second method, the video frames in the first video and the second video are sorted according to a time sequence, and the matrix rows of the similarity matrix are sequentially constructed corresponding to the video frames in the first video (in other words, the horizontal axis of the matrix is ​​the frame number in the first video), and the matrix columns of the similarity matrix are sequentially constructed corresponding to the video frames in the second video (in other words, the vertical axis of the matrix is ​​the frame number in the second video). Alternatively, the matrix rows of the similarity matrix are sequentially constructed corresponding to the video frames in the second video (in other words, the horizontal axis of the matrix is ​​the frame number in the second video), and the matrix columns of the similarity matrix are sequentially constructed corresponding to the video frames in the first video (in other words, the vertical axis of the matrix is ​​the frame number in the first video), and then the corresponding feature similarities between the video frames are filled in the corresponding positions in the matrix, thereby obtaining a matrix of feature similarities between each video frame of the first video and each video frame of the second video.

[0072] Through the above two methods or other methods that can obtain the feature similarity between each video frame of the first video and each video frame of the second video, a feature similarity matrix between the video frames is obtained.

[0073] Exemplarily, the feature cosine similarity matrix between each video frame of the first video and each video frame of the second video is obtained as follows:

[0074]

[0075] Wherein, the length of the first video and the second video are both 5 seconds, and the frame is extracted at 0 seconds, and then the two videos are extracted at 1 frame per second. After the video frames in the first video and the second video are sorted according to the time sequence relationship, the matrix rows of the similarity matrix are constructed in sequence corresponding to the video frames in the first video (that is, the horizontal coordinate of the matrix is ​​the frame number in the first video), and the 0th to 5th frames of the first video correspond to the 0th to 5th columns of the similarity matrix (arranged from left to right), and the matrix columns of the similarity matrix are constructed in sequence corresponding to the video frames in the second video (that is, the vertical coordinate of the matrix is ​​the frame number in the second video), and the 0th to 5th frames of the second video correspond to the 0th to 5th rows of the similarity matrix (arranged from top to bottom). For example, the feature similarity between the 0th frame of the first video and the 0th frame of the second video is 0.95, and the feature similarity between the 2nd frame of the first video and the 3rd frame of the second video is 0.23.

[0076] After obtaining the feature similarity matrix, the position of the highest feature similarity corresponding to each frame of the first video in each frame of the second video can be identified to determine the best matching frame. For example, the highest feature similarity corresponding to the first frame of the first video is 0.89, and its position in the second video is the first frame of the second video. Therefore, the best matching frame of the first frame of the first video in the second video is the first frame of the second video. Similarly, the result of obtaining at least one best matching frame from the above feature similarity matrix is ​​as follows:

[0077]

[0078] Then, the best matching frames corresponding to the multiple video frames in the first video can be sorted according to the time sequence of the multiple video frames in the first video, and based on the position of each best matching frame in the second video frame, it is determined whether the sorted best matching frames have a linear time sequence relationship. For example, the position of each of the best matching frames in the matrix can be abstracted as a point, and then a regression algorithm or the like is used to determine whether the points have a linear time sequence relationship.

[0079] A numerical matrix similar to the above matrix intuitively shows the feature similarity between each frame of the first video and the second video. Optionally, a heat map matrix can be used to replace the numerical matrix, and the heat map color blocks can be used to replace the numerical values ​​in a limited manner to improve the readability of the feature similarity matrix and the operability of determining whether there is a linear temporal relationship between the best matching frames. Figure 3-5 The process of determining whether the best matching frames have a linear temporal relationship based on the heat map matrix is ​​schematically shown.

[0080] Figure 3 is an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix according to an embodiment of the present disclosure.

[0081] exist Figure 3 In the figure, the length of the first video and the second video are both 9 seconds, and the frame is extracted at 0 seconds, and then the two videos are extracted at 1 frame per second. It is assumed here that each video frame in the first video corresponds to a column of the similarity matrix, and each video frame in the second video corresponds to a row of the similarity matrix. It can be seen that the heat map matrix is ​​composed of thermal blocks of different shades, where different thermal blocks represent different similarity values, where lighter blocks represent higher thermal values, and higher thermal values ​​represent higher feature similarities. The highest thermal block represents the highest feature similarity, that is, the optimal matching frame.

[0082] Figure 4 It shows that according to Figure 3An exemplary schematic diagram of the optimal matching frame of the heat map matrix of . Each video frame in the first video corresponds to a column of the similarity matrix, and each video frame in the second video corresponds to a row of the similarity matrix. Figure 3 In the heat map matrix generated in the , thermal blocks of different shades are obtained, and the darkest color block, that is, the highest thermal block, that is, the best matching frame is further identified and marked. Then, the best matching frames corresponding to the multiple video frames are sorted according to the time sequence of the multiple video frames in the first video, and the position of the best matching frame in the second video frame is found. As shown in the figure, the position of the best matching frame corresponding to the 0th frame of the first video in the second video frame is the 0th frame of the second video, the position of the best matching frame corresponding to the 3rd frame of the first video in the second video frame is the 3rd frame of the second video, the position of the best matching frame corresponding to the 6th frame of the first video in the second video frame is the 6th frame of the second video, the position of the best matching frame corresponding to the 7th frame of the first video in the second video frame is the 6th and 7th frames of the second video, and the position of the best matching frame corresponding to the 8th frame of the first video in the second video frame is the 6th and 8th frames of the second video, and so on.

[0083] Figure 5 It shows that according to Figure 4 An exemplary schematic diagram of the optimal matching frame and the fitted ideal linear relationship.

[0084] As can be seen from the description of the optional implementation of step S6 above, the best matching frames corresponding to the multiple video frames in the first video can be sorted according to the timing of the multiple video frames in the first video, and then based on the position of each best matching frame in the second video frame, it is determined whether the sorted best matching frames have a linear timing relationship.

[0085] In the present disclosure, the definition and application of the best matching frame in the feature similarity matrix include but are not limited to the following two methods:

[0086] In the first mode, when each video frame in the first video corresponds to a column of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding column of the similarity matrix; or when each video frame in the first video corresponds to a row of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding row of the similarity matrix. Then, based on the position of each best matching frame in the second video frame, determining whether the sorted best matching frames have a linear temporal relationship includes: determining whether the highest similarities of each column or row in the similarity matrix have a linear temporal relationship based on a regression algorithm.

[0087] In the second method, a heat map matrix is ​​generated based on the similarity matrix, wherein different similarities correspond to different colors in the heat map, wherein, when each video frame in the first video corresponds to a column of the similarity matrix, its best matching frame corresponds to the highest thermal color block in the corresponding column of the heat map matrix, or when each video frame in the first video corresponds to a row of the similarity matrix, its best matching frame corresponds to the highest thermal color block in the corresponding row of the heat map matrix. Then, based on the position of each best matching frame in the second video frame, it is determined whether the sorted best matching frames have a linear temporal relationship, including: determining whether the highest thermal color blocks in each column or row in the heat map matrix have a linear temporal relationship based on a regression algorithm or heat map recognition.

[0088] Common regression algorithms include least squares regression, etc. Least squares regression finds the best regression function by minimizing the sum of squares of errors. Its essence is unbiased estimation, so the existence of outliers has a great impact on effectiveness. Optionally, based on a robust regression algorithm, it is determined whether the optimal matching frames after sorting have a linear temporal relationship. Compared with least squares regression, which is very sensitive to outliers, robust regression is to apply a robust estimation method to a regression model to fit the structure in which most of the data exists. At the same time, it can also identify potential outliers, strong influence points or structures that deviate from model assumptions, so it is more accurate. Optionally, based on the RANSAC algorithm, it is determined whether the optimal matching frames after sorting have a linear temporal relationship. RANSAC, also known as the random sampling consensus algorithm, estimates the parameters of the model in an iterative manner in a data set containing outliers, and can achieve more robust model parameter estimation, for example, it can estimate higher precision parameters from a data set containing a large number of outliers, reducing the impact of outliers on model results.

[0089] exist Figure 5 : The best matching frame after sorting is obtained by using the heat map matrix based on feature similarity in the second method, and then the ideal linear relationship of the best matching frame is fitted by the RANSAC algorithm, where the blue point represents the best matching frame and the orange point represents the ideal linear relationship. By calculating the overlap (overlap) of the blue point and the orange point, it can be determined whether the best matching frame after sorting is within the range of its ideal linear relationship. Figure 5 The blue points in the image highly overlap with the orange points, so it can be determined that the sorted best matching frames have a temporal relationship, so that the first video and the second video are determined to be repeated.

[0090] Fig. 6AIt is an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix of non-repeating videos according to an embodiment of the present disclosure. It can be seen that the color depth of the heat color blocks in the heat map matrix is ​​randomly and irregularly distributed, and the best matching frames after sorting are scattered and do not have a linear relationship. Therefore, it can be determined that there is no linear temporal relationship between the best matching frames after sorting, so that the first video and the second video are determined to be non-repeating.

[0091] Figure 6B is an exemplary schematic diagram showing a heat map matrix generated based on a similarity matrix of repeated videos according to an embodiment of the present disclosure. It can be seen that although the color depth of the heat map blocks in the heat map matrix is ​​randomly distributed, the lightest color blocks, that is, the highest heat color blocks, are almost distributed on the diagonal line, and there is an obvious linear relationship. Therefore, it can be determined that there is a linear temporal relationship between the sorted best matching frames, so that the first video and the second video are determined to be repeated. Figure 6B In the schematic heat map matrix shown, the linear distribution of the highest thermal color blocks is more significant. In other embodiments, regression algorithms, heat map recognition or other means can be used to determine whether there is a linear time series relationship between the highest thermal color blocks in each column or row in the heat map matrix.

[0092] Figure 7 It is an exemplary schematic diagram showing another heat map matrix generated based on a similarity matrix of repeated videos according to an embodiment of the present disclosure. The left side (a) shows a heat map matrix generated based on a similarity matrix, and the right side (b) uses dots to mark the highest thermal color blocks identified in the heat map matrix on the left. It is assumed here that each video frame in the first video corresponds to a column of the similarity matrix, and each video frame in the second video corresponds to a row of the similarity matrix. It can be seen that the length of the first video is 53 seconds, and the length of the second video is 48 seconds, and the lengths of the two videos are not the same. In addition, not every video frame within the entire length of the first video has a linear relationship with the highest thermal color block corresponding to the second video, that is, not all the best matching frames within the entire first video or second video range have a linear temporal relationship, but the best matching frames corresponding to the 10th to 50th video frames of the first video have a more obvious linear relationship after sorting. However, it can still be obtained from the figure that part of the first video and part of the second video have a repeated relationship. That is to say, regardless of whether the lengths of the videos to be verified are the same or whether the linear temporal relationship of the sorted optimal matching frames is global or local, the method disclosed in the present invention can be applied to determine the repetitive relationship between the video to be verified and the original video, or to mark the video to be verified as requiring further verification.

[0093] Figure 8 8 is a schematic diagram showing an apparatus 800 for checking duplicate videos according to an embodiment of the present disclosure.

[0094] According to an embodiment of the present disclosure, an apparatus 800 for verifying duplicate videos may include an acquisition module 801 , a frame extraction module 802 , an extraction module 803 , a feature similarity determination module 804 , an optimal matching frame determination module 805 , and a verification module 806 .

[0095] The acquisition module 801 may be configured to acquire a first video and a second video to be verified.

[0096] Optionally, the first video and the second video to be verified are obtained from a primary duplicate video detection result, wherein the primary duplicate video detection result is a preliminary detection result of duplicate video detection on a massive video database.

[0097] The frame extraction module 802 may be configured to extract frames from the first video and the second video at predetermined time intervals to obtain a plurality of video frames of the first video and a plurality of video frames of the second video.

[0098] The extraction module 803 may be configured to extract frame features of each of the multiple video frames of the first video and the multiple video frames of the second video.

[0099] Optionally, a first neural network is used to extract frame features of the video frames, wherein in the massive video database, repeated video detection is performed using the frame features of each video frame extracted by the first neural network.

[0100] Optionally, a second neural network is used to extract frame features of the video frame, and the scale of the second neural network is smaller than the scale of the first neural network.

[0101] Optionally, frame features of the video frame are extracted from a frame feature pool associated with the massive video database.

[0102] The feature similarity determination module 804 may be configured to determine feature similarities between each video frame of the first video and each video frame of the second video based on frame features of each video frame of the first video and frame features of each video frame of the second video.

[0103] Optionally, the feature similarity between each frame of the first video and each frame of the second video is determined by calculating the cosine similarity between each frame of the first video and each frame of the second video.

[0104] The best matching frame determination module 805 can be configured to determine, for each video frame in the first video, at least one best matching frame of the video frame among multiple video frames in the second video based on feature similarities between the respective video frames of the first video and the respective video frames of the second video.

[0105] Optionally, among the multiple video frames of the second video, at least one video frame having the highest cosine similarity with the video frame in the first video frame is determined as the at least one optimal matching frame.

[0106] The verification module 806 may be configured to perform a duplicate video verification on the first video and the second video based on the position of the best matching frame corresponding to each video frame in the first video in the second video.

[0107] Optionally, according to the timing of the multiple video frames in the first video, the best matching frames corresponding to the multiple video frames in the first video are sorted; based on the position of each best matching frame in the second video frame, it is determined whether the sorted best matching frames have a linear timing relationship; if there is a linear timing relationship, the first video and the second video are determined to be repeated videos; if there is no linear timing relationship, the first video and the second video are determined to be non-repeating.

[0108] According to yet another aspect of the present disclosure, a duplicate video verification device is also provided. Fig. 9 A schematic diagram of a repeated video checking device 2000 according to an embodiment of the present disclosure is shown.

[0109] like Fig. 9 As shown, the duplicate video checking device 2000 may include one or more processors 2010 and one or more memories 2020. The memory 2020 stores a computer readable code, and when the computer readable code is run by the one or more processors 2010, the method described above may be executed.

[0110] The processor in the embodiments of the present disclosure may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc., and may be an X86 architecture or an ARM architecture.

[0111] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0112] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Fig.10 The architecture of the computing device 3000 shown in FIG. Fig.10 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the method provided in the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Fig. 9 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig.10 One or more components of a computing device are shown.

[0113] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided. Fig.11 A schematic diagram 4000 of a storage medium according to the present disclosure is shown.

[0114] like Fig.11As shown, the computer storage medium 4020 stores computer readable instructions 4010. When the computer readable instructions 4010 are executed by the processor, the method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0115] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method according to the embodiments of the present disclosure.

[0116] Embodiments of the present disclosure provide a duplicate video verification method, apparatus, device, computer program product, and computer-readable storage medium.

[0117] Different from the method of comparing absolute similarity with a threshold in the prior art, the method disclosed in the present invention extracts the best matching frame of each column (row) from the similarity matrix regardless of whether the video to be verified is repeated, thereby improving the reliability of the verification method. In addition, the method disclosed in the present invention combines the characteristics of repeated videos, that is, repeated videos should have continuous repeated frames in the time series, and further improves the accuracy of the verification method by determining whether the best matching frames have a linear time series relationship.

[0118] The method disclosed in the present invention can not only be used alone, but also can be applied to secondary duplicate video verification after large-scale primary duplicate video detection. The method is also conducive to further refinement of the duplicate video verification process.

[0119] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0121] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A repeated video verification method based on frame features, comprising: Obtain a first video and a second video to be verified; Extracting frames from the first video and the second video at predetermined time intervals to obtain a plurality of video frames of the first video and a plurality of video frames of the second video; For each of the plurality of video frames of the first video and the plurality of video frames of the second video, extracting a frame feature of the video frame; Determine, based on frame features of each video frame of the first video and frame features of each video frame of the second video, feature similarities between each video frame of the first video and each video frame of the second video; Based on feature similarities between each video frame of the first video and each video frame of the second video, for each video frame in the first video, determining at least one best matching frame of the video frame among multiple video frames of the second video; as well as Based on whether there is a linear temporal relationship between positions of the best matching frames corresponding to the video frames sorted in time sequence in the first video and in the second video, a duplicate video check is performed on the first video and the second video.

2. The method according to claim 1, wherein: The performing duplicate video verification on the first video and the second video based on whether there is a linear time sequence relationship between positions of the best matching frames corresponding to the video frames sorted in time sequence in the first video and in the second video includes: Sorting the best matching frames corresponding to the multiple video frames in the first video according to the time sequence of the multiple video frames in the first video; Based on the position of each best matching frame in the second video, determining whether the sorted best matching frames have a linear temporal relationship; If there is a linear temporal relationship, the first video and the second video are determined to be repeated videos; if there is no linear temporal relationship, the first video and the second video are determined to be non-repeating.

3. The method according to claim 1, wherein: The feature similarity between each frame of the first video and each frame of the second video is determined by calculating the cosine similarity between each frame of the first video and each frame of the second video.

4. The method according to claim 3, wherein: For each video frame in the first video, determining at least one best matching frame of the video frame among multiple video frames in the second video, comprising: Among the multiple video frames of the second video, at least one video frame having the highest cosine similarity with the video frame in the first video is determined as the at least one optimal matching frame.

5. The method according to claim 2, wherein: Determining feature similarities between each video frame of the first video and each video frame of the second video based on frame features of each video frame of the first video and frame features of each video frame of the second video, respectively, includes: constructing a video feature matrix of the first video using frame features of each video frame of the first video; constructing a video feature matrix of the second video using frame features of each video frame of the second video; and Based on the video feature matrix of the first video and the video feature matrix of the second video, feature similarities between each video frame of the first video and each video frame of the second video are determined by matrix multiplication to obtain a similarity matrix, and a similarity matrix between the first video and the second video is determined based on the similarity matrix.

6. The method according to claim 5, wherein: In the case where each video frame in the first video corresponds to a column of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding column of the similarity matrix; or In the case where each video frame in the first video corresponds to a row of the similarity matrix, its best matching frame corresponds to the highest similarity in the corresponding row of the similarity matrix, Wherein, determining whether the sorted best matching frames have a linear temporal relationship based on the position of each best matching frame in the second video includes: Based on the regression algorithm, it is determined whether there is a linear temporal relationship between the highest similarities of each column or each row in the similarity matrix.

7. The method according to claim 5, further comprising: Based on the similarity matrix, a heat map matrix is ​​generated, where different similarities correspond to different colors in the heat map. Wherein, when each video frame in the first video corresponds to a column of the similarity matrix, its best matching frame corresponds to the highest thermal color block in the corresponding column of the heat map matrix, or when each video frame in the first video corresponds to a row of the similarity matrix, its best matching frame corresponds to the highest thermal color block in the corresponding row of the heat map matrix, Wherein, determining whether the sorted best matching frames have a linear temporal relationship based on the position of each best matching frame in the second video includes: Based on the regression algorithm or the heat map recognition, it is determined whether there is a linear time series relationship between the highest heat color blocks in each column or row in the heat map matrix.

8. The method according to claim 1, wherein: The obtaining of the first video and the second video to be verified comprises: From the primary duplicate video detection result, obtain the first video and the second video to be checked, The primary duplicate video detection result is a preliminary detection result of duplicate video detection on a massive video database.

9. The method according to claim 8, wherein: For each of the multiple video frames of the first video and the multiple video frames of the second video, extracting a frame feature of the video frame includes any of the following: Extracting frame features of the video frames using a first neural network, wherein in the massive video database, repeated video detection is performed using the frame features of each video frame extracted by the first neural network; extracting frame features of the video frame using a second neural network, the scale of the second neural network being smaller than the scale of the first neural network; or Frame features of the video frame are extracted from a frame feature pool associated with the massive video database.

10. A repeated video verification device, comprising: An acquisition module, used to acquire a first video and a second video to be verified; A frame extraction module, used for extracting frames of the first video and the second video at a predetermined time interval to obtain a plurality of video frames of the first video and a plurality of video frames of the second video; An extraction module, configured to extract frame features of each of the plurality of video frames of the first video and the plurality of video frames of the second video; a feature similarity determination module, configured to determine feature similarities between each video frame of the first video and each video frame of the second video based on frame features of each video frame of the first video and frame features of each video frame of the second video; An optimal matching frame determination module, configured to determine, for each video frame in the first video, at least one optimal matching frame of the video frame among multiple video frames in the second video based on feature similarities between each video frame in the first video and each video frame in the second video; as well as A verification module is used to perform duplicate video verification on the first video and the second video based on whether there is a linear time sequence relationship between the positions of the best matching frames corresponding to the video frames sorted in time sequence in the first video and in the second video.

11. A repeated video verification device, comprising: one or more processors; as well as One or more memories, wherein a computer executable program is stored, and when the computer executable program is executed by the processor, the method of any one of claims 1-9 is performed.

12. A computer program product, comprising computer software code, wherein the computer software code is used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the instructions are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images