Video duplication checking method, device, storage medium and computer program product

Abstract

The present application discloses a video duplication checking method, device, storage medium and computer program product, the method comprising: obtaining N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from a target video to be checked for duplication, and the L reference video frames are determined from H reference videos for duplication checking and comparison; based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames are determined from the L reference video frames as similar video frames; a similar video frame is similar to at least one to-be-processed video frame; based on the M similar video frames, K reference videos are determined from the H reference videos as similar videos; any similar video contains at least one similar video frame; determining the audio feature similarity between the target video and each similar video, and judging whether the target video and each similar video are repeated based on the audio feature similarity; the efficiency of video duplication checking can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video duplication detection method, device, storage medium and computer program product. Background Art

[0002] Video duplication checking usually requires finding videos that are duplicates of the video to be checked for duplication from the videos used for duplication checking and comparison. In existing duplication checking methods, it is usually necessary to extract multiple types of features from the video, and then compare the multiple types of features extracted to achieve video duplication checking. The features referenced for video duplication checking are complicated, which leads to low duplication checking efficiency. Summary of the invention

[0003] The embodiments of the present application provide a video duplication checking method, apparatus, device, storage medium and computer program product, which can improve the efficiency of video duplication checking.

[0004] On the one hand, an embodiment of the present application provides a video duplication checking method, comprising:

[0005] Obtain N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from the target video to be checked for plagiarism, and the L reference video frames are determined from H reference videos for checking for plagiarism, where N, L, and H are all positive integers;

[0006] Based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames are determined from the L reference video frames as similar video frames; a similar video frame is similar to at least one to-be-processed video frame, M∈[1,L];

[0007] According to the M similar video frames, K reference videos are determined from the H reference videos as similar videos; any similar video contains at least one similar video frame, and K is a positive integer;

[0008] The audio feature similarity between the target video and each similar video is determined, and whether the target video and each similar video are repeated is determined according to the audio feature similarity between the target video and each similar video.

[0009] On the one hand, an embodiment of the present application provides a video duplication checking device, comprising:

[0010] An acquisition unit is used to acquire N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from the target video to be checked for plagiarism, and the L reference video frames are determined from H reference videos for checking for plagiarism, and N, L and H are all positive integers;

[0011] A processing unit, configured to determine M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame; a similar video frame is similar to at least one to-be-processed video frame, M∈[1,L];

[0012] The processing unit is further used to determine K reference videos as similar videos from the H reference videos according to the M similar video frames; any similar video contains at least one similar video frame, and K is a positive integer;

[0013] The processing unit is further used to determine the audio feature similarity between the target video and each similar video, and determine whether the target video and each similar video are repeated according to the audio feature similarity between the target video and each similar video.

[0014] On the one hand, an embodiment of the present application provides a video duplication checking device, characterized in that the video duplication checking device includes an input interface and an output interface, and further includes:

[0015] a processor adapted to implement one or more instructions; and,

[0016] Computer storage medium, wherein the computer storage medium stores one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the above-mentioned video duplication checking method.

[0017] On the one hand, an embodiment of the present application provides a computer storage medium, characterized in that computer program instructions are stored in the computer storage medium, and when the computer program instructions are executed by a processor, they are used to execute the above-mentioned video duplication checking method.

[0018] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer storage medium; a processor of a video duplication checking device reads the computer program from the computer storage medium, and the processor executes the computer program, so that the video duplication checking device performs the above-mentioned video duplication checking method.

[0019] In an embodiment of the present application, N to-be-processed video frames determined from a target video to be checked for duplicates can be obtained, and L reference video frames determined from H reference videos for checking for duplicates can be obtained; then, based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames can be determined from the L reference video frames as similar video frames, and one similar video frame is similar to at least one to-be-processed video frame; and then, based on the M similar video frames, K reference videos can be determined from the H reference videos as similar videos, and any similar video contains at least one similar video frame; then, based on the audio feature similarity between the target video and each similar video, it can be determined whether the target video is repeated with respect to each similar video. That is to say, based on the similarity of image features between each video frame to be processed and each reference video frame, similar videos to the target video can be recalled from the reference videos; and further, based on the similarity of audio features between the target video and each similar video, it is determined whether the target video and each similar video are duplicates, so as to realize duplicate checking of the target video; that is, in the process of realizing duplicate checking of the target video, two types of features, image features of video frames and audio features of videos, are referred to, and the types of features referred to for video duplicate checking are concise, which can improve the efficiency of video duplicate checking. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a structural schematic diagram of a video duplication checking system provided in an embodiment of the present application;

[0022] Figure 2 It is a structural schematic diagram of another video duplication checking system provided in an embodiment of the present application;

[0023] Figure 3 It is a structural schematic diagram of another video duplication checking system provided in an embodiment of the present application;

[0024] Figure 4 It is a flowchart of a video duplication checking method provided in an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of a process of checking for plagiarism in a target video provided by an embodiment of the present application;

[0026] Figure 6 It is a flowchart of another video duplication checking method provided in an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of extracting image features of any video frame through an optimized image feature extraction model provided by an embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of checking for duplicate content in a target video based on video clips provided in an embodiment of the present application;

[0029] Fig. 9 It is a flowchart of an optimization method of an image feature extraction model provided in an embodiment of the present application;

[0030] Fig.10 It is a comparison diagram between a sample video frame, a video frame with similar annotations, and a video frame with difference annotations provided in an embodiment of the present application;

[0031] Fig.11 It is a schematic diagram of determining a model loss value through an image feature extraction model provided by an embodiment of the present application;

[0032] Fig.12 It is a structural schematic diagram of a video duplication checking device provided in an embodiment of the present application;

[0033] Fig.13 It is a structural schematic diagram of a video duplication checking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] In order to improve the efficiency of video duplication checking, the present application proposes a video duplication checking scheme, which can obtain N to-be-processed video frames determined from a target video to be checked for duplication, and obtain L reference video frames determined from H reference videos for duplication checking and comparison; then, based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames can be determined from the L reference video frames as similar video frames, and a similar video frame is similar to at least one to-be-processed video frame; then, based on the M similar video frames, K reference videos can be determined from the H reference videos as similar videos, and any similar video contains at least one similar video frame; then, whether the target video is repeated with each similar video can be determined based on the audio feature similarity between the target video and each similar video; wherein N, L, H and K are all positive integers, M∈[1, L].

[0036] In one embodiment, N to-be-processed video frames and L reference video frames are extracted from the target video and H reference videos respectively based on the same frame extraction rule. The frame extraction rule can be set according to specific needs; for example, the frame extraction rule can indicate: when extracting frames for any video, extract each frame of the video; for another example, the frame extraction rule can indicate: when extracting frames for any video, extract from the first frame of the video and extract every certain number of frames; for another example, the frame extraction rule can indicate: when extracting frames for any video, extract from the first frame of the video and extract every certain time interval. For example, if video A includes 30 frames, when extracting frames from video A, if the frame extraction rule indicates that each frame of the video is to be extracted, 30 frames can be extracted from video A; if the frame extraction rule indicates that the extraction starts from the first frame of the video and is performed every 4 frames, the 1st frame, 6th frame, 11th frame, 16th frame, 21st frame and 26th frame of video A can be extracted from video A. Furthermore, since different videos may contain different numbers of video frames, when different videos are extracted based on the same frame extraction rule, the number of extracted video frames may be different; therefore, when L reference video frames are extracted from H reference videos, the number of reference video frames extracted from different reference videos may be different.

[0037] In a specific implementation, the video duplication checking scheme proposed in the present application can be executed by a video duplication checking device, which can be a terminal device or a server; the terminal device here may include but is not limited to: computers, smart phones, tablet computers, laptops, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, smart wearable devices, etc.; the server here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0038] In one embodiment, the database corresponding to the video duplication checking device may store in advance H reference videos for duplication checking and L reference video frames determined from the H reference videos. Then, after obtaining the N to-be-processed video frames of the target video, the video duplication checking device may check the target video for duplication based on the H reference videos and L reference video frames stored in the database corresponding to the video duplication checking device.

[0039] In one embodiment, the image feature similarity refers to: the similarity determined based on the image features of two video frames, the image features of any video frame are obtained by performing image feature extraction processing on any video frame; the image feature similarity between the to-be-processed video frame and the reference video frame is the similarity determined based on the image features of the to-be-processed video frame and the reference video frame, the image features of the to-be-processed video frame and the reference video frame are obtained by performing image feature extraction processing on the optimized image feature extraction model; the video duplication checking device can obtain the image features of each to-be-processed video frame and the image features of each reference video frame before determining M reference video frames as similar video frames from L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame. In a feasible implementation, the image features of each reference video frame can be pre-acquired by the video duplication checking device and stored in a database corresponding to the video duplication checking device. The video duplication checking device can obtain the image features of each reference video frame from the database corresponding to the video duplication checking device before determining M reference video frames as similar video frames from L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame. Optionally, the H reference videos used for duplicate checking and the L reference video frames determined from the H reference videos can be stored in other databases, and the image features of the reference video frames are stored in the database corresponding to the video duplicate checking device; and an index can be constructed based on the image features of the reference video frames, the video frame identifiers of the reference video frames, and the video identifiers of the reference videos containing the reference video frames to achieve the determination of the reference video frames and the reference videos containing the reference video frames, and then perform video duplicate checking based on the image features of the reference video frames, the reference video frames, and the reference videos. Furthermore, in order to reduce the memory usage of indexing a large number of image features, the inverted index and product quantization technology in the Faiss framework that provides efficient similarity search and clustering for dense vectors can be used to compress image features, that is, to compress feature vectors used to characterize image features.

[0040] In a feasible implementation, the extraction of image features of the video frames to be processed and the extraction of image features of the reference video frames can both be performed by the video duplication checking device; that is, the video duplication checking device can perform image feature extraction processing on each reference video frame to obtain the image features of each reference video frame, and store the image features of each reference video frame in a database corresponding to the video duplication checking device; after obtaining N video frames to be processed, the video duplication checking device can perform image feature extraction processing on each video frame to be processed to obtain the image features of each video frame to be processed, and obtain the image features of each reference video frame from the database corresponding to the video duplication checking device, and then determine the image feature similarity between each video frame to be processed and each reference video frame based on the image features of each video frame to be processed and the image features of each reference video frame, and then perform duplication checking on the target video.

[0041] In another feasible implementation manner, the extraction of image features of the to-be-processed video frame and the extraction of image features of the reference video frame can both be performed by a feature extraction device, and the video duplication checking device can obtain the image features of the to-be-processed video frame and the image features of the reference video frame extracted by the feature extraction device; based on this, the video duplication checking scheme proposed in the present application can be performed by the video duplication checking device in collaboration with the feature extraction device; see Figure 1 , is a structural diagram of a video duplication checking system provided in an embodiment of the present application, Figure 1 The video duplication checking system shown may include a video duplication checking device 101 and a feature extraction device 102; the feature extraction device 102 may be a terminal device or a server; the terminal device here may include but is not limited to: a computer, a smart phone, a tablet computer, a laptop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an intelligent wearable device, etc.; the server here may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The video duplication checking device 101 and the feature extraction device 102 can be directly or indirectly connected to each other by wired or wireless communication, and this application is not limited here.

[0042] Furthermore, in order to improve the efficiency of extracting the image features of the processed video frame and the image features of the reference video frame, multiple feature extraction devices can be used to perform collaborative execution; further, in order to horizontally expand the feature extraction device that performs image feature extraction, the multiple feature extraction devices can be deployed in a producer-consumer mode. Figure 2 , is a structural diagram of another video duplication checking system provided in an embodiment of the present application, Figure 2 The video duplication checking system shown may include a video duplication checking device 201, a video distribution device 202, and a plurality of feature extraction devices 203; wherein the video distribution device 202 may be a central node device for realizing data distribution, and the video distribution device 202 and each feature extraction device 203 communicate based on a message queue, that is, the video distribution device 202 may act as a producer to upload data to the message queue; each feature extraction device 203 may act as a consumer to receive data sent by the message queue. After acquiring the target video, the video distribution device 202 may upload the target video to the message queue; after the message queue receives the target video, the target video may be sent to any feature extraction device 203; the feature extraction device 203 receiving the target video sent by the message queue may extract N to-be-processed video frames from the target video, and extract the image features of each to-be-processed video frame, and then may send the obtained to-be-processed video frames and the image features of each to-be-processed video frame to the video duplication checking device 201 through the video distribution device 202. After obtaining any reference video, the video distribution device 202 can upload the reference video to the message queue; after the message queue receives the reference video, it can send the reference video to any feature extraction device 203; the feature extraction device 203 that receives the reference video sent by the message queue can extract the reference video frame corresponding to the reference video from the reference video, and extract the image features of each reference video frame, and then the obtained reference video frames and the image features of each reference video frame can be sent to the video duplication checking device 201 through the video distribution device 202.

[0043] Furthermore, the video duplication checking scheme proposed in this application can be executed by a video duplication checking device to check the target video for duplication in the database corresponding to the video duplication checking device, that is, to check the target video for duplication in the reference video indicated by the database corresponding to the video duplication checking device; in order to further improve the data storage capacity and video duplication checking efficiency, the reference video, the reference video frame determined from the reference video, and the image features of the reference video frame can be stored in a dispersed manner based on the databases corresponding to multiple video duplication checking devices. Figure 3 , is a structural diagram of another video duplication checking system provided in an embodiment of the present application, Figure 3The video duplication checking system shown may include a content distribution device 301, multiple video duplication checking devices 302, a video distribution device 303 and multiple feature extraction devices 304; wherein, the content distribution device 301 may be a central node device for realizing data distribution, and the content distribution device 301 and each video duplication checking device 302 may be directly or indirectly communicated with each other via wired or wireless communication. This application does not make any restrictions here. The content distribution device 301 and multiple video duplication checking devices 302 constitute a distributed system.

[0044] After receiving the reference video, any feature extraction device 304 can extract the reference video frame corresponding to the reference video from the reference video, and extract the image features of each reference video frame, and then send the reference video, each reference video frame corresponding to the reference video, and the image features of each reference video frame to the content distribution device 301 through the video distribution device 303; the content distribution device 301 can send the reference video, each reference video frame corresponding to the reference video, and the image features of each reference video frame to the database corresponding to any video duplication checking device 302 for storage. After receiving the image features of each to-be-processed video frame of the target video sent by the feature extraction device 304 through the video distribution device 303, the content distribution device 301 can send the image features of each to-be-processed video frame to each video duplication checking device 302, so that each video duplication checking device 302 can check the target video for duplication in its corresponding database based on the image features of each to-be-processed video frame.

[0045] Optionally, when the reference video and the reference video frame corresponding to the reference video are not stored in the database corresponding to the video duplication check device 302, that is, the image features of the reference video frame are stored in the database corresponding to the video duplication check device, and when an index is constructed based on the image features of the reference video frame, the video frame identifier of the reference video frame and the video identifier of the reference video containing the reference video frame, the image features of any reference video frame corresponding to any reference video can be randomly distributed to the database corresponding to any video duplication check device 302 for storage; that is, the image features of the reference video frames stored in the databases corresponding to each video duplication check device 302 can be stored in disorder; further optionally, in order to balance the time spent by each video duplication check device 302 to perform video duplication check in its corresponding database, all image features of the reference video frames used for duplication check and comparison can be relatively evenly distributed to the databases corresponding to each video duplication check device 302 for storage, that is, it can be ensured that the amount of data stored in the databases corresponding to each video duplication check device 302 is as similar as possible.

[0046] It is particularly important to note that in the specific implementation of the present application, user-related data is involved. For example, when the target video is a video generated by the user, when the embodiment of the present application is applied to a specific product or technology, the user's permission or consent must be obtained, and the collection, use and processing of the relevant data must comply with local laws, regulations and standards.

[0047] Based on the above video duplication checking solution, the present application embodiment provides a video duplication checking method. Figure 4 , which is a flow chart of a video duplication checking method provided in an embodiment of the present application. Figure 4 The video duplication checking method shown can be executed by a video duplication checking device. Figure 4 The video duplication checking method shown may include the following steps:

[0048] S401, obtaining N to-be-processed video frames and L reference video frames.

[0049] Among them, N to-be-processed video frames are determined from the target video to be checked for duplicate content, and L reference video frames are determined from H reference videos for checking for duplicate content, and N, L, and H are all positive integers; the N to-be-processed video frames and the L reference video frames are extracted from the target video and the H reference videos respectively based on the same frame extraction rule, and the frame extraction rule can be set according to specific needs; because the number of video frames contained in different videos may be different, when different videos are extracted based on the same frame extraction rule, the number of extracted video frames may be different, so when L reference video frames are extracted from H reference videos, the number of reference video frames extracted from different reference videos may be different. Optionally, the H reference videos and the L reference video frames determined from the H reference videos can be pre-stored in a database corresponding to the video duplication checking device, or can be stored in other databases.

[0050] S402 : Based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames are determined from L reference video frames as similar video frames.

[0051] Among them, the image feature similarity between the video frame to be processed and the reference video frame is a similarity determined based on the image features of the video frame to be processed and the reference video frame, and the image features of the video frame to be processed and the reference video frame are obtained by performing image feature extraction processing on the optimized image feature extraction model; a similar video frame is similar to at least one video frame to be processed, M∈[1,L]; that is, based on the image feature similarity between a video frame to be processed and each reference video frame, at least one reference video frame similar to the video frame to be processed can be determined from the L reference video frames as a similar video frame, but it is also possible that a reference video frame similar to the video frame to be processed cannot be determined from the L reference video frames; based on different video frames to be processed , it is possible to determine the same reference video frames from L reference video frames as similar video frames corresponding to different video frames to be processed. For example, if based on video frame 1 to be processed, reference video frame 1, reference video frame 2 and reference video frame 3 similar to video frame 1 to be processed are determined from L reference video frames as similar video frames corresponding to video frame 1 to be processed; if based on video frame 2 to be processed, reference video frame 2, reference video frame 4 and reference video frame 5 similar to video frame 2 to be processed are determined from L reference video frames as similar video frames corresponding to video frame 2 to be processed, then it indicates that reference video frame 2 is similar to both video frame 1 to be processed and video frame 2 to be processed, so a similar video frame is similar to at least one video frame to be processed.

[0052] In one embodiment, if a reference video frame similar to any of the video frames to be processed cannot be determined from the L reference video frames based on the image feature similarity between each video frame to be processed and each reference video frame, it is determined that there is no video that is repeated with the target video among the H reference videos.

[0053] S403 : According to the M similar video frames, K reference videos are determined from the H reference videos as similar videos.

[0054] Wherein, any similar video contains at least one similar video frame, and K is a positive integer.

[0055] In one embodiment, any similar video may refer to: a reference video including at least one similar video frame; that is, a reference video including at least one similar video frame among the H reference videos may be determined as a similar video similar to the target video.

[0056] For example, if the M video frames to be processed determined from the target video are: video frame 1 to be processed, video frame 2 to be processed, video frame 3 to be processed, video frame 4 to be processed, video frame 5 to be processed, and video frame 6 to be processed; if the H reference videos are: reference video 1, reference video 2, reference video 3, and reference video 4; the L reference video frames determined from the H reference videos are: reference video frame 1, reference video frame 2, reference video frame 3, reference video frame 4, reference video frame 5, reference video frame 6, reference video frame 7, reference video frame 8, reference video frame 9, and reference video frame 10.

[0057] The corresponding relationship between the to-be-processed video frame, the similar video frame similar to the to-be-processed video frame, and the reference video to which the similar video frame belongs can be given by the following Table 1, where “√” indicates that there is a belonging relationship between the reference video frame and the reference video:

[0058] Table 1

[0059]

[0060] Table 1 indicates: a similar video frame similar to the video frame 1 to be processed is reference video frame 1, and reference video frame 1 belongs to both reference video 1 and reference video 2; a similar video frame similar to the video frame 2 to be processed is reference video frame 2, and reference video frame 2 belongs to both reference video 1 and reference video 2; a similar video frame similar to the video frame 3 to be processed is reference video frame 3, and reference video frame 3 belongs to both reference video 1, reference video 2 and reference video 3; a similar video frame similar to the video frame 4 to be processed is reference video frame 4, and reference video frame 4 belongs to both reference video 1 and reference video 3; a similar video frame similar to the video frame 5 to be processed is reference video frame 5, and reference video frame 5 belongs to both reference video 1 and reference video 2; a similar video frame similar to the video frame 6 to be processed is reference video frame 6, and reference video frame 6 belongs to both reference video 1 and reference video 2; then reference video 1, reference video 2 and reference video 3 can be determined as similar videos similar to the target video.

[0061] S404, determining the audio feature similarity between the target video and each similar video, and determining whether the target video and each similar video are repeated according to the audio feature similarity between the target video and each similar video.

[0062] In one embodiment, when the video duplication checking device determines the audio feature similarity between the target video and each similar video, it can first obtain the audio features of the target video and the audio features of each similar video, and then determine the audio feature similarity between the target video and each similar video based on the audio features of the target video and the audio features of each similar video. In a feasible implementation, the acquisition of the audio features of the target video and the audio features of each similar video can be obtained after the video duplication checking device performs audio feature extraction processing on the target video and the audio features extraction processing on each similar video. In another feasible implementation, the acquisition of the audio features of the target video and the audio features of each similar video by the video duplication checking device can be obtained after the feature extraction device performs audio feature extraction processing on the target video and the audio features extraction processing on each similar video. In another feasible implementation, the audio features of each reference video can be extracted in advance and stored in a database corresponding to the video duplication checking device. Then, after the video duplication checking device determines K reference videos as similar videos from H reference videos based on M similar video frames, the audio features of the K reference videos can be directly obtained from the database corresponding to the video duplication checking device as the audio features of K similar videos.

[0063] Optionally, when the video duplication checking device determines the audio feature similarity between the target video and each similar video based on the audio features of the target video and the audio features of each similar video, the cosine similarity between the audio features of the target video and the audio features of each similar video may be used as the audio feature similarity between the target video and each similar video, or the audio feature similarity between the target video and each similar video may be determined based on the feature distance (for example, Euclidean distance, Manhattan distance, etc.) between the audio features of the target video and the audio features of each similar video, and so on; the embodiment of the present application does not limit the specific representation method of the audio feature similarity between any videos.

[0064] Furthermore, the video duplication checking device can determine whether the target video and each similar video are repeated based on the audio feature similarity between the target video and each similar video; in a specific implementation, when determining whether the target video and any similar video are repeated, if the audio feature similarity between the target video and any similar video is greater than a preset audio similarity threshold, then the target video is determined to be repeated with the any similar video; if the audio feature similarity between the target video and any similar video is less than or equal to the preset audio similarity threshold, then the target video is determined to be not repeated with the any similar video; wherein, the preset audio similarity threshold can be set according to specific needs.

[0065] See also Figure 5, is a flow chart of checking for duplicate content of a target video provided in an embodiment of the present application, wherein a video duplication checking device obtains N to-be-processed video frames determined from the target video and L reference video frames determined from H reference videos; if, based on the image feature similarity between each to-be-processed video frame and each reference video frame, no reference video frame similar to any to-be-processed video frame can be determined from the L reference video frames, then it is determined that there is no video in the H reference videos that is duplicated with the target video; if, based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames are determined from the L reference video frames as similar video frames, then, based on the M similar video frames, K reference videos are determined from the H reference videos as similar videos, the audio feature similarity between the target video and each similar video is determined, and whether the target video is duplicated with each similar video is determined based on the audio feature similarity between the target video and each similar video.

[0066] In an embodiment of the present application, N to-be-processed video frames determined from a target video to be checked for duplicates can be obtained, and L reference video frames determined from H reference videos for checking for duplicates can be obtained; then, based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames can be determined from the L reference video frames as similar video frames, and one similar video frame is similar to at least one to-be-processed video frame; and then, based on the M similar video frames, K reference videos can be determined from the H reference videos as similar videos, and any similar video contains at least one similar video frame; then, based on the audio feature similarity between the target video and each similar video, it can be determined whether the target video is repeated with respect to each similar video. That is to say, based on the similarity of image features between each video frame to be processed and each reference video frame, similar videos to the target video can be recalled from the reference videos; and further, based on the similarity of audio features between the target video and each similar video, it is determined whether the target video and each similar video are duplicates, so as to realize duplicate checking of the target video; that is, in the process of realizing duplicate checking of the target video, two types of features, image features of video frames and audio features of videos, are referred to, and the types of features referred to for video duplicate checking are concise, which can improve the efficiency of video duplicate checking.

[0067] Based on the above embodiments, the present application provides another video duplication checking method. Figure 6 , which is a flow chart of another video duplication checking method provided in an embodiment of the present application. Figure 6 The video duplication checking method shown can be executed by a video duplication checking device. Figure 6 The video duplication checking method shown may include the following steps:

[0068] S601, obtaining N to-be-processed video frames and L reference video frames.

[0069] Among them, N video frames to be processed are determined from the target video to be checked for plagiarism, and L reference video frames are determined from H reference videos used for plagiarism checking and comparison. N, L and H are all positive integers; step S601 is consistent with the relevant process of the above-mentioned step S401, and will not be repeated here.

[0070] S602: Obtain image features of each video frame to be processed.

[0071] In one embodiment, the image features of any video frame to be processed are obtained by performing image feature extraction processing on the video frame to be processed. In a feasible implementation, the image features of the video frame to be processed obtained by the video duplication checking device can be obtained by performing image feature extraction processing on the video frame to be processed by the video duplication checking device; or the image features of the video frame to be processed obtained by the feature extraction device after performing image feature extraction processing on the video frame to be processed; in such a case, Figure 2 or Figure 3 In the video duplication checking system shown, the image features of each to-be-processed video frame acquired by the video duplication checking device may be extracted by a first feature extraction device among a plurality of feature extraction devices, and the first feature extraction device is a feature extraction device that receives a target video via a message queue, and the target video is uploaded to the message queue by a video distribution device; for ease of explanation, in subsequent embodiments, the image features of the to-be-processed video frame are extracted by the video duplication checking device for explanation.

[0072] In one embodiment, the video duplication checking device can call the optimized image feature extraction model to extract the image features of any video frame, that is, the video duplication checking device can call the optimized image feature extraction model to perform image feature extraction processing on any video frame to obtain the image features of any video frame. The optimized image feature extraction model is obtained by optimizing the model parameters of the image feature extraction model. The optimized image feature extraction model has the same model structure as the image feature extraction model, but different model parameters. The optimization process of the image feature extraction model will be introduced in subsequent embodiments.

[0073] The optimized image feature extraction model can include an image feature extraction layer and a global average pooling layer, see Figure 7, is a schematic diagram of extracting image features of any video frame through an optimized image feature extraction model provided in an embodiment of the present application; a high-dimensional image feature vector of any video frame can be extracted through the image feature extraction layer in the optimized image feature extraction model, and then the high-dimensional image feature vector of any video frame is subjected to dimensionality reduction processing through the global average pooling layer in the optimized image feature extraction model to obtain a low-dimensional image feature vector of the any video frame, and the obtained low-dimensional image feature vector of the any video frame is used as the image feature of the any video frame. Optionally, the image feature extraction layer in the optimized image feature extraction model can be a convolutional neural network (CNN), for example, a deep residual network (ResNet) in a convolutional neural network, and exemplary examples can be ResNet50, ResNet101, etc. in a deep residual network; if the image feature extraction layer in the optimized image feature extraction model is ResNet101, the dimension of the low-dimensional image feature vector of any video frame obtained by dimensionality reduction processing through the global average pooling layer in the optimized image feature extraction model can be 1×2048.

[0074] S603: Obtain image features of each reference video frame.

[0075] In one embodiment, the image features of each reference video frame may be pre-acquired by a video duplication checking device and stored in a database corresponding to the video duplication checking device. During the process of checking the target video for duplication, the video duplication checking device may obtain the image features of each reference video frame from the database corresponding to the video duplication checking device. In a feasible implementation, the image features of the reference video frames pre-acquired and stored by the video duplication checking device may be those obtained by the video duplication checking device performing image feature extraction processing on the reference video frames; or they may be those obtained by the video duplication checking device performing image feature extraction processing on the reference video frames by a feature extraction device; in such a case, Figure 2 or Figure 3 In the video duplication checking system shown, the image features of any reference video frame pre-acquired and stored by the video duplication checking device are extracted by a second feature extraction device among multiple feature extraction devices, and the second feature extraction device is a feature extraction device that receives the reference video to which any reference video frame belongs through a message queue, and the reference video to which any reference video frame belongs is uploaded to the message queue by a video distribution device; for ease of explanation, in subsequent embodiments, the image features of the reference video frame are extracted by the video duplication checking device for explanation.

[0076] S604: Determine the image feature similarity between each to-be-processed video frame and each reference video frame.

[0077] In one embodiment, taking the example of determining the image feature similarity between a video frame to be processed and a reference video frame based on the image features of the video frame to be processed and the image features of the reference video frame, the video duplication checking device may use the cosine similarity between the image features of the video frame to be processed and the image features of the reference video frame as the image feature similarity between the video frame to be processed and the reference video frame; or may determine the image feature similarity between the video frame to be processed and the reference video frame based on the feature distance (such as Euclidean distance, Manhattan distance, etc.) between the image features of the video frame to be processed and the image features of the reference video frame, and so on; the embodiment of the present application does not limit the specific representation method of the image feature similarity between any video frames.

[0078] S605 , based on the image feature similarity between each to-be-processed video frame and each reference video frame, determine M reference video frames from the L reference video frames as similar video frames.

[0079] In one embodiment, taking determining similar video frames corresponding to a video frame to be processed as an example; the video duplication checking device can determine the image feature similarity between any video frame to be processed and each reference video frame in L reference video frames for any video frame to be processed among N video frames to be processed; and use a second number of reference video frames whose image feature similarity is greater than a first similarity threshold and whose image feature similarity is ranked first as similar video frames to any video frame to be processed; wherein the first similarity threshold can be set according to specific needs, and the second number can be set according to specific needs. In a feasible implementation, when the video duplication checking device uses a second number of reference video frames whose image feature similarity is greater than a first similarity threshold and whose image feature similarity is ranked first as similar video frames to any video frame to be processed, it can first use the reference video frames whose image feature similarity is greater than the first similarity threshold among the L reference video frames as candidate video frames corresponding to any video frame to be processed based on the image feature similarity between any video frame to be processed and each reference video frame; and then select a second number of candidate video frames from the candidate video frames corresponding to any video frame to be processed in descending order of image feature similarity as similar video frames similar to any video frame to be processed. Optionally, the video duplication checking device can also use reference video frames whose image feature similarity is greater than the first similarity threshold as similar video frames to any video frame to be processed, that is, the video duplication checking device can also use the candidate video frames corresponding to any video frame to be processed as similar video frames similar to the any video frame to be processed.

[0080] For example, if a video frame 1 to be processed is extracted from the target video, the L reference video frames determined from the H reference videos are: reference video frame 1, reference video frame 2, reference video frame 3, reference video frame 4, reference video frame 5, reference video frame 6, reference video frame 7, reference video frame 8, reference video frame 9, and reference video frame 10; if based on the image feature similarity between the video frame 1 to be processed and each reference video frame, the reference video frames whose image feature similarity is greater than the first similarity threshold are determined, and are arranged in order from large to small in terms of image feature similarity, are: reference video frame 1, reference video frame 2, reference video frame 3, and reference video frame 4, then the candidate video frames corresponding to the video frame 1 to be processed are arranged in order from large to small in terms of image feature similarity, are: reference video frame 1, reference video frame 2, reference video frame 3, and reference video frame 4. If the candidate video frame corresponding to the video frame to be processed 1 is taken as a similar video frame similar to the video frame to be processed 1, the similar video frames similar to the video frame to be processed 1 are: reference video frame 1, reference video frame 2, reference video frame 3 and reference video frame 4.

[0081] If, from the candidate video frames corresponding to the video frame 1 to be processed, one candidate video frame is selected as a similar video frame similar to the video frame 1 to be processed in order of image feature similarity from large to small, then the similar video frame similar to the video frame 1 to be processed is: reference video frame 1; if, from the candidate video frames corresponding to the video frame 1 to be processed, two candidate video frames are selected as similar video frames similar to the video frame 1 to be processed in order of image feature similarity from large to small, then the similar video frames similar to the video frame 1 to be processed are: reference video frame 1 and reference video frame 2; the embodiments of the present application will be described later by taking as an example the selection of one candidate video frame as a similar video frame similar to any video frame to be processed from the candidate video frames corresponding to any video frame to be processed in order of image feature similarity from large to small.

[0082] S606 , determining K reference videos from the H reference videos as similar videos according to the M similar video frames.

[0083] Wherein, any similar video contains at least one similar video frame, and K is a positive integer.

[0084] In one embodiment, a video duplication checking device determines K reference videos as similar videos from H reference videos based on M similar video frames, which may include: determining at least one candidate video from H reference videos based on M similar video frames; a candidate video refers to a reference video containing at least one similar video frame; for any candidate video in at least one candidate video, dividing any candidate video into P video segments according to the positions of each similar video frame contained in any candidate video in any candidate video; the first frame and the last frame of any video segment are both similar video frames, and the arrangement order of each similar video frame contained in any video segment in any video segment is the same as the arrangement order of the to-be-processed video frames corresponding to each similar video frame in the target video, and P is a positive integer; if there is at least one first video segment among the P video segments of any candidate video, any candidate video is regarded as a similar video; wherein the first video segment refers to a video segment including at least a first number of similar video frames, and the first number can be set according to specific needs.

[0085] For example, if the correspondence between each to-be-processed video frame determined from the target video, the similar video frame corresponding to each to-be-processed video frame, and the reference video to which each similar video frame belongs is shown in Table 1; if the reference video contains each reference video frame, and the position of each reference video frame in the reference video is given by the following Table 2:

[0086] Table 2

[0087] Reference Video 1 Reference Video 2 Reference Video 3 Reference Video 4 Reference video frame 1 Reference video frame 1 Reference video frame 7 Reference video frame 7 Reference video frame 2 Reference video frame 2 Reference video frame 9 Reference video frame 8 Reference video frame 3 Reference video frame 3 Reference video frame 10 Reference video frame 9 Reference video frame 4 Reference video frame 8 Reference video frame 4 Reference video frame 10 Reference video frame 5 Reference video frame 5 Reference video frame 3 Reference video frame 8 Reference video frame 6 Reference video frame 6 Reference video frame 7 Reference video frame 10

[0088] The first column in Table 2 indicates that the reference video frames contained in the reference video 1 from front to back are: reference video frame 1, reference video frame 2, reference video frame 3, reference video frame 4, reference video frame 5, reference video frame 6, and so on; if the to-be-processed video frames contained in the target video from front to back are: to-be-processed video frame 1, to-be-processed video frame 2, to-be-processed video frame 3, to-be-processed video frame 4, to-be-processed video frame 5 and to-be-processed video frame 6; then for the reference video 1 among the multiple candidate videos, according to the positions of the similar video frames contained in the candidate video (i.e., reference video 1) in the candidate video, a video segment is divided from the candidate video, the first frame of the video segment is the reference video frame 1, and the last frame is the reference video frame 6; for the reference video 1 among the multiple candidate videos 2. According to the positions of each similar video frame contained in the candidate video (i.e., reference video 2) in the candidate video, two video segments are divided from the candidate video, wherein the first frame of one video segment is reference video frame 1 and the last frame is reference video frame 3, and the first frame of the other video segment is reference video frame 5 and the last frame is reference video frame 6; for reference video 3 among multiple candidate videos, according to the positions of each similar video frame contained in the candidate video (i.e., reference video 3) in the candidate video, since the arrangement order of reference video frame 3 and reference video frame 4 contained in the candidate video in the candidate video is different from the arrangement order of to-be-processed video frame 3 corresponding to reference video frame 3 and to-be-processed video frame 4 corresponding to reference video frame 4 in the target video, video segments cannot be divided from the candidate video.

[0089] Further, if the first number is set to 2; since the video segment determined from the candidate video as reference video 1 includes 6 similar video frames, it is determined that the candidate video has 1 first video segment, and the candidate video (i.e., reference video 1) is taken as a similar video; since 1 video segment determined from the candidate video as reference video 2 includes 3 similar video frames, and another video segment determined includes 2 similar video frames, it is determined that the candidate video has 2 first video segments, and the candidate video (i.e., reference video 2) is taken as a similar video.

[0090] S607, determining the audio feature similarity between the target video and each similar video, and determining whether the target video and each similar video are repeated according to the audio feature similarity between the target video and each similar video.

[0091] In one embodiment, the video duplication checking device determines the audio feature similarity between the target video and each similar video, which may include: determining the audio feature similarity between the target video and each similar video based on the audio feature of the target video and the audio feature of each similar video. In another embodiment, the video duplication checking device determines the audio feature similarity between the target video and each similar video, which may include: for any similar video among the similar videos, from each first video segment corresponding to any similar video, selecting the first video segment containing the largest number of similar video frames as the second video segment; determining the to-be-processed video segment corresponding to the second video segment in the target video; and using the audio feature similarity between the to-be-processed video segment and the second video segment as the audio feature similarity between the target video and any similar video.

[0092] For example, when the similar video is reference video 1, the first video segment corresponding to reference video 1, whose first frame is reference video frame 1 and whose last frame is reference video frame 6, can be used as the second video segment; and the video segment in the target video, whose first frame is to-be-processed video frame 1 and whose last frame is to-be-processed video frame 6, can be determined as the to-be-processed video segment corresponding to the second video segment. The audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and the similar video, that is, the audio feature similarity between the to-be-processed video segment and the second video segment can be determined based on the audio features of the to-be-processed video segment and the audio features of the second video segment, and the audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and the similar video.

[0093] When the similar video is reference video 2, the first video segment containing the largest number of similar video frames can be selected from the first video segments corresponding to the similar video as the second video segment, that is, the first video segment corresponding to reference video 2, containing the largest number of similar video frames, with the first frame being reference video frame 1 and the last frame being reference video frame 3, can be used as the second video segment; and the video segment in the target video, with the first frame being the to-be-processed video frame 1 and the last frame being the to-be-processed video frame 3, can be determined as the to-be-processed video segment corresponding to the second video segment; and the audio feature similarity between the to-be-processed video segment and the second video segment can be used as the audio feature similarity between the target video and the similar video.

[0094] Furthermore, the relevant process of determining whether the target video and each similar video are duplicates based on the audio feature similarity between the target video and each similar video has been introduced in the above step S404 and will not be repeated here.

[0095] See also Figure 8, is a schematic diagram of checking for duplicate content of a target video based on a video clip provided in an embodiment of the present application, and is described based on the relationship between the to-be-processed video frames, similar video frames, similar videos, and the reference video frames contained in the similar videos shown in Tables 1 and 2; image features of each to-be-processed video frame in the N to-be-processed video frames of the target video can be extracted, and image features of each reference video frame in the L reference video frames of the H reference videos used for checking for duplicate content can be extracted from the database corresponding to the video duplication checking device, wherein the target video includes 6 to-be-processed video frames, namely, to-be-processed video frame 1, to-be-processed video frame 2, to-be-processed video frame 3, to-be-processed video frame 4, to-be-processed video frame 5, and to-be-processed video frame 6, and each to-be-processed video frame has Figure 8 801, 802, 803, 804, 805 and 806 respectively; based on the image features of each video frame to be processed and the image features of each reference video frame, the image feature similarity between each video frame to be processed and each reference video frame is determined; based on the image feature similarity between each video frame to be processed and each reference video frame, M reference video frames are determined from L reference video frames as similar video frames; according to the M similar video frames, K reference videos are determined from H reference videos as similar videos; wherein, one similar video among the K similar videos can be indicated by 807, the similar video is reference video 1, including 1 video clip, the video clip can be indicated by 808, the first frame of which is reference video frame 1, the last frame is reference video frame 6, the video clip The similar video frames included are, from front to back, reference video frame 1, reference video frame 2, reference video frame 3, reference video frame 4, reference video frame 5, and reference video frame 6; if the first number is set to 2, the video segment indicated by the 808 mark is determined to be the first video segment, and from the first video segments corresponding to the similar video, the first video segment containing the largest number of similar video frames is selected as the second video segment as the video segment indicated by the 808 mark; further, the to-be-processed video segment corresponding to the second video segment in the target video can be determined, and the audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and the similar video, and whether the target video and the similar video are repeated is determined based on the audio feature similarity.

[0096] In an embodiment of the present application, based on the image features of each of the N to-be-processed video frames of the target video and the image features of each of the L reference video frames of the H reference videos used for duplicate checking and comparison, the image feature similarity between each to-be-processed video frame and each of the reference video frames can be determined, and based on the image feature similarity, M reference video frames can be determined from the L reference video frames as similar video frames; and then, based on the M similar video frames, K reference videos can be determined from the H reference videos as similar videos, wherein a reference video containing at least one similar video frame among the H reference videos can be determined as For any candidate video, whether to regard the candidate video as a similar video can be determined based on the relationship between the positions of each similar video frame contained in the candidate video in the candidate video and the positions of each to-be-processed video corresponding to each similar video contained in the candidate video in the target video; and then, whether the target video and each similar video are duplicates can be determined based on the similarity of the audio features between the target video and each similar video; in the process of checking the duplicate of the target video, two types of features, image features of the video frame and audio features of the video, are referred to, and the types of features referred to for video duplication checking are concise, which can improve the efficiency of video duplication checking.

[0097] Since the image features of any video frame can be obtained by calling the optimized image feature extraction model to perform image feature extraction processing on any video frame, the optimized image feature extraction model is obtained by optimizing the model parameters of the image feature extraction model. The optimized image feature extraction model has the same model structure as the image feature extraction model, but different model parameters. Based on this, the embodiment of the present application elaborates on the optimization process of the image feature extraction model, see Fig. 9 , is a flow chart of an optimization method for an image feature extraction model provided in an embodiment of the present application; Fig. 9 The optimization method of the image feature extraction model shown can be executed by a video duplication checking device, or by any electronic device that can optimize the image feature extraction model, for example, it can be executed by a feature extraction device. The embodiments of the present application will be described later by taking the execution by a video duplication checking device as an example; Fig. 9 The optimization method of the image feature extraction model shown may include the following steps:

[0098] S901, obtaining training samples.

[0099] The training samples include sample video frames, labeled similar video frames corresponding to the sample video frames, and labeled difference video frames corresponding to the sample video frames; the sample video frame may refer to: a video frame in the sample video, the labeled similar video frame corresponding to the sample video frame may refer to: a video frame with a similar content background and a similar content subject to the sample video frame, and the labeled difference video frame corresponding to the sample video frame may refer to: a video frame with a similar content background but a different content subject to the sample video frame. For example, see Fig.10 , which is a comparison diagram among a sample video frame, annotated similar video frame and annotated difference video frame provided in an embodiment of the present application; if the sample video frame is as indicated by mark 1001, the content background is the sky, and the content body is the airplane as indicated by mark 1002; the annotated similar video frame corresponding to the sample video frame can be as indicated by mark 1003, which has a similar content background and a similar content body to the sample video frame, except that a new identifier as indicated by mark 1004 is added, which is "A"; the annotated difference video frame corresponding to the sample video frame can be as indicated by mark 1005, which has a similar content background to the sample video frame, both of which are the sky, but the content body is different, and its content body is a rocket as indicated by mark 1006.

[0100] In one embodiment, the annotated similar video frame corresponding to the sample video frame and the annotated difference video frame corresponding to the sample video frame can be determined from L reference video frames; in this case, the video duplication checking device obtains the training sample, which may include: selecting a video frame from the sample video as the sample video frame; calling the image feature extraction model to extract the image features of the sample video frame and the image features of each reference video frame; based on the image features of the sample video frame and the image features of each reference video frame, determining the image feature similarity between the sample video frame and each reference video frame; based on the image feature similarity between the sample video frame and each reference video frame, extracting the image features of the L reference video frames; In the frame, the reference video frame whose image feature similarity is greater than the second similarity threshold is used as the recalled video frame corresponding to the sample video frame; the recalled video frames with similar content background and similar content body in the recalled video frames corresponding to the sample video frame are added to the first video frame set, and the recalled video frames with similar content background but different content body in the recalled video frames corresponding to the sample video frame are added to the second video frame set; any recalled video frame in the first video frame set is selected as the labeled similar video frame, and any recalled video frame in the second video frame set is selected as the labeled difference video frame, and training samples are constructed based on the sample video frames, the labeled similar video frames and the labeled difference video frames.

[0101] Among them, when selecting a video frame from the sample video as the sample video frame, the selected video frame can be any video frame. The video duplicate checking device calls the image feature extraction model to extract the image features of the sample video frame, and extracts the image features of each reference video frame, which is similar to the above-mentioned related process of extracting the image features of any video frame through the optimized image feature extraction model, and will not be repeated here. The video duplicate checking device determines the image feature similarity between the sample video frame and each reference video frame based on the image features of the sample video frame and the image features of each reference video frame, which is similar to the related process of step S604, and will not be repeated here. Based on the image feature similarity between the sample video frame and each reference video frame, the video duplicate checking device uses the reference video frame whose image feature similarity is greater than the second similarity threshold among the L reference video frames as the recalled video frame corresponding to the sample video frame, which is similar to the above-mentioned related process of using the reference video frame whose image feature similarity is greater than the first similarity threshold among the L reference video frames as the candidate video frame corresponding to any video frame to be processed based on the image feature similarity between any video frame to be processed and each reference video frame, and will not be repeated here.

[0102] Among them, the image feature extraction model can be a model pre-trained based on an existing image data set, and the existing image data set can be ImageNet; the second similarity threshold can be set according to specific needs. Generally speaking, in order to obtain more training samples, the second similarity threshold can be set to be smaller than the first similarity threshold. Optionally, the recalled video frames with similar content backgrounds and similar content bodies in the recalled video frames corresponding to the sample video frames are added to the first video frame set, and the recalled video frames with similar content backgrounds but different content bodies in the recalled video frames corresponding to the sample video frames are added to the second video frame set, which can be manually screened.

[0103] In one embodiment, when multiple training samples are obtained based on multiple sample video frames of multiple sample videos (for example, 100,000 sample videos), a portion of the training samples can be used to optimize the image feature extraction model, and the remaining training samples can be used for model verification; for example, 80% of the training samples can be selected from multiple training samples for model optimization, and the remaining 20% ​​of the training samples can be used for model verification.

[0104] S902, calling an image feature extraction model to extract image features of a sample video frame, image features of annotated similar video frames, and image features of annotated difference video frames.

[0105] Among them, step S902 is similar to the above step S604 and will not be repeated here.

[0106] S903, calculating a model loss value based on a feature distance between an image feature of the sample video frame and an image feature of annotated similar video frames, and a feature distance between an image feature of the sample video frame and an image feature of annotated different video frames.

[0107] In one embodiment, a triplet loss function (Triplet Loss) can be used as an optimization constraint, and the feature distance between the image features of the sample video frame and the image features of the annotated similar video frame, as well as the feature distance between the image features of the sample video frame and the image features of the annotated difference video frame are used as function parameters of the triplet loss function, and the function value of the triplet loss function is calculated as the model loss value; the feature distance can be Euclidean distance, Manhattan distance, etc., which is not limited in the embodiments of the present application.

[0108] S904, optimizing the model parameters of the image feature extraction model in a direction of reducing the model loss value to obtain an optimized image feature extraction model.

[0109] In one embodiment, since the triplet loss function is used as the optimization constraint, the model parameters of the image feature extraction model are optimized in the direction of reducing the model loss value, that is, the model parameters of the image feature extraction model are optimized in the direction of reducing the feature distance between the image features of the sample video frame and the image features of the annotated similar video frame, and increasing the feature distance between the image features of the sample video frame and the image features of the annotated difference video frame.

[0110] See also Fig.11 , which is a schematic diagram of determining a model loss value through an image feature extraction model provided in an embodiment of the present application; the image feature extraction model can be called to extract image features of a sample video frame, image features of annotated similar video frames, and image features of annotated difference video frames, respectively; the feature distance between the image features of the sample video frame and the image features of the annotated similar video frame, and the feature distance between the image features of the sample video frame and the image features of the annotated difference video frame are used as function parameters of a triplet loss function, and the function value of the triplet loss function is calculated as the model loss value.

[0111] In an embodiment of the present application, a training sample including a sample video frame, an annotated similar video frame corresponding to the sample video frame, and an annotated difference video frame corresponding to the sample video frame can be used to optimize the image feature extraction model; wherein the annotated similar video frame corresponding to the sample video frame can refer to: a video frame with a similar content background and a similar content body to the sample video frame, and the annotated difference video frame corresponding to the sample video frame can refer to: a video frame with a similar content background but a different content body to the sample video frame; the image feature extraction model can be called to extract image features of the sample video frame, image features of the annotated similar video frame, and image features of the annotated difference video frame, respectively; based on the image features of the sample video frame and the annotated similar video frame, the image features of the annotated difference video frame are extracted. The model loss value is calculated based on the feature distance between the image features of the sample video frame and the image features of the labeled difference video frame. The model parameters of the image feature extraction model are optimized in the direction of reducing the model loss value to obtain the optimized image feature extraction model. The triplet loss function can be used as an optimization constraint to optimize the model parameters of the image feature extraction model in the direction of reducing the feature distance between the image features of the sample video frame and the image features of the labeled similar video frame, and increasing the feature distance between the image features of the sample video frame and the image features of the labeled difference video frame. The optimized image feature extraction model can be made more accurate in image feature extraction.

[0112] Based on the above-mentioned related embodiments of the video duplication checking method, the present application embodiment provides a video duplication checking device. Fig.12 , is a structural schematic diagram of a video duplication checking device provided in an embodiment of the present application, and the video duplication checking device may include an acquisition unit 1201 and a processing unit 1202. Fig.12 The video duplication checking device shown can run the following units:

[0113] The acquisition unit 1201 is used to acquire N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from the target video to be checked for plagiarism, and the L reference video frames are determined from H reference videos for checking for plagiarism, and N, L and H are all positive integers;

[0114] The processing unit 1202 is configured to determine M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame; a similar video frame is similar to at least one to-be-processed video frame, M∈[1,L];

[0115] The processing unit 1202 is further configured to determine K reference videos as similar videos from the H reference videos according to the M similar video frames; any similar video contains at least one similar video frame, and K is a positive integer;

[0116] The processing unit 1202 is further configured to determine the audio feature similarity between the target video and each similar video, and determine whether the target video and each similar video are duplicates based on the audio feature similarity between the target video and each similar video.

[0117] In one embodiment, the N to-be-processed video frames and the L reference video frames are extracted from the target video and the H reference videos respectively based on the same frame extraction rule;

[0118] When the processing unit 1202 determines K reference videos as similar videos from the H reference videos according to the M similar video frames, the processing unit 1202 specifically performs the following operations:

[0119] Determine at least one candidate video from the H reference videos according to the M similar video frames; the candidate video refers to a reference video containing at least one similar video frame;

[0120] For any candidate video among the at least one candidate video, according to the positions of the similar video frames contained in the any candidate video in the any candidate video, P video segments are divided from the any candidate video; the first frame and the last frame of any video segment are both similar video frames, and the arrangement order of the similar video frames contained in the any video segment in the any video segment is the same as the arrangement order of the to-be-processed video frames corresponding to the similar video frames in the target video, and P is a positive integer;

[0121] If there is at least one first video segment among the P video segments of any candidate video, then any candidate video is regarded as a similar video; wherein the first video segment refers to a video segment including at least a first number of similar video frames.

[0122] In one embodiment, when the processing unit 1202 determines the audio feature similarity between the target video and each similar video, it specifically performs the following operations:

[0123] For any similar video among the similar videos, selecting, from the first video segments corresponding to the similar video, a first video segment containing the largest number of similar video frames as the second video segment;

[0124] Determining a to-be-processed video segment corresponding to the second video segment in the target video;

[0125] The audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and any of the similar videos.

[0126] In one embodiment, when the processing unit 1202 determines M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame, the processing unit 1202 specifically performs the following operations:

[0127] For any one of the N to-be-processed video frames, determine the image feature similarity between the any one of the to-be-processed video frames and each of the L reference video frames;

[0128] A second number of reference video frames whose image feature similarity is greater than the first similarity threshold and whose image feature similarity is ranked first are used as similar video frames to any one of the video frames to be processed.

[0129] In one embodiment, the image feature similarity between the to-be-processed video frame and the reference video frame is a similarity determined based on image features of the to-be-processed video frame and the reference video frame, and the image features of the to-be-processed video frame and the reference video frame are obtained by performing image feature extraction processing on an optimized image feature extraction model;

[0130] The processing unit 1202 is further configured to:

[0131] Acquire training samples; the training samples include sample video frames, annotated similar video frames corresponding to the sample video frames, and annotated difference video frames corresponding to the sample video frames;

[0132] Calling an image feature extraction model to respectively extract image features of the sample video frame, image features of the annotated similar video frame, and image features of the annotated difference video frame;

[0133] Calculating a model loss value based on a feature distance between an image feature of the sample video frame and an image feature of the annotated similar video frame, and a feature distance between an image feature of the sample video frame and an image feature of the annotated difference video frame;

[0134] The model parameters of the image feature extraction model are optimized in the direction of reducing the model loss value to obtain the optimized image feature extraction model.

[0135] In one embodiment, when the processing unit 1202 obtains the training sample, it specifically performs the following operations:

[0136] Selecting a video frame from the sample video as a sample video frame;

[0137] Calling the image feature extraction model to extract image features of the sample video frame and extracting image features of each reference video frame;

[0138] Determining the image feature similarity between the sample video frame and each reference video frame based on the image feature of the sample video frame and the image feature of each reference video frame;

[0139] Based on the image feature similarity between the sample video frame and each of the reference video frames, taking a reference video frame whose image feature similarity is greater than a second similarity threshold among the L reference video frames as a recalled video frame corresponding to the sample video frame;

[0140] Adding the recalled video frames with similar content backgrounds and similar content bodies among the recalled video frames corresponding to the sample video frames to the first video frame set, and adding the recalled video frames with similar content backgrounds but different content bodies among the recalled video frames corresponding to the sample video frames to the second video frame set;

[0141] Any recalled video frame in the first video frame set is selected as the labeled similar video frame, and any recalled video frame in the second video frame set is selected as the labeled difference video frame, and the training sample is constructed based on the sample video frame, the labeled similar video frame and the labeled difference video frame.

[0142] In one embodiment, the acquisition unit 1201 is further used to acquire the image features of each to-be-processed video frame; the image features of each to-be-processed video frame are extracted by a first feature extraction device among a plurality of feature extraction devices, the first feature extraction device is a feature extraction device that receives the target video through a message queue, and the target video is uploaded to the message queue by a video distribution device;

[0143] The acquisition unit 1201 is further used to acquire the image features of each reference video frame; the image features of any reference video frame are extracted by a second feature extraction device among multiple feature extraction devices, the second feature extraction device is a feature extraction device that receives the reference video to which any reference video frame belongs through the message queue, and the reference video to which any reference video frame belongs is uploaded to the message queue by the video distribution device;

[0144] The processing unit 1202 is further configured to determine the image feature similarity between each to-be-processed video frame and each reference video frame based on the image features of each to-be-processed video frame and the image features of each reference video frame.

[0145] According to one embodiment of the present application, Figure 4 , Figure 6 as well as Fig. 9 The steps involved in the video duplication checking method shown can be Fig.12The video duplication checking device shown in the figure is executed by each unit. For example, Figure 4 Step S401 shown can be performed by Fig.12 The acquisition unit 1201 in the video duplication checking device shown is used to perform the operation. Figure 4 Steps S402 to S404 shown in FIG. Fig.12 The processing unit 1202 in the video duplication checking device shown in the figure is used for execution. Figure 6 Steps S601 to S603 shown in FIG. Fig.12 The acquisition unit 1201 in the video duplication checking device shown is used to perform the operation. Figure 6 Steps S604 to S607 shown in FIG. Fig.12 The processing unit 1202 in the video duplication checking device shown in the figure is used for execution. Fig. 9 Steps S901 to S904 shown in FIG. Fig.12 The processing unit 1202 in the video duplication checking device shown is used for execution.

[0146] According to another embodiment of the present application, Fig.12 The various units in the video duplication checking device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the video duplication checking device based on logical function division can also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0147] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 4 , Figure 6 as well as Fig. 9 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Fig.12 The video duplication checking device shown in and the video duplication checking method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable storage medium, and loaded into the above-mentioned computing device through the computer readable storage medium, and run therein.

[0148] In an embodiment of the present application, N to-be-processed video frames determined from a target video to be checked for duplicates can be obtained, and L reference video frames determined from H reference videos for checking for duplicates can be obtained; then, based on the image feature similarity between each to-be-processed video frame and each reference video frame, M reference video frames can be determined from the L reference video frames as similar video frames, and one similar video frame is similar to at least one to-be-processed video frame; and then, based on the M similar video frames, K reference videos can be determined from the H reference videos as similar videos, and any similar video contains at least one similar video frame; then, based on the audio feature similarity between the target video and each similar video, it can be determined whether the target video is repeated with respect to each similar video. That is to say, based on the similarity of image features between each video frame to be processed and each reference video frame, similar videos to the target video can be recalled from the reference videos; and further, based on the similarity of audio features between the target video and each similar video, it is determined whether the target video and each similar video are duplicates, so as to realize duplicate checking of the target video; that is, in the process of realizing duplicate checking of the target video, two types of features, image features of video frames and audio features of videos, are referred to, and the types of features referred to for video duplicate checking are concise, which can improve the efficiency of video duplicate checking.

[0149] Based on the above-mentioned video duplication checking method and video duplication checking device embodiments, the present application also provides a video duplication checking device. Fig.13 , which is a structural diagram of a video duplication checking device provided in an embodiment of the present application. Fig.13 The video duplication checking device shown may include at least a processor 1301, an input interface 1302, an output interface 1303, and a computer storage medium 1304. The processor 1301, the input interface 1302, the output interface 1303, and the computer storage medium 1304 may be connected via a bus or other means.

[0150] The computer storage medium 1304 can be stored in the memory of the video duplication checking device. The computer storage medium 1304 is used to store a computer program. The computer program includes program instructions. The processor 1301 is used to execute the program instructions stored in the computer storage medium 1304. The processor 1301 (or CPU (Central Processing Unit)) is the computing core and control core of the video duplication checking device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the above-mentioned video duplication checking method process or corresponding functions.

[0151] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in a video duplication checking device for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the terminal and the extended storage medium supported by the terminal. The computer storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1301 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0152] In one embodiment, the processor 1301 and the input interface 1302 may load and execute one or more instructions stored in the computer storage medium to implement the above-mentioned Figure 4 , Figure 6 as well as Fig. 9 In the corresponding steps of the method in the embodiment of the video duplication checking method, in a specific implementation, one or more instructions in the computer storage medium are loaded by the processor 1301 and the input interface 1302 and execute the following steps:

[0153] Input interface 1302, used to obtain N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from the target video to be checked for plagiarism, and the L reference video frames are determined from H reference videos for plagiarism checking and comparison, where N, L and H are all positive integers;

[0154] The processor 1301 is configured to determine M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame; a similar video frame is similar to at least one to-be-processed video frame, M∈[1,L];

[0155] The processor 1301 is further configured to determine K reference videos as similar videos from the H reference videos according to the M similar video frames; any similar video includes at least one similar video frame, and K is a positive integer;

[0156] The processor 1301 is further configured to determine the audio feature similarity between the target video and each similar video, and determine whether the target video and each similar video are duplicates based on the audio feature similarity between the target video and each similar video.

[0157] In one embodiment, the N to-be-processed video frames and the L reference video frames are extracted from the target video and the H reference videos respectively based on the same frame extraction rule;

[0158] When the processor 1301 determines K reference videos as similar videos from the H reference videos according to the M similar video frames, the processor 1301 specifically performs the following operations:

[0159] Determine at least one candidate video from the H reference videos according to the M similar video frames; the candidate video refers to a reference video containing at least one similar video frame;

[0160] For any candidate video among the at least one candidate video, according to the positions of the similar video frames contained in the any candidate video in the any candidate video, P video segments are divided from the any candidate video; the first frame and the last frame of any video segment are both similar video frames, and the arrangement order of the similar video frames contained in the any video segment in the any video segment is the same as the arrangement order of the to-be-processed video frames corresponding to the similar video frames in the target video, and P is a positive integer;

[0161] If there is at least one first video segment among the P video segments of any candidate video, then any candidate video is regarded as a similar video; wherein the first video segment refers to a video segment including at least a first number of similar video frames.

[0162] In one embodiment, when the processor 1301 determines the audio feature similarity between the target video and each similar video, it specifically performs the following operations:

[0163] For any similar video among the similar videos, selecting, from the first video segments corresponding to the similar video, a first video segment containing the largest number of similar video frames as the second video segment;

[0164] Determining a to-be-processed video segment corresponding to the second video segment in the target video;

[0165] The audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and any of the similar videos.

[0166] In one embodiment, the processor 1301 specifically performs the following operations when determining M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame:

[0167] For any one of the N to-be-processed video frames, determine the image feature similarity between the any one of the to-be-processed video frames and each of the L reference video frames;

[0168] A second number of reference video frames whose image feature similarity is greater than the first similarity threshold and whose image feature similarity is ranked first are used as similar video frames to any one of the video frames to be processed.

[0169] In one embodiment, the image feature similarity between the to-be-processed video frame and the reference video frame is a similarity determined based on image features of the to-be-processed video frame and the reference video frame, and the image features of the to-be-processed video frame and the reference video frame are obtained by performing image feature extraction processing on an optimized image feature extraction model;

[0170] The processor 1301 is further configured to:

[0171] Acquire training samples; the training samples include sample video frames, annotated similar video frames corresponding to the sample video frames, and annotated difference video frames corresponding to the sample video frames;

[0172] Calling an image feature extraction model to respectively extract image features of the sample video frame, image features of the annotated similar video frame, and image features of the annotated difference video frame;

[0173] Calculating a model loss value based on a feature distance between an image feature of the sample video frame and an image feature of the annotated similar video frame, and a feature distance between an image feature of the sample video frame and an image feature of the annotated difference video frame;

[0174] The model parameters of the image feature extraction model are optimized in the direction of reducing the model loss value to obtain the optimized image feature extraction model.

[0175] In one embodiment, when the processor 1301 obtains the training sample, it specifically performs the following operations:

[0176] Selecting a video frame from the sample video as a sample video frame;

[0177] Calling the image feature extraction model to extract image features of the sample video frame and extracting image features of each reference video frame;

[0178] Determining the image feature similarity between the sample video frame and each reference video frame based on the image feature of the sample video frame and the image feature of each reference video frame;

[0179] Based on the image feature similarity between the sample video frame and each of the reference video frames, taking a reference video frame whose image feature similarity is greater than a second similarity threshold among the L reference video frames as a recalled video frame corresponding to the sample video frame;

[0180] Adding the recalled video frames with similar content backgrounds and similar content bodies among the recalled video frames corresponding to the sample video frames to the first video frame set, and adding the recalled video frames with similar content backgrounds but different content bodies among the recalled video frames corresponding to the sample video frames to the second video frame set;

[0181] Any recalled video frame in the first video frame set is selected as the labeled similar video frame, and any recalled video frame in the second video frame set is selected as the labeled difference video frame, and the training sample is constructed based on the sample video frame, the labeled similar video frame and the labeled difference video frame.

[0182] In one embodiment, the input interface 1302 is further used to obtain the image features of each to-be-processed video frame; the image features of each to-be-processed video frame are extracted by a first feature extraction device among a plurality of feature extraction devices, the first feature extraction device is a feature extraction device that receives the target video through a message queue, and the target video is uploaded to the message queue by a video distribution device;

[0183] The input interface 1302 is further used to obtain the image features of each reference video frame; the image features of any reference video frame are extracted by a second feature extraction device among multiple feature extraction devices, and the second feature extraction device is a feature extraction device that receives the reference video to which any reference video frame belongs through the message queue, and the reference video to which any reference video frame belongs is uploaded to the message queue by the video distribution device;

[0184] The processor 1301 is further configured to determine the image feature similarity between each to-be-processed video frame and each reference video frame based on the image features of each to-be-processed video frame and the image features of each reference video frame.

[0185] The embodiment of the present application provides a computer program product, which includes a computer program stored in a computer storage medium; a processor of a video duplication checking device reads the computer program from the computer storage medium, and the processor executes the computer program, so that the video duplication checking device performs the above-mentioned Figure 4 , Figure 6 as well as Fig. 9 The method embodiment shown in the figure. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0186] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video duplication checking method, characterized in that: include: Obtain N to-be-processed video frames and L reference video frames; the N to-be-processed video frames are determined from a target video to be checked for duplicate content, and the L reference video frames are determined from H reference videos for checking for duplicate content, N, L, and H are all positive integers, and the N to-be-processed video frames and the L reference video frames are extracted from the target video and the H reference videos, respectively, based on the same frame extraction rule; Based on the image feature similarity between each to-be-processed video frame and each reference video frame, determining M reference video frames from the L reference video frames as similar video frames; A similar video frame is similar to at least one video frame to be processed, M∈[1,L]; According to the M similar video frames, determining K reference videos from the H reference videos as similar videos; Any similar video contains at least one similar video frame, and K is a positive integer; Determine the audio feature similarity between the target video and each similar video, and determine whether the target video and each similar video are repeated according to the audio feature similarity between the target video and each similar video; The method of determining K reference videos as similar videos from the H reference videos based on the M similar video frames includes: determining at least one candidate video from the H reference videos based on the M similar video frames; the candidate video refers to a reference video containing at least one similar video frame; for any candidate video among the at least one candidate video, dividing P video segments from the any candidate video according to the positions of each similar video frame contained in the any candidate video in the any candidate video; the first frame and the last frame of any video segment are both similar video frames, and the arrangement order of each similar video frame contained in the any video segment in the any video segment is the same as the arrangement order of the to-be-processed video frames corresponding to the each similar video frame in the target video, and P is a positive integer; if there is at least one first video segment among the P video segments of any candidate video, the any candidate video is regarded as a similar video; wherein the first video segment refers to a video segment including at least a first number of similar video frames.

2. The method according to claim 1, characterized in that The determining of the audio feature similarity between the target video and each similar video includes: For any similar video among the similar videos, selecting, from the first video segments corresponding to the similar video, a first video segment containing the largest number of similar video frames as the second video segment; Determining a to-be-processed video segment corresponding to the second video segment in the target video; The audio feature similarity between the to-be-processed video segment and the second video segment is used as the audio feature similarity between the target video and any of the similar videos.

3. The method according to claim 1, characterized in that The determining M reference video frames as similar video frames from the L reference video frames based on the image feature similarity between each to-be-processed video frame and each reference video frame comprises: For any one of the N to-be-processed video frames, determine the image feature similarity between the any one of the to-be-processed video frames and each of the L reference video frames; A second number of reference video frames whose image feature similarity is greater than the first similarity threshold and whose image feature similarity is ranked first are used as similar video frames to any one of the video frames to be processed.

4. The method according to claim 1, characterized in that The image feature similarity between the to-be-processed video frame and the reference video frame is a similarity determined based on the image features of the to-be-processed video frame and the reference video frame, and the image features of the to-be-processed video frame and the reference video frame are obtained by performing image feature extraction processing on the optimized image feature extraction model; The method further comprises: Acquire training samples; the training samples include sample video frames, annotated similar video frames corresponding to the sample video frames, and annotated difference video frames corresponding to the sample video frames; Calling an image feature extraction model to respectively extract image features of the sample video frame, image features of the annotated similar video frame, and image features of the annotated difference video frame; Calculating a model loss value based on a feature distance between an image feature of the sample video frame and an image feature of the annotated similar video frame, and a feature distance between an image feature of the sample video frame and an image feature of the annotated difference video frame; The model parameters of the image feature extraction model are optimized in the direction of reducing the model loss value to obtain the optimized image feature extraction model.

5. The method according to claim 4, characterized in that The obtaining of training samples comprises: Selecting a video frame from the sample video as a sample video frame; Calling the image feature extraction model to extract image features of the sample video frame and extracting image features of each reference video frame; Determining the image feature similarity between the sample video frame and each reference video frame based on the image feature of the sample video frame and the image feature of each reference video frame; Based on the image feature similarity between the sample video frame and each of the reference video frames, taking a reference video frame whose image feature similarity is greater than a second similarity threshold among the L reference video frames as a recalled video frame corresponding to the sample video frame; Adding the recalled video frames with similar content backgrounds and similar content bodies among the recalled video frames corresponding to the sample video frames to the first video frame set, and adding the recalled video frames with similar content backgrounds but different content bodies among the recalled video frames corresponding to the sample video frames to the second video frame set; Any recalled video frame in the first video frame set is selected as the labeled similar video frame, and any recalled video frame in the second video frame set is selected as the labeled difference video frame, and the training sample is constructed based on the sample video frame, the labeled similar video frame and the labeled difference video frame.

6. The method according to claim 1, characterized in that The method further comprises: Acquire the image features of each to-be-processed video frame; the image features of each to-be-processed video frame are extracted by a first feature extraction device among a plurality of feature extraction devices, the first feature extraction device is a feature extraction device that receives the target video through a message queue, and the target video is uploaded to the message queue by a video distribution device; Obtaining image features of each reference video frame; the image features of any reference video frame are extracted by a second feature extraction device among a plurality of feature extraction devices, the second feature extraction device is a feature extraction device that receives a reference video to which any reference video frame belongs through the message queue, and the reference video to which any reference video frame belongs is uploaded to the message queue by the video distribution device; Based on the image features of each to-be-processed video frame and the image features of each reference video frame, the image feature similarity between each to-be-processed video frame and each reference video frame is determined.

7. A video duplication checking device, characterized in that: The video duplication checking device includes an input interface and an output interface, and also includes: a processor adapted to implement one or more instructions; and, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the video duplication detection method as described in any one of claims 1-6.

8. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, which, when executed by a processor, are used to execute the video duplication checking method as described in any one of claims 1-6.

9. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, is used to load and execute the video duplication detection method as described in any one of claims 1-6.

Images