Video matching method, device and system, electronic equipment and storage medium

Abstract

The invention provides a video matching method and device, electronic equipment, a storage medium and a system, and relates to the technical field of computers. The video matching method comprises the following steps: extracting a first video frame from a first video by using a preset sampling algorithm; extracting a second video frame from the second video by using a preset sampling algorithm; wherein the first video and the second video correspond to different coding parameters; determining a hash value of each first video frame and a hash value of each second video frame; and determining whether the contents of the first video and the second video are consistent or not according to the hash value of each first video frame and the hash value of each second video frame. According to the invention, the matching task of massive video files can be accurately and efficiently executed.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a video matching method, apparatus, electronic device, storage medium, and system. Background Technology

[0002] With the rapid development of internet technology, the amount of video data has exploded. This trend poses a severe challenge to video management, retrieval, and copyright protection: on the one hand, traditional manual review methods are inefficient and unable to cope with massive amounts of data, resulting in long content review cycles and high costs; on the other hand, existing automated technologies have significant shortcomings. For example, while file hash-based matching methods can quickly identify completely identical videos, they are helpless against content that is "visually similar but with different files" due to differences in encoding parameters (such as different resolutions and bitrates) or minor editing (such as adding watermarks or cutting clips).

[0003] Therefore, how to accurately and efficiently perform matching tasks for massive amounts of video files is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] This invention provides a video matching method, apparatus, electronic device, storage medium, and system to overcome the aforementioned deficiencies in the prior art, thereby enabling accurate and efficient execution of matching tasks for massive amounts of video files.

[0005] This invention provides a video matching method, comprising the following steps.

[0006] Using a preset sampling algorithm, a first video frame is extracted from a first video; using the same preset sampling algorithm, a second video frame is extracted from a second video; wherein the first video and the second video correspond to different encoding parameters; the hash value of each first video frame and the hash value of each second video frame are determined; based on the hash values ​​of each first video frame and the hash values ​​of each second video frame, it is determined whether the content of the first video and the second video is consistent.

[0007] According to a video matching method provided by the present invention, determining whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and each second video frame includes: obtaining a first hash vector based on the hash values ​​of each first video frame; obtaining a second hash vector based on the hash values ​​of each second video frame; and determining whether the content of the first video and the second video is consistent based on the first hash vector and the second hash vector.

[0008] According to a video matching method provided by the present invention, determining whether the content of the first video and the second video is consistent based on the first hash vector and the second hash vector includes: Determine the similarity between the first hash vector and the second hash vector; based on the similarity, determine whether the content of the first video and the second video is consistent.

[0009] According to a video matching method provided by the present invention, determining the similarity between the first hash vector and the second hash vector includes: calculating the difference between the first hash vector and the second hash vector; determining whether the content of the first video and the second video is consistent based on the similarity includes: if the difference is less than a preset threshold, determining that the content of the first video and the second video is consistent.

[0010] According to a video matching method provided by the present invention, the step of extracting a first video frame from a first video using a preset sampling algorithm includes: The first video frame is sampled sequentially from the first video according to the interval frame number; the second video frame is extracted from the second video using the preset sampling algorithm, which includes: sampling the second video frame sequentially from the second video according to the interval frame number.

[0011] According to a video matching method provided by the present invention, the method further includes: extracting a video from a first video set as a first video; determining the number of video frames contained in the first video; and filtering at least one second video from a second video set based on the number of video frames contained in the first video.

[0012] The present invention also provides a video matching device, comprising the following modules: A first sampling module is used to extract a first video frame from a first video using a preset sampling algorithm; a second sampling module is used to extract a second video frame from a second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters; a first determining module is used to determine the hash value of each first video frame and the hash value of each second video frame; a second determining module is used to determine whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and the hash values ​​of each second video frame.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the video matching method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video matching method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the video matching method as described above.

[0016] The present invention also provides a video matching system, the system comprising a server and multiple clients; the server uses a preset hash algorithm to determine matching videos in two sets of videos to be matched; removes the determined matching videos from the two sets of videos to be matched, obtaining a first set of videos and a second set of videos; and sends the first set of videos and the second set of videos to the clients, so that the clients, for the first set of videos and the second set of videos, execute the method as described in claim 6 to continue filtering out matching videos from the first set of videos and the second set of videos.

[0017] The video matching method, apparatus, electronic device, storage medium, and system provided by this invention utilize a preset sampling algorithm to extract a first video frame from a first video and a second video frame from a second video (wherein the first and second videos correspond to different encoding parameters), thereby effectively sampling video content with different encoding parameters. By determining the hash values ​​of each first and second video frame and further comparing these hash values, it is possible to accurately determine whether the content of the first and second videos is consistent, thus solving the problem that video content cannot be directly matched through file hashes due to different encoding parameters. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the video matching method provided by the present invention.

[0020] Figure 2 This is the second flowchart of the video matching method provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the architecture of the video matching system provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the video matching device provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined Figures 1-3 The video matching method of the present invention is described.

[0026] Figure 1 This is one of the flowcharts illustrating the video matching method provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Extract the first video frame from the first video using a preset sampling algorithm.

[0027] Preset sampling algorithms are methods for selecting specific frames from a video according to predetermined rules, aiming to reduce computation and improve matching efficiency.

[0028] In some embodiments, the first video frames may be sampled sequentially from the first video according to the interval frame number.

[0029] In the specific implementation process, the sampling ratio and the number of frames at intervals can be determined first. For example, it can be set that one frame is extracted every 10 frames as a sampling frame. Then, based on the total number of frames in the video, the number of frames to be extracted is calculated. Starting from the first frame of the first video, each frame is checked according to the set number of frames at intervals, and frames that meet the interval conditions are extracted to obtain the set of the first video frames. For example, if the total number of frames in the video is 1000, and one frame is extracted every 10 frames, then a total of 100 frames are extracted as the first video frames.

[0030] Step 102: Extract the second video frame from the second video using a preset sampling algorithm.

[0031] The first and second videos use different encoding parameters. Encoding parameters refer to the parameters used during the video encoding process, such as frame rate, bit rate, and resolution. Different encoding parameters will result in different binary data in the video files, but the video content may be the same.

[0032] In some embodiments, second video frames may be sampled sequentially from the second video according to the interval frame number.

[0033] In the specific process, firstly, the sampling ratio and frame interval for the second video are determined. This ratio and frame interval must be the same as the sampling ratio and frame interval for the first video. Then, based on the total number of frames in the second video, the number of frames to be extracted is calculated. Starting from the first frame of the second video, each frame is checked according to the set frame interval, and frames that meet the interval conditions are extracted as the set of second video frames. For example, if the second video has a total of 1000 frames, and one frame is extracted every 10 frames, then a total of 100 frames are extracted as the second video frames.

[0034] Step 103: Determine the hash value of each first video frame and the hash value of each second video frame.

[0035] A hash value is a fixed-length binary value calculated from a video frame using a specific algorithm. It is used to uniquely identify the content characteristics of a video frame.

[0036] In the specific implementation process, for the set of first video frames and the set of second video frames extracted in steps 101 and 102, the same hash algorithm (e.g., MD5, SHA-1, etc.) is used to calculate the hash value of each video frame.

[0037] For example, for each frame in the set of first video frames, its MD5 hash value is calculated to obtain the hash value of the first video frame; similarly, for each frame in the set of second video frames, its MD5 hash value is also calculated to obtain the hash value of the second video frame.

[0038] In practice, the hash values ​​of the first video frame and each of the second video frames can be pre-calculated and stored in a database, and can be directly retrieved from the database.

[0039] Step 104: Determine whether the content of the first video and the second video is consistent based on the hash value of each first video frame and the hash value of each second video frame.

[0040] In some embodiments, a first hash vector can be obtained based on the hash value of each first video frame; a second hash vector can be obtained based on the hash value of each second video frame; and the content of the first video and the second video can be determined to be consistent based on the first hash vector and the second hash vector.

[0041] In the specific implementation process, the hash values ​​of all the first video frames can be combined into a vector to obtain the first hash vector; the hash values ​​of all the second video frames can be combined into a vector to obtain the second hash vector.

[0042] Next, the similarity between the first hash vector and the second hash vector is determined; based on the similarity, it is determined whether the content of the first video and the second video are consistent.

[0043] For example, the difference between the first hash vector and the second hash vector can be calculated; if the difference is less than a preset threshold, it is determined that the content of the first video and the second video are consistent.

[0044] Figure 2 This is the second flowchart illustrating the video matching method provided by the present invention, as shown below. Figure 2 As shown, the method includes the following: Step 201: Extract one video from the first video set and use it as the first video.

[0045] A video collection refers to a set of video files collected according to certain rules. For example, the first video collection could be a set of movie clips downloaded from website A.

[0046] In the specific implementation process, all video files in the first video set can be traversed; one video file can be extracted using a random selection algorithm or a sequential selection algorithm; and the extracted video file can be used as the first video for subsequent processing.

[0047] Step 202: Determine the number of video frames contained in the first video.

[0048] The number of video frames refers to the number of still image frames contained in a video file, used to measure the length or richness of the video content. For example, a 10-second video with 30 frames per second has a total of 300 frames.

[0049] Step 203: Based on the number of video frames contained in the first video, select at least one second video from the second video set.

[0050] The second video set refers to a collection of video files that differs from the first video set but may contain similar content, gathered according to certain rules. For example, the second video set could be a collection of movie clips downloaded from website B.

[0051] In the specific implementation process, all video files in the second video set can be traversed; for each video file, the number of video frames it contains is loaded using a video processing library and calculated; the number of video frames is compared with the number of video frames in the first video; video files with the same number of video frames are selected as the second video; the selection result is recorded as at least one second video for subsequent content comparison with the first video.

[0052] Step 204: Extract the first video frame from the first video using a preset sampling algorithm.

[0053] Step 205: Extract the second video frame from the second video using a preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters.

[0054] Step 206: Determine the hash value of each first video frame and the hash value of each second video frame; Step 207: Determine whether the content of the first video and the second video is consistent based on the hash value of each first video frame and the hash value of each second video frame.

[0055] For details on the specific implementation methods and technical effects of steps 204 to 207, please refer to the relevant content in steps 101 to 104, which will not be repeated here.

[0056] In the embodiments provided by this invention, a video is first extracted from a first video set as the first video, and the number of video frames it contains is determined. Based on this, at least one second video with the same number of video frames is selected from a second video set, achieving preliminary and efficient screening of large-scale video data based on the number of video frames. Subsequently, video frames are extracted from the first and second videos using the same preset sampling algorithm, ensuring consistency and comparability of sampling even if the two videos correspond to different encoding parameters. Furthermore, by calculating the hash value of each sampled video frame and determining whether the content of the first and second videos is consistent based on these hash values, the problem of traditional content comparison methods failing due to encoding differences is effectively solved. This provides strong technical support for applications such as copyright protection, content review, and similar video retrieval of video content.

[0057] The following is combined Figure 3 The video matching system of the present invention is described.

[0058] Figure 3 This is a schematic diagram of the architecture of the video matching system provided by the present invention.

[0059] like Figure 3 As shown, the video matching system 300 includes a server 310 and multiple clients: client 320, client 321, client 322… Any one of the server 310 and the multiple clients performs the following interactive operation: The server uses a preset hash algorithm to determine the matching video from two sets of videos to be matched.

[0060] In practice, a pre-defined hash algorithm can be used to calculate the hash value of the video data to compare the consistency of any two video contents. Pre-defined hash algorithms may include, but are not limited to, MD5, perceptual hashing, etc.

[0061] The server can load two sets of videos to be matched; for each video in each set, a preset hash algorithm is applied to calculate its hash value; the calculated hash values ​​are compared, and videos with the same or similar hash values ​​are identified as matching videos; the identification information of these matching videos is recorded.

[0062] In the specific implementation process, the server can use the hash calculation module to generate the file hash and video frame hash value of each video in the video set to be matched in batch processing, and store the calculated hash value in the database for subsequent efficient matching.

[0063] For example, batch hash calculations can be implemented using the UrlVideoHash class. After loading a video file, this class generates file hashes and frame hashes in parallel. The file hashes are directly stored in the database, while the frame hashes generate hash vectors based on the video ID and frame sequence number and are updated to a distributed cache (such as Redis). Batch processing optimizes IO and computing resources, improving the processing efficiency of a single node.

[0064] The videos that are determined to be matched are removed from the two sets of videos to be matched, resulting in the first set of videos and the second set of videos.

[0065] In the specific implementation process, the server, based on the matching video identification information recorded above, traverses the two sets of videos to be matched, locates and deletes these matching videos, and identifies the sets after deleting the matching videos as the first video set and the second video set, respectively, for subsequent processing.

[0066] The first video set and the second video set are sent to the client, so that the client can perform the following actions on the first video set and the second video set: Figure 2 The method is to continue filtering out matching videos from the first video set and the second video set.

[0067] A client is a computing device or software that receives data sent by a server and performs specific tasks.

[0068] In the specific implementation process, the server packages the data of the first video set and the second video set via a network connection; it then sends the packaged data to the designated client using a data transmission protocol; the client receives the data packets and unpacks them to recover the first video set and the second video set; the client then performs the following actions on these two video sets: Figure 2 The method is used to continue filtering out matching videos; the client sends the filtering results back to the server, completing the entire video matching process.

[0069] In the specific implementation process, the client can use the matching module (for example, implemented using the VideoMatch class) to perform a comparison between the first video and the second video based on the hash library (which stores the hash values ​​of the calculated video frames) (steps 101 to 104), identify duplicate or similar video content, and support exact matching (same hash value) and fuzzy matching (hash difference less than the threshold).

[0070] In the embodiments provided by this invention, hash calculation tasks are distributed to multiple nodes through a client-server architecture (TaskServer / TaskClient), and the results are collected and aggregated, which can effectively solve the performance bottleneck of a single node and support massive video processing.

[0071] The video matching device provided by the present invention is described below. The video matching device described below and the video matching method described above can be referred to in correspondence.

[0072] Figure 4 This is a schematic diagram of the video matching device provided by the present invention. Figure 4 As shown, the video matching device 400 includes the following modules.

[0073] The first sampling module 410 is used to extract the first video frame from the first video using a preset sampling algorithm.

[0074] The second sampling module 420 is used to extract a second video frame from the second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters.

[0075] The first determining module 430 is used to determine the hash value of each first video frame and the hash value of each second video frame.

[0076] The second determining module 440 is used to determine whether the content of the first video and the second video is consistent based on the hash value of each first video frame and the hash value of each second video frame.

[0077] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a video matching method, which includes: extracting a first video frame from a first video using a preset sampling algorithm; extracting a second video frame from a second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters; determining the hash value of each first video frame and the hash value of each second video frame; and determining whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and the hash values ​​of each second video frame.

[0078] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video matching method provided by the above methods. The method includes: extracting a first video frame from a first video using a preset sampling algorithm; extracting a second video frame from a second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters; determining the hash value of each first video frame and the hash value of each second video frame; and determining whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and the hash values ​​of each second video frame.

[0080] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the video matching method provided by the above methods. The method includes: extracting a first video frame from a first video using a preset sampling algorithm; extracting a second video frame from a second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters; determining the hash value of each first video frame and the hash value of each second video frame; and determining whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and the hash values ​​of each second video frame.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video matching method, characterized in that, include: The first video frame is extracted from the first video using a preset sampling algorithm; Using the preset sampling algorithm, a second video frame is extracted from the second video; wherein the first video and the second video correspond to different encoding parameters; Determine the hash value of each first video frame and the hash value of each second video frame; Based on the hash values ​​of each first video frame and each second video frame, determine whether the content of the first video and the second video is consistent.

2. The video matching method according to claim 1, characterized in that, Determining whether the content of the first video and the second video is consistent based on the hash values ​​of each first video frame and each second video frame includes: The first hash vector is obtained based on the hash value of each first video frame; The second hash vector is obtained based on the hash value of each second video frame; Based on the first hash vector and the second hash vector, determine whether the content of the first video and the second video are consistent.

3. The video matching method according to claim 2, characterized in that, The step of determining whether the content of the first video and the second video are consistent based on the first hash vector and the second hash vector includes: Determine the similarity between the first hash vector and the second hash vector; Based on the similarity, it is determined whether the content of the first video and the second video are consistent.

4. The video matching method according to claim 3, characterized in that, Determining the similarity between the first hash vector and the second hash vector includes: Calculate the difference between the first hash vector and the second hash vector; The step of determining whether the content of the first video and the second video is consistent based on the similarity includes: If the difference is less than a preset threshold, it is determined that the content of the first video and the second video are consistent.

5. The video matching method according to any one of claims 1 to 4, characterized in that, The step of extracting the first video frame from the first video using a preset sampling algorithm includes: The first video frames are sampled sequentially from the first video according to the interval frame number; The step of extracting a second video frame from the second video using the preset sampling algorithm includes: The second video frames are sampled sequentially from the second video according to the stated interval frame number.

6. The video matching method according to claim 5, characterized in that, The method further includes: Extract one video from the first video set and use it as the first video; Determine the number of video frames contained in the first video; Based on the number of video frames contained in the first video, at least one second video is selected from the second video set.

7. A video matching device, characterized in that, include: The first sampling module is used to extract the first video frame from the first video using a preset sampling algorithm; The second sampling module is used to extract a second video frame from the second video using the preset sampling algorithm; wherein the first video and the second video correspond to different encoding parameters; The first determining module is used to determine the hash value of each first video frame and the hash value of each second video frame; The second determining module is used to determine whether the content of the first video and the second video is consistent based on the hash value of each first video frame and the hash value of each second video frame.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the video matching method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the video matching method as described in any one of claims 1 to 6.

10. A video matching system, characterized in that, The system includes a server and multiple clients; The server uses a preset hash algorithm to determine the matching video in the two sets of videos to be matched. Remove the videos that are determined to be matched from the two sets of videos to be matched, and obtain the first set of videos and the second set of videos; The first video set and the second video set are sent to the client, so that the client performs the method of claim 6 on the first video set and the second video set to continue filtering out matching videos from the first video set and the second video set.

Images