Video processing method and device, electronic equipment and medium

Abstract

The present disclosure provides a video processing method and device, electronic equipment and medium, relates to the technical field of artificial intelligence, and in particular to the field of intelligent search. It can be applied to a video search scene or a video recommendation scene. The specific implementation scheme is: selecting a target video clustering cluster to which a to-be-identified video belongs from a candidate video clustering cluster; wherein the candidate video clustering cluster is obtained by clustering at least two candidate videos; and determining whether at least two to-be-identified videos are repeated according to the target video clustering cluster to which the at least two to-be-identified videos belong. The present disclosure can improve the timeliness and accuracy of video duplication checking.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the field of intelligent search, which can be applied to video search or video recommendation scenarios. Background Technology

[0002] With the development of information technology, especially the rapid development of the Internet, video production has become increasingly easier, and video has gradually become the mainstream means of information dissemination.

[0003] In the fields of video search and video recommendation, massive amounts of videos need to be processed, many of which contain duplicates. Duplicate videos not only consume additional storage and computing resources but also negatively impact user experience. Summary of the Invention

[0004] This disclosure provides a video processing method, apparatus, electronic device, and medium.

[0005] According to one aspect of this disclosure, a video processing method is provided, the method comprising:

[0006] Select the target video cluster to which the video to be identified belongs from the candidate video clusters; wherein, the candidate video clusters are obtained by clustering at least two candidate videos;

[0007] Determine whether the at least two videos to be identified are duplicates based on the target video clusters to which they belong.

[0008] According to another aspect of this disclosure, a video processing apparatus is provided, comprising:

[0009] The video clustering cluster determination module is used to select the target video cluster to which the video to be identified belongs from the candidate video clusters; wherein, the candidate video clusters are obtained by clustering at least two candidate videos;

[0010] The duplicate video recognition module is used to determine whether the at least two videos to be recognized are duplicates based on the target video cluster to which they belong.

[0011] According to another aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0012] At least one processor; and

[0013] A memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the video processing method described in any embodiment of this disclosure.

[0015] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the video processing method described in any embodiment of this disclosure.

[0016] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the video processing method described in any embodiment of this disclosure.

[0017] The technology disclosed herein can improve the timeliness and accuracy of video plagiarism detection.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0020] Figure 1 This is a flowchart of a video processing method provided according to an embodiment of the present disclosure;

[0021] Figure 2 This is a flowchart of another video processing method provided according to an embodiment of the present disclosure;

[0022] Figure 3 This is a flowchart of another video processing method provided according to an embodiment of the present disclosure;

[0023] Figure 4 This is a flowchart of another video processing method provided according to an embodiment of the present disclosure;

[0024] Figure 5A This is a flowchart of another video processing method provided according to an embodiment of the present disclosure;

[0025] Figure 5B A flowchart of an image frame matching path determination method provided according to an embodiment of this disclosure;

[0026] Figure 6 This is a schematic diagram of the structure of a video processing apparatus provided according to an embodiment of the present disclosure;

[0027] Figure 7This is a block diagram of an electronic device used to implement the video processing method of the embodiments of this disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Figure 1 This is a flowchart illustrating a video processing method according to an embodiment of this disclosure. This embodiment is applicable to situations involving duplicate video recognition in video search or video recommendation scenarios. The method can be executed by a video processing device, which can be implemented in software and / or hardware and integrated into an electronic device that performs video processing functions. Figure 1 As shown, the video processing method in this embodiment may include:

[0030] S101, Select the target video cluster to which the video to be identified belongs from the candidate video clusters; wherein, the candidate video clusters are obtained by clustering at least two candidate videos;

[0031] S102, determine whether the at least two videos to be identified are duplicates based on the target video clusters to which the at least two videos to be identified belong.

[0032] The videos to be identified have not yet undergone video deduplication. Video deduplication is used to determine whether two videos are duplicates. Optionally, two videos with similar content but different sources are considered duplicates. The number of videos to be identified is determined based on actual business needs and is not limited here. Optionally, there are generally at least two videos to be identified.

[0033] Candidate video clusters are obtained by clustering at least two candidate videos. These candidate videos can be collected from the internet. Optionally, candidate video clusters are pre-constructed based on video repetition relationships between candidate videos. These repetition relationships are used to determine whether candidate videos from different sources are repetitive. Optionally, candidate videos that are repetitive are grouped into the same video cluster, resulting in candidate video clusters. Optionally, a topology graph is drawn, using candidate videos as nodes and connecting repetitive candidate videos with undirected edges. Candidate video clusters are determined based on the node connections in the topology graph.

[0034] The target video cluster refers to the candidate video cluster to which the candidate video that is a duplicate of the video to be identified belongs.

[0035] Based on the target video clusters to which at least two videos to be identified belong, it is determined whether the videos to be identified belong to the same video cluster. If the two videos to be identified belong to the same video cluster, then it can be determined that the two videos to be identified are duplicates.

[0036] Optionally, after clustering at least two candidate videos to obtain candidate video clusters, a unique identifier is assigned to each candidate video cluster to distinguish different video clusters.

[0037] Optionally, the target video cluster to which the video to be identified belongs can be determined by identifying the unique identifier of the video cluster to which it belongs. Optionally, whether different videos to be identified are duplicates can be determined by determining whether the unique identifiers of the video clusters to which the videos to be identified belong are the same. Specifically, if the unique identifiers of the video clusters to which they belong are the same, then the videos to be identified are determined to be duplicate videos.

[0038] Optionally, the video to be identified can be a candidate response result in a video recommendation scenario or a video search scenario, responding to a video recommendation request or a video search operation. Based on the target video cluster to which the video to be identified belongs, it is determined whether at least two videos to be identified are duplicates. Optionally, if the videos to be identified are duplicates, deduplication processing is performed.

[0039] In related technologies, image frame features of the video to be identified are typically extracted. By comparing these features and based on a pre-set threshold for the number of matching frames, it is determined whether the videos are duplicates. However, the methods provided by these technologies for video deduplication suffer from low coverage. In the multimedia era of widespread video, the total number of videos is countless, making it difficult to find duplicates within such a massive dataset. Therefore, these technologies are ill-suited for deduplication of large volumes of videos. Especially with new videos added daily, these technologies struggle to consistently generate duplicate video relationships.

[0040] This disclosure, through pre-constructing video clusters and maintaining them, updates the video clusters even when new videos are added, ensuring the continuous generation of video duplication relationships. This can be used for deduplication of massive amounts of video data. Furthermore, this disclosure selects a target video cluster from candidate video clusters to determine whether at least two videos to be identified are duplicates based on the target video clusters to which they belong. This reduces the complexity of video deduplication and improves the timeliness and accuracy of video deduplication detection.

[0041] Figure 2This is a flowchart of another video processing method provided according to an embodiment of the present disclosure; this embodiment is an optional solution proposed based on the above embodiments. Specifically, the present disclosure embodiment refines the operation of "selecting the target video cluster to which the video to be identified belongs from the candidate video clusters".

[0042] See Figure 2 The video processing method provided in this embodiment includes:

[0043] S201, Based on the video global features of the video to be identified, determine the recall video that matches the video to be identified from the at least two candidate videos.

[0044] The video to be identified extracts both global video features and image frame features. Optionally, the global video features are obtained by extracting features from the entire video using a video understanding model. The video understanding model is selected based on actual business needs and is not limited here. For example, a Time Sensitive Networking (TSN) model can be used. Optionally, keyframes are first extracted from the video to be identified using a keyframe extraction algorithm, and then image frame features are extracted from the keyframes using an image feature extraction model. For example, a convolutional neural network can be used.

[0045] Video global features are obtained by extracting features from the entire video. Video global features refer to features at the video granularity level; they represent the overall feature representation of the video. This differs from image frame features, which are obtained by extracting features from keyframes in the video. Image frame features are frame-level features; they represent the feature representation of keyframes in the video. Video global features include inter-frame features, which are the content variation features between adjacent image frames.

[0046] In this context, the recalled video refers to the candidate video that matches the video to be identified. The global video features of the recalled video are matched with the global video features of the video to be identified.

[0047] Optionally, feature matching is performed between the global video features of the video to be identified and the global video features of the candidate videos to determine the recall videos that match the video to be identified from the candidate videos. The recalled videos may or may not be identical to the video to be identified. Determining whether the recalled videos are identical to the video to be identified requires further refined feature comparison.

[0048] As we know, the number of candidate videos is massive, often in the hundreds of millions. Utilizing global video features for initial screening of candidate videos yields recalled videos. This significantly reduces the resource consumption required for subsequent refined feature comparisons.

[0049] S202, determine the image frame matching relationship between the video to be identified and the video to be recalled based on the image frame features of the video to be identified and the image frame features of the video to be recalled.

[0050] The image frame matching relationship is determined based on the image frame similarity between key frames in the video to be identified and key frames in the recalled video.

[0051] Optionally, the image frame features of the video to be identified and the image frame features of the recalled video are compared to determine the image frame matching relationship between the video to be identified and the recalled video.

[0052] S203, Based on the image frame matching relationship, determine the video duplication relationship between the recalled video and the video to be identified.

[0053] When there are a massive number of candidate videos, the number of videos selected for recall is usually multiple. Based on the image frame matching relationship, similar image frames in each recalled video that match the keyframes in the video to be identified, as well as the relative positions of the similar image frames in the video to be identified and the recalled videos, can be determined.

[0054] Based on similar image frames and their relative positions in the video to be identified and the recalled video, the video duplication relationship between the recalled video and the video to be identified is determined.

[0055] S204, based on the video repetition relationship and the video cluster to which the recalled video belongs, determine the target video cluster to which the video to be identified belongs.

[0056] Each recalled video belongs to a specific video cluster, and each recalled video has its own video cluster.

[0057] Based on video duplication relationships, it can be determined whether the recalled video and the video to be identified are duplicates of each other. Reference videos that are duplicates of the video to be identified are selected from the recalled videos. Based on the video clusters to which the reference videos belong, the target video cluster to which the video to be identified belongs is determined.

[0058] Optionally, if it is determined that the reference video and the video to be identified are duplicates, the video to be identified is added to the video cluster to which the reference video belongs. The video clusters are then updated to ensure the continuous generation of video duplication relationships, making it suitable for deduplication of massive amounts of videos.

[0059] S205, determine whether the at least two videos to be identified are duplicates based on the target video clusters to which the at least two videos to be identified belong.

[0060] Based on the above steps, the target video cluster to which each video to be identified belongs can be determined. By comparing whether the target video clusters to which each video to be identified belongs are consistent, it can be determined whether any two videos to be identified are duplicates.

[0061] This disclosed technical solution, by initially screening candidate videos based on the global video features at the video granularity level, determines the recall video that matches the video to be identified from the candidate videos, greatly reducing the resource consumption required for subsequent fine feature matching, and is conducive to improving the timeliness of video deduplication; this disclosed technical solution, by performing fine feature comparison between the video to be identified and the recall video based on the image frame features at the frame granularity level, is conducive to ensuring the accuracy of video deduplication.

[0062] Figure 3 This is a flowchart of another video processing method provided according to an embodiment of the present disclosure; this embodiment is an optional solution proposed based on the above embodiments. Specifically, the present disclosure embodiment refines the operation of "determining a recall video that matches the video to be identified from at least two candidate videos based on the video global features of the video to be identified".

[0063] See Figure 3 The video processing method provided in this embodiment includes:

[0064] S301, determine the bucket center of the candidate video bucket based on the global video features of the candidate videos in the candidate video bucket.

[0065] It is known that the number of candidate videos is massive. To save resources during video retrieval, alternatively, candidate videos can be binned based on their global video features. Specifically, candidate videos with similar global video features are grouped into the same video bin, resulting in candidate video bins.

[0066] The data format of the video global features is a vector. Optionally, the bin center of the candidate video bins is obtained by averaging the video global features of the candidate videos in the candidate video bins.

[0067] S302, Based on the global video features of the video to be identified, determine the relative distance between the bin center of the candidate video bin and the video to be identified.

[0068] The bin center for each candidate video bin is determined based on the global video features of the candidate videos within that bin. The bin center, from the perspective of global video features, reflects the common characteristics of the candidate videos within that bin.

[0069] The relative distance between the bin center of the candidate video bin and the video to be identified can reflect the feature similarity between the video to be identified and the candidate videos in the candidate video bin from the perspective of global video features.

[0070] The relative distance is inversely proportional to the feature similarity between the video to be identified and the candidate videos in the candidate video bucket. The smaller the relative distance, the greater the feature similarity between the video to be identified and the candidate videos in the candidate video bucket.

[0071] S303, determine the target video bucket for the video to be identified from the candidate video buckets according to the relative distance, and determine the candidate videos belonging to the target video bucket as the recall videos that match the video to be identified.

[0072] The candidate video buckets are sorted according to their relative distances. The n candidate video buckets with the smallest relative distances are selected as the target video buckets. Here, n is a positive integer.

[0073] Candidate videos belonging to the target video bucket are identified as recall videos that match the video to be identified.

[0074] S304, determine the image frame matching relationship between the video to be identified and the video to be recalled based on the image frame features of the video to be identified and the image frame features of the video to be recalled.

[0075] Optionally, the candidate video buckets store image frame features of the candidate videos. The image frame features of the recalled video are obtained from the target video bucket to which the recalled video belongs. Based on the image frame features of the video to be identified and the image frame features of the recalled video, the image frame matching relationship between the video to be identified and the recalled video is determined.

[0076] S305, Based on the image frame matching relationship, determine the video duplication relationship between the recalled video and the video to be identified.

[0077] S306, Based on the video repetition relationship and the video cluster to which the recalled video belongs, determine the target video cluster to which the video to be identified belongs.

[0078] S307, determine whether the at least two videos to be identified are duplicates based on the target video clusters to which the at least two videos to be identified belong.

[0079] The technical solution disclosed herein divides candidate videos into buckets based on their global video features. Video retrieval is then performed based on these buckets, and matching recall videos are determined at the bucket granularity for the video to be identified. This effectively saves resources during the video retrieval process and improves the timeliness of video deduplication.

[0080] Figure 4 This is a flowchart of another video processing method provided according to an embodiment of the present disclosure; this embodiment is an optional solution proposed based on the above embodiments. Specifically, the present disclosure embodiment refines the operation of "determining the image frame matching relationship between the video to be identified and the recalled video based on the image frame features of the video to be identified and the image frame features of the recalled video".

[0081] See Figure 4 The video processing method provided in this embodiment includes:

[0082] S401, based on the video global features of the video to be identified, determine the recall video that matches the video to be identified from the at least two candidate videos.

[0083] S402, determine the image frame similarity between the video to be identified and the video to be recalled based on the image frame features of the video to be identified and the image frame features of the video to be recalled.

[0084] Based on frame-level image frame features, a refined feature comparison is performed between the video to be identified and the recalled video to determine the image frame similarity between the two videos.

[0085] S403, Based on the image frame similarity and the similarity threshold, determine similar image frames from the video to be identified and the recalled video.

[0086] The similarity threshold is used to measure the similarity between keyframes in the video to be identified and keyframes in the recalled video. Optionally, the image frame similarity is compared with the similarity threshold to obtain similar image frames. Image frames with similar similarity values ​​are greater than the similarity threshold.

[0087] S404, determine the image frame matching relationship between the video to be identified and the video to be recalled based on the relative positions of the similar image frames in the video to be identified and the video to be recalled.

[0088] The relative positions of similar image frames in the target video and the recalled video can determine the temporal characteristics of similar image frames in both videos. Based on these temporal characteristics, the matching duration of the target video and the recalled video can be determined.

[0089] S405, Based on the image frame matching relationship, determine the video duplication relationship between the recalled video and the video to be identified.

[0090] The matching duration between the recalled video and the video to be identified can be determined based on the image frame matching relationship. Optionally, the matching duration between the recalled video and the video to be identified is compared with a preset duration threshold. Based on the duration comparison result, the video duplication relationship between the recalled video and the video to be identified is determined, thus determining whether the recalled video duplicates the video to be identified.

[0091] Optionally, recall videos with a matching duration exceeding a preset duration threshold can be identified as duplicate videos of the video to be identified.

[0092] S406, Based on the video repetition relationship and the video cluster to which the recalled video belongs, determine the target video cluster to which the video to be identified belongs.

[0093] S407, determine whether the at least two videos to be identified are duplicates based on the target video clusters to which the at least two videos to be identified belong.

[0094] This disclosed technical solution determines the image frame similarity between the video to be identified and the recalled video based on image frame features. According to the image frame similarity and similarity threshold, similar image frames are identified from the video to be identified and the recalled video. Based on the relative positions of similar image frames in the video to be identified and the recalled video, the image frame matching relationship between the video to be identified and the recalled video is determined. The image frame matching relationship is used for video deduplication. At the same time, the spatial features and temporal features of image frames are considered to ensure the accuracy of video deduplication.

[0095] Figure 5A This is a flowchart of another video processing method provided according to an embodiment of the present disclosure; this embodiment is an optional solution proposed based on the above embodiments. Specifically, the present disclosure embodiment refines the operation "determining the video duplication relationship between the recalled video and the video to be identified based on the image frame matching relationship".

[0096] See Figure 5A The video processing method provided in this embodiment includes:

[0097] S501, based on the video global features of the video to be identified, determine the recall video that matches the video to be identified from the at least two candidate videos.

[0098] S502, determine the image frame matching relationship between the video to be identified and the video to be recalled based on the image frame features of the video to be identified and the image frame features of the video to be recalled.

[0099] S503, based on the image frame matching relationship, determine the relative positions of similar image frames in the video to be identified and the video to be recalled, and respectively use them as the first relative position and the second relative position.

[0100] Image frame matching relationships are determined based on the relative positions of similar image frames in the video to be identified and the video to be recalled.

[0101] The first relative position refers to the relative position of similar image frames within the image to be identified. Based on the first relative position, it can be determined which frame of the video to be identified the similar image frame belongs to.

[0102] The second relative position refers to the relative position of the similar image frame in the recall video. Based on the second relative position, it can be determined which frame of the recall video the similar image frame is in.

[0103] S504, determine the image frame matching path between the video to be identified and the video to be recalled based on the first relative position and the second relative position.

[0104] Image frame matching relationships are used to map frame-level matching to video-level duration matching. Image frame matching paths are used to quantify the matching duration of the video to be identified and the recalled video.

[0105] Optionally, a dynamic programming algorithm can be used to determine the image frame matching path between the video to be identified and the video to be recalled, based on the first relative position and the second relative position.

[0106] Figure 5B A flowchart of an image frame matching path determination method provided according to an embodiment of this disclosure is shown below. Figure 5B Based on the image frame matching relationship between the video to be identified (A) and the recalled video (B), the image frame matching matrix can be determined. Where A... n Used to distinguish keyframes in video A to be identified, where n is a natural number, determined by the number of keyframes in video A, and A0 represents the first keyframe in video A. Where B... m Used to distinguish keyframes in recall video B, m is a natural number, B0 represents the first keyframe in recall video B, and m is determined based on the number of keyframes in recall video B.

[0107] The size of the image frame matching matrix is ​​determined by the number of keyframes in the target video A and the recall video B. If the target video A has 10 keyframes, the image frame matching matrix has 10 columns; if the recall video B has 8 keyframes, the image frame matching matrix has 8 rows. Therefore, the image frame matching matrix is ​​a 10-column, 8-row matrix. The matrix elements represent the image frame matching relationships. A matrix element of 1 indicates that the corresponding image frames in the target video A and the recall video B match, and are considered similar image frames. A matrix element of 0 indicates that the corresponding image frames in the target video A and the recall video B do not match.

[0108] Based on the image frame matching matrix, a dynamic programming algorithm, such as the Smith-Waterman algorithm, is used to determine the longest matching path between the two videos, video A to be identified and video B to be recalled. See also Figure 5B The longest matching path between the two videos, video A to be identified and video B to be recalled, is the image frame matching path indicated by the arrow.

[0109] S505, determine the video duplication relationship between the recalled video and the video to be identified based on the image frame matching path.

[0110] Based on the path length of the image frame matching path, the matching duration of the video to be identified and the recalled video is determined, and the video duplication relationship between the two is determined based on the matching duration. Optionally, if the matching duration exceeds a preset duration threshold, the recalled video is determined to be a duplication of the video to be identified.

[0111] S506, Based on the video repetition relationship and the video cluster to which the recalled video belongs, determine the target video cluster to which the video to be identified belongs.

[0112] S507, determine whether the at least two videos to be identified are duplicates based on the target video clusters to which the at least two videos to be identified belong.

[0113] The technical solution disclosed herein determines the relative positions of similar image frames in the video to be identified and the video to be recalled based on the image frame matching relationship. Based on the relative positions of similar video frames, it determines the image frame matching path between the video to be identified and the video to be recalled. It maps frame-level matching to video-level duration matching, so that video deduplication is not affected by the length of the video itself, thus ensuring the accuracy of video deduplication.

[0114] Figure 6This is a schematic diagram of a video processing apparatus according to an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to situations involving duplicate video recognition in video search or video recommendation scenarios. This apparatus can be implemented using software and / or hardware, and can implement the video processing method described in any embodiment of the present disclosure. Figure 6 As shown, the video processing apparatus 600 includes:

[0115] The video clustering cluster determination module 601 is used to select the target video cluster to which the video to be identified belongs from the candidate video clusters; wherein, the candidate video clusters are obtained by clustering at least two candidate videos;

[0116] The duplicate video recognition module 602 is used to determine whether the at least two videos to be recognized are duplicates based on the target video cluster to which the at least two videos to be recognized belong.

[0117] This disclosure, through pre-constructing video clusters and maintaining them, updates the video clusters even when new videos are added, ensuring the continuous generation of video duplication relationships. This can be used for deduplication of massive amounts of video data. Furthermore, this disclosure selects a target video cluster from candidate video clusters to determine whether at least two videos to be identified are duplicates based on the target video clusters to which they belong. This reduces the complexity of video deduplication and improves the timeliness and accuracy of video deduplication detection.

[0118] Optionally, the video cluster determination module 601 includes: a recall video determination submodule, configured to determine a recall video matching the video to be identified from at least two candidate videos based on the video global features of the video to be identified; an image frame matching relationship determination submodule, configured to determine an image frame matching relationship between the video to be identified and the recall video based on the image frame features of the video to be identified and the image frame features of the recall video; a video repetition relationship determination submodule, configured to determine a video repetition relationship between the recall video and the video to be identified based on the image frame matching relationship; and a video cluster determination submodule, configured to determine the target video cluster to which the video to be identified belongs based on the video repetition relationship and the video cluster to which the recall video belongs.

[0119] Optionally, the recall video determination submodule includes: a bin center determination unit, used to determine the bin center of the candidate video bin based on the video global features of the candidate videos in the candidate video bins; a relative distance determination unit, used to determine the relative distance between the bin center of the candidate video bin and the video to be identified based on the video global features of the video to be identified; and a recall video determination unit, used to determine a target video bin for the video to be identified from the candidate video bins based on the relative distance, and determine the candidate videos belonging to the target video bin as recall videos that match the video to be identified.

[0120] Optionally, the image frame matching relationship determination submodule includes: an image frame similarity determination unit, configured to determine the image frame similarity between the video to be identified and the recalled video based on the image frame features of the video to be identified and the image frame features of the recalled video; a similar image frame determination unit, configured to determine similar image frames from the video to be identified and the recalled video based on the image frame similarity and a similarity threshold; and an image frame matching relationship determination unit, configured to determine the image frame matching relationship between the video to be identified and the recalled video based on the relative positions of the similar image frames in the video to be identified and the recalled video.

[0121] Optionally, the video duplication relationship determination submodule includes: a relative position determination unit, configured to determine the relative positions of similar image frames in the video to be identified and the recalled video based on the image frame matching relationship, respectively designated as a first relative position and a second relative position; an image frame matching path determination unit, configured to determine the image frame matching path between the video to be identified and the recalled video based on the first relative position and the second relative position; and a video duplication relationship determination unit, configured to determine the video duplication relationship between the recalled video and the video to be identified based on the image frame matching path.

[0122] The video processing apparatus provided in this disclosure can execute the video processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the video processing method.

[0123] The collection, storage, use, processing, transmission, provision, and disclosure of user information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0124] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0125] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0126] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the electronic device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0127] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as video processing methods. For example, in some embodiments, the video processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform video processing methods by any other suitable means (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable video processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0133] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0134] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0135] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0136] Cloud computing refers to a technology system that enables access to a shared pool of physical or virtual resources via a network. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for applications such as artificial intelligence and blockchain, as well as for model training.

[0137] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0138] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video processing method, the method comprising: selecting a target video cluster to which a to-be-identified video belongs from candidate video cluster; wherein the candidate video cluster is obtained by clustering at least two candidate videos; determining whether at least two to-be-identified videos are duplicated according to target video clusters to which the at least two to-be-identified videos belong; wherein the selecting a target video cluster to which a to-be-identified video belongs from candidate video cluster comprises: determining a recall video matching the to-be-identified video from the at least two candidate videos according to a video global feature of the to-be-identified video; wherein the video global feature is obtained by feature extraction on a whole video; the video global feature comprises inter-frame features; determining an image frame matching relationship between the to-be-identified video and the recall video according to image frame features of the to-be-identified video and the recall video; determining a video duplication relationship between the recall video and the to-be-identified video according to the image frame matching relationship; wherein the image frame matching relationship is used to determine a matching duration between the recall video and the to-be-identified video; determining a target video cluster to which the to-be-identified video belongs according to the video duplication relationship and a video cluster to which the recall video belongs.

2. The method of claim 1, wherein, the determining a recall video matching the to-be-identified video from the at least two candidate videos according to a video global feature of the to-be-identified video comprises: determining a bucket center of a candidate video bucket according to video global features of candidate videos in the candidate video bucket; determining a relative distance between the bucket center of the candidate video bucket and the to-be-identified video according to the video global feature of the to-be-identified video; determining a target video bucket for the to-be-identified video from the candidate video bucket according to the relative distance, and determining candidate videos belonging to the target video bucket as recall videos matching the to-be-identified video.

3. The method of claim 1, wherein, the determining an image frame matching relationship between the to-be-identified video and the recall video according to image frame features of the to-be-identified video and the recall video comprises: determining an image frame similarity between the to-be-identified video and the recall video according to the image frame features of the to-be-identified video and the recall video; determining similar image frames from the to-be-identified video and the recall video according to the image frame similarity and a similarity threshold; determining an image frame matching relationship between the to-be-identified video and the recall video according to relative positions of the similar image frames in the to-be-identified video and the recall video.

4. The method of claim 1, wherein, the determining a video duplication relationship between the recall video and the to-be-identified video according to the image frame matching relationship comprises: determining relative positions of similar image frames in the to-be-identified video and the recall video as a first relative position and a second relative position respectively according to the image frame matching relationship; determining an image frame matching path of the to-be-identified video and the recall video according to the first relative position and the second relative position; According to the image frame matching relationship, a video repetition relationship between the recall video and the to-be-identified video is determined.

5. A video processing apparatus, comprising: a video cluster determination module configured to select a target video cluster to which a to-be-identified video belongs from candidate video clusters, wherein the candidate video clusters are obtained by clustering at least two candidate videos; a duplicate video identification module configured to determine whether at least two to-be-identified videos are duplicates according to target video clusters to which the at least two to-be-identified videos belong; wherein the video cluster determination module comprises a recall video determination submodule configured to determine a recall video matching the to-be-identified video from the at least two candidate videos according to a video global feature of the to-be-identified video, wherein the video global feature is obtained by performing feature extraction on a video as a whole, and the video global feature comprises inter-frame features; an image frame matching relationship determination submodule configured to determine an image frame matching relationship between the to-be-identified video and the recall video according to image frame features of the to-be-identified video and image frame features of the recall video; a video repetition relationship determination submodule configured to determine a video repetition relationship between the recall video and the to-be-identified video according to the image frame matching relationship, wherein the image frame matching relationship is used to determine a matching duration between the recall video and the to-be-identified video; a video cluster determination submodule configured to determine the target video cluster to which the to-be-identified video belongs according to the video repetition relationship and a video cluster to which the recall video belongs.

6. The apparatus of claim 5, wherein, The recall video determination submodule comprises: a bucket center determination unit configured to determine a bucket center of a candidate video bucket according to video global features of candidate videos in the candidate video bucket; a relative distance determination unit configured to determine a relative distance between the bucket center of the candidate video bucket and the to-be-identified video according to the video global feature of the to-be-identified video; a recall video determination unit configured to determine a target video bucket for the to-be-identified video from the candidate video bucket according to the relative distance, and determine candidate videos belonging to the target video bucket as the recall video matching the to-be-identified video.

7. The apparatus of claim 5, wherein, The image frame matching relationship determination submodule comprises: an image frame similarity determination unit configured to determine an image frame similarity between the to-be-identified video and the recall video according to image frame features of the to-be-identified video and image frame features of the recall video; a similar image frame determination unit configured to determine similar image frames from the to-be-identified video and the recall video according to the image frame similarity and a similarity threshold; an image frame matching relationship determination unit configured to determine an image frame matching relationship between the to-be-identified video and the recall video according to relative positions of the similar image frames in the to-be-identified video and the recall video.

8. The apparatus of claim 5, wherein, The video repetition relationship determination submodule comprises: A relative position determination unit is configured to determine relative positions of similar image frames in the video to be identified and in the recall video according to the image frame matching relationship, as a first relative position and a second relative position respectively; An image frame matching path determination unit is configured to determine an image frame matching path of the video to be identified and the recall video according to the first relative position and the second relative position; A video repetition relationship determination unit is configured to determine a video repetition relationship between the video to be identified and the recall video according to the image frame matching path. 9.An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the video processing method of any one of claims 1-4.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the video processing method of any one of claims 1-4. 11.A computer program product comprising a computer program which, when executed by a processor, implements the video processing method of any one of claims 1-4.

Images