Video detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a video detection method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring N videos to be classified; performing frame extraction on each video in the N videos based on a predetermined frame extraction interval, performing feature extraction frame by frame, and determining N video frame sequences corresponding to each video in the N videos; searching similar video frames from the first group of video frame sequences based on seed video frames in the first video frame sequence; determining the distance between the first video frame sequence and each second video frame sequence in the second group of video frame sequences to obtain a target distance matrix; and performing clustering operation on the N videos based on the target distance matrix, and determining at least one target video set. The technical problem that the accuracy of video detection is low due to the fact that the difference of video local features is not considered is solved, and the video detection method and device can be applied to various scenes such as the cloud technology, artificial intelligence, intelligent traffic and auxiliary driving.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a video detection method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] Currently, related technologies mainly classify and retrieve videos by mapping them to feature vectors. However, this classification method, which is based on the overall features of the video, is coarse-grained and cannot capture the similarity of local features in individual video frames. Therefore, during video clustering, some similar videos may be incorrectly classified into different categories, resulting in the technical problem of low accuracy in video detection.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a video detection method and apparatus, storage medium and electronic device to at least solve the technical problem of low accuracy in video detection due to failure to consider the differences in local features of the video.

[0005] According to one aspect of the embodiments of this application, a video detection method is provided, comprising: acquiring N videos to be classified, wherein N is a positive integer; extracting frames from each of the N videos based on a predetermined frame extraction interval, and extracting features frame by frame to determine N video frame sequences corresponding to each of the N videos, wherein one video frame sequence in the N video frame sequences includes a set of feature vectors, each feature vector representing a video frame in a video; and finding similar video frames from a first set of video frame sequences based on a seed video frame in a first video frame sequence, wherein the N video frame sequences consist of the first video frame sequence and the first set of video frame sequences, and the similar video frames indicate that the frame similarity between them and the seed video frames meets a preset value. The similarity criteria are used to identify video frames; the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences is determined to obtain a target distance matrix, wherein the second group of video frame sequences represents the video frame sequence to which the similar video frames belong, and the value in the i-th row and j-th column of the target distance matrix is ​​used to represent the distance between the i-th video and the j-th video in the N videos, where i and j are both positive integers less than or equal to N; based on the target distance matrix, a clustering operation is performed on the N videos to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

[0006] According to another aspect of the embodiments of this application, a video detection apparatus is also provided, comprising: an acquisition module, configured to acquire N videos to be classified, wherein N is a positive integer; a determination module, configured to extract frames from each of the N videos based on a predetermined frame extraction interval, and perform feature extraction frame by frame to determine N video frame sequences corresponding to each of the N videos, wherein one video frame sequence in the N video frame sequences includes a set of feature vectors, each feature vector representing a video frame in a video; and a query module, configured to find similar video frames from a first set of video frame sequences based on a seed video frame in a first video frame sequence, wherein the N video frame sequences consist of the first video frame sequence and the first set of video frame sequences, and the similar video frames represent the frame similarity between the seed video frame and the seed video frame. The system comprises: a video frame set satisfying a preset similarity condition; an execution module for determining the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences to obtain a target distance matrix, wherein the second group of video frame sequences represents the video frame sequence to which the similar video frames belong, and the value of the i-th row and j-th column in the target distance matrix represents the distance between the i-th video and the j-th video in the N videos, where i and j are both positive integers less than or equal to N; and a clustering module for performing clustering operations on the N videos based on the target distance matrix to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

[0007] Optionally, the device is configured to find similar video frames from a first group of video frame sequences based on seed video frames in a first video frame sequence in the following manner: each video frame sequence in the N video frame sequences is sequentially regarded as the first video frame sequence, and each video frame in the first video frame sequence is sequentially regarded as the seed video frame and the following operations are performed: obtaining a first feature vector corresponding to the seed video frame; when the preset similarity condition indicates that the M video frames with the highest similarity have been found, determining the M second feature vectors that satisfy the preset similarity condition through neighbor feature vector search; and determining the video frames corresponding to the M second feature vectors as the similar video frames.

[0008] Optionally, the device is configured to determine M second feature vectors that satisfy the preset similarity condition by means of a neighbor feature vector search when the M video frames with the highest similarity are found according to the preset similarity condition: when the M video frames with the highest similarity are found according to the preset similarity condition, determine the reference feature vector corresponding to each video frame in each video frame sequence in the first group of video frame sequences; determine the dot product between each reference feature vector and the first feature vector; and determine the M reference feature vectors with the largest dot product values ​​as the M second feature vectors.

[0009] Optionally, the apparatus is configured to determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences in the following manner to obtain a target distance matrix: Each video frame sequence in the N video frame sequences is sequentially regarded as the first video frame sequence, and each video frame in the first video frame sequence is sequentially regarded as the seed video frame. The following operations are performed: an initial distance matrix is ​​created, wherein the initial distance matrix includes N rows and N columns; the target frame similarity between the seed video frame and each of the similar video frames is determined; each target frame similarity is superimposed on the corresponding second video frame sequence; the distance between the first video frame sequence and each of the second video frame sequences is determined, wherein the frame similarity includes the target frame similarity; the initial distance matrix is ​​filled according to the distance between the first video frame sequence and each of the second video frame sequences to obtain the target distance matrix.

[0010] Optionally, the device is configured to determine the target frame similarity between the seed video frame and each of the similar video frames by superimposing each of the target frame similarities onto the corresponding second video frame sequence, and determining the distance between the first video frame sequence and each of the second video frame sequences by: obtaining the target frame similarity between each video frame in the i-th video frame sequence and each of the similar video frames in the j-th second video frame sequence; performing a superposition operation on each of the target frame similarities in the j-th second video frame sequence to determine a target distance value, wherein the target distance value represents the distance between the i-th video corresponding to the i-th video frame sequence and the j-th video corresponding to the j-th second video frame sequence; the device is configured to fill the initial distance matrix according to the distance between the first video frame sequence and each of the second video frames to obtain the target distance matrix by: filling the target distance value into the i-th row and j-th column of the target distance matrix as the value of the element in the i-th row and j-th column of the target distance matrix.

[0011] Optionally, the apparatus is configured to perform a superposition operation on the similarity of each target frame in the j-th second video frame sequence to determine the target distance value by: obtaining the video length corresponding to the i-th video; and performing a superposition operation on the similarity of each target frame based on the video length to determine the target distance value.

[0012] Optionally, the apparatus is configured to determine the target frame similarity between the seed video frame and each of the similar video frames by: superimposing each target frame similarity onto the corresponding second video frame sequence; determining the distance between the first video frame sequence and each of the second video frame sequences by: determining the target frame similarity between the seed video frame and each of the similar video frames; superimposing each target frame similarity onto the corresponding second video frame sequence; determining the video similarity between each of the second video frame sequences in the first video frame sequence and the second group of video frame sequences; performing an exponential decreasing transformation operation on each video similarity; and determining the distance between each of the second video frame sequences in the first video frame sequence and the second group of video frame sequences.

[0013] Optionally, the device is configured to acquire N videos to be classified in the following manner, including at least one of the following: acquiring the N videos uploaded by a target account within a first time period; acquiring multiple videos uploaded by a target account within different time periods; determining the N videos belonging to the same time period from the multiple videos according to timestamps; acquiring a group of videos uploaded by a group of accounts within a second time period; determining the N videos from the group of videos based on account identifiers, wherein the N videos represent videos uploaded by the same account; acquiring multiple groups of videos uploaded by a group of accounts within different time periods; determining the N videos belonging to the same time period and belonging to the same account from the multiple groups of videos according to timestamps and based on account identifiers.

[0014] Optionally, the apparatus is configured to perform a clustering operation on the N videos based on the target distance matrix to determine at least one target video set by: obtaining a predetermined neighborhood radius parameter and a minimum number of points parameter, wherein the neighborhood radius parameter is used to determine the neighboring videos of the target core video, and the minimum number of points parameter is used to determine the minimum number of the neighboring videos of the target core video; and performing a clustering operation on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set.

[0015] Optionally, the apparatus is configured to perform clustering operations on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set: for each unvisited target video among the N videos, the following operations are performed: accessing the target video and determining the first neighbor video of the target video based on the neighborhood radius parameter; if the number of the first neighbor videos is greater than or equal to the minimum number of points parameter, marking the target video as a first core video, wherein the target core video includes the first core video; recursively checking each of the first neighbor videos and determining the second neighbor video corresponding to each of the first neighbor videos that satisfies the neighborhood radius parameter; if the number of the second neighbor videos is less than the minimum number of points parameter, determining that the target core video, the first neighbor video, and the second neighbor video belong to the same target video set; if the number of the second neighbor videos is greater than or equal to the minimum number of points parameter, determining the first neighbor video as a second core video, wherein the target core video includes the second core video.

[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described video detection method when running.

[0017] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the video detection method described above.

[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the video detection method described above through the computer program.

[0019] In this embodiment, N videos to be classified are obtained, where N is a positive integer; frames are extracted from each of the N videos based on a predetermined frame extraction interval, and features are extracted frame by frame to determine N video frame sequences corresponding to each of the N videos. Each video frame sequence in the N video frame sequences includes a set of feature vectors, each feature vector representing a video frame in a video; similar video frames are found from a first set of video frame sequences based on seed video frames in the first video frame sequence, where the N video frame sequences consist of the first video frame sequence and the first set of video frame sequences, and similar video frames represent video frames whose frame similarity with the seed video frame meets a preset similarity condition; the distance between each second video frame sequence in the first video frame sequence and the second set of video frame sequences is determined to obtain a target distance matrix, where the second set of video frame sequences represents similar video frames. The value in the i-th row and j-th column of the target distance matrix represents the distance between the i-th and j-th videos out of N videos, where i and j are positive integers less than or equal to N. Based on the target distance matrix, clustering is performed on the N videos to determine at least one target video set. This target video set includes a target core video, and the distance between the target core video and other videos in the target video set satisfies a preset distance condition, as does the number of other videos. By acquiring each video frame in each video, feature vectors are determined on a frame-by-frame basis. Similarity judgment is then performed on each video frame based on these feature vectors. This approach further enables effective management and classification of a large number of videos, improving the efficiency of video similarity classification. Thus, it solves the technical problem of low accuracy in video detection due to the failure to consider the differences in local video features. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a schematic diagram of an application environment for an optional video detection method according to an embodiment of this application;

[0022] Figure 2 This is a flowchart illustrating an optional video detection method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of an optional video detection method according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of another optional video detection method according to an embodiment of this application;

[0032] Figure 12 This is a schematic diagram of an optional video detection device according to an embodiment of this application;

[0033] Figure 13 This is a schematic diagram of the structure of an optional video detection product according to an embodiment of this application;

[0034] Figure 14 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0038] Milvus Database: A vector database used to handle large-scale vector search tasks. Using vector data storage and search techniques, it can quickly retrieve vectors similar to the query vector. Milvus is suitable for various application scenarios, such as computer vision, natural language processing, and recommendation systems.

[0039] DBSCAN Clustering Algorithm: Density-Based Clustering (DBSCAN) is a density-based unsupervised clustering method that can discover clusters of arbitrary shapes and exhibits good robustness to noise. The core idea of ​​DBSCAN is to find regions with sufficiently high density in the data space and divide the points within these regions into clusters. In image processing, DBSCAN can be used for tasks such as image segmentation and anomaly detection.

[0040] ResNet101 Model: Residual Networks (ResNet) are a type of deep convolutional neural network that addresses the vanishing and exploding gradient problems in deep network training by introducing a residual learning framework. ResNet-101 is a variant of ResNet containing 101 network layers. ResNet-101 has achieved significant performance improvements on multiple visual recognition tasks, such as image classification, object detection, and semantic segmentation.

[0041] Faiss Database: A fast similarity search database, this is a C++ library for efficient similarity search and clustering. It is specifically designed to handle large-scale vector sets, supports various index structures and search algorithms, and can help users quickly find the most similar vectors in large datasets.

[0042] The present application will be described below with reference to embodiments:

[0043] According to one aspect of the embodiments of this application, a video detection method is provided. Optionally, in this embodiment, the video detection method can be applied to, for example... Figure 1 The hardware environment shown consists of server 101 and terminal device 103. For example... Figure 1 As shown, server 101 is connected to terminal 103 via a network and can be used to provide services to terminal devices or applications installed on terminal devices. The applications can be video applications, instant messaging applications, browser applications, educational applications, game applications, etc. Database 105 can be set up on the server or independently of the server to provide data storage services for server 101, such as a game data storage server. The network mentioned above can include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks, metropolitan area networks, and wide area networks. The wireless network includes Bluetooth, WIFI, and other networks that enable wireless communication. Terminal device 103 can be a terminal configured with an application, and can include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, virtual reality (VR) terminals, augmented reality (AR) terminals, mixed reality (MR) terminals, and other computer devices. The server mentioned above can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0044] Combination Figure 1 As shown, the embodiments of this application can be executed by an electronic device, which can be a terminal device or a server. Furthermore, the embodiments of this application can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0045] The above is merely an example, and this embodiment does not impose any specific limitations.

[0046] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the above video detection method includes:

[0047] For example, the embodiments of this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0048] S202, Obtain N videos to be classified, where N is a positive integer;

[0049] Optionally, in the embodiments of this application, the above-mentioned N videos may include, but are not limited to, videos on video sharing platforms, local storage devices, or videos stored in a database.

[0050] In one exemplary embodiment, Figure 3 This is a schematic diagram of an optional video detection method according to an embodiment of this application, such as... Figure 3 As shown, users enter the video sharing platform, select to upload a video (including but not limited to uploading a video), or enter the video address of a third-party application platform. Then, they click the submit button to complete the video upload operation. The video is then identified as one of the aforementioned N videos, and these N videos can be submitted by different or the same users.

[0051] In another exemplary embodiment, if the user's account on the video sharing platform and the third-party application platform are interchangeable, that is, the user can log in to the video sharing platform and the third-party application using the same account and password, then after the user enters the video sharing platform, the video sharing platform's backend server will automatically retrieve the videos that the user has taken or collected in the third-party application, and ask the user whether they need to upload them to the video sharing platform synchronously through message pop-ups or other means. There may be a cooperative relationship or integration relationship between the third-party application and the video sharing platform.

[0052] S204, based on a predetermined frame extraction interval, each video in the N videos is extracted, and features are extracted frame by frame to determine the N video frame sequences corresponding to each video in the N videos. Among them, a video frame sequence in the N video frame sequences includes a set of feature vectors, and each feature vector is used to represent a video frame in a video.

[0053] For example, after obtaining the above N videos, video frames can be extracted from each video as a unit, and feature extraction operations can be performed on the video frames to obtain the video frame sequence corresponding to each video, as in the embodiments of this application.

[0054] Optionally, in the embodiments of this application, the predetermined frame extraction interval can be flexibly set, for example, frame extraction every 1 second, frame extraction every 0.5 seconds, etc. This application does not limit it. The video frame refers to a single image in the video. The feature vector is used to describe the key attributes of the video frame, such as the color, texture, shape, etc. of the object in the video frame.

[0055] It should be noted that the aforementioned predetermined frame skipping intervals for different video sets may be the same or different, and this application does not impose specific limitations. For example, for videos that are predefined as having rich content (game videos, dance videos, etc.), a smaller frame skipping interval is set to capture more details; while for videos that are predefined as having less variation (landscape videos, shopping videos, etc.), a larger frame skipping interval is set.

[0056] In one exemplary embodiment, a video is used as an example for illustration. Figure 4 This is a schematic diagram of another optional video detection method according to an embodiment of this application, such as... Figure 4 As shown, including but not limited to:

[0057] S402, determine the frame extraction interval. For example, if the interval is 10 seconds, then extract one video frame every 10 seconds.

[0058] S404, Read video, for example, including but not limited to reading video data using video processing libraries (OpenCV, etc.);

[0059] S406, Perform video frame extraction operation. For example, if the video is 100 seconds long and the frame extraction interval is 10 seconds, then extract the 1st, 11th, 21st...91st frames.

[0060] S408, Feature Extraction, extracting features from each video frame, that is, extracting multimodal features from each frame image, including but not limited to using deep learning models, such as using the ISC model (a deep learning model for extracting multimodal features) which has lower computational cost and better performance, or the ResNet101 model (residual network model), etc.

[0061] S410, Construct a feature vector sequence. For each video, combine the feature vectors of the extracted video frames in order to form a feature frame sequence.

[0062] S206, find similar video frames from the first group of video frame sequences based on the seed video frames in the first video frame sequence, wherein the N video frame sequences consist of the first video frame sequence and the first group of video frame sequences, and the similar video frames represent video frames whose frame similarity with the seed video frames meets the preset similarity conditions.

[0063] For example, after obtaining the video frame sequences corresponding to each of the above videos, the feature vectors corresponding to each video frame in each video frame sequence can be saved to the database. Then, as in the embodiments of this application, the seed video frame in the first video frame sequence is determined, and the neighboring feature vectors of the first feature vector corresponding to the seed video frame are quickly found through the database. Similar video frames are found based on the neighboring feature vectors, including but not limited to using the Milvus database (vector similarity search engine database). The Milvus database can implement various distance calculation methods between vectors, including Euclidean distance, dot product, etc.

[0064] Optionally, in this embodiment of the application, the first video frame sequence is any one of the N video frame sequences, the first group of video frame sequences is the video frame sequence other than the first video frame sequence among the N video frame sequences, and the seed video frame refers to the video frame selected one by one among the N video frame sequences. The seed video frame will be used to compare with other video frames to find similar video frames.

[0065] In one exemplary embodiment, Figure 5 This is a schematic diagram of another optional video detection method according to an embodiment of this application. The first video frame sequence in the above N video frame sequences can be as follows: Figure 5 As shown, next, similar video frames are found from the first group of video frame sequences based on the seed video frames in the first video frame sequence.

[0066] In yet another exemplary embodiment, Figure 6 This is a schematic diagram of another optional video detection method according to an embodiment of this application. The process of obtaining similar video frames described above may include, but is not limited to, the following: Figure 6 As shown:

[0067] S602, Create a video similarity matrix corresponding to the first video frame sequence of a single user;

[0068] For example, if a user a has n videos v1, v2...vn, then there will be a similarity value between each pair of videos. Several similarity values ​​can form a similarity matrix, denoted as similar_matrix. The size of the similarity matrix can be represented as n*n, where similar_matrix[i,j] = similar_matrix[j,i], and the similarity between the same video and itself is 1. When i == j, similar[i,j] = 1. When i != j initially, similar_matrix[i,j] = 0, waiting to be filled in subsequent steps.

[0069] S604, obtain the seed video frame in the first video frame sequence, and the first feature vector corresponding to the seed video frame;

[0070] S606, using the first feature vector, search for similar video frames in the feature vectors of each of the first group of video frame sequences in the first group of video frame sequences, and determine the neighboring feature vectors and the similar video frames corresponding to the neighboring feature vectors.

[0071] S608, fill the similarity matrix with neighboring feature vectors. For example, for a certain user a certain video v1, the feature vectors of the video frames of this video are f1, f2, f3 in sequence. For each feature vector, the 10 neighboring feature vectors with the closest dot product (that is, the largest dot product value) can be found respectively.

[0072] S208, determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences to obtain the target distance matrix, where the second group of video frame sequences represents the video frame sequence to which similar video frames belong, and the value of the i-th row and j-th column in the target distance matrix is ​​used to represent the distance between the i-th video and the j-th video in N videos, where i and j are both positive integers less than or equal to N;

[0073] Optionally, in this embodiment, the distances between the other video frame sequences in the first group of video frame sequences (excluding the second group of video frame sequences) and the first video frame sequence are also filled into the target distance matrix, and the values ​​of the corresponding elements in the target distance matrix are either preset values ​​or determined by performing an exponentially decreasing transformation operation.

[0074] S210, perform clustering operation on N videos based on the target distance matrix to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

[0075] For example, after obtaining the above-mentioned similar video frames, the target distance matrix can be generated based on the video frame sequence (the second group of video frame sequences) and the first video frame sequence where the above-mentioned similar video frames are located, as in the embodiments of this application. The target core video can be used to cluster similar videos, accurately identify other videos with similar features, and perform clustering to complete the determination of similar videos.

[0076] Optionally, in the embodiments of this application, some matrix elements in the target distance matrix describe the distance between each frame in the first video frame sequence and the second group of video frame sequences, while other matrix elements describe the distance between the first video frame sequence and other video frame sequences in the first group of video frame sequences excluding the second group of video frame sequences.

[0077] In an exemplary embodiment, the corresponding video frames in the above N video frame sequences are first clustered according to similarity values. The clustering method includes, but is not limited to, the DBSCAN clustering algorithm. Unlike k-means and hierarchical classification clustering algorithms, it does not require preset categories and seed points. By clustering video frames based on distance values, clusters of arbitrary shapes can be obtained, and it is robust to noise points. The usage parameters are as follows:

[0078] DBSCAN(eps=threshold,min_samples=1,metric='precomputed');

[0079] labels=db.fit_predict(distances_matrix);

[0080] Here, eps is the neighborhood radius (epsilon) parameter, that is, video frames whose distance between video frames is less than eps will be considered as the above similar video frames; min_samples indicates that if a video frame contains at least min_samples of the above similar video frames (including seed video frames) in its neighborhood, then the video frame is considered as the core video frame, and the video containing the core video frame is the above target core video; distances_matrix represents the above target distance matrix.

[0081] In one exemplary embodiment, Figure 7 This is a schematic diagram of another optional video detection method according to an embodiment of this application. The video detection method proposed in this application can be applied to application scenarios such as video content management, video classification, and topic discovery. For example, the video detection method can be applied to the application scenario of video content management, such as... Figure 7 As shown:

[0082] S702, the video sharing platform's system backend obtained data showing that 100 users submitted 1 million videos within a month, with each user submitting 10,000 videos, and the number of videos submitted by each user could be different;

[0083] S704 categorizes the videos submitted by each user, resulting in multiple video types. The same user can submit multiple types of videos, and the specific classification of each video category can be manually labeled.

[0084] S706, retrieve N videos uploaded by user A;

[0085] S708, extract frames from N videos one by one to obtain 100*N images (the images are the video frames mentioned above. Assume each video is 100 seconds long, and one frame is extracted per second. The length of different videos can be different).

[0086] S710 uses the ISC model to extract features from 100*N images, obtaining 100*N feature vectors, which are then saved to the feature database, and the feature database is labeled with the video corresponding to each feature vector.

[0087] S712 creates an N*N target distance matrix using 100*N feature vectors. For example, an element (2,3) in the matrix refers to the distance value between the original second and third videos.

[0088] S714. Perform clustering using the DBSCAN algorithm on the target distance matrix to obtain a 1*N vector. If the second and third elements of the vector have the same value, it means that the second and third videos belong to the same class. If the value is -1, it means that they do not belong to any class.

[0089] S716, based on the clustering classification results, video type and other information are labeled. For example, video A in these N videos has 10 similar videos.

[0090] S718, the video sharing platform's system backend determines the video creation quality of each user based on the clustering classification results, and distributes rewards such as virtual resources to users with higher video creation quality. For example, if user A uploads 100 videos in a month and there are 10 similar videos, while user B uploads 100 videos in a month and there are 20 similar videos, then user A's video creation quality is higher than user B's, and user A can be given more virtual resources.

[0091] In yet another exemplary embodiment, the video detection method proposed in this application can be applied to application scenarios such as game recommendation, including but not limited to:

[0092] S1, the promotion channel backend obtained 1 million game applications downloaded by 100 users within a month;

[0093] S2 categorizes the game applications downloaded by each user, resulting in various types of game applications;

[0094] S3, retrieve the N game applications downloaded by user A;

[0095] S4: Extract frames from the promotional videos displayed on the download pages of N game applications one by one to obtain 100*N images (assuming each video is 100 seconds long, one frame is extracted per second, and the length of different videos can be different).

[0096] S5. Use the ISC model to extract features from 100*N images to obtain 100*N feature vectors, which are then saved to the feature database. The feature database is labeled with the video corresponding to each feature vector.

[0097] S6. Create an N*N target distance matrix using 100*N feature vectors. For example, an element (2,3) in the matrix refers to the distance value between the original second and third videos.

[0098] S7. Perform clustering using the DBSCAN algorithm on the target distance matrix to obtain a 1*N vector. If the second and third elements of the vector have the same value, it means that the second and third videos belong to the same class. If the value is -1, it means that they do not belong to any class.

[0099] S8. Based on the clustering classification results, label information such as game application type. For example, among these N game applications, game application A has 10 similar game applications.

[0100] S9's promotion channel backend determines each user's game type preference based on clustering results, providing targeted marketing strategies and personalized recommendations for each user.

[0101] In this embodiment, N videos to be classified are obtained, where N is a positive integer; frames are extracted from each of the N videos based on a predetermined frame extraction interval, and features are extracted frame by frame to determine N video frame sequences corresponding to each of the N videos. Each video frame sequence in the N video frame sequences includes a set of feature vectors, each feature vector representing a video frame in a video; similar video frames are found from a first set of video frame sequences based on seed video frames in the first video frame sequence, where the N video frame sequences consist of the first video frame sequence and the first set of video frame sequences, and similar video frames represent video frames whose frame similarity with the seed video frame meets a preset similarity condition; the distance between each second video frame sequence in the first video frame sequence and the second set of video frame sequences is determined to obtain a target distance matrix, where the second set of video frame sequences represents similar video frames. The value in the i-th row and j-th column of the target distance matrix represents the distance between the i-th and j-th videos out of N videos, where i and j are positive integers less than or equal to N. Based on the target distance matrix, clustering is performed on the N videos to determine at least one target video set. This target video set includes a target core video, and the distance between the target core video and other videos in the target video set satisfies a preset distance condition, as does the number of other videos. By acquiring each video frame in each video, feature vectors are determined on a frame-by-frame basis. Similarity judgment is then performed on each video frame based on these feature vectors. This approach further enables effective management and classification of a large number of videos, improving the efficiency of video similarity classification. Thus, it solves the technical problem of low accuracy in video detection due to the failure to consider the differences in local video features.

[0102] As an optional approach, the above-mentioned method of finding similar video frames from a first set of video frame sequences based on seed video frames in the first video frame sequence includes: treating each video frame sequence in the above N video frame sequences as the above first video frame sequence, and treating each video frame in the above first video frame sequence as the above seed video frame, and performing the following operations: obtaining the first feature vector corresponding to the above seed video frame; if the above-mentioned preset similarity condition indicates that the M video frames with the highest similarity have been found, determining the M second feature vectors that satisfy the above-mentioned preset similarity condition through neighbor feature vector search; and determining the video frames corresponding to the above-mentioned M second feature vectors as the above-mentioned similar video frames.

[0103] For example, after obtaining the above N video frame sequences, as in the embodiments of this application, the feature vectors of the first video frame and the seed video frame can be compared by means of neighbor feature vector search to find the M video frames with the highest similarity to the seed video frame as similar video frames.

[0104] Optionally, in the embodiments of this application, the first feature vector may include, but is not limited to, key visual information of the seed video frame, such as the color, texture, edge, and shape of the object in the seed video frame. The neighboring feature vector can be understood as other vectors that are close to the first feature vector. It may include, but is not limited to, using methods such as Euclidean distance and cosine similarity to measure the degree of similarity between the first feature vector and the neighboring feature vector.

[0105] In one exemplary embodiment, Figure 8 This is a schematic diagram of another optional video detection method according to an embodiment of this application, such as... Figure 8 As shown, the process of finding similar video frames described above includes, but is not limited to:

[0106] S802, Start;

[0107] S804, obtain the first feature vector of the seed video frame;

[0108] S806, set similarity conditions, such as similarity threshold or similarity ranking;

[0109] S808 uses a neighboring feature vector search algorithm to search for and determine M second feature vectors that meet the preset similarity conditions in the first set of video frame sequences based on the first feature vector of the seed video frame.

[0110] S810, determine the video frames corresponding to these M second feature vectors as similar video frames;

[0111] S812, repeat the above S804 to S810 until each video frame sequence in the N video frame sequences has been executed. That is, treat each video frame sequence as the first video frame sequence in turn, and treat each video frame as the seed video frame in turn, and repeat the above process of finding similar video frames.

[0112] S814, End.

[0113] Through the embodiments of this application, by comparing the feature vectors of the first video frame and the seed video frame through the nearest neighbor feature vector search method, the M video frames with the highest similarity to the seed video frame are found as similar video frames, thereby achieving the purpose of fast video frame similarity search. While improving search efficiency, by adjusting the value of M, the number of similar video frames found can be flexibly controlled to meet the actual needs of different application scenarios, so as to achieve the technical effect of flexibly adjusting the search range of similar video frames.

[0114] As an optional approach, when the M video frames with the highest similarity are found according to the preset similarity condition, the M second feature vectors that satisfy the preset similarity condition are determined by neighbor feature vector search, including: when the M video frames with the highest similarity are found according to the preset similarity condition, determining the reference feature vector corresponding to each video frame in each video frame sequence in the first group of video frame sequences; determining the dot product between each of the reference feature vectors and the first feature vector; and determining the M reference feature vectors with the largest dot product values ​​as the M second feature vectors.

[0115] For example, when the M video frames with the highest similarity are found, as in the embodiments of this application, the reference feature vector corresponding to each video frame in the first group of video frame sequences can be determined first, and then the dot product between each reference feature vector and the first feature vector can be calculated. The M reference feature vectors with the largest dot product values ​​can be selected as the M second feature vectors.

[0116] Optionally, in the embodiments of this application, the reference feature vector may include, but is not limited to, local features, global features, or mixed features of video frames, and the dot product is used to describe the similarity between different feature vectors, and may include, but is not limited to, using dot product (inner product) to calculate.

[0117] In an exemplary embodiment, firstly, feature extraction is performed on each video frame in the first set of video frame sequences to obtain the corresponding reference feature vector. The reference feature vector can be a local feature, a global feature, or a mixed feature of the video frame. Next, the dot product between each reference feature vector and the first feature vector (i.e., the feature vector of the target video frame) is calculated. The values ​​of these dot products are sorted, and the M reference feature vectors with the largest values ​​are selected. Further, the M reference feature vectors with the largest values ​​are used as M second feature vectors. The similarity between these M second feature vectors and the first feature vector is higher than the similarity between the feature vectors corresponding to other video frames in the above video frame sequences and the first feature vector.

[0118] In this embodiment, a reference feature vector corresponding to each video frame in the first group of video frame sequences is first determined. Then, the dot product between each reference feature vector and the first feature vector is calculated. The M reference feature vectors with the largest dot product values ​​are selected as the M second feature vectors. This improves the acquisition rate of the second feature vectors, thereby reducing the amount of data for similar video comparison and achieving the technical effect of improving the processing efficiency of feature vectors. In addition, by selecting the M reference feature vectors with the largest dot product with the first feature vector, it can be ensured that the selected second feature vectors are closest to the first feature vector in the feature space, effectively improving the accuracy of video comparison.

[0119] As an optional approach, determining the distance between each second video frame sequence in the first video frame sequence and the second set of video frame sequences to obtain the target distance matrix includes: treating each video frame sequence in the N video frame sequences as the first video frame sequence in turn, and treating each video frame in the first video frame sequence as the seed video frame in turn, and performing the following operations: creating an initial distance matrix, wherein the initial distance matrix includes N rows and N columns; determining the target frame similarity between the seed video frame and each of the similar video frames, superimposing each of the target frame similarities on the corresponding second video frame sequence, determining the distance between the first video frame sequence and each of the second video frames, wherein the frame similarity includes the target frame similarity; filling the initial distance matrix according to the distance between the first video frame sequence and each of the second video frames to obtain the target distance matrix.

[0120] For example, after obtaining the first video frame sequence and the second set of video frame sequences, each video frame sequence in the N video frame sequences can be regarded as the first video frame sequence in sequence, and the distance between each video frame sequence and other video frame sequences can be calculated to fill the target distance matrix, as in the embodiments of this application.

[0121] Optionally, in this embodiment, the initial distance matrix can be an N-row, N-column blank matrix, the target frame similarity can include, but is not limited to, the similarity between video frames, and the distance between the first video frame sequence and each of the second video frame sequences is used to represent the degree of similarity between the first video frame sequence and each of the second video frame sequences.

[0122] In an exemplary embodiment, the feature vectors of the first video frame and each of the aforementioned second video frames are first compared. That is, the distance between the first video frame sequence and all other video frame sequences is determined by comparing the frame similarity between the first feature vector and the second feature vector. Further, an N*N initial distance matrix is ​​first created. Each element (i, j) in the initial distance matrix can be understood as follows: if i equals j, it can be set to 0 (indicating that the distance between the same video frame sequence and itself is 0); if i does not equal j, the calculated distance is filled into the (i, j) position.

[0123] Specifically, assuming the second video frame sequence includes the second video frame sequence V2 and the second video frame sequence V3, the calculated distance between the first video frame sequences V1 and V2 is D(V1,V2) = 5, and the distance between the first video frame sequences V1 and V3 is D(V1,V3) = 10, indicating that the first video frame sequences V1 and V2 are more similar.

[0124] In this embodiment, each of the N video frame sequences is sequentially regarded as the first video frame sequence. The distance between each video frame sequence and other video frame sequences is calculated, and the distance matrix is ​​filled to obtain the target distance matrix. The target distance matrix can be used for clustering, classification, or finding the most similar video frame sequence, thereby improving the accuracy of the target distance matrix. Furthermore, in this embodiment, different distance metrics, such as Euclidean distance and Manhattan distance, can be selected according to different needs to adapt to different application scenarios.

[0125] As an optional approach, determining the target frame similarity between the seed video frame and each of the similar video frames, and superimposing each target frame similarity onto the corresponding second video frame sequence to determine the distance between the first video frame sequence and each of the second video frame sequences, includes: obtaining the target frame similarity between each video frame in the i-th video frame sequence and each of the similar video frames in the j-th second video frame sequence; performing a superposition operation on each of the target frame similarities in the j-th second video frame sequence to determine a target distance value, wherein the target distance value represents the distance between the i-th video corresponding to the i-th video frame sequence and the j-th video corresponding to the j-th second video frame sequence; filling the initial distance matrix based on the distance between the first video frame sequence and each of the second video frames to obtain the target distance matrix includes: filling the target distance value into the i-th row and j-th column of the target distance matrix as the value of the element in the i-th row and j-th column of the target distance matrix.

[0126] For example, after determining the seed video frames, as in the embodiments of this application, the distance between the first video frame sequence and the second video frame sequence can be calculated first, and then the target frame similarity between each video frame in the first video frame sequence and similar video frames in the second video frame sequence can be calculated. The target similarity is then superimposed to obtain the target distance value. Finally, the initial distance matrix is ​​filled according to the target distance value to obtain the target distance matrix.

[0127] Optionally, in the embodiments of this application, the above-mentioned superposition operation includes, but is not limited to, summing the similarity of the above-mentioned target frames, and the above-mentioned target distance is used to describe the degree of similarity between different videos.

[0128] In an exemplary embodiment, assume the i-th video is VideoA and the j-th video is VideoB. VideoA has 5 video frames, denoted as A1, A2, A3, A4, and A5. VideoB also has 5 video frames, denoted as B1, B2, B3, B4, and B5. First, A1 is selected as the seed video frame. Then, the similarity between A1 and B1, B2, B3, B4, and B5 is calculated, assuming they are 0.8, 0.7, 0.6, 0.9, and 0.5 respectively. These similarity values ​​are then summed to obtain the similarity for each video. An exponential decreasing transformation is then performed on the similarity for each video to obtain a target distance of 3.5. Alternatively, the average distance can be taken, resulting in a value of 0.7.

[0129] Furthermore, A2, A3, A4, and A5 can be used as the seed video frames mentioned above to obtain the similarity between A2, A3, A4, and A5 and B1, B2, B3, B4, and B5 respectively. These similarities are then superimposed to obtain the target distance between VideoA and VideoB. Similarly, the distance between VideoA and other second videos can be calculated. Then, the target distance values ​​are filled into a 5*5 initial distance matrix to obtain the target distance matrix. The value of one element in the matrix is ​​the target distance value obtained by superimposing the similarities mentioned above.

[0130] It should be noted that the distance between the videos mentioned above can be understood as the distance between video frame sequences.

[0131] Through the embodiments of this application, the target frame similarity between each video frame in the first video frame sequence and similar video frames in the second video frame sequence is calculated. The target similarity is then superimposed to obtain the target distance value. Subsequently, the initial distance matrix is ​​filled according to the target distance value to obtain the target distance matrix, thereby achieving the purpose of quantifying the similarity or difference between different video frame sequences. Through the embodiments of this application, the technical effect of automatically calculating the similarity between video frame sequences is realized, effectively reducing manual intervention and improving the automation level of video content analysis.

[0132] As an optional approach, the above-mentioned method of performing a superposition operation on the similarity of each of the target frames in the j-th second video frame sequence to determine the target distance value includes: obtaining the video length corresponding to the i-th video; and performing a superposition operation on the similarity of each of the target frames based on the video length to determine the target distance value.

[0133] For example, in determining the target distance value, as in the embodiments of this application, the length of the i-th video can be obtained first, and then the similarity of each target frame can be superimposed to determine the target distance value.

[0134] In an exemplary embodiment, the similarity of the target frames can be weighted according to the video length, or the similarity of the target frames can be directly added together.

[0135] By employing the method of first obtaining the length of the i-th video and then performing an overlay operation on the similarity of each target frame to determine the target distance value, the technical effect of improving the accuracy and efficiency of video detection is achieved.

[0136] As an optional approach, the above-mentioned determination of the target frame similarity between the seed video frame and each of the similar video frames, superimposing each of the target frame similarities onto the corresponding second video frame sequence, and determining the distance between the first video frame sequence and each of the second video frame sequences includes: determining the target frame similarity between the seed video frame and each of the similar video frames; superimposing each of the target frame similarities onto the corresponding second video frame sequence to determine the video similarity between each of the second video frame sequences in the first video frame sequence and the second group of video frame sequences; performing an exponential decreasing transformation operation on each of the video similarities to determine the distance between each of the second video frame sequences in the first video frame sequence and the second group of video frame sequences.

[0137] For example, after obtaining the target frame similarity between the seed video frame and each of the similar video frames, the target frame similarity can be superimposed on the second video frame sequence as in the embodiments of this application. Then, the video similarity is calculated on the superimposed video frame sequence, and an exponential decreasing transformation operation is performed to obtain the distance between each video frame sequence in the first video frame sequence and the second group of video frame sequences.

[0138] Optionally, in the embodiments of this application, the above-mentioned exponential decrease transformation operation refers to the process of gradually reducing the exponential term in the exponential function.

[0139] In an exemplary embodiment, an exponentially decreasing transformation is used to transform the similarity matrix composed of the target frame similarity between the seed video frame and each of the similar video frames into a distance matrix. After the exponentially decreasing transformation, the target frame similarity is P, and the distance is exp(-p), that is, the negative target frame similarity value of the natural logarithm raised to the power of exp(-p).

[0140] In this embodiment, the target frame similarity is superimposed with the second video frame sequence, and an exponentially decreasing transformation operation is performed to determine the distance between the superimposed video frame sequences. This simplifies the calculation process and, through the exponentially decreasing transformation operation, yields a more accurate video similarity result. This makes the distance calculation between the first video frame sequence and the second set of video frame sequences more accurate and reliable. The exponentially decreasing transformation operation simplifies the distance calculation process between video frame sequences, reducing the consumption of computing resources while also achieving the technical effect of improving the calculation speed of video similarity.

[0141] As an optional approach, obtaining the N videos to be categorized includes at least one of the following: obtaining the N videos uploaded by the target account within a first time period; obtaining multiple videos uploaded by the target account within different time periods; determining the N videos belonging to the same time period from the multiple videos based on timestamps; obtaining a group of videos uploaded by a group of accounts within a second time period; determining the N videos from the group of videos based on account identifiers, wherein the N videos represent videos uploaded by the same account; obtaining multiple groups of videos uploaded by a group of accounts within different time periods; determining the N videos belonging to the same time period and belonging to the same account from the multiple groups of videos based on timestamps and account identifiers.

[0142] For example, the N videos to be classified may be videos uploaded by the target account within a time period, as in the embodiments of this application, or videos within different time periods. They may also be videos uploaded by a group of accounts (i.e., different accounts) within a time period, or videos within different time periods.

[0143] Optionally, in this embodiment of the application, the aforementioned group of accounts may include, but is not limited to, several different accounts.

[0144] In an exemplary embodiment, firstly, N videos uploaded by target account A within a first time period are obtained; and multiple videos uploaded by target account A within different time periods can be obtained, and N videos belonging to the same time period are determined based on the timestamp; similarly, a group of videos uploaded by account A within a second time period are obtained, and N videos among them are determined based on the account identifier; and multiple groups of videos uploaded by account A within different time periods can be obtained, and N videos belonging to the same time period and belonging to the same account are determined based on the timestamp and account identifier.

[0145] Through the embodiments of this application, videos uploaded by the target account within a time period, or videos within different time periods, or videos uploaded by a group of accounts (i.e., different accounts) within a time period, or videos within different time periods, are identified as the aforementioned N videos. This broadens the sources of video frames, enabling the video detection method proposed in this application to be applied to various application scenarios, ensuring the robustness and accuracy of video similarity classification in each application scenario.

[0146] As an optional approach, the above-mentioned clustering operation on the N videos based on the target distance matrix to determine at least one target video set includes: obtaining a predetermined neighborhood radius parameter and a minimum number of points parameter, wherein the neighborhood radius parameter is used to determine the neighboring videos of the target core video, and the minimum number of points parameter is used to determine the minimum number of the neighboring videos of the target core video; and performing a clustering operation on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set.

[0147] For example, after obtaining the target distance matrix, video clustering can be performed according to the predetermined neighborhood radius parameter and minimum number of points parameter, as in the embodiments of this application, to determine at least one set of the target videos.

[0148] Optionally, in the embodiments of this application, the aforementioned neighborhood radius parameter refers to a preset parameter used to determine the range of neighboring videos of the target core video, the aforementioned minimum number of points parameter refers to a preset parameter used to determine the minimum number of neighboring videos that must be included in the target core video, the aforementioned neighboring videos refer to videos whose distance from the target core video is within the range threshold corresponding to the neighborhood radius parameter, and the aforementioned target video set may include, but is not limited to, several similar videos.

[0149] In an exemplary embodiment, assuming the neighborhood radius parameter R is 10 and the minimum number of points M is 3, for each video, first find all videos whose target distance values ​​differ by no more than 10 as the aforementioned neighbor videos. Then, when the number of neighbor videos is greater than or equal to 3, the neighbor videos and the target core video are determined as the aforementioned target video set.

[0150] Through the embodiments of this application, video clustering is performed based on the predetermined neighborhood radius parameter and minimum number of points parameter to determine at least one of the target video sets, ensuring the accuracy of the target video sets. Furthermore, in the embodiments of this application, the neighborhood radius and minimum number of points parameter can be adjusted according to different application scenarios and needs, making the clustering method more flexible and adaptable, and achieving the technical effect of efficient and accurate identification of target video sets.

[0151] As an optional approach, the above-mentioned clustering operation on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set includes: for each unvisited target video among the N videos, performing the following operations: accessing the target video and determining the first neighbor video of the target video based on the neighborhood radius parameter; if the number of the first neighbor videos is greater than or equal to the minimum number of points parameter, marking the target video as a first core video, wherein the target core video includes the first core video; recursively checking each of the first neighbor videos and determining the second neighbor video corresponding to each of the first neighbor videos that satisfies the neighborhood radius parameter; if the number of the second neighbor videos is less than the minimum number of points parameter, determining that the target core video, the first neighbor video, and the second neighbor video belong to the same target video set; if the number of the second neighbor videos is greater than or equal to the minimum number of points parameter, determining the first neighbor video as a second core video, wherein the target core video includes the second core video.

[0152] For example, during the clustering operation on the above N videos, as in the embodiments of this application, the target video can be accessed first to find the first neighbor video. If the number of first neighbor videos is greater than or equal to the minimum number of points, the target video is marked as the first core video and added to the target video set. Then, each first neighbor video is recursively checked to find the second neighbor video that satisfies the neighborhood radius parameter. If the number of second neighbor videos is less than the minimum number of points, the target core video, the first neighbor video, and the second neighbor video are determined to be in the same target video set. If the number of second neighbor videos is greater than or equal to the minimum number of points, the first neighbor video is marked as the second core video and added to the target video set.

[0153] Optionally, in this embodiment of the application, several videos in the first neighbor video will be identified as the second neighbor video.

[0154] In an exemplary embodiment, assuming the above N videos are videos A, B, C, D, and E, the above neighborhood radius parameter is 2, and the above minimum number of points parameter is 2, video A is first accessed, and it is determined that the first neighboring videos of video A are B and C. At this time, the number of the first neighboring videos is equal to the minimum number of points parameter 2. Therefore, video A is marked as the first core video, and the core video set of video A is {A}. Next, videos B and C are recursively checked, and it is found that the first neighboring videos of videos B and C are A and D and A, D, and E, respectively. Further, videos A, B, C, D, and E are determined to be the same video set, namely {A, B, C, D, E}. For video B, the number of the first neighboring videos is still greater than or equal to the minimum number of points parameter 2. Therefore, video B can be marked as the second core video.

[0155] In this embodiment, the number of first neighbor videos and the value of the minimum number of points parameter are determined. If the number of first neighbor videos is greater than or equal to the minimum number of points parameter, the target video is marked as the first core video and added to the target video set. Then, each first neighbor video is recursively checked to find the second neighbor video that satisfies the neighborhood radius parameter. If the number of second neighbor videos is less than the minimum number of points parameter, the target core video, the first neighbor videos, and the second neighbor videos are determined to be in the same target video set. If the number of second neighbor videos is greater than or equal to the minimum number of points parameter, the first neighbor video is marked as the second core video and added to the target video set. This ensures the accuracy of video clustering and achieves the technical effect of improving the quality of video clustering.

[0156] For example, the video detection method proposed in this application can be used to construct similar video clustering analysis based on video frame similarity and clustering algorithms (e.g., DBSCAN clustering algorithm (density clustering algorithm)). By integrating multiple video features for deduplication and utilizing video frame similarity for clustering, the accuracy and efficiency of video clustering are improved. This includes, but is not limited to, using video frame similarity for clustering, that is, using the similarity between video frames as the basis for clustering, rather than the clustering methods based on overall video features in related technologies. Using the video frame-based clustering method in this application embodiment can more finely characterize the similarity of videos, improving the accuracy of clustering. Simultaneously, it improves the processing effect of editing and mixing similarity. In addition to achieving better results with video, the DBSCAN clustering algorithm can also be used to cluster videos. Compared with related technologies such as K-means and hierarchical clustering algorithms (K-means clustering algorithm), the DBSCAN algorithm does not require a preset total number of categories, can automatically identify the number of clusters, and has good robustness to noisy data. Furthermore, feature storage can be achieved based on databases such as Milvus (vector similarity search engine) or Faiss (fast similarity search) databases. The feature vectors of video frames are stored in the Milvus database, which has the function of quickly finding neighboring feature vectors, enabling efficient searching and clustering of similar videos.

[0157] In one exemplary embodiment, in order to expand the influence of the target application, the marketing management platform has set up a user incentive mechanism to encourage users to submit video works and reward users based on data such as the number of views of the video works. Since there are often a large number of identical or similar videos among the video works submitted by users, if the marketing management platform administrators rely solely on manual screening and removal of invalid videos with high similarity, the processing time is long and the accuracy is poor. For example, manual processing often results in the omission of similar video works. Based on this, the embodiments of this application can effectively screen out similar videos submitted by the same user, thereby improving the efficiency of video screening, reducing errors and omissions in manual screening, and providing a novel and efficient video management method for the marketing management platform.

[0158] Specifically, first, users enter the URL (link address) of a video work from a third-party platform in the input box on the video creation and sharing platform's (managed by the marketing management platform) upload page. Then, they click the upload button to send a request to the video creation and sharing platform's backend server to submit the work. If the backend receives the user's submitted video work, it will save the video work in the database. Next, the marketing management platform can periodically extract video works from the database and perform similar video clustering to filter out invalid works for the administrator's reference. After the administrator confirms, similar video works will not be considered valid video works for user reward settlement.

[0159] In one exemplary embodiment, Figure 9 This is a schematic diagram of another optional video detection method according to an embodiment of this application, such as... Figure 9 As shown, after a user submits a video link to the video creation and sharing platform, the video is automatically saved to the database. By processing the submitted video works in the database, extracting features, and clustering them, clustering results for similar videos are obtained. This includes, but is not limited to, feature extraction and saving of video frames, and clustering of similar videos among users.

[0160] S902, Feature extraction and storage of video frames:

[0161] S1, Batch download user-submitted video works, using video links stored in the database to batch download all third-party video works submitted by users within a recent period (e.g., one month) to local storage space;

[0162] S2, extract frame images from each video work, including but not limited to using ffmpeg (audio and video codec), specifically, adopt the isochronous frame extraction method, extract one frame of video per second to preserve the temporal sequence characteristics of the video and ensure the similar video detection effect;

[0163] S3, extract the feature vectors of each frame image, that is, extract the multimodal features of each frame image, including but not limited to using deep learning models, such as using the ISC model (a deep learning model for extracting multimodal features) which has lower computational cost and better performance, or the ResNet101 model (residual network model), etc.

[0164] S4. Save the extracted feature vectors to the database. This database needs to record more than just the feature vectors of the images. Figure 10 This is a schematic diagram of another optional video detection method according to an embodiment of this application. The database stores the following content: Figure 10As shown, when a first feature vector exists, neighboring feature vectors can be quickly found through a database, including but not limited to the use of the Milvus database (vector similarity search engine database). The Milvus database can implement various distance calculation methods between vectors, including Euclidean distance, dot product, etc. The feature vectors of each frame image output by the ISC algorithm model in S1-3 above are also standardized feature vectors, and each feature vector is a unit vector. Therefore, the value of the dot product between any two feature vectors is between 0 and 1. This value can be regarded as the similarity value between two frame images. This includes, but is not limited to, using the dot product to implement various distance calculation methods between vectors. On the one hand, it is relatively simple to understand, and on the other hand, the dot product distance calculation method is also more convenient for subsequent similarity threshold adjustment.

[0165] The first feature vector (Seed Feature Vector) is an initial feature vector used for data clustering and similarity search, which can be used to query neighboring feature vectors that are similar to the first feature vector.

[0166] S904, similar video clustering Figure 11 This is a schematic diagram of another optional video detection method according to an embodiment of this application. The clustering process can be as follows: Figure 11 As shown:

[0167] S1. Create a video similarity matrix for a single user. If a user a has n videos v1, v2, ..., vn, then there will be a similarity value between each pair of videos. Several similarity values ​​can form a similarity matrix, denoted as similar_matrix. The size of the similarity matrix can be represented as n*n, where similar_matrix[i,j] = similar_matrix[j,i], and the similarity between the same video and itself is 1. When i == j, similar[i,j] = 1. When i != j initially, similar_matrix[i,j] = 0, waiting to be filled in subsequent steps.

[0168] S2, search for similar video frames one by one for the frame vectors of all videos of the user, and fill the similarity matrix. For example, for a certain user a certain video v1, the feature vectors of the video frames of this video are f1, f2, f3 in sequence. For each feature vector, the 10 feature vectors with the largest dot product value can be found respectively.

[0169] In the process of finding the 10 feature vectors with the largest dot product, since this application embodiment performs clustering operation on the videos of a single user, it is necessary to limit the search range to uid (user ID) as 'a'. At the same time, each video frame should not be compared with the video frames of its own video. Therefore, conditions such as ignoring the video frames of the current video can also be added. The syntax of the above two search conditions in Milvus is uid='a'&&! (video_uuid_frame_num like'v%'), based on this, the ten most similar video frames of each video frame in the video are obtained as sv1, sv2, sv3...sv10, and their corresponding similarity values ​​s1, s2, s3....s10; at this time, the similarity matrix is ​​filled according to sv1 and the similarity values. In addition, since the length of each video is not consistent, the influence of the video duration on the similarity value also needs to be considered. If two completely identical videos have a similarity value of 1 for each frame image, then after searching for similar video frames one by one for each frame vector of all the user's videos, the value of this video in the similarity matrix should be 1. That is, the cumulative similarity value for each frame should be s / len(v), where s represents the image similarity value and len(v) represents the length of the video (seconds).

[0170] Furthermore, since the similarity matrix requires similar_matrix[i,j] = similar_matrix[j,i], the similarity matrix determination operation needs to be performed twice for each similarity result. That is, the similarity value of each frame image should be divided by 2 again. For each query result sv1, sv2, sv3...sv10, deduplication processing can also be performed, including but not limited to, ensuring that each video can only be matched once for each frame, so that the final video similarity value is less than or equal to 1. In summary, the videos to which the ten video frames searched for each frame belong are sv1, sv2, sv3...sv10, and their corresponding similarity values ​​s1, s2, s3...s10.

[0171] S3 clusters video frames based on similarity values. Clustering methods include, but are not limited to, the DBSCAN clustering algorithm. Unlike k-means and hierarchical classification clustering algorithms, it does not require pre-setting categories or seed points. It clusters video frames based on distance values, resulting in clusters of arbitrary shapes. Furthermore, it is robust to noise points. The usage parameters are shown below:

[0172] DBSCAN(eps=threshold,min_samples=1,metric='precomputed');

[0173] labels=db.fit_predict(distances_matrix);

[0174] Here, eps is the neighborhood radius (epsilon) parameter, that is, video frames whose distance to each other is less than eps will be considered neighbors; min_samples indicates that a video frame is considered a core video frame if its neighborhood contains at least min_samples video frames (including itself); distances_matrix represents the distance matrix between video frames.

[0175] In summary, the similarity matrix is ​​first transformed into a distance matrix using an exponential decreasing transformation without losing its similarity correlation. If the similarity value is x, after the exponential decreasing transformation, the similarity value is updated to exp(-x), which is the natural logarithm raised to the power of -x. Then, the DBSCAN clustering algorithm is used for clustering. If the size of the input distances_matrix matrix is ​​n*n, the output label is a list of length n. When labels[i] == labels[j] and labels[i] >= 0, then i and j belong to the same cluster, thus obtaining the final clustering result of the user's video works.

[0176] This application's embodiments use the similarity between video frames as the basis for clustering. Compared to existing clustering methods based on overall video features, this approach can more finely characterize video similarity, thereby improving clustering accuracy. This is especially true when processing similar videos with mixed effects editing, enabling more accurate identification of similar videos. By clustering video frames based on similarity, subtle changes and features within the video can be better captured, allowing for more accurate grouping of similar videos into the same cluster. Furthermore, the use of the DBSCAN clustering algorithm, which automatically identifies the number of clusters, avoids the limitation of pre-setting the number of clusters in related technologies. In other words, pre-setting the number of clusters in related technologies can lead to inaccurate clustering results or an inability to adapt to variations in different datasets. In this application's embodiments, the DBSCAN clustering algorithm can automatically identify the number of clusters based on the data distribution, eliminating the need for pre-setting. This adaptive characteristic makes this application's embodiments more flexible and accurate when processing video data of different scales and feature distributions.

[0177] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0178] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0179] According to another aspect of the embodiments of this application, a video detection apparatus for implementing the above-described video detection method is also provided. For example... Figure 12 As shown, the device includes:

[0180] The acquisition module 1202 is used to acquire N videos to be classified, where N is a positive integer;

[0181] The determination module 1204 is used to extract frames from each of the N videos based on a predetermined frame extraction interval, and extract features frame by frame to determine the N video frame sequences corresponding to each of the N videos. Among the N video frame sequences, a video frame sequence includes a set of feature vectors, and each feature vector is used to represent a video frame in a video.

[0182] The query module 1206 is used to find similar video frames from the first group of video frame sequences based on the seed video frames in the first video frame sequence. The N video frame sequences consist of the first video frame sequence and the first group of video frame sequences. The similar video frames are video frames whose frame similarity with the seed video frames meets the preset similarity conditions.

[0183] The execution module 1208 is used to determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences, and to obtain a target distance matrix. The second group of video frame sequences represents the video frame sequence to which similar video frames belong. The value of the i-th row and j-th column in the target distance matrix is ​​used to represent the distance between the i-th video and the j-th video in N videos. i and j are both positive integers less than or equal to N.

[0184] Clustering module 1210 is used to perform clustering operations on N videos based on a target distance matrix to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

[0185] As an optional solution, the above-mentioned device is used to find similar video frames from a first set of video frame sequences based on seed video frames in a first video frame sequence in the following manner: each video frame sequence in the N video frame sequences is regarded as a first video frame sequence in turn, and each video frame in the first video frame sequence is regarded as a seed video frame in turn and the following operations are performed: obtaining the first feature vector corresponding to the seed video frame; if the preset similarity condition indicates that the M video frames with the highest similarity are found, the M second feature vectors that satisfy the preset similarity condition are determined by neighbor feature vector search; and the video frames corresponding to the M second feature vectors are determined as similar video frames.

[0186] As an optional approach, the aforementioned device is used to determine M second feature vectors that satisfy the preset similarity conditions by means of neighbor feature vector search, provided that the M video frames with the highest similarity are found under the preset similarity conditions: Given that the M video frames with the highest similarity are found under the preset similarity conditions, a reference feature vector is determined for each video frame in each video frame sequence of the first group of video frame sequences; the dot product between each reference feature vector and the first feature vector is determined; and the M reference feature vectors with the largest dot product values ​​are determined as the M second feature vectors.

[0187] As an optional approach, the aforementioned apparatus is used to determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences in the following manner to obtain a target distance matrix: Each video frame sequence in the N video frame sequences is sequentially regarded as a first video frame sequence, and each video frame in the first video frame sequence is sequentially regarded as a seed video frame. The following operations are performed: an initial distance matrix is ​​created, wherein the initial distance matrix includes N rows and N columns; the target frame similarity between the seed video frame and each similar video frame is determined; each target frame similarity is superimposed on the corresponding second video frame sequence; the distance between the first video frame sequence and each second video frame sequence is determined, wherein the frame similarity includes the target frame similarity; the initial distance matrix is ​​filled according to the distance between the first video frame sequence and each second video frame sequence to obtain the target distance matrix.

[0188] As an optional solution, the above-mentioned device is used to determine the target frame similarity between the seed video frame and each similar video frame in the following manner, and to superimpose each target frame similarity onto the corresponding second video frame sequence to determine the distance between the first video frame sequence and each second video frame sequence: obtaining the target frame similarity between each video frame in the i-th video frame sequence and each similar video frame in the j-th second video frame sequence; performing a superposition operation on each target frame similarity in the j-th second video frame sequence to determine the target distance value, wherein the target distance value represents the distance between the i-th video corresponding to the i-th video frame sequence and the j-th video corresponding to the j-th second video frame sequence; the above-mentioned device is used to fill the initial distance matrix according to the distance between the first video frame sequence and each second video frame sequence in the following manner to obtain the target distance matrix: filling the target distance value into the i-th row and j-th column of the target distance matrix as the value of the element in the i-th row and j-th column of the target distance matrix.

[0189] As an optional approach, the above-mentioned device is used to perform a superposition operation on the similarity of each target frame in the j-th second video frame sequence to determine the target distance value by: obtaining the video length corresponding to the i-th video; performing a superposition operation on the similarity of each target frame based on the video length to determine the target distance value.

[0190] As an optional approach, the aforementioned apparatus is used to determine the target frame similarity between a seed video frame and each similar video frame in the following manner: superimposing each target frame similarity onto the corresponding second video frame sequence, and determining the distance between the first video frame sequence and each second video frame sequence; determining the target frame similarity between a seed video frame and each similar video frame; superimposing each target frame similarity onto the corresponding second video frame sequence, and determining the video similarity between each second video frame sequence in the first video frame sequence and the second group of video frame sequences; performing an exponential decreasing transformation operation on each video similarity, and determining the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences.

[0191] As an optional solution, the above-mentioned device is used to acquire N videos to be classified in the following ways, including at least one of the following: acquiring N videos uploaded by a target account within a first time period; acquiring multiple videos uploaded by a target account within different time periods; determining N videos belonging to the same time period from the multiple videos according to timestamps; acquiring a group of videos uploaded by a group of accounts within a second time period; determining N videos from the group of videos based on account identifiers, wherein the N videos represent videos uploaded by the same account; acquiring multiple groups of videos uploaded by a group of accounts within different time periods; determining N videos belonging to the same time period and belonging to the same account from the multiple groups of videos according to timestamps and based on account identifiers.

[0192] As an optional approach, the aforementioned apparatus is used to perform clustering operations on N videos based on a target distance matrix to determine at least one target video set by: obtaining a predetermined neighborhood radius parameter and a minimum number of points parameter, wherein the neighborhood radius parameter is used to determine the neighboring videos of the target core video, and the minimum number of points parameter is used to determine the minimum number of neighboring videos of the target core video; and performing clustering operations on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine at least one target video set.

[0193] As an optional approach, the aforementioned apparatus is used to perform clustering operations on N videos based on a neighborhood radius parameter and a minimum number of points parameter to determine at least one target video set: For each unvisited target video among the N videos, the following operations are performed: accessing the target video and determining the first neighboring videos of the target video based on the neighborhood radius parameter; if the number of first neighboring videos is greater than or equal to the minimum number of points parameter, marking the target video as a first core video, wherein the target core video includes the first core video; recursively checking each first neighboring video and determining the second neighboring videos corresponding to each first neighboring video that satisfy the neighborhood radius parameter; if the number of second neighboring videos is less than the minimum number of points parameter, determining that the target core video, the first neighboring videos, and the second neighboring videos belong to the same target video set; if the number of second neighboring videos is greater than or equal to the minimum number of points parameter, determining the first neighboring videos as the second core videos, wherein the target core videos include the second core videos.

[0194] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0195] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0196] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program.

[0197] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0198] Figure 13A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0199] It should be noted that, Figure 13 The computer system 1300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0200] like Figure 13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1302 or programs loaded from storage section 1308 into random access memory (RAM). The RAM 1303 also stores various programs and data required for system operation. The CPU 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output interface 1305 (I / O interface) is also connected to the bus 1304.

[0201] The following components are connected to the input / output interface 1305: an input section 1306 including a keyboard, mouse, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a local area network card, modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output interface 1305 as needed. Removable media 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on the drive 1310 as needed so that computer programs read from them can be installed into the storage section 1308 as needed.

[0202] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1309, and / or installed from removable medium 1311. When the computer program is executed by central processing unit 1301, it performs various functions defined in the system of this application.

[0203] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processing unit 1301, it performs various functions provided in the embodiments of this application.

[0204] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described video detection method is also provided. This electronic device may be... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 14 As shown, the electronic device includes a memory 1402 and a processor 1404. The memory 1402 stores a computer program, and the processor 1404 is configured to execute the steps of any of the above method embodiments via the computer program.

[0205] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0206] Optionally, in this embodiment, the processor may be configured to execute the methods in the embodiments of this application via a computer program.

[0207] Alternatively, as those skilled in the art will understand, Figure 14 The structure shown is for illustrative purposes only. Figure 14 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 14 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 14 The different configurations shown.

[0208] The memory 1402 can be used to store software programs and modules, such as the program instructions / modules corresponding to the video detection method and apparatus in this embodiment. The processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, thereby realizing the aforementioned video detection method. The memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1402 may further include memory remotely located relative to the processor 1404, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1402 may be used, but is not limited to, to store the aforementioned N video and other information. As an example, such as... Figure 14 As shown, the memory 1402 may include, but is not limited to, the acquisition module 1202, determination module 1204, query module 1206, execution module 1208, and clustering module 1210 from the video detection device. Furthermore, it may include, but is not limited to, other module units from the video detection device, which will not be elaborated upon in this example.

[0209] Optionally, the transmission device 1406 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1406 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1406 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0210] In addition, the aforementioned electronic device also includes: a display 1408 for displaying the aforementioned N videos; and a connection bus 1410 for connecting the various module components in the aforementioned electronic device.

[0211] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0212] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of an electronic device reads computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the electronic device to perform the video detection method provided in the various alternative implementations of the above-described video detection aspect.

[0213] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store methods for performing the embodiments of this application.

[0214] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0215] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0216] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0217] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0218] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0219] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0220] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0221] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A video detection method, characterized in that, include: Obtain N videos to be classified, where N is a positive integer; Based on a predetermined frame extraction interval, each of the N videos is sampled, and features are extracted frame by frame to determine the N video frame sequences corresponding to each of the N videos. Among the N video frame sequences, one video frame sequence includes a set of feature vectors, and each feature vector is used to represent a video frame in a video. Based on the seed video frames in the first video frame sequence, similar video frames are found from the first group of video frame sequences. The N video frame sequences consist of the first video frame sequence and the first group of video frame sequences. The similar video frames represent video frames whose frame similarity with the seed video frames meets a preset similarity condition. Determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences to obtain a target distance matrix, wherein the second group of video frame sequences represents the video frame sequence to which the similar video frames belong, and the value of the i-th row and j-th column in the target distance matrix is ​​used to represent the distance between the i-th video and the j-th video in the N videos, where i and j are both positive integers less than or equal to N; Clustering is performed on the N videos based on the target distance matrix to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

2. The method according to claim 1, characterized in that, The step of finding similar video frames from the first set of video frame sequences based on seed video frames in the first video frame sequence includes: Each of the N video frame sequences is sequentially treated as the first video frame sequence, and each video frame in the first video frame sequence is sequentially treated as the seed video frame, and the following operations are performed: Obtain the first feature vector corresponding to the seed video frame; If the preset similarity condition indicates that the M video frames with the highest similarity are found, then the M second feature vectors that satisfy the preset similarity condition are determined by neighbor feature vector search. The video frames corresponding to the M second feature vectors are identified as the similar video frames.

3. The method according to claim 2, characterized in that, When the M video frames with the highest similarity are found according to the preset similarity condition, the M second feature vectors that satisfy the preset similarity condition are determined by neighbor feature vector search, including: When the preset similarity condition indicates that the M video frames with the highest similarity are found, the reference feature vector corresponding to each video frame in each video frame sequence in the first group of video frame sequences is determined. Determine the dot product between each of the reference feature vectors and the first feature vector; The M reference feature vectors with the largest dot product values ​​are determined as the M second feature vectors.

4. The method according to claim 1, characterized in that, Determining the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences to obtain the target distance matrix includes: Each of the N video frame sequences is sequentially treated as the first video frame sequence, and each video frame in the first video frame sequence is sequentially treated as the seed video frame, and the following operations are performed: Create an initial distance matrix, wherein the initial distance matrix comprises N rows and N columns; Determine the target frame similarity between the seed video frame and each of the similar video frames, superimpose each target frame similarity onto the corresponding second video frame sequence, and determine the distance between the first video frame sequence and each of the second video frames, wherein the frame similarity includes the target frame similarity; The target distance matrix is ​​obtained by filling the initial distance matrix with the distance between the first video frame sequence and each of the second video frame sequences.

5. The method according to claim 4, characterized in that, The step of determining the target frame similarity between the seed video frame and each of the similar video frames, superimposing each target frame similarity onto the corresponding second video frame sequence, and determining the distance between the first video frame sequence and each of the second video frame sequences includes: obtaining the target frame similarity between each video frame in the i-th video frame sequence and each of the similar video frames in the j-th second video frame sequence; performing a superposition operation on each target frame similarity in the j-th second video frame sequence to determine a target distance value, wherein the target distance value represents the distance between the i-th video corresponding to the i-th video frame sequence and the j-th video corresponding to the j-th second video frame sequence; The step of filling the initial distance matrix with the distance between the first video frame sequence and each of the second video frame sequences to obtain the target distance matrix includes: filling the target distance value into the i-th row and j-th column of the target distance matrix as the value of the element in the i-th row and j-th column of the target distance matrix.

6. The method according to claim 5, characterized in that, The step of performing a superposition operation on the similarity of each target frame in the j-th second video frame sequence to determine the target distance value includes: Get the length of the i-th video; Based on the video length, the similarity of each target frame is superimposed to determine the target distance value.

7. The method according to claim 4, characterized in that, The step of determining the target frame similarity between the seed video frame and each of the similar video frames, superimposing each target frame similarity onto the corresponding second video frame sequence, and determining the distance between the first video frame sequence and each of the second video sequences includes: Determine the target frame similarity between the seed video frame and each of the similar video frames; The similarity of each target frame is superimposed on the corresponding second video frame sequence to determine the video similarity between each second video frame sequence in the first video frame sequence and the second group of video frame sequences. An exponentially decreasing transformation operation is performed on each of the video similarities to determine the distance between each of the second video frame sequences in the first video frame sequence and the second group of video frame sequences.

8. The method according to claim 1, characterized in that, The acquisition of N videos to be classified includes at least one of the following: Obtain the N videos uploaded by the target account within the first time period; Get multiple videos uploaded by the target account within different time periods; Based on the timestamps, determine the N videos that belong to the same time period from the plurality of videos; Get a set of videos uploaded by a group of accounts within the second time period; The N videos are determined from the set of videos based on account identifiers, wherein the N videos represent videos uploaded by the same account; Get multiple sets of videos uploaded by a group of accounts within different time periods; determine the N videos belonging to the same time period and the same account from the multiple sets of videos according to the timestamp and account identifier.

9. The method according to claim 1, characterized in that, The step of performing clustering operations on the N videos based on the target distance matrix to determine at least one target video set includes: Obtain a predetermined neighborhood radius parameter and a minimum number of points parameter, wherein the neighborhood radius parameter is used to determine the neighboring videos of the target core video, and the minimum number of points parameter is used to determine the minimum number of the neighboring videos of the target core video; Clustering operations are performed on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set.

10. The method according to claim 9, characterized in that, The step of performing clustering operations on the N videos based on the neighborhood radius parameter and the minimum number of points parameter to determine the at least one target video set includes: For each unvisited target video among the N videos, perform the following operations: Access the target video and determine the first neighbor video of the target video based on the neighborhood radius parameter; If the number of the first neighbor videos is greater than or equal to the minimum number of points parameter, the target video is marked as the first core video, wherein the target core video includes the first core video; Recursively examine each first neighbor video to determine the second neighbor video that satisfies the neighborhood radius parameter for each first neighbor video. If the number of the second neighbor videos is less than the minimum number of points parameter, the target core video, the first neighbor video, and the second neighbor video are determined to belong to the same target video set; If the number of the second neighbor videos is greater than or equal to the minimum number of points parameter, the first neighbor video is determined as the second core video, wherein the target core video includes the second core video.

11. A video detection device, characterized in that, include: The acquisition module is used to acquire N videos to be classified, where N is a positive integer; The determination module is used to extract frames from each of the N videos based on a predetermined frame extraction interval, and extract features frame by frame to determine the N video frame sequences corresponding to each of the N videos. Among the N video frame sequences, one video frame sequence includes a set of feature vectors, and each feature vector is used to represent a video frame in a video. The query module is used to find similar video frames from a first group of video frame sequences based on seed video frames in a first video frame sequence. The N video frame sequences consist of the first video frame sequence and the first group of video frame sequences. The similar video frames represent video frames whose frame similarity with the seed video frames meets a preset similarity condition. An execution module is used to determine the distance between each second video frame sequence in the first video frame sequence and the second group of video frame sequences, and to obtain a target distance matrix. The second group of video frame sequences represents the video frame sequence to which the similar video frames belong. The value of the i-th row and j-th column in the target distance matrix is ​​used to represent the distance between the i-th video and the j-th video in the N videos. i and j are both positive integers less than or equal to N. The clustering module is used to perform clustering operations on the N videos based on the target distance matrix to determine at least one target video set, wherein the target video set includes a target core video, the distance between the target core video and other videos in the target video set satisfies a preset distance condition, and the number of other videos satisfies a preset number condition.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 10.

13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 10.

14. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 10 through the computer program.

Images