A video detection method and related apparatus

Abstract

The application discloses a video detection method and related device, and relates to the technical field of software, which comprises the following steps: extracting first segment features and first single-frame features of a video to be examined; calling a pre-constructed copyright database, wherein the copyright database comprises a vector database and a key-value library, the vector database records second segment features of a copyright video and time intervals thereof, and the key-value library records second single-frame features of the copyright video and time stamps thereof; searching the second segment features by using the first segment features, and determining a target time interval hit by the video to be examined according to a search result; pulling part of the single-frame features in the second single-frame features with time stamps in the target time interval, and determining a detection result of the video to be examined by comparing the first single-frame features with the pulled part of the single-frame features. Through the cooperative processing of coarse search positioning and precision checking, the application can improve the accuracy and reliability of video copyright detection, and realizes accurate judgment on video infringement behavior.

Description

Technical Field

[0001] This application relates to the field of software technology, and in particular to a video detection method and related apparatus. Background Technology

[0002] With the rapid development of internet technology and the widespread adoption of smart terminals, digital content such as short videos and long videos has experienced explosive growth, and video copyright protection has become a focus of industry attention.

[0003] To address the need to compare massive amounts of user-uploaded content with a vast copyright database, current methods primarily utilize convolutional neural networks or Transformer models to extract high-dimensional feature vectors from video frames. These vectors are then compressed into a single global feature vector through operations such as global average pooling, and finally, cosine similarity is calculated to determine the similarity of video content.

[0004] However, due to the lack of refined temporal verification logic in the feature compression method, it is difficult to accurately distinguish real continuous infringing segments when faced with edited, re-encoded or videos containing common redundant segments, resulting in a large number of false alarms and false negatives, which seriously affects the accuracy and reliability of video copyright confirmation. Summary of the Invention

[0005] In view of the above problems, this application provides a video detection method and related apparatus to improve the accuracy and reliability of video copyright detection. The specific solution is as follows:

[0006] A first aspect of this application provides a video detection method, the video detection method comprising:

[0007] Extract the features of the first segment and the first single frame of the video submitted for review;

[0008] The pre-built copyright database is retrieved. The copyright database includes a vector database and a key-value database. The vector database records the second segment features of the copyrighted video and its time interval. The key-value database records the second single-frame features of the copyrighted video and its timestamp.

[0009] The first segment feature is used to search for the second segment feature, and the target time interval of the video to be reviewed is determined based on the search results;

[0010] Extract a portion of the single-frame features whose timestamps are located within the target time interval from the second single-frame features, and determine the detection result of the video to be reviewed by comparing the first single-frame features with the extracted portion of the single-frame features.

[0011] In one possible implementation, the process of constructing the copyright database includes:

[0012] Redundancy cleaning is performed on the copyrighted video by extracting the perceptual hash value of the copyrighted video per unit time.

[0013] Extract the second segment features and the second single frame features from the copyrighted video after redundancy cleaning;

[0014] The second segment features and their time intervals are stored in the vector database by establishing a hierarchical navigation small-world index;

[0015] The second single-frame feature and its timestamp are stored in the key-value library.

[0016] In one possible implementation, determining the detection result of the video to be reviewed by comparing the first single-frame features with the retrieved partial single-frame features includes:

[0017] Among the retrieved single-frame features, identify the target single-frame feature that has the highest similarity to the first single-frame feature and a similarity greater than the corresponding threshold.

[0018] Obtain the timestamp of the first single-frame feature in the video under review, and sort the target single-frame features in ascending order of the timestamp of the first single-frame feature;

[0019] Based on the sorting result of the target single-frame features, detect whether the timestamps of the target single-frame features have temporal monotonicity;

[0020] If the timestamp of the target single-frame feature has temporal monotonicity, the detection result of the video to be reviewed is determined based on the number of consecutive frames of the target single-frame feature.

[0021] In one possible implementation, determining the target single-frame feature with the highest similarity to the first single-frame feature among the retrieved single-frame features includes:

[0022] Among the extracted single-frame features, candidate single-frame features with a similarity greater than the corresponding threshold to the first single-frame feature are identified.

[0023] The corresponding time offset is determined based on the timestamps of the first single-frame feature and the candidate single-frame feature;

[0024] The candidate single-frame features are filtered using the time offset, and the single-frame feature with the highest similarity among the filtered candidate single-frame features is determined as the target single-frame feature.

[0025] In one possible implementation, detecting whether the timestamps of the target single-frame features have temporal monotonicity according to the sorting result of the target single-frame features includes:

[0026] According to the sorting result of the target single-frame features, traverse two consecutive single-frame features in the target single-frame features;

[0027] If the timestamp of the latter single-frame feature is later than the timestamp of the former single-frame feature among the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature has temporal monotonicity.

[0028] If the timestamp of the latter single-frame feature is earlier than the timestamp of the former single-frame feature in the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity.

[0029] If the timestamp of the latter single-frame feature is equal to the timestamp of the former single-frame feature among the two single-frame features traversed, the cumulative value of the number of pauses is incremented by 1, and if the current cumulative value of the number of pauses is greater than the corresponding threshold, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity.

[0030] A second aspect of this application provides a video detection device, the video detection device comprising:

[0031] A database construction module is used to construct a copyright database, which includes a vector database and a key-value library. The vector database records the second segment features of the copyrighted video and its time interval, and the key-value library records the second single-frame features of the copyrighted video and its timestamp.

[0032] The video detection module is used to extract the first segment features and the first single-frame features of the video under review; retrieve the copyright database; search the second segment features using the first segment features, and determine the target time interval of the video under review based on the search results; pull out a portion of the single-frame features whose timestamps are located in the target time interval from the second single-frame features, and determine the detection result of the video under review by comparing the first single-frame features with the pulled portion of the single-frame features.

[0033] In one possible implementation, the database construction module is specifically used for:

[0034] The copyrighted video is redundancy cleaned by extracting its perceptual hash value per unit time; the second segment feature and the second single frame feature are extracted from the redundancy cleaned copyrighted video; the second segment feature and its time interval are stored in the vector database by establishing a hierarchical navigation small-world index; and the second single frame feature and its timestamp are stored in the key-value library.

[0035] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the video detection method described in the first aspect or any implementation thereof.

[0036] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0037] The memory is used to store computer programs;

[0038] The processor is used to execute the computer program so that the electronic device can implement the video detection method of the first aspect or any implementation thereof.

[0039] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the video detection method described in the first aspect or any implementation thereof.

[0040] Using the above technical solution, this application provides a video detection method and related apparatus, comprising: extracting first segment features and first single-frame features of the video under review; retrieving a pre-constructed copyright database, the copyright database including a vector database and a key-value database, the vector database recording second segment features of the copyrighted video and their time intervals, and the key-value database recording second single-frame features of the copyrighted video and their timestamps; retrieving the second segment features using the first segment features, and determining the target time interval of the video under review based on the retrieval results; extracting a portion of the single-frame features whose timestamps are located within the target time interval from the second single-frame features, and determining the detection result of the video under review by comparing the first single-frame features with the extracted portion of the single-frame features. This application can quickly locate the target time interval that the video under review may hit by searching the vector database. This process avoids the huge computational overhead caused by scanning the entire database. In addition, it only pulls some single-frame features from the key value library that are within the target time interval for fine-grained comparison. This effectively avoids the problem of loss of temporal information caused by global feature compression. Through the coordinated processing of coarse search and location and precision verification, the accuracy and reliability of video copyright detection can be improved, and accurate determination of video infringement can be achieved. Attached Figure Description

[0041] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0042] Figure 1 A flowchart illustrating a video detection method provided in an embodiment of this application;

[0043] Figure 2 This is a partial flowchart illustrating a video detection method provided in an embodiment of this application;

[0044] Figure 3 This is another schematic flowchart of a video detection method provided in an embodiment of this application;

[0045] Figure 4 This is a partial flowchart illustrating a video detection method provided in an embodiment of this application.

[0046] Figure 5 This is another part of the flowchart of a video detection method provided in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the structure of a video detection device provided in an embodiment of this application;

[0048] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0049] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0050] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0051] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0052] To improve the accuracy and reliability of video copyright detection, this application provides a video detection method. The video detection method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0053] See Figure 1 , Figure 1 This is a flowchart illustrating a video detection method provided in an embodiment of this application. Figure 1 As shown in the figure, a video detection method provided in this application embodiment may include steps S101 to S104, which are described in detail below.

[0054] S101, extract the features of the first segment and the first single frame of the video to be reviewed.

[0055] In this embodiment, the first segment feature can characterize the overall trend and salient features of the image within a continuous time period in the video under review, while the first single-frame feature is a frame-level feature characterizing the absolute image content at a specific moment in the video under review. Each frame or keyframe of the video under review is independently encoded using a pre-trained visual model to obtain the first single-frame feature. Specifically, the visual model can be a self-supervised visual Transformer model, which can extract high-quality, robust high-dimensional feature vectors from images to characterize the semantic and visual information of the image. Furthermore, the video under review is divided into multiple sliding windows, and a pooling algorithm is used to compress and fuse all the first single-frame features within each sliding window to obtain the corresponding first segment feature.

[0056] For example, assuming a sliding window size of 10 frames and a stride of 5 frames, a fusion operation combining mean pooling and max pooling is performed on the 10 first single-frame features within the sliding window to generate a first segment feature with fixed dimensions. This first segment feature retains both local salient texture information and reflects the dynamic change trend within the sliding window. This local multi-frame fusion method effectively avoids the complete loss of temporal information caused by global average pooling of the entire long video, ensuring that subsequent retrieval can perceive the temporal progression logic of the video content.

[0057] S102, retrieve the pre-built copyright database. The copyright database includes a vector database and a key-value database. The vector database records the second segment features of the copyrighted video and its time interval, while the key-value database records the second single-frame features of the copyrighted video and its timestamp.

[0058] In this embodiment, the copyright database is used to store preprocessed massive amounts of copyright video feature data, including a vector database and a key-value database. The vector database stores dense feature vectors and supports approximate nearest neighbor search. It records the second segment features of the copyright video, which are segment-level features isomorphic to the first segment features of the video under review. It also records the corresponding copyright video ID and time interval, including the start and end frame numbers. The key-value database is a high-speed data storage structure based on key-value pairs. It records the second single-frame features of the copyright video, which are frame-level features of the copyright video. The key is a unique identifier, and the value contains the second single-frame feature and its precise timestamp.

[0059] It should be noted that vector databases can use open-source vector search engines such as Milvus, and utilize hierarchical navigation small-world index (HNSW) technology to store second-segment features, so as to achieve massive vector retrieval in milliseconds.

[0060] In one possible implementation, a redundancy cleaning scheme based on perceptual hash values ​​is introduced. This effectively removes common opening and closing sequences and advertising content from videos within the same album, preventing these frequently repeated segments from being misjudged as core infringing content in subsequent detections and significantly reducing the false positive rate. Building upon this, a second segment feature generated by local sliding window fusion is used to replace the global average pooling feature, preserving temporal information within short video segments and resolving the temporal loss problem caused by feature compression in long videos. Furthermore, a vector database is constructed using a hierarchical navigation small-world index to achieve millisecond-level coarse-grained localization, and a key-value library is used to store high-precision second single-frame features to support memory-level frame-by-frame fine-grained localization, forming a collaborative processing mechanism for coarse-grained localization and precision verification.

[0061] See Figure 2 , Figure 2 This is a partial flowchart illustrating a video detection method provided in an embodiment of this application. Figure 2 As shown in the embodiment of this application, a video detection method is provided, wherein the process of constructing a copyright database may include steps S201 to S204, which are described in detail below.

[0062] S201 performs redundancy cleaning on copyrighted videos by extracting the perceptual hash value of the copyrighted video per unit time.

[0063] In this embodiment, the unit of time is a preset time granularity. Taking 1 second as an example, all copyrighted videos under the same album are traversed, and the perceptual hash value of each copyrighted video per second is extracted. The frequency of the same hash value appearing in different copyrighted videos is counted. If the frequency of a certain hash value is greater than a preset redundancy threshold, the corresponding scene is determined to be common content, such as an intro, outro, or advertisement. Furthermore, common content that appears repeatedly in different copyrighted videos of the same album is removed, thereby achieving redundancy cleaning of each copyrighted video. By using cross-video deduplication based on perceptual hash values, it can be ensured that the features added to the database only come from core content with distinguishability, thus avoiding false positives in subsequent detections caused by common content from the source.

[0064] S202, extract the second segment features and the second single frame features from the copyrighted video after redundancy cleaning.

[0065] In this embodiment of the application, the second segment feature can characterize the overall trend and salient features of the picture within a continuous time period in the copyrighted video, while the second single frame feature is a frame-level feature that characterizes the absolute picture content at a specific moment in the copyrighted video.

[0066] For copyrighted videos that have undergone redundancy cleaning, each frame or keyframe of the video can be independently encoded using a pre-trained visual model to obtain second single-frame features. Specifically, the visual model can be a self-supervised visual Transformer model. Then, the video is divided into multiple sliding windows, and a pooling algorithm is used to compress and fuse all second single-frame features within each sliding window to obtain the corresponding second segment features.

[0067] It should be noted that the extraction method for the second single-frame feature can refer to the extraction method for the first single-frame feature, and the extraction method for the second segment feature can refer to the extraction method for the first segment feature. The second segment feature is used to quickly locate the approximate time interval during the retrieval stage, while the second single-frame feature is used for precise frame-level comparison during the detailed examination stage. The two are used together to preserve local temporal details and achieve effective compression of data volume.

[0068] S203, by establishing a hierarchical navigation small-world index, stores the features of the second segment and its time interval in the vector database.

[0069] A vector database is a database system specifically designed for storing and retrieving high-dimensional dense vectors. In this embodiment, the second segment feature vector is bound to its corresponding time interval and inserted into the vector database as a record. During the insertion process, a hierarchical navigation small-world index is automatically constructed. This hierarchical navigation small-world index is used to construct a multi-level navigation graph structure to achieve rapid retrieval of the most similar results from massive vectors. The time interval includes the start and end timestamps of the video corresponding to the second segment feature.

[0070] S204, store the second single-frame feature and its timestamp into the key-value library.

[0071] A key-value store is a high-speed storage system that stores data in key-value pairs. In this embodiment, the timestamp of the second single-frame feature in the copyrighted video is used as the key in the key-value pair, and the second single-frame feature is used as the value and stored in the key-value store. Specifically, after the second single-frame feature is extracted, it can be written into the key-value store according to its timestamp order.

[0072] In this embodiment, a copyright database with anti-redundancy interference capability and high retrieval efficiency is constructed. The vector database is responsible for coarse-grained, rapid time interval positioning, while the key-value database is responsible for quickly retrieving specific single-frame features based on timestamps after determining the time interval. By storing the second single-frame feature independently in the key-value database, the retrieval latency and storage cost pressure caused by storing massive amounts of fine-grained data in the vector database can be avoided, achieving efficient collaboration between coarse and fine searches.

[0073] S103, use the first segment feature to search for the second segment feature, and determine the target time interval of the video to be reviewed based on the search results.

[0074] In this embodiment, the first segment feature is used as a query vector and input into the vector database. By calculating the similarity (e.g., cosine similarity) between the query vector and the second segment features in the vector database, at least one second segment feature with the highest similarity score is recalled. The time interval of the recalled second segment feature in the copyrighted video is then used as the target time interval. When searching the vector database, the efficiency of the hierarchical navigation small-world index is utilized to quickly filter out the potentially infringing target time interval from a massive number of second segment features without traversing the entire database. This significantly narrows the data search range for subsequent fine-grained comparisons and substantially reduces computational overhead.

[0075] S104: Extract some single-frame features whose timestamps are located in the target time interval from the second single-frame features, and determine the detection result of the video to be reviewed by comparing the first single-frame features with the extracted partial single-frame features.

[0076] In this embodiment, based on a determined target time interval, all second single-frame features with timestamps within that target time interval can be accurately extracted from the key-value library and retrieved into application memory. Further, in application memory, the first single-frame features are compared one by one with each retrieved second single-frame feature, and the frame-level matching degree is determined through similarity calculation. In addition, the detection result is a comprehensive judgment based on frame-level comparison, considering not only the similarity score of a single frame but also implicitly assessing the number and distribution density of matching frames. For example, after locking the target time interval to 100-110 seconds of the copyrighted video, all second single-frame features with timestamps within that target time interval are retrieved from the key-value library, and then the cosine similarity between the first single-frame feature of the video under review and all retrieved second single-frame features is calculated. If the cosine similarity of multiple consecutive frames is greater than the corresponding similarity threshold, and the number of matching frames reaches the corresponding minimum number, then the video under review is determined to have infringing behavior, and a detection result confirming the copyright is output.

[0077] In this embodiment, through progressive processing from coarse localization at the fragment level to precise comparison at the single frame level, and hierarchical calling of vector database and key value library, not only can extremely fast retrieval from massive video libraries be achieved, but also through a frame-level secondary verification mechanism, the true continuous infringing segments and accidental feature noise collisions can be effectively distinguished, significantly improving the precision and recall of video copyright detection.

[0078] In one possible implementation, by filtering the target single-frame features with the highest similarity and exceeding the threshold from the extracted single features, the purification from massive coarse search results to a precise candidate set is completed, eliminating the interference of low-confidence matching. Then, by using the ascending order of timestamps, a time mapping sequence that conforms to the playback logic is forcibly constructed, providing a structured data foundation for time series analysis. Furthermore, the temporal monotonicity detection is introduced, using the physical law that video content must develop unidirectionally with time, to accurately identify and eliminate accidental collision noises that are similar in single frames but have disordered temporal order (such as time reversal), thereby accurately distinguishing between real continuous theft and scattered feature coincidences. Finally, a quantitative judgment is made based on the number of features that pass the verification, ensuring that only matching segments with a certain duration and temporal continuity are identified as infringement.

[0079] See Figure 3 , Figure 3 This is another schematic flowchart illustrating a video detection method provided in an embodiment of this application. Figure 3 As shown in the embodiment of this application, a video detection method is provided, wherein step S104, "determining the detection result of the video to be reviewed by comparing the first single-frame feature with the partially extracted single-frame feature", may include steps S301 to S304, which are described in detail below.

[0080] S301, among the retrieved single-frame features, determine the target single-frame feature that has the highest similarity to the first single-frame feature and is greater than the corresponding threshold.

[0081] In this embodiment, the similarity, such as cosine similarity, between the first single-frame feature and each second single-frame feature in the retrieved partial single-frame features is calculated. Then, the second single-frame feature with the highest similarity (greater than a corresponding threshold) is selected as the target single-frame feature. This selection ensures that the frame-level features entering the subsequent temporal verification stage are the optimal match at the current moment, facilitating the subsequent construction of accurate timeline correspondences.

[0082] In one possible implementation, a coarse screening is first performed using a similarity threshold, followed by the introduction of time offset calculations to capture temporal alignment patterns between videos. Finally, a secondary fine screening is conducted based on the consistency of the main offsets to lock in the optimal solution. This approach not only utilizes the visual semantic correlation between the first single-frame features and candidate single-frame features but also deeply explores their causal dependency on the timeline. By using clustering filtering based on time offsets, false matches caused by similar scenes but asynchronous plot developments can be automatically identified and eliminated, ensuring that the final target single-frame features are truly consecutive frames belonging to the same infringing segment. This improves the accuracy of the video copyright detection system in complex editing and noisy environments.

[0083] See Figure 4 , Figure 4 This is a further schematic flowchart of a video detection method provided in an embodiment of this application. Figure 4 As shown in the embodiment of this application, a video detection method is provided, wherein step S301, "determining the target single-frame feature that has the highest similarity to the first single-frame feature and a similarity greater than the corresponding threshold among the extracted single-frame features", may include steps S401 to S403, which are described in detail below.

[0084] S401, among the retrieved partial single-frame features, determine candidate single-frame features whose similarity to the first single-frame feature is greater than the corresponding threshold.

[0085] In this embodiment, the similarity between the first single-frame feature and each second single-frame feature in the retrieved partial single-frame features is calculated, and the second single-frame features with a similarity greater than the corresponding threshold are selected as candidate single-frame features. This threshold truncation method can significantly reduce the data load of subsequent time offset calculations while retaining all potential matching possibilities.

[0086] S402, determine the corresponding time offset based on the timestamps of the first single-frame feature and the candidate single-frame feature.

[0087] In this embodiment, the time offset is the difference between the timestamp of the first single-frame feature in the video under review and the timestamp of the candidate single-frame feature in the copyrighted video, used to characterize the relative misalignment of the two videos on the timeline. For example, if the 10-second frame of the video under review is similar to the 15-second frame of the copyrighted video, the time offset is -5 seconds. By introducing the time offset, simple image similarity comparison can be transformed into a dual verification of image content and temporal rhythm, providing key temporal evidence for identifying real continuous infringing segments.

[0088] S403 filters candidate single-frame features by time offset and selects the single-frame feature with the highest similarity from the filtered candidate single-frame features as the target single-frame feature.

[0089] Based on the principle that genuine infringing videos typically exhibit stable linear temporal transformations, such as overall fast-forwarding, slow-motion, or fixed clip offsets, while time offsets generated by random noise collisions present a discrete and disordered distribution, this application embodiment statistically analyzes the distribution frequency of time offsets corresponding to all candidate single-frame features, extracts the most frequent principal offset, and filters candidate single-frame features whose time offsets differ from the principal offset within a preset tolerance range. Specifically, firstly, cluster analysis or histogram statistics are performed on all calculated time offsets to find the mode as the principal offset; then, candidate single-frame features whose time offsets deviate too much from the principal offset, such as exceeding ±2 seconds, are eliminated, determining that these candidate single-frame features are accidental visual similarities rather than true temporal correspondences.

[0090] Based on this, the single-frame feature with the highest similarity among the selected candidate single-frame features is determined as the target single-frame feature. This effectively avoids misjudgment caused by similar local images but conflicting temporal logic, and ensures that the selected target single-frame feature is not only highly similar visually, but also maintains strict consistency with the video under review in terms of temporal evolution logic.

[0091] S302, obtain the timestamp of the first single-frame feature in the video under review, and sort the target single-frame features in ascending order of the timestamp of the first single-frame feature.

[0092] In this embodiment, the source data information corresponding to the target single-frame feature is read, that is, the timestamp of its corresponding first single-frame feature on the original timeline of the video under review is obtained. Based on this timestamp, the target single-frame features are reordered in ascending order of the timestamp. This ascending order not only simulates the process of playing the video frame by frame, but also enables subsequent steps to traverse and check the continuity of the timeline in a linear manner.

[0093] S303, based on the sorting results of the target single-frame features, detect whether the timestamps of the target single-frame features have temporal monotonicity.

[0094] In this embodiment, temporal monotonicity means that as the timeline of the video under review progresses, the timeline of the corresponding copyrighted video segment should also show a synchronous forward progression or a brief pause. Based on the sorting result of the target single-frame features, it is determined whether the timestamp of the subsequent single-frame feature in the copyrighted video is not earlier than the timestamp of the preceding single-frame feature. If the timestamp of the subsequent single-frame feature is not earlier than the timestamp of the preceding single-frame feature, then the target single-frame feature is determined to have temporal monotonicity; conversely, if the timestamp of the subsequent single-frame feature is earlier than the timestamp of the preceding single-frame feature, then the target single-frame feature is determined not to have temporal monotonicity.

[0095] In one possible implementation, the sorted sequence of target single-frame features is traversed, and the timestamps of two consecutive single-frame features in the copyrighted video are compared sequentially. If the timestamp of the later single-frame feature is greater than or equal to the timestamp of the earlier single-frame feature, it is determined to have temporal monotonicity; if the timestamp of the later single-frame feature is less than the timestamp of the earlier single-frame feature, it is determined not to have temporal monotonicity, indicating a time reversal. This matching result is most likely scattered noise generated by the visual model's misjudgment of different scenes, rather than continuous stolen segments. In addition, for cases where timestamps are equal (i.e., the scene is frozen), the number of freezes can be accumulated. Temporal monotonicity is determined only when the accumulated number of freezes does not exceed the corresponding threshold (e.g., allowing 3 consecutive frozen frames).

[0096] See Figure 5 , Figure 5 This is another schematic flowchart illustrating a video detection method provided in an embodiment of this application. Figure 5 As shown in the embodiment of this application, a video detection method is provided, wherein step S303, "detecting whether the timestamp of the target single frame feature has temporal monotonicity according to the sorting result of the target single frame feature", may include steps S501 to S504, which are described in detail below.

[0097] S501, traverse two consecutive single-frame features in the target single-frame features according to the sorting result of the target single-frame features.

[0098] In this embodiment, a traversal pointer is initialized. Starting from the beginning position of the sorted target single-frame features, two adjacent single-frame features are selected sequentially as the current comparison objects, defined as the previous single-frame feature and the next single-frame feature, respectively. Through this sliding window-style pairwise comparison method, minute changes on the timeline can be captured frame by frame.

[0099] S502, if the timestamp of the latter single-frame feature is later than the timestamp of the former single-frame feature among the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature has temporal monotonicity.

[0100] In this embodiment of the application, the timestamps of two adjacent single-frame features in the copyrighted video are compared. If the timestamp of the later single-frame feature is greater than the timestamp of the earlier single-frame feature, it is determined that the timestamp of the target single-frame feature has temporal monotonicity, indicating that the order of the scene evolution in the video under review is consistent with the original shooting order in the copyrighted video, and no editing error has occurred.

[0101] S503, if the timestamp of the latter single-frame feature is earlier than the timestamp of the former single-frame feature among the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity.

[0102] In this embodiment of the application, the timestamps of two adjacent single-frame features in the copyrighted video are compared. If the timestamp of the later single-frame feature is less than the timestamp of the earlier single-frame feature, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity. This may be due to the non-linear editing and splicing of the video under review, or due to accidental collision noise generated by the visual model.

[0103] S504, if the timestamp of the latter single-frame feature is equal to the timestamp of the former single-frame feature among the two single-frame features traversed, increment the cumulative value of the number of pauses by 1, and if the current cumulative value of the number of pauses is greater than the corresponding threshold, determine that the timestamp of the target single-frame feature does not have temporal monotonicity.

[0104] In this embodiment, a fault-tolerance mechanism is introduced to prevent false positives in static scenes or looping segments in the video, while an upper limit is set to filter abnormal stillness. The timestamps of two adjacent single-frame features in the copyrighted video are compared. If the timestamp of the later single-frame feature equals the timestamp of the earlier single-frame feature, a counter for the number of pauses is started, and its current cumulative value is incremented by 1. If the current cumulative value of the number of pauses is greater than the corresponding threshold, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity; if the current cumulative value of the number of pauses is less than or equal to the corresponding threshold, it is considered a legitimate brief pause, and the determination of temporal monotonicity continues until the target single-frame feature has been traversed. If the current cumulative value of the number of pauses is still less than or equal to the corresponding threshold, then it is determined that the timestamp of the target single-frame feature has temporal monotonicity.

[0105] For example, setting a threshold of 3 times corresponds to approximately 0.1 to 0.3 seconds of stillness. If four consecutive pairs of single-frame features have identical timestamps, it indicates that the video under review might be an infinitely stretched static image or experience severe signal interruption in that segment, rather than normal video stream content. In this case, it is judged as lacking temporal monotonicity. This approach encompasses normal film and television language, such as stop-motion effects, while eliminating false high-similarity matches caused by using static images to evade detection or pure noise.

[0106] S304. If the timestamp of the target single-frame feature has temporal monotonicity, the detection result of the video to be reviewed is determined based on the number of consecutive frames of the target single-frame feature.

[0107] In this embodiment, if the timestamp of the target single-frame feature has temporal monotonicity, the number of consecutive frames of the target single-frame feature is counted. This number of consecutive frames can characterize the duration and certainty of the infringing segment. If the number of consecutive frames of the target single-frame feature is greater than or equal to the corresponding threshold, the video under review is determined to constitute copyright infringement; conversely, if the number of consecutive frames of the target single-frame feature is less than the corresponding threshold, the video under review is determined not to constitute copyright infringement.

[0108] Based on the above description, the video detection method provided in this application, through the coordinated processing of coarse location and precision verification, can improve the accuracy and reliability of video copyright detection and achieve accurate determination of video infringement.

[0109] The above describes a video detection method provided by the embodiments of this application. The following will describe the apparatus for performing the above video detection method.

[0110] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a video detection device provided in an embodiment of this application. Figure 6 As shown in the figure, an embodiment of this application provides a video detection device, comprising:

[0111] The database construction module 601 is used to construct the copyright database. The copyright database includes a vector database and a key-value library. The vector database records the second segment features of the copyrighted video and its time interval, and the key-value library records the second single frame features of the copyrighted video and its timestamp.

[0112] The video detection module 602 is used to extract the first segment features and the first single-frame features of the video under review; retrieve the copyright database; search the second segment features using the first segment features, and determine the target time interval of the video under review based on the search results; pull the partial single-frame features whose timestamps are located in the target time interval from the second single-frame features, and determine the detection result of the video under review by comparing the first single-frame features with the pulled partial single-frame features.

[0113] In one possible implementation, database construction module 601 is specifically used for:

[0114] The copyrighted video is redundancy cleaned by extracting the perceptual hash value of the copyrighted video per unit time; the second segment feature and the second single frame feature are extracted from the redundancy cleaned copyrighted video; the second segment feature and its time interval are stored in the vector database by establishing a hierarchical navigation small-world index; the second single frame feature and its timestamp are stored in the key-value library.

[0115] In one possible implementation, the video detection module 602, used to determine the detection result of the video to be reviewed by comparing the features of the first single frame with the features of the partially retrieved single frames, is specifically used for:

[0116] Among the extracted single-frame features, identify the target single-frame feature with the highest similarity to the first single-frame feature, whose similarity is greater than the corresponding threshold. Obtain the timestamp of the first single-frame feature in the video under review, and sort the target single-frame features in ascending order of the timestamp of the first single-frame feature. According to the sorting result of the target single-frame features, detect whether the timestamp of the target single-frame feature has temporal monotonicity. If the timestamp of the target single-frame feature has temporal monotonicity, determine the detection result of the video under review based on the number of consecutive frames of the target single-frame feature.

[0117] In one possible implementation, a video detection module 602, used to determine the target single-frame feature among the retrieved partial single-frame features that has a similarity greater than a corresponding threshold and the highest similarity, is specifically used for:

[0118] Among the retrieved single-frame features, candidate single-frame features with a similarity greater than the corresponding threshold to the first single-frame feature are identified; the corresponding time offset is determined based on the timestamps of the first single-frame feature and the candidate single-frame features; the candidate single-frame features are filtered using the time offset, and the single-frame feature with the highest similarity among the filtered candidate single-frame features is selected as the target single-frame feature.

[0119] In one possible implementation, the video detection module 602, used to detect whether the timestamps of the target single-frame features have temporal monotonicity according to the sorting result of the target single-frame features, is specifically used for:

[0120] According to the sorting result of the target single-frame features, it iterates through two consecutive single-frame features in the target single-frame features; if the timestamp of the latter single-frame feature is later than the timestamp of the former single-frame feature, it is determined that the timestamp of the target single-frame feature has temporal monotonicity; if the timestamp of the latter single-frame feature is earlier than the timestamp of the former single-frame feature, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity; if the timestamp of the latter single-frame feature is equal to the timestamp of the former single-frame feature, the cumulative value of the number of pauses is incremented by 1, and if the current cumulative value of the number of pauses is greater than the corresponding threshold, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity.

[0121] It should be noted that the detailed functions of each module in the embodiments of this application can be found in the corresponding disclosures of the above video detection method embodiments, and will not be repeated here.

[0122] This application also provides an electronic device in its embodiments. See also... Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device in this embodiment may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. When the electronic device is powered on, the RAM 703 also stores various programs and data required for the operation of the electronic device. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0124] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, memory cards, hard drives, etc.; and communication devices 709. Communication device 709 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0125] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the video detection methods provided in this application.

[0126] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the video detection methods provided in this application.

[0127] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0129] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0130] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A video detection method, characterized in that, The video detection method includes: Extract the features of the first segment and the first single frame of the video submitted for review; The pre-built copyright database is retrieved. The copyright database includes a vector database and a key-value database. The vector database records the second segment features of the copyrighted video and its time interval. The key-value database records the second single-frame features of the copyrighted video and its timestamp. The first segment feature is used to search for the second segment feature, and the target time interval of the video to be reviewed is determined based on the search results; Extract a portion of the single-frame features whose timestamps are located within the target time interval from the second single-frame features, and determine the detection result of the video to be reviewed by comparing the first single-frame features with the extracted portion of the single-frame features.

2. The video detection method according to claim 1, characterized in that, The process of constructing the copyright database includes: Redundancy cleaning is performed on the copyrighted video by extracting the perceptual hash value of the copyrighted video per unit time. Extract the second segment features and the second single frame features from the copyrighted video after redundancy cleaning; The second segment features and their time intervals are stored in the vector database by establishing a hierarchical navigation small-world index; The second single-frame feature and its timestamp are stored in the key-value library.

3. The video detection method according to claim 1, characterized in that, The step of determining the detection result of the video to be reviewed by comparing the first single-frame features with the retrieved partial single-frame features includes: Among the retrieved single-frame features, identify the target single-frame feature that has the highest similarity to the first single-frame feature and a similarity greater than the corresponding threshold. Obtain the timestamp of the first single-frame feature in the video under review, and sort the target single-frame features in ascending order of the timestamp of the first single-frame feature; Based on the sorting result of the target single-frame features, detect whether the timestamps of the target single-frame features have temporal monotonicity; If the timestamp of the target single-frame feature has temporal monotonicity, the detection result of the video to be reviewed is determined based on the number of consecutive frames of the target single-frame feature.

4. The video detection method according to claim 3, characterized in that, The step of determining the target single-frame feature with the highest similarity to the first single-frame feature from the retrieved single-frame features includes: Among the extracted single-frame features, candidate single-frame features with a similarity greater than the corresponding threshold to the first single-frame feature are identified. The corresponding time offset is determined based on the timestamps of the first single-frame feature and the candidate single-frame feature; The candidate single-frame features are filtered using the time offset, and the single-frame feature with the highest similarity among the filtered candidate single-frame features is determined as the target single-frame feature.

5. The video detection method according to claim 3, characterized in that, The step of detecting whether the timestamps of the target single-frame features have temporal monotonicity according to the sorting result of the target single-frame features includes: According to the sorting result of the target single-frame features, traverse two consecutive single-frame features in the target single-frame features; If the timestamp of the latter single-frame feature is later than the timestamp of the former single-frame feature among the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature has temporal monotonicity. If the timestamp of the latter single-frame feature is earlier than the timestamp of the former single-frame feature in the two single-frame features traversed, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity. If the timestamp of the latter single-frame feature is equal to the timestamp of the former single-frame feature among the two single-frame features traversed, the cumulative value of the number of pauses is incremented by 1, and if the current cumulative value of the number of pauses is greater than the corresponding threshold, it is determined that the timestamp of the target single-frame feature does not have temporal monotonicity.

6. A video detection device, characterized in that, The video detection device includes: A database construction module is used to construct a copyright database, which includes a vector database and a key-value library. The vector database records the second segment features of the copyrighted video and its time interval, and the key-value library records the second single-frame features of the copyrighted video and its timestamp. The video detection module is used to extract the first segment features and the first single-frame features of the video under review; retrieve the copyright database; search the second segment features using the first segment features, and determine the target time interval of the video under review based on the search results; pull out a portion of the single-frame features whose timestamps are located in the target time interval from the second single-frame features, and determine the detection result of the video under review by comparing the first single-frame features with the pulled portion of the single-frame features.

7. The video detection device according to claim 6, characterized in that, The database construction module is specifically used for: The copyright video is redundancy cleaned by extracting the perceptual hash value of the copyright video per unit time; the second segment feature and the second single frame feature are then extracted from the redundancy cleaned copyright video. The second segment features and their time intervals are stored in the vector database by establishing a hierarchical navigation small-world index; the second single-frame features and their timestamps are stored in the key-value library.

8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the video detection method as described in any one of claims 1 to 5.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the video detection method as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the video detection method as described in any one of claims 1 to 5.

Images