Material video duplicate checking method and device, electronic equipment and computer readable medium

By uniformly extracting the keyframes of video frames and performing feature fusion, combining local feature extraction and keyframe feature comparison, the problems of large computing power overhead, low efficiency and large error in the existing video plagiarism check method are solved, and efficient and accurate Video check.

CN119089003AActive Publication Date: 2024-12-06CHENGDU GUANGCHANG CREATIVE TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411153255.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-06
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

The existing video plagiarism check method compares feature similarity frame by frame, resulting in high computing overhead, low efficiency, and long-term performance. The video frame feature characterization ability extracted from conventional convolution operations is poor, resulting in the edited video video. The missed check rate is high and the error is large.

Method used

The keyframes of video frames are uniformly extracted, and the feature information of video frames is generated through local feature extraction and feature fusion. The keyframe feature comparison is used for video latch checking, reducing frame-by-frame feature extraction and comparison, improving representation ability, and reducing errors.

Benefits of technology

It reduces computing resource overhead and video diversion checking time, reduces video diversion checking errors, and improves the accuracy of diversion checking of edited and processed videos.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089003A_ABST
    Figure CN119089003A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a material video duplicate checking method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: receiving a material video as a video to be subjected to duplicate checking; selecting each comparison material video from a material video library; analyzing the video to be subjected to duplicate checking to obtain a video frame sequence to be subjected to duplicate checking; for each comparison material video, the following steps are executed: generating feature information of each key frame to be subjected to duplicate checking; according to the feature information of each key frame to be subjected to duplicate checking, determining the comparison material video as a similar material video in response to determining that the comparison material video meets a key frame matching condition; based on each target key frame to be subjected to duplicate checking, performing duplicate checking processing on the similar material video to determine material video duplicate checking information; and sending each piece of material duplicate checking information to a user side, so that the user side displays each piece of material duplicate checking information. According to the embodiment, the computing power resource overhead and the video duplicate checking time consumption can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and specifically to a material video duplication detection method, device, electronic device, and computer-readable medium. Background Art

[0002] Video duplication detection is a common video matching technology used to find a target video that is highly similar to the video to be detected or contains repeated video clips among multiple videos. At present, when performing video duplication detection, the usual method is to extract the feature information of the video or video frame image, and then determine the duplicate video by feature comparison.

[0003] However, the inventors have found that when the above method is used to check for duplicate content in a video, the following technical problems often occur:

[0004] First, video duplication checking is performed by comparing feature similarity frame by frame, which consumes a lot of computing power, has low duplication checking efficiency, and takes a long time.

[0005] Second, in order to improve the representation ability of the extracted video frame feature information, it is often necessary to continuously deepen the network depth of the vital sign extraction model, which leads to increased computing resource overhead and longer feature extraction time.

[0006] Third, the representation ability of video frame features extracted by conventional convolution operations is relatively poor, and the missed detection rate of material videos that have been edited (such as perspective transformation, image cropping) is high, resulting in large actual errors in video duplication detection.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present disclosure propose a material video duplication checking method, device, electronic device and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0010] In a first aspect, some embodiments of the present disclosure provide a material video duplication checking method, the method comprising: receiving a material video uploaded by a user terminal as a video to be checked for duplication, wherein the video to be checked for duplication corresponds to video identification information and at least one video keyword tag; selecting at least one material video from a material video library as each comparison material video according to the video to be checked for duplication and the at least one video keyword tag, wherein the material video library is a database storing each material video uploaded by each user terminal; parsing the video to be checked for duplication to obtain a video frame sequence to be checked for duplication, wherein each video frame to be checked for duplication in the video frame sequence to be checked for duplication corresponds to the time information of the video frame to be checked for duplication; for each comparison material video in the comparison material videos, executing The following duplication screening steps are as follows: based on the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplication, each key frame feature information to be checked for duplication is generated; based on the above-mentioned key frame feature information to be checked for duplication, in response to determining that the above-mentioned comparison material video meets the key frame matching condition, the above-mentioned comparison material video is determined as a similar material video; based on each target key frame to be checked for duplication, the above-mentioned similar material video is checked for duplication to determine the material video duplication information, wherein the above-mentioned each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the above-mentioned key frame feature information to be checked for duplication; the generated each material duplication checking information is sent to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned each material duplication checking information.

[0011] In a second aspect, some embodiments of the present disclosure provide a material video duplication checking method and device, the device comprising: a receiving unit, configured to receive a material video uploaded by a user terminal as a video to be checked for duplication, wherein the video to be checked for duplication corresponds to video identification information and at least one video keyword tag; a selecting unit, configured to select at least one material video from a material video library as each comparison material video according to the video to be checked for duplication and the at least one video keyword tag, wherein the material video library is a database storing each material video uploaded by each user terminal; a parsing unit, configured to parse the video to be checked for duplication to obtain a video frame sequence to be checked for duplication, wherein each video frame to be checked for duplication in the video frame sequence to be checked for duplication corresponds to time information of the video frame to be checked for duplication; a duplication screening unit, configured to parse each of the above-mentioned comparison material videos; For each comparison material video in the video, the following duplication screening steps are performed: based on the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplication, each key frame feature information to be checked for duplication is generated; based on the above-mentioned key frame feature information to be checked for duplication, in response to determining that the above-mentioned comparison material video meets the key frame matching condition, the above-mentioned comparison material video is determined as a similar material video; based on each target key frame to be checked for duplication, the above-mentioned similar material video is checked for duplication to determine the material video duplication information, wherein the above-mentioned each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the above-mentioned key frame feature information to be checked for duplication; the sending unit is configured to send the generated each material duplication checking information to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned each material duplication checking information.

[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the material video duplication checking method of some embodiments of the present disclosure can reduce the computing power resource overhead and reduce the time consumption of video duplication checking. Specifically, the reason for the large computing power resource overhead and the long video duplication checking time is that the video duplication checking is performed by comparing the feature similarity frame by frame, which has a large computing power overhead, low duplication checking efficiency, and a long duplication checking time. Based on this, the material video duplication checking method of some embodiments of the present disclosure first receives the material video uploaded by the user end as the video to be checked for duplication. Among them, the above-mentioned video to be checked for duplication corresponds to video identification information and at least one video theme word label. Then, according to the above-mentioned video to be checked for duplication and the above-mentioned at least one video theme word label, at least one material video is selected from the material video library as each comparison material video. Among them, the above-mentioned material video library is a database storing each material video uploaded by each user end. Thus, each video that may be similar to the above-mentioned video to be checked for duplication or contain repeated segments can be preliminarily selected as the comparison material video. Afterwards, the above-mentioned video to be checked for duplication is parsed to obtain a video frame sequence to be checked for duplication. Among them, each video frame to be checked for duplicates in the above-mentioned video frame sequence to be checked for duplicates corresponds to the time information of the video frame to be checked for duplicates. Thus, by parsing the video to be checked for duplicates, image processing can be performed with the video frame to be checked for duplicates as the basic unit in the subsequent processing process. Then, for each of the above-mentioned comparison material videos, the following duplicate checking and screening steps are performed: the first step is to generate each key frame feature information to be checked for duplicates according to the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplicates. Thus, the key frames in the video frame sequence to be checked for duplicates can be uniformly extracted, and the extracted key frames to be checked for duplicates can be feature extracted to obtain the feature information of each key frame to be checked for duplicates that characterizes the image content in each corresponding video frame. The second step is to determine the above-mentioned comparison material video as a similar material video in response to determining that the above-mentioned comparison material video meets the key frame matching condition according to the above-mentioned key frame feature information to be checked for duplicates. Thus, by comparing the features a few times, it can be determined whether the comparison material video has a video frame similar to the key frame to be checked for duplicates, so that the comparison material video can be preliminarily screened. The third step is to perform a duplication check on the above-mentioned similar material videos based on each target key frame to be checked for duplication, so as to determine the material video duplication check information. Among them, the above-mentioned each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the feature information of each key frame to be checked for duplication. Therefore, when there are video frames similar to each key frame to be checked for duplication in the above-mentioned comparison material video, the comparison material video is checked for duplication based on the key frame to be checked for duplication, and the material video duplication check information is generated. Finally, the generated material duplication check information is sent to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned material duplication check information.Also, because of the method of uniformly extracting key frames and comparing key frame features, it is possible to reduce the large amount of computing power overhead caused by frame-by-frame feature extraction and frame-by-frame feature comparison of the video to be checked for duplicates and each comparison material video, thereby reducing the waste of computing power resources and reducing the time spent on video duplicate checking. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flow chart of some embodiments of the material video duplication checking method according to the present disclosure;

[0017] Figure 2 It is a structural schematic diagram of some embodiments of the material video duplication checking device according to the present disclosure;

[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The process 100 of some embodiments of the material video duplication checking method according to the present disclosure is shown. The material video duplication checking method comprises the following steps:

[0026] Step 101, receiving a material video uploaded by a user as a video to be checked for duplicate content.

[0027] In some embodiments, the execution subject (such as a computing device) of the material video duplication checking method can receive the material video uploaded by the user end as the video to be checked for duplicates through a wired connection or a wireless connection. Among them, the above-mentioned video to be checked for duplicates corresponds to video identification information and at least one video theme word tag. The above-mentioned material video may refer to a video used as a material for video editing and video creation. The above-mentioned video identification information may be a character string generated by a digital signature algorithm and representing the above-mentioned video to be checked for duplicates. The above-mentioned at least one video theme word tag may be various keywords selected by the user of the above-mentioned user end when uploading the video, representing the video theme of the above-mentioned video to be checked for duplicates. For example, the above-mentioned at least one video theme word tag may be "animation", "hot-blooded teenager" and "inspirational".

[0028] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future.

[0029] Step 102: Select at least one material video from the material video library as each comparison material video according to the video to be checked for duplicates and at least one video keyword tag.

[0030] In some embodiments, the execution subject may select at least one material video from a material video library as each comparison material video according to the video to be checked for duplicate content and the at least one video keyword tag. The material video library is a database storing material videos uploaded by each user terminal.

[0031] In some optional implementations of some embodiments, the execution subject may select at least one material video from the material video library as each comparison material video according to the video to be checked for duplicate content and the at least one video keyword tag through the following steps:

[0032] The first step is to determine the video size of the video to be checked for plagiarism as the size of the video to be checked for plagiarism.

[0033] The second step is to determine the duration of the video with the duplicate check video as the duration of the duplicate check video. The duration of the duplicate check video has a time unit by default. The time unit can be seconds.

[0034] The third step is to select the material videos corresponding to the at least one video keyword tag from the material videos included in the material video library as the comparison material videos. The comparison material videos can be material videos whose corresponding video keyword tags have at least one identical video keyword tag with the video to be checked for duplicates.

[0035] In the fourth step, in response to determining that there is at least one material video in the material video library that satisfies a size matching condition corresponding to the size of the duplicate check video, at least one material video that satisfies the size matching condition is determined as each comparison material video. The size matching condition may be that the video size of the comparison material video is the same as the size of the duplicate check video.

[0036] In step 5, in response to determining that there is at least one material video in the material video library that satisfies the duration matching condition corresponding to the duration of the target video, at least one material video that satisfies the duration matching condition is determined as each comparison material video. The duration matching condition may be that the video duration of the comparison material video is the same as the duration of the duplicate checking video.

[0037] Step 103, parse the video to be checked for duplicates to obtain a frame sequence of the video to be checked for duplicates.

[0038] In some embodiments, the above-mentioned execution subject can parse the above-mentioned video to be checked for duplicates to obtain a sequence of video frames to be checked for duplicates. Among them, each video frame to be checked for duplicates in the above-mentioned sequence of video frames to be checked for duplicates corresponds to the time information of the video frame to be checked for duplicates. The above-mentioned time information of the video frame to be checked for duplicates can be the time corresponding to each video frame to be checked for duplicates in the above-mentioned video to be checked for duplicates. In practice, the above-mentioned execution subject can split the above-mentioned video to be checked for duplicates frame by frame through a video decoder to obtain a sequence of video frames to be checked for duplicates. As an example, the above-mentioned video decoder can be a FFmpeg video decoder.

[0039] Step 104, for each of the comparison material videos, perform the following duplication screening steps:

[0040] Step 1041, generating feature information of each key frame to be checked for duplication based on the comparison material video and the frame sequence of the video to be checked for duplication.

[0041] In some embodiments, the execution subject may generate feature information of each key frame to be checked for duplication based on the comparison material video and the video frame sequence to be checked for duplication. The generated feature information of the key frame to be checked for duplication may be a video frame feature vector of the corresponding key frame to be checked for duplication. The key frame to be checked for duplication may be a key frame in the video sequence to be checked for duplication.

[0042] In some optional implementations of some embodiments, the execution subject may generate feature information of each key frame to be checked for duplication according to the comparison material video and the frame sequence of the video to be checked for duplication through the following steps:

[0043] The first step is to determine the video length of the comparison material video as the comparison video length.

[0044] In the second step, the ratio of the length of the comparison video to the length of the duplicate checking video is determined as the frame extraction adjustment coefficient.

[0045] The third step is to select each video frame to be checked for duplication from the above-mentioned video frame sequence to be checked for duplication according to the above-mentioned frame extraction adjustment coefficient and the number of frames extracted corresponding to the duration of the above-mentioned video to be checked for duplication as each key frame to be checked for duplication. The above-mentioned number of frames extracted may be the number of key frames to be extracted from the above-mentioned video frame sequence to be checked for duplication. The above-mentioned number of frames extracted may be the product of a preset number of frames extracted per unit time and the duration of the above-mentioned video to be checked for duplication. As an example, the above-mentioned preset number of frames extracted per unit time may be 1 frame / second.

[0046] The fourth step is to perform feature extraction processing on each of the key frames to be checked for duplicate content, so as to generate feature information of each of the key frames to be checked for duplicate content.

[0047] In some optional implementations of some embodiments, the execution subject may select each video frame to be checked for duplication as each key frame to be checked for duplication from the sequence of video frames to be checked for duplication according to the frame extraction adjustment coefficient and the number of frames extracted corresponding to the duration of the video to be checked for duplication through the following steps:

[0048] In the first step, the product of the frame extraction adjustment coefficient and the frame extraction quantity is determined as the target frame extraction quantity.

[0049] In the second step, for each two adjacent video frames to be checked for duplicates in the above-mentioned sequence of video frames to be checked for duplicates, optical flow feature extraction is performed on the above-mentioned two video frames to be checked for duplicates to generate optical flow information of the video frames to be checked for duplicates. Among them, the above-mentioned optical flow information of the video frames to be checked for duplicates can be the optical flow vectors of the above-mentioned two video frames to be checked for duplicates. The generated optical flow information of the video frames to be checked for duplicates corresponds to optical flow time information. The above-mentioned optical flow time information can be the time information of the video frames to be checked for duplicates corresponding to the first video frame to be checked for duplicates in the above-mentioned two video frames to be checked for duplicates. In practice, the above-mentioned execution subject can extract optical flow features of the above-mentioned two video frames to be checked for duplicates through relevant library functions (such as the cv2.calcOpticalFlowPyrLK() method in the OpenCV library) to generate optical flow information of the video frames to be checked for duplicates. The above-mentioned optical flow information of the video frames to be checked for duplicates can characterize the displacement direction and displacement of each pixel in the above-mentioned two video frames to be checked for duplicates. The larger the optical flow information of the above-mentioned video frames to be checked for duplicates, the greater the change in image content in the above-mentioned two video frames to be checked for duplicates can be represented.

[0050] The third step is to sort the generated optical flow information of each video frame to be checked for duplicates to obtain an optical flow information sequence of the video frames to be checked for duplicates. In practice, the above execution subject can sort the generated optical flow information of each video frame to be checked for duplicates in chronological order to obtain an optical flow information sequence of the video frames to be checked for duplicates.

[0051] The fourth step is to determine the optical flow information of each to-be-checked duplicate video frame in the above-mentioned optical flow information sequence of the to-be-checked duplicate video frame that meets the preset optical flow threshold condition as the target video frame optical flow information sequence. In practice, first, for each to-be-checked duplicate video frame optical flow information in each to-be-checked duplicate video frame optical flow information, the above-mentioned execution subject can determine the sum of the squares of each vector element in the above-mentioned to-be-checked duplicate video frame optical flow information (i.e., optical flow vector) as the total amount of optical flow change. The above-mentioned preset optical flow threshold condition can be that the total amount of optical flow change corresponding to the target video frame optical flow information is greater than or equal to the preset optical flow change total amount threshold. Then, the above-mentioned execution subject can use the above-mentioned optical flow information of each to-be-checked duplicate video frame that meets the preset optical flow threshold condition as each target to-be-checked duplicate video frame optical flow information, and determine it as the target video frame optical flow information sequence in chronological order. Here, the specific setting of the preset optical flow change total amount threshold is not limited.

[0052] The fifth step is to determine the preset video start time and the time information of each video frame corresponding to the above-mentioned target video frame optical flow information sequence as the target time information sequence. The above-mentioned preset video start time can be "00:00:00". For example, the above-mentioned target video frame optical flow information sequence can be (a30-31, a90-91, ..., a300-301, a360-361). Among them, the above-mentioned "a23-24" can be the optical flow information of the video frame to be checked for duplicates corresponding to the 30th video frame to be checked for duplicates and the 31st video frame to be checked for duplicates in the above-mentioned video frame sequence to be checked for duplicates. The target time information sequence corresponding to the above-mentioned target video frame optical flow information sequence can be (00:00:00, 00:00:01, 00:00:03, ..., 00:00:10, 00:00:12).

[0053] Step 6: Based on each target time information in the above target time information sequence, perform the following key frame extraction steps:

[0054] The first sub-step, in response to determining that the above-mentioned target time information does not meet the position sequence condition, determines the frame segment time ratio according to the above-mentioned target time information and the preceding target time information, wherein the preceding target time information is the previous target time information adjacent to the above-mentioned target time information in the above-mentioned target time information sequence. The above-mentioned position sequence condition may be that the above-mentioned target time information is the first target time information in the above-mentioned target time information sequence. In practice, first, the above-mentioned execution subject may determine the time difference between the above-mentioned target time information and the above-mentioned preceding target time information as the time difference value. Then, the above-mentioned execution subject may determine the ratio of the above-mentioned time difference value to the above-mentioned duplication check video duration as the frame segment time ratio.

[0055] In the second sub-step, the product of the frame segment time ratio and the target frame extraction number is determined as the frame segment frame extraction number.

[0056] The third sub-step is to extract, according to the above-mentioned target time and the above-mentioned previous target time, the video frames to be checked for duplication, which are equal to the number of frames extracted from the above-mentioned frame segment, as the key frames to be checked for duplication from the above-mentioned video frame sequence to be checked for duplication. In practice, firstly, the above-mentioned execution subject can use the video frames to be checked for duplication within the time range represented by the above-mentioned target time and the above-mentioned previous target time in the above-mentioned video frame sequence to be checked for duplication as the sub-sequence of video frames to be extracted. Then, the above-mentioned execution subject can uniformly extract (i.e., the intervals of the video frames to be extracted are the same) the video frames to be checked for duplication, which are equal to the number of frames extracted from the above-mentioned frame segment, as the key frames to be checked for duplication in the determined sub-sequence of video frames to be extracted.

[0057] In some optional implementations of some embodiments, the execution subject may perform feature extraction processing on each of the key frames to be checked for duplicate content through the following steps to generate feature information of each of the key frames to be checked for duplicate content:

[0058] In the first step, for each of the above key frames to be checked for duplicates, the following multi-feature extraction steps are performed:

[0059] In the first sub-step, local feature point detection is performed on the key frames to be checked for duplicates to obtain local feature position information of each key frame. The obtained local feature position information of the key frames can be the image position coordinates of the corresponding feature points in the key frames to be checked for duplicates. In practice, the execution subject can perform feature point detection on the key frames to be checked for duplicates by using a preset feature point detection operator to obtain local feature position information of each key frame. As an example, the feature point detection operator can be but is not limited to a DoG (Difference of Gaussians) operator.

[0060] In the second sub-step, based on the local feature position information of each key frame, the key frame to be checked for duplicates is subjected to regional cropping processing to obtain regional images of each key frame. In practice, for each key frame local feature position information in the local feature position information of each key frame, the execution subject can take the local feature position information of the key frame as the center and perform image cropping according to a preset cropping size to obtain a key frame regional image. It should be noted that the preset cropping size can be a preset size, which is not limited here.

[0061] The third sub-step is to perform deep feature extraction processing on each of the above-mentioned key frame area images to obtain feature information of each key frame area. Among them, the feature dimensions of each key frame area feature information obtained are equal. The above-mentioned key frame area feature information may be an image feature vector of the key frame area image. In practice, the above-mentioned execution subject may perform scaling processing on each key frame area image, uniformly scaling it to a preset input size, and obtain each scaled key frame area image. As an example, the above-mentioned preset input size may be 32×32. Then, the above-mentioned execution subject may perform image preprocessing on each scaled key frame area image to obtain each processed key frame area image. The above-mentioned image preprocessing may include image normalization and grayscale conversion. Finally, the above-mentioned execution subject may perform deep feature extraction processing on each of the above-mentioned processed key frame area images through a pre-trained deep feature extraction model to obtain feature information of each key frame area. The above-mentioned deep feature extraction model may be a neural network model that takes the processed key frame area image as input and the corresponding key frame area feature information as output. The above-mentioned deep feature extraction model may include a first convolutional network, a second convolutional network, a third convolutional network, a fourth convolutional network, a fifth convolutional network, a sixth convolutional network, a seventh convolutional network and an output layer. The above-mentioned first convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 32, a stride of 1, a normalization layer and a ReLu activation layer. The above-mentioned second convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 64, a stride of 1 / 2, a normalization layer and a ReLu activation layer. The above-mentioned third convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 64, a stride of 1, a normalization layer and a ReLu activation layer. The above-mentioned fourth convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 128, a stride of 1 / 2, a normalization layer and a ReLu activation layer. The fifth convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 128, a stride of 1, a normalization layer, and a ReLu activation layer. The sixth convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 256, a stride of 1 / 2, a normalization layer, and a ReLu activation layer. The seventh convolutional network may include a convolutional layer with a convolution kernel size of 3×3, a number of channels of 512, a stride of 1 / 2, a normalization layer, and a ReLu activation layer. The output layer may include a convolutional layer with a convolution kernel size of 8×8, a stride of 1, a normalization layer, and a Dropout layer.

[0062] The fourth sub-step is to determine the key frame global feature matrix according to the above key frame region feature information. In practice, the above execution subject may determine the matrix obtained by horizontally splicing the above key frame region feature information as the key frame global feature matrix.

[0063] The fifth sub-step is to perform dimensionality reduction processing on the above-mentioned key frame global feature matrix to generate the key frame feature information to be checked for duplicates. In practice, the above-mentioned execution subject can perform dimensionality reduction processing on the above-mentioned key frame global feature matrix through a dimensionality reduction algorithm to generate the key frame feature information to be checked for duplicates of a preset dimension. As an example, the above-mentioned dimensionality reduction algorithm can be but is not limited to the PCA dimensionality reduction algorithm. Here, the specific setting of the preset dimension is not limited.

[0064] The relevant content of the above-mentioned embodiment, as an inventive point of the embodiment of the present disclosure, can solve the second technical problem "In order to improve the representation ability of the extracted video frame feature information, it is often necessary to continuously deepen the network depth of the vital sign extraction model, which leads to increased computing power resource overhead and longer feature extraction time". The factors that lead to increased computing power resource overhead and longer feature extraction time are often as follows: In order to improve the representation ability of the extracted video frame feature information, it is often necessary to continuously deepen the network depth of the vital sign extraction model. If the above-mentioned factors are solved, the effect of reducing computing power resource overhead and reducing feature extraction time can be achieved. In order to achieve this effect, the present disclosure generates video frame feature information by extracting local features in the video frame, and performing feature fusion and feature dimensionality reduction on the local features, thereby improving the representation ability of the generated video frame feature information, and further reducing computing power resource overhead and reducing feature extraction time.

[0065] Step 1042: Based on the feature information of each key frame to be checked for duplication, in response to determining that the compared material video meets the key frame matching condition, the compared material video is determined as a similar material video.

[0066] In some embodiments, the execution subject may determine the comparison material video as a similar material video in response to determining that the comparison material video satisfies a key frame matching condition based on the key frame feature information to be checked for duplication. The key frame matching condition may be that there is at least one comparison material video frame feature information satisfying the key frame feature similarity condition in the comparison material video frame feature information sequence corresponding to the comparison material video frame sequence.

[0067] In some optional implementations of some embodiments, the execution subject may determine the comparison material video as a similar material video in response to determining that the comparison material video meets the key frame feature similarity condition according to the feature information of each key frame to be checked for duplication through the following steps:

[0068] The first step is to parse the comparison material video to obtain a comparison material video frame sequence. The comparison material video frame in the comparison material video frame sequence corresponds to the comparison material video frame time information. The comparison material video frame time information can be the time of the corresponding comparison material video frame in the comparison material video. In practice, the execution subject can parse the comparison material video through the video decoder to obtain the comparison material video frame sequence.

[0069] The second step is to perform feature extraction processing on each comparison material video frame in the above comparison material video frame sequence to generate feature information of each comparison material video frame and obtain a comparison material video frame feature information sequence. It should be noted that the above “perform feature extraction processing on each comparison material video frame in the above comparison material video frame sequence to generate feature information of each comparison material video frame” can refer to the implementation method of “generating feature information of key frames to be checked for duplication”, which will not be repeated here.

[0070] In the third step, for each of the key frame feature information to be checked for duplicates, the following steps are performed:

[0071] The first sub-step is to determine the video frame feature similarity between the key frame feature information to be checked for duplicates and each of the video frame feature information of the comparison material in the video frame feature information sequence of the comparison material, and obtain a video frame similarity set. In practice, the execution subject may determine the video frame feature similarity between the key frame feature information to be checked for duplicates and each of the video frame feature information of the comparison material in the video frame feature information sequence of the comparison material by a similarity algorithm, and obtain a video frame similarity set. As an example, the similarity algorithm may be, but is not limited to, a cosine similarity algorithm.

[0072] In the first sub-step, in response to determining that there is at least one video frame similarity satisfying the key frame feature similarity condition in the above video frame similarity set, the following steps are performed:

[0073] Sub-step 1: Determine the at least one video frame similarity as each target video frame similarity. The key frame feature similarity condition may be that the target video frame similarity is greater than or equal to a preset similarity threshold. As an example, the preset similarity threshold may be 0.9.

[0074] Sub-step 2: for each target video frame similarity among the above target video frame similarities, determine the to-be-checked duplication video frame corresponding to the above target video frame similarity as the target to-be-checked duplication key frame.

[0075] The fourth step is to determine the above-mentioned comparison material videos as similar material videos.

[0076] Step 1043, based on each target key frame to be checked for duplication, similar material videos are checked for duplication to determine the material video duplication information.

[0077] In some embodiments, the execution subject may perform a duplication check on the similar material video based on each target key frame to be checked for duplication to determine the material video duplication check information. The target key frames to be checked for duplication are the video frames to be checked for duplication corresponding to the key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the feature information of the key frames to be checked for duplication.

[0078] In some optional implementations of some embodiments, the above-mentioned execution entity can perform a duplication check on the above-mentioned similar material videos based on each target key frame to be checked for duplication through the following steps to determine the material video duplication check information.

[0079] In the first step, for each of the above target key frames to be checked for duplicates, perform the following steps:

[0080] The first sub-step is to select two corresponding target time information from the target time information sequence as frame segment interception information according to the time information of the video frame to be checked for duplication corresponding to the target key frame to be checked for duplication. In practice, the execution subject can select two target time information adjacent to the time information of the video frame to be checked for duplication corresponding to the target key frame to be checked for duplication from the target time information sequence as frame segment interception information. For example, the time information of the video frame to be checked for duplication corresponding to the target key frame to be checked for duplication can be "00:00:21". The target time sequence can be (00:00:00, ..., 00:00:10, 00:00:40, ...). Then the frame segment interception information corresponding to the time information of the video frame to be checked for duplication corresponding to the target key frame to be checked for duplication can be (00:00:10, 00:00:40).

[0081] The second sub-step is to intercept each video frame to be checked for duplication corresponding to the frame segment interception information from the above-mentioned video frame sequence to be checked for duplication as a sub-sequence of video frames to be checked for duplication. In practice, the above-mentioned execution subject can intercept each video frame to be checked for duplication within the time range represented by the frame segment interception information from the above-mentioned video frame sequence to be checked for duplication as a sub-sequence of video frames to be checked for duplication.

[0082] The third sub-step is to intercept each comparison material video frame from the comparison material video frame sequence as a comparison material video frame sub-sequence according to the comparison material video frame corresponding to the target key frame to be checked for duplicates. The number of the video frames to be checked for duplicates in the sub-sequence of the video frame to be checked for duplicates is the same as the number of the comparison material video frames in the sub-sequence of the comparison material video frame. In practice. The execution subject can take the comparison material video frame corresponding to the target key frame to be checked for duplicates (that is, the comparison material video frame that meets the similarity condition of the key frame features) as the center frame, and intercept each comparison material video frame with the same total number of frames as the sub-sequence of the video frame to be checked for duplicates (the number of comparison material video frames before and after the center frame is the same as the number of key frames to be checked before and after the target key frame to be checked for duplicates) from the comparison material video frame sequence as a comparison material video frame sub-sequence.

[0083] The fourth sub-step is to perform regional feature extraction processing on each video frame to be checked for duplicates in the above-mentioned sub-sequence of video frames to be checked for duplicates, so as to generate regional feature information of each video frame to be checked for duplicates. It should be noted that the implementation method of the above-mentioned "performing regional feature extraction processing on the above-mentioned video frames to be checked for duplicates, so as to generate regional feature information of each video frame to be checked for duplicates" can refer to the implementation method of "key frame regional feature information", which will not be repeated again.

[0084] The fifth sub-step is to construct a video frame region feature matching model based on the generated feature information of each video frame region to be checked for duplicates. The video frame region feature matching model may be a tree model for feature comparison. As an example, the video frame region feature matching model may be a KD tree model.

[0085] The sixth sub-step is to construct a material video region feature matching model based on the above-mentioned comparison material video frame sub-sequences. The above-mentioned material video region feature matching model can be a tree model constructed by using the regional features of each comparison material video frame. As an example, the above-mentioned material video region feature matching model can be a KD tree model.

[0086] The seventh sub-step is to determine the frame segment similarity according to the constructed material video region feature matching model and the constructed video frame region feature matching model. In practice, the execution subject may determine the structural similarity of the video frame region feature matching model and the material video region feature matching model as the frame segment similarity through a tree similarity algorithm. As an example, the tree similarity algorithm may be, but is not limited to, a maximum common subtree algorithm, a topological distance algorithm, and a tree edit distance algorithm.

[0087] In the eighth sub-step, in response to determining that the frame segment similarity is greater than or equal to a preset similarity threshold, the comparison material video frame subsequence is determined to be a repeated comparison material video frame subsequence. As an example, the preset similarity threshold may be 0.80.

[0088] The ninth sub-step is to determine the material video duplication checking information according to the determined repeated comparison material video frame sub-sequences. In practice, the execution subject may determine the frame segment similarity and the frame segment interception information as the material video duplication checking information.

[0089] The relevant contents of the above embodiments, as an inventive point of the embodiments of the present disclosure, can solve the second technical problem "the representation ability of the video frame features extracted by conventional convolution operations is relatively poor, and the missed detection rate of the material video after editing (such as perspective change, image capture) is high, resulting in a large actual error in video duplication detection". The factors that lead to large actual errors in video duplication detection are often as follows: the representation ability of the video frame features extracted by conventional convolution operations is relatively poor, and the missed detection rate of the material video after editing (such as perspective change, image capture) is high. If the above factors are solved, the effect of reducing the error of video duplication detection can be achieved. In order to achieve this effect, the present disclosure performs video duplication detection by extracting local features of video frame subsequences and using the extracted local features of each video frame to construct a feature matching model. Therefore, even if some video clips in the comparison material video are edited by perspective change, image capture, etc., when the local features contained in each comparison material video frame and the corresponding video frame to be checked for duplication in the video clip are highly similar, video duplication determination can also be performed, thereby greatly reducing the actual error of video duplication detection.

[0090] Step 105, sending the generated duplicate checking information of each material to the user terminal, so that the user terminal can display the duplicate checking information of each material.

[0091] In some embodiments, the execution entity may send the generated material duplication check information to the user terminal, so that the user terminal can display the material duplication check information. In practice, the execution entity may send the generated material duplication check information to the user terminal, so that the user terminal can display the material duplication check information on the associated display screen.

[0092] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the material video duplication checking method of some embodiments of the present disclosure can reduce the computing power resource overhead and reduce the time consumption of video duplication checking. Specifically, the reason for the large computing power resource overhead and the long video duplication checking time is that the video duplication checking is performed by comparing the feature similarity frame by frame, which has a large computing power overhead, low duplication checking efficiency, and a long duplication checking time. Based on this, the material video duplication checking method of some embodiments of the present disclosure first receives the material video uploaded by the user end as the video to be checked for duplication. Among them, the above-mentioned video to be checked for duplication corresponds to video identification information and at least one video theme word label. Then, according to the above-mentioned video to be checked for duplication and the above-mentioned at least one video theme word label, at least one material video is selected from the material video library as each comparison material video. Among them, the above-mentioned material video library is a database storing each material video uploaded by each user end. Thus, each video that may be similar to the above-mentioned video to be checked for duplication or contain repeated segments can be preliminarily selected as the comparison material video. Afterwards, the above-mentioned video to be checked for duplication is parsed to obtain a video frame sequence to be checked for duplication. Among them, each video frame to be checked for duplicates in the above-mentioned video frame sequence to be checked for duplicates corresponds to the time information of the video frame to be checked for duplicates. Thus, by parsing the video to be checked for duplicates, image processing can be performed with the video frame to be checked for duplicates as the basic unit in the subsequent processing process. Then, for each of the above-mentioned comparison material videos, the following duplicate checking and screening steps are performed: the first step is to generate each key frame feature information to be checked for duplicates according to the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplicates. Thus, the key frames in the video frame sequence to be checked for duplicates can be uniformly extracted, and the extracted key frames to be checked for duplicates can be feature extracted to obtain the feature information of each key frame to be checked for duplicates that characterizes the image content in each corresponding video frame. The second step is to determine the above-mentioned comparison material video as a similar material video in response to determining that the above-mentioned comparison material video meets the key frame matching condition according to the above-mentioned key frame feature information to be checked for duplicates. Thus, by comparing the features a few times, it can be determined whether the comparison material video has a video frame similar to the key frame to be checked for duplicates, so that the comparison material video can be preliminarily screened. The third step is to perform a duplication check on the above-mentioned similar material videos based on each target key frame to be checked for duplication, so as to determine the material video duplication check information. Among them, the above-mentioned each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the feature information of each key frame to be checked for duplication. Therefore, when there are video frames similar to each key frame to be checked for duplication in the above-mentioned comparison material video, the comparison material video is checked for duplication based on the key frame to be checked for duplication, and the material video duplication check information is generated. Finally, the generated material duplication check information is sent to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned material duplication check information.Also, because of the method of uniformly extracting key frames and comparing key frame features, it is possible to reduce the large amount of computing power overhead caused by frame-by-frame feature extraction and frame-by-frame feature comparison of the video to be checked for duplicates and each comparison material video, thereby reducing the waste of computing power resources and reducing the time spent on video duplicate checking.

[0093] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a material video duplication detection device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the material video duplication checking device can be specifically applied to various electronic devices.

[0094] like Figure 2 As shown, the material video duplication checking device 200 of some embodiments includes: a receiving unit 201, a selection unit 202, a parsing unit 203, a duplication checking screening unit 203 and a sending unit 205. The receiving unit 201 is configured to receive the material video uploaded by the user terminal as the video to be checked for duplication, wherein the video to be checked for duplication corresponds to video identification information and at least one video keyword tag; the selecting unit 202 is configured to select at least one material video from the material video library as each comparison material video according to the video to be checked for duplication and the at least one video keyword tag, wherein the material video library is a database storing each material video uploaded by each user terminal; the parsing unit 203 is configured to parse the video to be checked for duplication to obtain a video frame sequence to be checked for duplication, wherein each video frame to be checked for duplication in the video frame sequence to be checked for duplication corresponds to the video frame time information to be checked for duplication; the duplication checking screening unit 204 is configured to perform, for each comparison material video in the comparison material videos, The following duplicate checking and screening steps are performed: based on the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplicates, each key frame feature information to be checked for duplicates is generated; based on the above-mentioned key frame feature information to be checked for duplicates, in response to determining that the above-mentioned comparison material video meets the key frame matching condition, the above-mentioned comparison material video is determined as a similar material video; based on each target key frame to be checked for duplicates, the above-mentioned similar material video is checked for duplicates to determine the material video duplicate checking information, wherein the above-mentioned each target key frame to be checked for duplicates is each video frame to be checked for duplicates corresponding to each key frame feature information to be checked for duplicates that meets the key frame feature similarity condition in the above-mentioned key frame feature information to be checked for duplicates; the sending unit 205 is configured to send the generated each material duplicate checking information to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned each material duplicate checking information.

[0095] It is understandable that the various units recorded in the material video duplication checking device 200 and the reference Figure 1Therefore, the operations, features and beneficial effects described above for the method are also applicable to the webpage generating device 200 and the units included therein, and will not be described in detail here.

[0096] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0098] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0099] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0100] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0101] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0102] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: receives the material video uploaded by the user terminal as the video to be checked for duplicates, wherein the above-mentioned video to be checked for duplicates corresponds to video identification information and at least one video keyword tag; selects at least one material video from the material video library as each comparison material video according to the above-mentioned video to be checked for duplicates and the above-mentioned at least one video keyword tag, wherein the above-mentioned material video library is a database storing each material video uploaded by each user terminal; parses the above-mentioned video to be checked for duplicates to obtain a sequence of video frames to be checked for duplicates, wherein each video frame to be checked for duplicates in the above-mentioned sequence of video frames to be checked for duplicates corresponds to the time information of the video frame to be checked for duplicates; for each comparison in the above-mentioned each comparison material video Material video, perform the following duplication screening steps: generate each key frame feature information to be checked for duplication according to the above-mentioned comparison material video and the above-mentioned video frame sequence to be checked for duplication; according to the above-mentioned key frame feature information to be checked for duplication, in response to determining that the above-mentioned comparison material video meets the key frame matching condition, determine the above-mentioned comparison material video as a similar material video; based on each target key frame to be checked for duplication, perform duplication checking on the above-mentioned similar material video to determine the material video duplication checking information, wherein the above-mentioned each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that meets the key frame feature similarity condition in the above-mentioned key frame feature information to be checked for duplication; send the generated each material duplication checking information to the above-mentioned user terminal, so that the above-mentioned user terminal can display the above-mentioned each material duplication checking information.

[0103] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a receiving unit, a selecting unit, a parsing unit, a duplicate checking and screening unit, and a sending unit. The names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the receiving unit may also be described as "a unit for receiving a material video uploaded by a user as a video to be checked for duplicate content".

[0106] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0107] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for checking duplicate content in a video, comprising: Receiving a material video uploaded by a user terminal as a video to be checked for duplicate content, wherein the video to be checked for duplicate content corresponds to video identification information and at least one video keyword tag; According to the video to be checked for duplicate content and the at least one video keyword tag, at least one material video is selected from a material video library as each comparison material video, wherein the material video library is a database storing each material video uploaded by each user terminal; Parsing the video to be checked for duplication to obtain a sequence of video frames to be checked for duplication, wherein each video frame to be checked for duplication in the sequence of video frames to be checked for duplication corresponds to time information of the video frame to be checked for duplication; For each of the comparison material videos, the following duplication screening steps are performed: Generate feature information of each key frame to be checked for duplicates according to the comparison material video and the frame sequence of the video to be checked for duplicates; According to the feature information of each key frame to be checked for duplication, in response to determining that the comparison material video meets the key frame matching condition, determining the comparison material video as a similar material video; Based on each target key frame to be checked for duplicates, the similar material video is checked for duplicates to determine the material video duplication check information, wherein each target key frame to be checked for duplicates is each video frame to be checked for duplicates corresponding to each key frame feature information to be checked for duplicates that meets the key frame feature similarity condition in the feature information of each key frame to be checked for duplicates; The generated duplicate checking information of each material is sent to the user terminal so that the user terminal can display the duplicate checking information of each material.

2. The method according to claim 1, wherein: The step of selecting at least one material video from a material video library as each comparison material video according to the video to be checked for duplicate content and the at least one video keyword tag includes: Determine the video size of the video to be checked for plagiarism as the size of the video to be checked for plagiarism; Determine the video duration of the video to be checked for plagiarism as the duration of the video to be checked for plagiarism; Selecting, from the material videos included in the material video library, material videos corresponding to the at least one video theme word tag as comparison material videos; In response to determining that there is at least one material video in the material video library that satisfies a size matching condition corresponding to the size of the duplicate check video, determining at least one material video that satisfies the size matching condition as each comparison material video; In response to determining that there is at least one material video in the material video library that meets a duration matching condition corresponding to the duration of the target video, at least one material video that meets the duration matching condition is determined as each comparison material video.

3. The method according to claim 2, wherein: The step of generating feature information of each key frame to be checked for duplicate content based on the comparison material video and the frame sequence of the video to be checked for duplicate content includes: Determine the video duration of the comparison material video as the comparison video duration; The ratio of the duration of the comparison video to the duration of the duplicate checking video is determined as a frame extraction adjustment coefficient; According to the frame extraction adjustment coefficient and the number of frames extracted corresponding to the duration of the video to be checked for duplication, each of the video frames to be checked for duplication is selected from the sequence of video frames to be checked for duplication as each of the key frames to be checked for duplication; A feature extraction process is performed on each of the key frames to be checked for duplicate content to generate feature information of each of the key frames to be checked for duplicate content.

4. The method according to claim 3, wherein: The step of selecting, according to the frame extraction adjustment coefficient and the number of frames extracted corresponding to the duration of the video to be checked for duplication, each of the video frames to be checked for duplication as each of the key frames to be checked for duplication from the sequence of video frames to be checked for duplication comprises: Determine the target frame extraction number by multiplying the frame extraction adjustment coefficient by the frame extraction number; For each two adjacent video frames to be checked for duplicates in the sequence of video frames to be checked for duplicates, optical flow feature extraction is performed on the two video frames to be checked for duplicates to generate optical flow information of the video frames to be checked for duplicates, wherein the generated optical flow information of the video frames to be checked for duplicates corresponds to optical flow time information; Sorting the generated optical flow information of each video frame to be checked for duplicates to obtain an optical flow information sequence of the video frames to be checked for duplicates; Determine the optical flow information of each to-be-checked duplicate video frame that meets the preset optical flow threshold condition in the to-be-checked duplicate video frame optical flow information sequence as the target video frame optical flow information sequence; Determine the preset video start time and each optical flow time information corresponding to the target video frame optical flow information sequence as the target time information sequence; Based on each target time information in the target time information sequence, the following key frame extraction steps are performed: In response to determining that the target time information does not satisfy the bit sequence condition, determining the frame segment time ratio according to the target time information and the preceding target time information, wherein the preceding target time information is the previous target time information adjacent to the target time information; The product of the frame segment time ratio and the target frame extraction number is determined as the frame segment frame extraction number; According to the target time and the preceding target time, the video frames to be checked for duplication, which are equal in number to the frame segment extracted, are extracted from the sequence of video frames to be checked for duplication as key frames to be checked for duplication.

5. The method according to claim 4, wherein: The step of determining the compared material video as a similar material video in response to determining that the compared material video satisfies a key frame matching condition based on the feature information of each key frame to be checked for duplication includes: Parsing the comparison material video to obtain a comparison material video frame sequence, wherein the comparison material video frames in the comparison material video frame sequence correspond to comparison material video frame time information; Performing feature extraction processing on each comparison material video frame in the comparison material video frame sequence to generate feature information of each comparison material video frame, and obtaining a comparison material video frame feature information sequence; For each piece of key frame feature information to be checked for duplicates in the key frame feature information to be checked for duplicates, the following steps are performed: Determine the video frame feature similarity between the key frame feature information to be checked for duplication and each comparison material video frame feature information in the comparison material video frame feature information sequence to obtain a video frame similarity set; In response to determining that at least one video frame similarity that satisfies the key frame feature similarity condition exists in the video frame similarity set, the following steps are performed: Determining the at least one video frame similarity as each target video frame similarity; For each target video frame similarity among the target video frame similarities, determining the to-be-checked duplicate video frame corresponding to the target video frame similarity as the target to-be-checked duplicate key frame; The comparison material video is determined as a similar material video.

6. A material video duplication checking device, comprising: A receiving unit is configured to receive a material video uploaded by a user terminal as a video to be checked for duplicate content, wherein the video to be checked for duplicate content corresponds to video identification information and at least one video keyword tag; A selection unit is configured to select at least one material video from a material video library as each comparison material video according to the video to be checked for duplication and the at least one video keyword tag, wherein the material video library is a database storing each material video uploaded by each user terminal; A parsing unit is configured to parse the video to be checked for duplication to obtain a sequence of video frames to be checked for duplication, wherein each video frame to be checked for duplication in the sequence of video frames to be checked for duplication corresponds to time information of the video frame to be checked for duplication; The duplication checking and screening unit is configured to perform the following duplication checking and screening steps for each of the comparison material videos: generating each key frame feature information to be checked for duplication according to the comparison material video and the frame sequence of the video to be checked for duplication; determining the comparison material video as a similar material video in response to determining that the comparison material video satisfies a key frame matching condition according to the key frame feature information to be checked for duplication; performing a duplication checking process on the similar material video based on each target key frame to be checked for duplication to determine the material video duplication checking information, wherein each target key frame to be checked for duplication is each video frame to be checked for duplication corresponding to each key frame feature information to be checked for duplication that satisfies the key frame feature similarity condition in the key frame feature information to be checked for duplication; The sending unit is configured to send the generated duplicate checking information of each material to the user terminal, so that the user terminal can display the duplicate checking information of each material.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Method and device for duplicate checking of videos

    CN107665261A

  • Video classification method, medium and device and computing equipment

    CN109862391A

  • Video duplicate checking method and device and electronic equipment

    CN112399236A

  • Video processing method and device, electronic equipment and readable storage medium

    CN113313065A

  • Method and device for detecting infringement video

    CN113779304A