Video matching method and device, electronic equipment and storage medium

Abstract

The application provides a video matching method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining a first image vector of a target video; the first image vector comprises a plurality of first sub-vectors, and each first sub-vector corresponds to a frame in the target video; obtaining a second image vector of a contrast video; the second image vector comprises a plurality of second sub-vectors, and each second sub-vector corresponds to a frame in the contrast video; obtaining a plurality of similarity values according to the plurality of first sub-vectors and the plurality of second sub-vectors; each similarity value is obtained according to a first sub-vector and a second sub-vector; performing Hough transformation on the plurality of similarity values to obtain a Hough transformation result; and determining a matching relationship between the target video and the contrast video according to the Hough transformation result. The video matching method provided by the application can accurately and quickly obtain a video matching result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, the technical field of multimedia and the technical field of video processing, and in particular relates to a video matching method and device, an electronic device and a storage medium. BACKGROUND

[0002] At present, users can create videos independently and upload the created videos to a video application platform.

[0003] With more and more users uploading videos, some of the uploaded videos are repeated with existing videos in the video application platform, or are low-creation videos or low-editing videos generated by simple processing of the existing videos. The increase in the number of such videos will affect the browsing experience of users. SUMMARY

[0004] The embodiments of the present application provide a video matching method and device, an electronic device and a storage medium to solve the problems in the related art. The technical solutions are as follows.

[0005] In a first aspect, the embodiments of the present application provide a video matching method, comprising:

[0006] obtaining a first image vector of a target video; the first image vector comprises a plurality of first sub-vectors, and each first sub-vector corresponds to a frame in the target video;

[0007] obtaining a second image vector of a comparison video; the second image vector comprises a plurality of second sub-vectors, and each second sub-vector corresponds to a frame in the comparison video;

[0008] obtaining a plurality of similarity values according to the plurality of first sub-vectors and the plurality of second sub-vectors; each similarity value is obtained according to one first sub-vector and one second sub-vector;

[0009] performing a Hough transform on the plurality of similarity values to obtain a Hough transform result;

[0010] determining a matching relationship between the target video and the comparison video according to the Hough transform result.

[0011] In a second aspect, the embodiments of the present application provide a video matching device, comprising:

[0012] a first image vector obtaining module configured to obtain a first image vector of a target video; the first image vector comprises a plurality of first sub-vectors, and each first sub-vector corresponds to a frame in the target video;

[0013] a second image vector obtaining module configured to obtain a second image vector of a comparison video; the second image vector comprises a plurality of second sub-vectors, and each second sub-vector corresponds to a frame in the comparison video;

[0014] The similarity value calculation module is configured to obtain a plurality of similarity values according to the plurality of first sub-vectors and the plurality of second sub-vectors, wherein each similarity value is obtained according to a first sub-vector and a second sub-vector.

[0015] The Hough transform module is configured to perform Hough transform on the plurality of similarity values to obtain a Hough transform result.

[0016] The result determination module is configured to determine the matching relationship between the target video and the control video according to the Hough transform result.

[0017] In a third aspect, an electronic device is provided, which includes at least one processor, and a memory connected with the at least one processor in communication; the memory stores instructions executable by the at least one processor, so that the at least one processor can execute the method of video matching.

[0018] In a fourth aspect, a computer readable storage medium is provided, which stores computer instructions, when the computer instructions are executed on a computer, the method in any one of the above aspects is executed.

[0019] The above technical solution has at least the following advantages or beneficial effects:

[0020] The embodiments of the present application determine the matching relationship between the target video and the control video by calculating the similarity values of the first sub-vectors and the second sub-vectors, and then performing Hough transform. The matching relationship is determined by accurate formula calculation, and the matching result can be obtained accurately and quickly.

[0021] The above summary is only for the purpose of the description and is not intended to limit in any way. In addition to the above described exemplary aspects, embodiments and features, further aspects, embodiments and features of the present application will be readily apparent to those skilled in the art by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In the drawings, the same reference numbers in the several figures indicate corresponding or similar components or elements. The drawings are not necessarily to scale, and the emphasis is normally on illustrating the principles of the application. It should be understood that the drawings are merely depictions of some embodiments of the application and should not be construed as limiting the scope of the application.

[0023] Figure 1 An application scenario schematic diagram of a video management method according to an embodiment of the present application is shown.

[0024] Figure 2 A flowchart of a video management method according to an embodiment of the present application is shown.

[0025] Figure 3 A flowchart illustrating a method for training a convolutional neural network model according to an embodiment of this application is shown.

[0026] Figure 4 A schematic diagram of a training sample according to an embodiment of this application is shown.

[0027] Figure 5 A schematic flowchart illustrating the process of obtaining a target frame image from an original frame image according to an embodiment of this application is shown.

[0028] Figure 6 This diagram illustrates the overall framework of the video management method according to an embodiment of this application.

[0029] Figure 7 A flowchart illustrating a video matching method according to an embodiment of this application is shown.

[0030] Figure 8 A flowchart illustrating the process of obtaining a similarity value in a video matching method according to an embodiment of this application is shown.

[0031] Figure 9 An embodiment according to this application is shown. Figure 9 Flowchart of the sub-steps in step S733.

[0032] Figure 10 The flowchart illustrates a video matching method according to an embodiment of this application, in which matching results are obtained based on Hough transform.

[0033] Figure 11 A schematic diagram of the Hough transform result according to an embodiment of this application is shown.

[0034] Figure 12 A schematic structural block diagram of a video management device according to an embodiment of this application is shown.

[0035] Figure 13 A schematic structural block diagram of a video matching device according to an embodiment of this application is shown.

[0036] Figure 14 This is a block diagram of an electronic device used to implement the video management method or video matching method of the embodiments of this application. Detailed Implementation

[0037] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0038] Figure 1An environment application diagram is shown schematically in accordance with an embodiment of the application. As shown in Figure 1

[0039] The provider network 2 can connect a plurality of mobile terminals 6 through a network 4. The provider network 2 can provide content services.

[0040] The content services can include content streaming services such as Internet Protocol video streaming services. The content streaming services can be configured to distribute content via various transmission technologies. The content services can be configured to provide content such as video, audio, text data, combinations thereof, and / or the like. The content can include content streams (e.g., video streams), content files (e.g., video files), and / or other data.

[0041] The provider network 2 can implement a video management service configured to allow users to upload video content and / or the like. The user uploaded video content can be rendered on a web-based presentation interface or a playback screen. The provider network 2 provides an interface or a path for the user to upload the video, the provider network 2 can be configured with a video management device for managing the user uploaded video, after the user uploaded video is analyzed, if the result of the analysis is that the user uploaded video is allowed to be uploaded, the user uploaded video can be uploaded to the video platform and can be rendered on the web-based presentation interface or the playback screen.

[0042] The provider network 2 can be located in a data center such as a single site or distributed in different geographical locations (e.g., in multiple sites). The provider network 2 can provide services via one or more networks 4. The network 4 includes various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 can include physical links such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and / or the like. The network 4 can include wireless links such as cellular links, satellite links, Wi-Fi links, and / or the like.

[0043] The plurality of mobile terminals 6 can be configured to access the content and services of the provider network 2. The plurality of mobile terminals 6 can include any type of electronic device such as mobile devices, tablet devices, laptop computers, workstations, virtual reality devices, gaming devices, set-top boxes, digital streaming media devices, vehicle terminals, smart televisions, set-top boxes, and / or the like.

[0044] The plurality of mobile terminals 6 can output (e.g., display, render, present) content (video, images, and / or the like) to users. In other embodiments, the mobile terminal 6 can also upload the target video to the video application platform to display the target video on the video application platform, so as to facilitate other users to browse and play the target video.

[0045] ​The video management scheme will be introduced through multiple embodiments. The scheme can be implemented by the electronic device 1500, which can be the provider network 2 or a computing node thereof, or the mobile terminal 6.

[0046] Figure 2 A flowchart of a video management method according to an embodiment of the application is shown. As shown in the figure, the video management method can include: Figure 2

[0047] S210, obtaining a first image vector of the target video.

[0048] The target video can be a video sent by a user to request uploading the target video to a video application platform. The video application platform can receive the video uploaded by the user and display the video uploaded by the user in a webpage for the user to click or browse and play. If the video application platform includes a plurality of videos with the same content and low editing degree, the user will repeatedly brush the same video content when browsing the videos, which affects the browsing experience of the user.

[0049] In other application scenarios, the target video can also be a video requesting to be added to a warehouse. In order to avoid repeated video content in the video warehouse, the video management method of the embodiment of the application can be used for management when requesting to add the target video to the warehouse.

[0050] In the embodiment of the application, whether to allow the target video to be uploaded or whether to allow the target video to be added to the warehouse is determined by managing the target video.

[0051] The target video can include video content originally created by the user, video created by the user based on an existing video, an existing video, or a video substantially the same as the content of the existing video, and the like.

[0052] If the target video is an existing video or a video substantially the same as the content of the existing video, the video is a repeated submission or a low-creative submission, and in order to avoid video repetition, the uploading or adding to the warehouse of the target video can be prevented.

[0053] In the embodiment of the application, the first image vector can be a feature vector representing the target video. Further, it can be a vector representing the image features of each frame of image in the target video.

[0054] In the embodiment of the application, by obtaining the first image vector of the target video, image-level feature extraction can be performed, thereby facilitating image-level feature matching. That is, each frame of image in the target video can be compared to determine whether the target video is repeated with the videos in the video warehouse.

[0055] ​S220, match the first image vector of the target video with second image vectors of a plurality of inventory videos based on the first image vector to obtain a first matching result.

[0056] The inventory videos can be videos in a video library, i.e., existing videos in the video library. The video library can be a database for storing videos in a video application platform. By matching and analyzing the first image vector of the target video with the inventory videos in the video library, it can be known whether the target video has duplicate content with the videos in the video library, avoiding duplicate content in the video library, thereby avoiding users encountering duplicate content and affecting user experience.

[0057] The second image vectors of the plurality of inventory videos can be stored in the video library together with the inventory videos. For example, the second image vectors can be marked in the tag information of the inventory videos, and when the inventory videos are obtained, the second image vectors of the inventory videos can be obtained.

[0058] In the embodiments of the present application, the first image vector of the target video is matched with the second image vectors of the inventory videos at the image level to obtain a first matching result. According to the first matching result, it can be read whether the target video has duplicate content with the inventory videos in the video library.

[0059] In one example, the first matching result can be a matching success or a matching failure. The matching success indicates that the content is duplicated, and the matching failure indicates that the video content is not duplicated.

[0060] In one example, the first matching result can also be a duplication rate. The higher the duplication rate, the higher the proportion of image frames in the target video that are duplicated with the image frames of the plurality of inventory videos in the video library.

[0061] In one example, the first matching result can be determined according to a first preset threshold. For example, when the duplication rate exceeds the first preset threshold, the first matching result is a matching success. The first preset threshold can be set according to specific application conditions.

[0062] In the embodiments of the present application, the target video is compared with the plurality of inventory videos in the video library, and the duplication rate can be obtained by comparing the image frames in the target video with the image frames of the plurality of inventory videos. That is, if the image frames in the target video are the same as the image frames in any inventory video, the image frames in the target video are duplicate images, and the proportion of the duplicate images in the total image frames in the target video is the duplication rate. When the target video is a low-creation video created based on the content reorganization of the plurality of inventory videos, the duplication rate is high. When the duplication rate exceeds the first preset threshold, it can be determined that the target video is duplicate content, so that the uploading or storage of the low-creation video can be prevented.

[0063] S230, determining a management policy of the target video according to the first matching result.

[0064] The management policy includes preventing the target video from being uploaded, or uploading the target video and adding a label including the first image vector to the target video, etc. The management policy can also be reported to a business backend for further manual review to ensure that the target video meets other requirements of the application platform.

[0065] In one example, when the first matching result is that the target video matches the inventory video, the target video is prevented from being uploaded, and relevant information is sent to the user end to inform the user.

[0066] In one example, when the first matching result is that the target video does not match the inventory video, upload information can be sent to the business backend for further manual review.

[0067] When the target video is uploaded, a classification result of the target video can also be obtained, which can be classified according to the first image vector to store the target video to a preset position in the video library.

[0068] In the embodiments of the present application, the obtained first image vector and second image vector can be obtained by analyzing the video content by using a convolutional neural network model. Existing or future applicable technical solutions for obtaining the first image vector and the second image vector can also be used.

[0069] In the embodiments of the present application, the first image vector of the target video is matched with the second image vector of the inventory video at the image level to obtain the first matching result, and the target video is managed according to the first matching result, which can avoid duplication of video content in the video library.

[0070] In one embodiment, step S210 includes: inputting the target video into a first feature extractor to obtain the first image vector by the first feature extractor; wherein the first feature extractor includes a pre-trained convolutional neural network model.

[0071] In the embodiments of the present application, the first feature extractor is used to obtain the first image vector of the target video. The first feature extractor can be a convolutional neural network model. That is, the image vector of the video is obtained by using the convolutional neural network model. The convolutional neural network model can use an existing or future applicable model structure.

[0072] In the embodiments of the present application, the first image vector of the target video is obtained by the convolutional neural network, which can quickly and accurately obtain the first image vector and improve the video management efficiency.

[0073] The second image vector of the plurality of inventory videos in the video library can also be obtained by the convolutional neural network model, and the second image vector is stored in the video library.

[0074] In the embodiments of the present application, the first image vector of the target video and the second image vector of the inventory video are obtained by the convolutional neural network model, so that the comparison and analysis at the image level can be performed based on the first image vector and the second image vector, and the repeated images between the target video and the inventory video are determined.

[0075] The traditional video matching method uses a classification model as a feature extractor to distinguish different types of images. For example, the frame image in the target video is a cat with a front face, and the inventory video can include a dog, a cat with the same appearance but a side face, and a cat with the same appearance but a front face but different colors; when the classification model is processed, the cat is regarded as the same class and the dog is regarded as another class. Therefore, the two images of the cat with the same appearance but a side face and the cat with the same appearance but a front face but different colors are regarded as matching, but the video including the two images in the image frame can not be the same video as the target video.

[0076] The embodiments of the present application use a convolutional neural network model as a feature extractor, which can perform more fine-grained feature extraction on images, so that the matching result is more accurate.

[0077] In one embodiment, as shown in Figure 3 The first feature extractor is trained multiple times by the following operations:

[0078] S410, obtaining a training sample, the training sample including a target sample, a positive sample and a plurality of negative samples.

[0079] As shown in Figure 4 The plurality of negative samples form a sequence, and the positive sample and the target sample form a sample pair. The negative sample is an image with different image content from the positive sample. The positive sample is an image with the same or similar image content as the target sample, wherein the image content of the positive sample includes an image obtained by blurring, adding text, and inverting the target sample to adjust the chroma.

[0080] S420, obtaining a positive similarity between the target sample and the positive sample, and obtaining a plurality of negative similarities between the target sample and the plurality of negative samples, respectively, the positive similarity and the plurality of negative similarities forming a prediction sequence.

[0081] The similarity between the positive sample and the target sample is high, close to 1. The similarity between the negative sample and the target sample is low, close to 0.

[0082] In the embodiments of the present application, the cosine similarity of two images is calculated to obtain the similarity of the two image samples.

[0083] AsFigure 4 As shown, there are K negative samples and 1 positive sample, and K negative similarities and 1 positive similarity with the target sample are obtained, that is, K+1 similarities are obtained in total; the K+1 similarities constitute the prediction sequence.

[0084] In S430, the target sequence is obtained. The target sequence is a sequence composed of the expected value of the positive similarity and the plurality of negative similarities.

[0085] The similarity between the positive sample and the target sample is high, close to 1, and when training the convolutional neural network model, the similarity is expected to be 1. The similarity between the negative sample and the target sample is low, close to 0, and when training the convolutional neural network model, the similarity is expected to be 0. The obtained target sequence can be , including K 0s and 1 1.

[0086] In S440, the convolutional neural network model is trained according to the target sequence and the prediction sequence.

[0087] In one example, the cross-entropy can be used as the loss function to train the convolutional neural network model.

[0088] In one example, when training the convolutional neural network model, K is set as a fixed value, the target sequence is determined, and the number of negative samples is determined, so that the convolutional neural network can be more conveniently trained. The number of negative samples is determined, but the image content of the negative samples can be different each time the training is performed.

[0089] In the embodiments of the present application, the training samples are constructed, and the model is supervised and trained in the manner of calculating the similarity, so as to train the convolutional neural network model.

[0090] In one embodiment, obtaining the training sample includes: updating the negative sample in the case of completing a single training; wherein the plurality of negative samples constitute a negative sample sequence, and updating the negative sample includes: removing the first N negative samples in the negative sample sequence, and adding N negative samples at the end of the negative sample sequence; N is a positive integer.

[0091] In the embodiments of the present application, the effect and accuracy of model training are improved by dynamically updating the negative sample.

[0092] In the embodiments of the present application, the number of removed negative samples is the same as the number of added negative samples when updating the negative sample, so that the number of updated negative samples remains unchanged, that is, the value of K is fixed, so that the same target sequence can be used for training, and the model can be conveniently trained while ensuring the training effect.

[0093] In one example, updating the negative sample can include removing the first negative sample in the negative sample sequence, and adding 1 negative sample at the end of the negative sample sequence.

[0094] In one example, the updating of the negative samples can also be removing the first two negative samples in the negative sample sequence and adding two negative samples at the end of the negative sample sequence.

[0095] In one example, the number of removed negative samples and the number of added negative samples can be the same after each single training. The number of removed negative samples after the first training can be different from the number of removed negative samples after the second training. The number of removed negative samples after each training can be determined by obtaining a random value, and the number of added negative samples is the same as the number of removed negative samples.

[0096] In one embodiment, obtaining the training samples includes updating the positive samples in the case of completing stage training; wherein one stage training includes multiple single trainings.

[0097] In the embodiments of the present application, one stage of training can be the training of one positive sample. Based on one target sample, multiple positive samples can be obtained, and the positive sample is an image obtained after adjusting the chroma based on the target sample after fuzzy processing, adding text, and reversing. For example, one target sample can obtain a corresponding number of positive samples, which is 5, and then 5 stages of training can be performed. First, the first positive sample and the target sample are used to form a positive sample pair, the negative sample sequence is dynamically updated, and multiple trainings are performed. After multiple trainings, the training result meets the model requirements, and the second stage of training is performed. In the second stage of training, the second positive sample and the target sample are used to form a positive sample pair; until the training of the 5 positive samples is completed.

[0098] After the training of the first target sample is completed, the target sample is updated and trained until an accurate convolutional neural network model is obtained.

[0099] In the embodiments of the present application, multiple trainings are performed by dynamically updating negative samples and updating positive samples to obtain an accurate convolutional neural network model.

[0100] In the embodiments of the present application, the convolutional neural network model is trained in the mode of negative sample sequence and positive sample pair, so that the obtained convolutional neural network model can distinguish the image obtained after adjusting the chroma based on the fuzzy processing, adding text, and reversing from the target sample similar image, and avoid the difficulty in distinguishing the repeated content when the target video is the video obtained after adjusting the chroma based on the fuzzy processing, adding text, and reversing of the inventory video.

[0101] In one embodiment, the positive sample includes an image obtained after one or more operations of cropping, framing, flipping, color changing, and blurring on the target sample.

[0102] The positive sample of the embodiment of the present application can be set based on application requirements. For example, in the application scene requirement, the target video obtained after the video frame is framed is regarded as a repeated video, and then the image obtained after the target sample is framed is regarded as a positive sample for training in the training of the convolutional neural network model. For another example, in some specific scene, the target video obtained after the video frame is flipped is regarded as a non-repeated video, and then the image obtained after the target sample is flipped is not regarded as a positive sample for training in the training of the convolutional neural network model, and the image obtained after the target sample is flipped is regarded as a negative sample for training.

[0103] In the embodiment of the present application, the positive sample is set for the training of the convolutional neural network model according to the actual application situation, so that the model obtained by training meets the specific application scene requirement, and the video management can be performed according to the specific application requirement.

[0104] In an implementation, the step S210 further includes:

[0105] Obtaining a plurality of original frame images of the target video;

[0106] Preprocessing the plurality of original frame images to obtain a plurality of target frame images;

[0107] Taking the plurality of target frame images as the input of the first feature extractor to obtain the output result of the first feature extractor.

[0108] The preprocessing of the original frame image can include improving the definition of the original frame image, performing chroma processing, and extracting the main object, etc.

[0109] In the embodiment of the present application, the original frame image is preprocessed, and then the target frame image after preprocessing is input into the convolutional neural network model to obtain the output result, so that the accurate output result can be obtained more efficiently.

[0110] In an implementation, the preprocessing of the plurality of original frame images to obtain the plurality of target frame images includes:

[0111] Converting the original frame image into a gray image;

[0112] Obtaining a plurality of boundary lines of the gray image;

[0113] Obtaining a rectangular frame based on the plurality of boundary lines;

[0114] According to the rectangular frame, the original frame image is cropped to obtain the target frame image corresponding to the original frame image.

[0115] In the embodiment of the present application, the preset algorithm can be a canny edge detection algorithm.

[0116] As shown in Figure 5 The original frame image is converted to obtain a gray image; a plurality of boundary lines are obtained by using a canny edge detection algorithm, to obtain an image including the plurality of boundary lines; wherein the plurality of boundary lines include a peripheral rectangular frame; the original frame image is cropped based on the rectangular frame to obtain a target frame image.

[0117] In the embodiments of the present application, after the plurality of original frame images of the target video are preprocessed, the target frame images obtained are input into the convolutional neural network model, so that the first image vectors obtained are all feature vectors expressing elements of the images themselves, unnecessary background information is avoided from being matched and processed, and the efficiency and accuracy of video matching are improved.

[0118] In an embodiment, the method further comprises:

[0119] obtaining a first video vector of the target video;

[0120] matching the first video vector with second video vectors of a plurality of inventory videos to obtain a second matching result;

[0121] determining a management strategy of the target video according to the second matching result.

[0122] In the embodiments of the present application, the first video vector is a video-level feature vector representing a feature of the video itself, which can be obtained by a recurrent neural network model.

[0123] In the embodiments of the present application, the matching of the video-level feature vector is performed simultaneously with the matching of the image-level feature vector to determine the management strategy of the target video according to the first matching result and the second matching result. For example, in the case that the first matching result is mismatching and the second matching result is matching, it is determined that the target video is duplicated with the inventory video. For another example, in the case that the first matching result is matching and the second matching result is mismatching, it is determined that the target video is duplicated with the inventory video. For another example, in the case that the first matching result is mismatching and the second matching result is mismatching, it is determined that the target video is not duplicated with the inventory video.

[0124] In one example, the management strategy of the target video can also be determined according to the matching result that is obtained faster between the first matching result and the second matching result. That is, if one of the first matching result and the second matching result is faster in obtaining the result and the result is that the target video is duplicated with the inventory video in the video library, the target video is prevented from being uploaded. In the case that the faster result shows that the target video is not duplicated with the inventory video in the video library, it is determined whether to prevent the target video from being uploaded according to the second result.

[0125] In the embodiments of the present application, the matching of the two levels of feature vectors is performed simultaneously to obtain matching results in two different ways, and a judgment is made according to the actual situation by combining the two matching results, so that the management strategy of the target video can be quickly determined.

[0126] In an embodiment, step S210 comprises:

[0127] The target number of continuous frame images are obtained from the target video.

[0128] The continuous frame images are input into the second feature extractor to obtain the first video vector by the second feature extractor.

[0129] In the embodiments of the present application, the target number of continuous frame images can be part of the continuous frame images in the target video. For example, the target video includes 100 frame images, and the target number of continuous frame images can be 30 frame images. Then, the 30 frame images are input into the second feature extractor to obtain the first video vector as the first video vector of the target video, and the first video vector is matched with the second video vectors of the plurality of inventory videos to obtain the second matching result.

[0130] The target number of continuous frame images obtained from the target video can be the target number of continuous frame images starting from the first frame image of the target video, or the target number of continuous frame images starting from a specified frame of the target video.

[0131] In the embodiments of the present application, the second feature extractor can be a recurrent neural network model, or other trained model capable of obtaining the video vector of the target video.

[0132] In the embodiments of the present application, by obtaining the target number of continuous frame images instead of all frame images of the target video, the number of frame images for matching can be reduced, and the matching efficiency can be improved.

[0133] In an embodiment, the video management method further comprises:

[0134] The first image vector is matched with the second image vectors of the plurality of restricted videos to obtain a third matching result.

[0135] The management strategy of the target video is determined according to the third matching result.

[0136] In the embodiments of the present application, the restricted video can be video content that is prohibited from being displayed and played in the video application platform, for example, a video related to illegal activities or a video content related to infringement of the exclusive broadcast rights of other video application platforms. The restricted video can be stored in the video library as a matching sample, wherein the restricted video can be stored in a different video library from the inventory video. By storing the restricted video in the video library, the target video is matched with the restricted video, and if the matching is successful, it can be determined that the target video is video content that is prohibited from being displayed and played in the video application platform, and the target video can be prevented from being uploaded.

[0137] In the embodiments of the present application, the third matching result can be obtained in a case where it is determined based on the first matching result and / or the second matching result that the target video is not repeated with the inventory video in the video library; or the third matching result can be obtained simultaneously with the first matching result and / or the second matching result. In either or any two matching results, the management strategy of the target video can be determined.

[0138] The video management method in the embodiments of the present application determines the management strategy of the target video based on one or more matching results of the first matching result, the second matching result and the third matching result; and one or more matching combinations can be used according to the specific application scenario requirements.

[0139] Figure 6 The overall framework structure of the video management method according to the embodiments of the present application is shown in the schematic diagram. Figure 6 As shown, in a case where the target video is obtained, the first image vector and the first video vector can be obtained simultaneously, and the first matching result and the second matching result can be obtained by matching the first image vector and the first video vector with the inventory video respectively; or the third matching result can be obtained by matching the first image vector with the restricted video simultaneously; and the management strategy of the target video can be determined according to one or more results of the first matching result, the second matching result and the third matching result. The management strategy of the target video can be that the video matching result is sent to the business backend, and the business backend determines whether to prevent the target video from being uploaded or to upload the target video by comparing and analyzing the comparison table.

[0140] The embodiments of the present application also provide a video matching method, as shown in Figure 7 The video matching method comprises the following steps.

[0141] S710, obtaining a first image vector of a target video; the first image vector comprises a plurality of first sub-vectors, and each first sub-vector corresponds to one frame in the target video.

[0142] In this embodiment, a first image vector of the target video is obtained based on a convolutional neural network. The first image vector includes a feature vector corresponding to each frame of the image, and multiple feature vectors from multiple frames constitute the first image vector. The first image vector represents the features of each frame of the target video and includes multiple first sub-vectors. Each first sub-vector corresponds to a frame in the target video, so that when performing video matching, matching can be performed on each frame. If multiple consecutive frames are matched, it indicates that the target video matches a corresponding reference video.

[0143] S720, Obtain the second image vector of the comparison video; the second image vector includes multiple second sub-vectors, each of which corresponds to a frame in the comparison video.

[0144] The comparison video can be an existing video from the above embodiments or a restricted video.

[0145] The second image vector of the comparison video is also obtained based on a convolutional neural network. Similar to the first image vector, the second image vector also includes multiple second sub-vectors, each corresponding to a frame in the comparison video.

[0146] S730, obtain multiple similarity values ​​based on multiple first sub-vectors and multiple second sub-vectors; wherein, each similarity value is obtained based on a first sub-vector and a second sub-vector.

[0147] In this embodiment, the similarity value between the first sub-vector and the second sub-vector is obtained by calculating the inner product of vectors. That is,

[0148]

[0149] in, ) is the first in the target video The first subvector corresponding to the frame image and the first subvector in the comparison video The similarity value between the second sub-vectors of the frame images; For the first in the target video Frame image, For the first The first sub-vector corresponding to the frame image; t is the t-th frame image in the comparison video. For the first The second sub-vector corresponding to the frame image.

[0150] S740 performs a Hough transform on multiple similarity values ​​to obtain the Hough transform result.

[0151] In this embodiment of the application, the formula for performing the Hough transform is:

[0152] S750, determining the matching relationship between the target video and the control video according to the Hough transform result.

[0153] In the Hough transform result, the video frame in the target video which is the same as the image frame of the control video can be determined. Based on the same video frame, the repetition rate can be determined, so that the matching relationship between the target video and the control video can be expressed in the repetition rate.

[0154] In one example, the matching relationship between the target video and the control video can also be expressed in a specific frame sequence repeated with the control video. For example, the first frame to the 51st frame can be repeated content, the 100th frame to the 120th frame can be repeated content, the 180th frame to the 250th frame can be repeated content, and so on.

[0155] In one example, the matching relationship between the target video and the control video can also be a conclusive description such as matching or not matching. By configuring a preset threshold, if the repetition rate exceeds the preset threshold, the output result is matching, which indicates that the target video is a low-impact video.

[0156] In the embodiment of the application, the similarity value of the first sub-vector and the second sub-vector is calculated, and then the Hough transform is performed to determine the matching relationship between the target video and the control video. The matching relationship is determined by accurate formula calculation, so that the matching result can be obtained accurately and quickly.

[0157] In one embodiment, the target video includes a plurality of target frames, wherein the plurality of target frames include a first target frame and a plurality of continuous target frames after the first target frame. As shown in Figure 8 The step S730 includes:

[0158] S731, determining a first control frame in the control video based on the first target frame, the first control frame being the video frame in the control video with the maximum similarity to the first target frame.

[0159] S732, obtaining a similarity value between the first target frame and the first control frame.

[0160] S733, determining a plurality of continuous control frames in the control video corresponding to the plurality of continuous target frames based on the plurality of continuous target frames.

[0161] S734, obtaining a plurality of similarity values corresponding to the plurality of continuous target frames based on the plurality of continuous target frames and the plurality of continuous control frames.

[0162] The steps S731 and S732 can be expressed by the following formula:

[0163]

[0164] wherein, It is a constant, a constant close to 0, or equal to 0.

[0165] In the above steps, the first target frame is the [missing information]. Based on the KNN algorithm, the image frame with the highest similarity to the first target frame in the comparison video is identified and designated as the first comparison frame. Then, several consecutive target frames in the target video are predicted, i.e., the [number of frames]. Frame, and several consecutive comparison frames in the comparison video, i.e., the first frame. The frame is the video frame with the highest similarity; where n is any positive integer. Then, by calculating the nth frame in the target video... Frame 1 and the first frame in the comparison video Frame similarity analysis can reduce the need for similarity calculations between other first and second sub-vectors, thus reducing computational load and increasing computational speed.

[0166] Accordingly, step S740 includes performing a Hough transform on the multiple similarity values ​​corresponding to the multiple target frames.

[0167]

[0168] in, This refers to the difference between the frame position of the first target frame in the target video and the frame position of the first reference frame in the reference video. The difference in frame position remains constant between several consecutive target frames and their corresponding consecutive reference frames.

[0169] In this embodiment of the application, the video frame with the highest similarity is obtained: the first frame in the target video. The frame and the t-th frame in the comparison video, i.e. the frame in the target video. The frame is matched with the t-th frame in the reference video, and then the possible matching video frames are obtained. The similarity values ​​between the possible matching video frames are subjected to Hough transform, which can reduce the calculation between other video frames with similarity values ​​close to 0, reduce the amount of computation, and improve the processing speed.

[0170] In one implementation, such as Figure 9 As shown, step S733 includes:

[0171] S7331 obtains multiple estimated speed-changing coefficients for the target video. These estimated speed-changing coefficients represent the ratio of the estimated playback speed of the target video to that of the original video.

[0172] In this embodiment, it is considered that users may adjust the speed of existing or original videos to obtain the target video. The estimated speed adjustment factor can be 0.5, 0.75, 1.25, 1.5, 1.75, and 2, which are commonly used speed adjustment factors when creating videos.

[0173] In the plurality of estimated speed change coefficients, it is assumed that the target video is one of the videos obtained based on the plurality of estimated speed change coefficients, and steps S7332 to S7334 are performed to obtain a plurality of Hough transform values.

[0174] S7332, based on the estimated speed change coefficient, a plurality of continuous target frames corresponding to a plurality of continuous reference frames in the case of no speed change are obtained.

[0175] Assuming that the target video is obtained based on the original video with a speed change of 1.5, the second frame in the target video should be the third frame of the original video in the case of no speed change. Therefore, when performing video matching, the second frame in the target video is matched with the third frame in the reference video.

[0176] S7333, based on the plurality of continuous target frames and the plurality of continuous reference frames corresponding to the plurality of continuous target frames in the case of no speed change, a plurality of similarity values corresponding to the plurality of continuous target frames are obtained.

[0177] Similarly, assuming that the target video is obtained based on the original video with a speed change of 1.5, the similarity value between the second frame in the target video and the third frame in the reference video should be obtained; the similarity value between the third frame in the target video and the 4.5th frame in the reference video; the similarity value between the fourth frame in the target video and the 6th frame in the reference video, and so on. Among them, the 4.5th frame can be determined as the 4th frame or the 5th frame based on the configuration.

[0178] S7334, Hough transform is performed on the plurality of similarity values corresponding to the plurality of target frames to obtain a Hough transform value.

[0179] S7335, based on the plurality of Hough transform values obtained based on the plurality of estimated speed change coefficients, a real speed change coefficient is determined.

[0180] Assuming that the target video is obtained based on the original video with a speed change of 1.5, after obtaining a plurality of similarities based on the above steps, Hough transform is performed to obtain a Hough transform value.

[0181] Again assuming that the target video is obtained based on the original video with a speed change of 1.25, after obtaining a plurality of similarities based on the above steps, Hough transform is performed to obtain another Hough transform value.

[0182] Based on the plurality of Hough transform values, the real speed change coefficient can be determined.

[0183] In the case of speed change, the similarity calculation formula between the first sub-vector and the second sub-vector is as follows:

[0184]

[0185] Among them, k is the speed change coefficient.

[0186] obtaining a Hough transform value based on the plurality of similarity values .

[0187] The real gear ratio can be determined based on the plurality of Hough transform values obtained based on the plurality of estimated gear ratios as follows:

[0188] .

[0189] wherein, the Hough transform value obtained based on the plurality of estimated gear ratios is the maximum Hough transform value. The Hough transform value obtained based on the plurality of estimated gear ratios is the maximum Hough transform value. The Hough transform value obtained based on the plurality of estimated gear ratios is the maximum Hough transform value. k The Hough transform value obtained based on the plurality of estimated gear ratios is the maximum Hough transform value.

[0190] S7336, determining a plurality of consecutive reference frames corresponding to the plurality of consecutive target frames in the reference video based on the real gear ratio.

[0191] In the embodiments of the present application, in the case that the target video is a variable speed video, similarity values are calculated based on a plurality of possible estimated gear ratios, and a plurality of corresponding Hough transform values are obtained. Finally, the real gear ratio is obtained based on the Hough transform values; and video matching is performed based on the real gear ratio to obtain the video matching result in the case that the target video is a variable speed video.

[0192] In one embodiment, as shown in Figure 10 , step S750 includes:

[0193] S751, obtaining a plurality of candidate clips according to the Hough transform result. The Hough transform result includes matched frame images and unmatched frame images, and the candidate clip includes a plurality of consecutive frame images.

[0194] Referring to Figure 11 , Figure 11 The Hough transform result of the embodiments of the present application is shown in the form of a data graph. The diagonal line can represent the image frames that can be matched in the target video and the reference video; the white points on the diagonal line can represent the matched image frames; the square box can represent the candidate clip, which is obtained based on the white points on the diagonal line and includes a plurality of consecutive matched frame images, and thus can be determined as a candidate clip. Different diagonal lines can represent the matching results of the target video and different reference videos.

[0195] In Figure 11 , the horizontal axis can represent the image frames of the reference video, and the vertical axis can represent the image frames of the target video.

[0196] S752, in the case that the plurality of candidate segments satisfy the preset condition, determining that the target video matches the control video.

[0197] The preset condition can be that the matching rate (repetition rate) exceeds a preset proportion. The matching rate can be a proportion of matched video frames in the target video to total video frames.

[0198] In the case that the plurality of candidate segments satisfy the preset condition, determining that the target video matches the control video. Figure 11 In the case that the plurality of candidate segments satisfy the preset condition, determining that the target video matches the control video.

[0199] In the case that the plurality of candidate segments satisfy the preset condition, determining that the target video matches the control video.

[0200] In one embodiment, step S751 comprises: in the case that the number of unmatched frame images in the continuous multiple frame images satisfies a preset threshold, determining that the continuous multiple frame images are candidate segments.

[0201] In one example, for example Figure 11 In the left first block in the case that the plurality of candidate segments satisfy the preset condition, a plurality of continuous white points can be included, but one or more points can not be white points, i.e., the frame images are not matched. In the case that the number of points that are not white points does not exceed a preset threshold, it can be determined that the continuous frame images in the block are matched candidate segments. The preset threshold can be a second preset threshold, which can be a specific value, such as 5 frames or 10 frames, etc. The second preset threshold can also be a specific value determined according to the total number of frames of the target video, for example, in the case that the total number of frames is 100 frames, the preset threshold can be determined as 5 frames according to a proportion of, for example, 5%.

[0202] In the case that the plurality of candidate segments satisfy the preset condition, determining that the target video matches the control video.

[0203] In one embodiment, step S751 comprises: in the case that the number of unmatched frame images in the continuous multiple frame images satisfies a preset threshold, determining that the continuous multiple frame images are candidate segments.

[0204] In the embodiments of the present application, when video matching is performed, individual frame images may be misaligned due to factors such as freezing, and a frame that should be matched in the current frame is matched in the previous frame or the next frame of the target video. In this case, the Hough transform result is that the current frame is not matched, but the adjacent frame is matched.

[0205] In one example, the target video may freeze, so the current frame is not matched with the corresponding frame of the comparison video, but the next frame of the current frame is matched in the comparison video, and it is considered that the current frame is matched with the comparison video.

[0206] In one example, the comparison video may freeze, so the frame image corresponding to the current frame in the comparison video is not matched with the current frame, but the next frame or the previous frame of the frame image is matched with the current frame, and it is determined that the current frame is matched.

[0207] In the embodiments of the present application, the current frame is considered to be a matched video frame by filling the current frame with adjacent frames, that is, moving the white points of the adjacent frames to the diagonal line to fill, so that the candidate segment omission caused by the freezing factor can be avoided, and the matching accuracy is affected.

[0208] In the embodiments of the present application, the current frame is filled with adjacent frames, which can be filled by using the following expression:

[0209]

[0210] wherein, i is the adjacent frame, i The range of the current frame during the filling operation, that is, the corrected similarity value, can be adjusted according to specific application conditions, for example, it can be . is the similarity value of the current frame during the filling operation, that is, the corrected similarity value.

[0211] In one embodiment, step S751 includes: in the case where one video segment of the comparison video is matched with multiple video segments of the target video, excluding the video segment as a candidate segment.

[0212] In the embodiments of the present application, it is considered that in some target videos, a theme frame or a theme segment may be used to connect segments. For example, in a target video, after playing a segment, a theme segment is played, then the next segment is played, then the theme segment is played, and then the subsequent segment is played. In this case, multiple video segments in the target video frame are matched with one video segment in the comparison video multiple times. The repetition of such theme video segments does not affect the editing degree of the target video. Therefore, it can be excluded as a candidate segment.

[0213] In one example, the video segment can be excluded as a candidate segment in a manner of non-maximum suppression.

[0214] Figure 12 A structural block diagram of a video management apparatus according to an embodiment of the present application is shown. As shown, the video management apparatus 1300 can include: Figure 12

[0215] A first image acquisition module 1301 configured to acquire a first image vector of a target video;

[0216] A first matching module 1302 configured to match the first image vector with second image vectors of a plurality of inventory videos, to obtain a first matching result;

[0217] A management module 1303 configured to determine a management strategy of the target video according to the first matching result.

[0218] In one embodiment, the first image acquisition module 1301 is configured to:

[0219] input the target video into a first feature extractor, to obtain the first image vector through the first feature extractor;

[0220] wherein the first feature extractor includes a pre-trained convolutional neural network model.

[0221] In one embodiment, the first feature extractor is obtained through multiple times of training by:

[0222] acquiring training samples, the training samples including a target sample, a positive sample, and a plurality of negative samples;

[0223] acquiring a positive similarity between the target sample and the positive sample, and respectively acquiring negative similarities between the target sample and the plurality of negative samples, the positive similarity and the plurality of negative similarities constituting a prediction sequence;

[0224] acquiring a target sequence, the target sequence being a sequence constituted by expected values of the positive similarity and the plurality of negative similarities;

[0225] training the convolutional neural network model according to the target sequence and the prediction sequence.

[0226] In one embodiment, acquiring training samples includes:

[0227] updating the negative samples upon completion of a single time of training; wherein the plurality of negative samples constitute a negative sample sequence, updating the negative samples includes removing the first N negative samples in the negative sample sequence, and adding N negative samples at the end of the negative sample sequence; N is a positive integer. ​

[0228] In an implementation, the obtaining the training samples comprises: in a case that a stage training is completed, updating the positive samples; wherein one stage training comprises a plurality of single trainings.

[0229] In an implementation, the obtaining the training samples comprises: obtaining positive samples, the positive samples comprising images obtained by performing one or more of cropping, framing, flipping, color changing, or blurring on the target samples.

[0230] In an implementation, the first image vector obtaining module is further configured to:

[0231] obtain a plurality of original frame images of the target video;

[0232] preprocess the plurality of original frame images to obtain a plurality of target frame images;

[0233] input the plurality of target frame images into the first feature extractor to obtain an output result of the first feature extractor.

[0234] In an implementation, the preprocessing the plurality of original frame images to obtain a plurality of target frame images comprises:

[0235] convert the original frame image into a grayscale image;

[0236] obtain a plurality of boundary lines of the grayscale image;

[0237] obtain a rectangular frame based on the plurality of boundary lines;

[0238] crop the original frame image according to the rectangular frame to obtain a target frame image corresponding to the original frame image.

[0239] In an implementation, the video management apparatus further comprises:

[0240] a first video vector obtaining module configured to obtain a first video vector of a target video;

[0241] a second matching module configured to match the first video vector with second video vectors of a plurality of inventory videos to obtain a second matching result;

[0242] a management module configured to determine a management strategy of the target video according to the second matching result.

[0243] In an implementation, the first video vector module is configured to:

[0244] obtain a target number of continuous frame images from the target video;

[0245] inputting the continuous frame images into a second feature extractor to obtain the first video vector by the second feature extractor.

[0246] In an embodiment, the video management apparatus further comprises:

[0247] a third matching module configured to match the first image vector with second image vectors of a plurality of limited videos to obtain a third matching result;

[0248] a management module configured to determine a management strategy of the target video according to the third matching result.

[0249] Figure 13 A structural block diagram of a video matching apparatus according to an embodiment of the present application is shown. As shown in the figure, the video matching apparatus 1400 can comprise: Figure 13

[0250] a first image vector obtaining module 1401 configured to obtain a first image vector of a target video; the first image vector comprises a plurality of first sub-vectors, each of which corresponds to a frame in the target video;

[0251] a second image vector obtaining module 1402 configured to obtain a second image vector of a control video; the second image vector comprises a plurality of second sub-vectors, each of which corresponds to a frame in the control video;

[0252] a similarity value obtaining module 1403 configured to obtain a plurality of similarity values according to the plurality of first sub-vectors and the plurality of second sub-vectors; each similarity value is obtained according to one first sub-vector and one second sub-vector;

[0253] a Hough transform module 1404 configured to perform Hough transform on the plurality of similarity values to obtain a Hough transform result;

[0254] a result determining module 1405 configured to determine a matching relationship between the target video and the control video according to the Hough transform result.

[0255] In an embodiment, the target video comprises a plurality of target frames, wherein the plurality of target frames comprises a first target frame and a plurality of continuous target frames after the first target frame;

[0256] The similarity value obtaining module 1403 is configured to:

[0257] determine a first control frame in the control video based on the first target frame, the first control frame being a frame in the control video that is most similar to the first target frame;

[0258] ​obtain a similarity value between the first target frame and the first reference frame;

[0259] based on the plurality of continuous target frames, determine a plurality of continuous reference frames in the reference video corresponding to the plurality of continuous target frames;

[0260] based on the plurality of continuous target frames and the plurality of continuous reference frames, obtain a plurality of similarity values corresponding to the plurality of continuous target frames;

[0261] based on the plurality of current frames, obtain a plurality of maximum similarity values;

[0262] correspondingly, the Hough transform module 1404 is configured to:

[0263] perform Hough transform on the similarity values corresponding to the plurality of target frames.

[0264] In an embodiment, the similarity value obtaining module 1403 is configured to:

[0265] obtain a plurality of estimated speedup coefficients of the target video; wherein the estimated speedup coefficient represents an estimated playback speed ratio of the target video and the original video;

[0266] based on the estimated speedup coefficient, obtain a plurality of continuous reference frames corresponding to the plurality of continuous target frames without speedup;

[0267] based on the plurality of continuous target frames and the plurality of continuous reference frames corresponding to the plurality of continuous target frames without speedup, obtain a plurality of similarity values corresponding to the plurality of continuous target frames;

[0268] perform Hough transform on the plurality of similarity values corresponding to the plurality of target frames to obtain a Hough transform value;

[0269] based on the plurality of Hough transform values obtained based on the plurality of estimated speedup coefficients, determine a real speedup coefficient;

[0270] based on the real speedup coefficient, determine a plurality of continuous reference frames in the reference video corresponding to the plurality of continuous target frames.

[0271] In an embodiment, the result determining module 1405 includes:

[0272] a candidate segment obtaining sub-module configured to obtain a plurality of candidate segments according to the Hough transform result; wherein the Hough transform result includes matched frame images and unmatched frame images, and the candidate segment includes a plurality of continuous frame images;

[0273] The matching relationship determination submodule is used to determine whether the target video matches the comparison video when multiple candidate segments meet preset conditions.

[0274] In one implementation, the candidate fragment acquisition submodule is used for:

[0275] If the number of unmatched frames in a series of consecutive images meets a preset threshold, the series of consecutive frames are determined to be candidate segments.

[0276] In one implementation, the candidate segment acquisition submodule is configured to: determine the current frame image as a matching frame image in a series of consecutive frames, where the current frame image does not match but includes at least one neighboring frame that matches; wherein the neighboring frame is a frame image before or after the current frame.

[0277] In one implementation, the candidate segment acquisition submodule is configured to: cancel the video segment as a candidate segment if a video segment of the comparison video matches multiple video segments of the target video.

[0278] The functions of each module in each device in the embodiments of this application can be found in the corresponding descriptions in the above methods, and will not be repeated here.

[0279] Figure 14 A structural block diagram of an electronic device 1500 according to an embodiment of this application is shown. Figure 14 As shown, the electronic device 1500 includes a memory 1510 and a processor 1520. The memory 1510 stores instructions that can be executed on the processor 1520. When the processor 1520 executes the instructions, it implements the video management method or video matching method described in the above embodiments. The number of memories 1510 and processors 1520 can be one or more. The electronic device 1500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1500 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0280] The electronic device 1500 can further include a communication interface 1530 for communicating with external devices, for data exchange and transmission. The various devices are connected to each other by different buses, and can be mounted on a common mainboard or in other ways as needed. The processor 1520 can process instructions executed within the electronic device 1500, including instructions stored in the memory or on the memory to display graphical information on a GUI on an external input / output device such as a display device coupled to the interface. In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple storage devices, if necessary. Similarly, multiple electronic devices 1500 can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 14 In the figure, only one thick line is used, but it does not mean that there is only one bus or one type of bus.

[0281] Optionally, in a specific implementation, if the memory 1510, the processor 1520 and the communication interface 1530 are integrated on a chip, the memory 1510, the processor 1520 and the communication interface 1530 can complete communication with each other through an internal interface.

[0282] It should be understood that the above processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It is worth noting that the processor can be a processor supporting advanced RISC machine (ARM) architecture.

[0283] The embodiment of the present application provides a computer readable storage medium (such as the memory 1510 described above), which stores computer instructions, and the program is executed by the processor to realize the method provided in the embodiment of the present application.

[0284] Optionally, the memory 1510 can include a program storage area and a data storage area. The program storage area can store the operating system, application programs required by at least one function, and the like. The data storage area can store data created by the electronic device 1500 according to the video management method or the video matching method, and the like. In addition, the memory 1510 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 1510 can optionally include a memory disposed remotely from the processor 1520, and these remote memories can be connected to the electronic device 1500 according to the video management method or the video matching method through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0285] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0286] In addition, the terms "first", "second", and the like are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0287] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or other processes. Also, the preferred embodiments of the present application can include additional implementations in which the order of steps can differ from that shown or discussed, including a process performed in substantially simultaneous fashion or in reverse order.

[0288] The logic and / or steps represented in the flow diagrams and / or otherwise described herein, for example, can be embodied in non-transitory computer-readable media, executed by an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed.

[0289] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-mentioned embodiment method can be instructed by a program to complete the relevant hardware, and the program can be stored in a computer readable storage medium, and the program includes one of the steps of the method embodiment or a combination thereof when executed.

[0290] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. The above-mentioned integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0291] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various changes or replacements within the technical scope disclosed in the present application, and these should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A video matching method, characterized in that, include: Obtain a first image vector of the target video; the first image vector includes a plurality of first sub-vectors, each of which corresponds to a frame in the target video. Obtain a second image vector from the comparison video; the second image vector includes multiple second sub-vectors, each of which corresponds to a frame in the comparison video. Multiple similarity values ​​are obtained based on multiple first sub-vectors and multiple second sub-vectors; wherein each similarity value is obtained based on one first sub-vector and one second sub-vector. Perform Hough transform on multiple similarity values ​​to obtain the Hough transform result; Based on the Hough transform results, the matching relationship between the target video and the reference video is determined; The process of obtaining multiple similarity values ​​based on multiple first sub-vectors and multiple second sub-vectors includes: obtaining the first reference frame with the highest similarity to the first target frame in the target video from the reference video, and comparing the similarity between several consecutive video frames after the first target frame and several consecutive reference frames after the first reference frame. The target video includes multiple target frames, wherein the multiple target frames include a first target frame and a plurality of consecutive target frames following the first target frame; determining the plurality of consecutive comparison frames includes: Multiple estimated speed-changing coefficients are obtained for the target video; wherein the estimated speed-changing coefficients represent the ratio of the estimated playback speed of the target video and the original video; Based on the estimated speed change coefficient, several consecutive reference frames are obtained for several consecutive target frames after the first target frame when the speed does not change. Based on the plurality of consecutive target frames and the plurality of consecutive reference frames corresponding to the plurality of consecutive target frames without speed change, a plurality of similarity values ​​corresponding to the plurality of consecutive target frames are obtained; Perform Hough transform on the multiple similarity values ​​corresponding to the multiple target frames to obtain Hough transform values; Based on the multiple Hough transform values ​​obtained from the multiple estimated gear ratios, the actual gear ratios are determined. Based on the actual speed change coefficient, determine a number of consecutive reference frames in the reference video that correspond to the number of consecutive target frames.

2. The method according to claim 1, characterized in that, The step of performing Hough transform on multiple similarity values ​​includes performing Hough transform on multiple similarity values ​​corresponding to the multiple target frames.

3. The method according to claim 1 or 2, characterized in that, Determining the matching relationship between the target video and the reference video based on the Hough transform result includes: Based on the Hough transform result, multiple candidate segments are obtained; wherein, the Hough transform result includes matching frame images and unmatched frame images, and the candidate segments include multiple consecutive frame images; If multiple candidate segments meet preset conditions, the target video is determined to match the comparison video.

4. The method according to claim 3, characterized in that, Based on the Hough transform result, multiple candidate segments are obtained, including: If the number of unmatched frames in a series of consecutive images meets a preset threshold, the series of consecutive frames are determined to be candidate segments.

5. The method according to claim 3 or 4, characterized in that, Based on the Hough transform result, multiple candidate segments are obtained, including: In a series of consecutive frames, if the current frame does not match but at least one neighboring frame does, the current frame is determined to be a matching frame; wherein the neighboring frame is a frame before or after the current frame.

6. The method according to claim 3, characterized in that, Based on the Hough transform result, multiple candidate segments are obtained, including: If a video segment from the comparison video matches multiple video segments from the target video, the video segment is removed as a candidate segment.

7. A video matching device, characterized in that, include: The first image vector acquisition module is used to acquire a first image vector of the target video; the first image vector includes a plurality of first sub-vectors, each of which corresponds to a frame in the target video. The second image vector acquisition module is used to acquire a second image vector of the comparison video; the second image vector includes a plurality of second sub-vectors, each of which corresponds to a frame in the comparison video. The similarity value calculation module is used to obtain multiple similarity values ​​based on multiple first sub-vectors and multiple second sub-vectors; wherein each similarity value is obtained based on one first sub-vector and one second sub-vector; The Hough transform module is used to perform Hough transform on multiple similarity values ​​to obtain the Hough transform result; The result determination module is used to determine the matching relationship between the target video and the reference video based on the Hough transform result; The process of obtaining multiple similarity values ​​based on multiple first sub-vectors and multiple second sub-vectors includes: obtaining the first reference frame with the highest similarity to the first target frame in the target video from the reference video, and comparing the similarity between several consecutive video frames after the first target frame and several consecutive reference frames after the first reference frame. The target video includes multiple target frames, wherein the multiple target frames include a first target frame and a plurality of consecutive target frames following the first target frame; determining the plurality of consecutive comparison frames includes: Multiple estimated speed-changing coefficients are obtained for the target video; wherein the estimated speed-changing coefficients represent the ratio of the estimated playback speed of the target video and the original video; Based on the estimated speed change coefficient, several consecutive reference frames are obtained for several consecutive target frames after the first target frame when the speed does not change. Based on the plurality of consecutive target frames and the plurality of consecutive reference frames corresponding to the plurality of consecutive target frames without speed change, a plurality of similarity values ​​corresponding to the plurality of consecutive target frames are obtained; Perform Hough transform on the multiple similarity values ​​corresponding to the multiple target frames to obtain Hough transform values; Based on the multiple Hough transform values ​​obtained from the multiple estimated gear ratios, the actual gear ratios are determined. Based on the actual speed change coefficient, determine a number of consecutive reference frames in the reference video that correspond to the number of consecutive target frames.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

10. A computer program product comprising computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images