Method, device and terminal for determining similar video clips, and storage medium

Abstract

The application discloses a method and device for determining similar video clips, a terminal and a storage medium. The method comprises the following steps: calculating the video similarity of a first video and a second video; if the video similarity meets a preset video similarity, determining the similar images corresponding to the first video and the second video by using a first preset algorithm; and determining the similar video clips corresponding to the first video and the second video based on the similar images. The application calculates the similarity of the two videos by using a similar matrix composed of the first video and the second video, and extracts the similar video clips corresponding to the first video and the second video by using a greedy algorithm, thereby improving the accuracy of the similar video clips and having wide adaptability, and being applicable not only to a single video but also to spliced and cut videos.

Description

Technical Field

[0001] This application relates to the field of video retrieval technology, and more specifically, to a method, apparatus, terminal, and storage medium for determining similar video clips. Background Technology

[0002] Video retrieval refers to searching for useful or needed information from videos. With the evolution and development of video retrieval technology, intelligent algorithms are being applied to video retrieval. However, how to improve the effectiveness of video retrieval has become a pressing issue.

[0003] Currently, machine learning methods are mainly used to achieve video retrieval. Specifically, the image features of two videos are first extracted through a neural network model, and then the cosine similarity of the image features is calculated to determine the similarity between the two videos. Finally, the videos that meet the preset similarity are compared with all videos in the database to determine similar video segments.

[0004] However, the above method suffers from low accuracy in retrieving similar video clips. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, terminal, and storage medium for determining similar video clips, so as to solve the problem of low accuracy in similar video clip retrieval in related technologies.

[0006] To achieve the above objectives, firstly, this application provides a method for determining similar video segments, comprising:

[0007] Calculate the video similarity between the first video and the second video;

[0008] If the video similarity meets the preset video similarity, the first preset algorithm is used to determine the similar images corresponding to the first video and the second video. The similar images are determined by calculating the image similarity between the frame images in the first video and the frame images in the second video. The image similarity is determined by the image features corresponding to the frame images in the first video and the frame images in the second video extracted by the contrastive learning model.

[0009] Identify similar video segments corresponding to the first and second videos based on similar images.

[0010] In one possible implementation, calculating the video similarity between the first video and the second video includes:

[0011] Determine the similarity matrix between the first video and the second video;

[0012] The similarity of the videos is determined using the second preset algorithm and the similarity matrix.

[0013] In one possible implementation, determining the similarity matrix between the first video and the second video includes:

[0014] Extract m frames from the first video and n frames from the second video respectively;

[0015] Perform an inner product operation on each frame of the m-frame images and each frame of the n-frame images to form a similarity matrix.

[0016] In one possible implementation, video similarity is determined using a second pre-defined algorithm and a similarity matrix, including:

[0017] Extract the maximum value of each column vector and the maximum value of each row vector in the similarity matrix;

[0018] The sum of the maximum values ​​of each column vector is divided by the number of column vectors to obtain the first quotient value, which is then used as the maximum value of the similarity matrix in the column direction.

[0019] The sum of the maximum values ​​of each row vector in the row vector is divided by the number of row vectors to obtain the second quotient value, which is then used as the maximum value of the similarity matrix in the row direction.

[0020] The maximum value between the maximum value in the listed directions and the maximum value in the row direction is selected as the video similarity.

[0021] In one possible implementation, if the video similarity satisfies a preset video similarity, a first preset algorithm is used to determine similar images corresponding to the first video and the second video, including:

[0022] If the video similarity meets the preset video similarity, select one frame image at a time, starting from the first frame image in the m-frame images.

[0023] Search for a frame image in n frames that has an image similarity greater than or equal to a preset image similarity with a given frame image, until no frame image with an image similarity greater than or equal to the preset image similarity can be found in the n frames.

[0024] Frames with similarity scores greater than or equal to a preset image similarity score are considered as similar images.

[0025] In one possible implementation, if a frame image with an image similarity greater than or equal to a preset image similarity is searched among n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes:

[0026] If there is an i-th frame in the n-frame images whose image similarity to the first frame in the m-frame images is greater than or equal to the preset image similarity, then continue to select the second frame in the m-frame images and traverse the frames after the i-th frame in the n-frame images, where i is greater than 0 and less than n;

[0027] Until there is no frame image after the i-th frame that has an image similarity to the second frame image greater than or equal to the preset image similarity, where j is greater than i and less than n.

[0028] In one possible implementation, if a frame image with an image similarity greater than or equal to a preset image similarity is searched among n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes:

[0029] If there is an i-th frame in the n-frame images whose image similarity to the first frame in the m-frame images is greater than or equal to the preset image similarity, then continue to select the second frame in the m-frame images and traverse the frames after the i-th frame in the n-frame images, where i is greater than 0 and less than n;

[0030] If there is a frame image j in the frame image after the i-th frame image that has an image similarity to the second frame image greater than or equal to the preset image similarity, continue to select the third frame image and traverse the frame images after the j-th frame image in the n-frame images until there is no frame image k in the frame image after the j-th frame image that has an image similarity to the third frame image greater than or equal to the preset image similarity, where j is greater than i and less than n, and k is greater than j and less than n.

[0031] Secondly, embodiments of the present invention provide a device for determining similar video segments, comprising:

[0032] The video similarity calculation module is used to calculate the video similarity between the first video and the second video.

[0033] A similar image determination module is used to determine similar images corresponding to a first video and a second video if the video similarity meets a preset video similarity. The similar images are determined by calculating the image similarity between frame images in the first video and frame images in the second video. The image similarity is determined by extracting image features corresponding to frame images in the first video and image features corresponding to frame images in the second video using a contrastive learning model.

[0034] The similar video segment determination module is used to determine similar video segments corresponding to the first video and the second video based on similar images.

[0035] Thirdly, embodiments of the present invention provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods for determining similar video segments.

[0036] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for determining similar video segments.

[0037] This invention provides a method, apparatus, terminal, and storage medium for determining similar video segments. The method includes: calculating the video similarity between a first video and a second video; if the video similarity satisfies a preset video similarity, determining similar images corresponding to the first video and the second video using a first preset algorithm; and then determining similar video segments corresponding to the first video and the second video based on the similar images. This invention calculates the similarity between two videos by using a similarity matrix composed of the first video and the second video, and then uses a greedy algorithm to extract similar video segments corresponding to the first video and the second video. This improves the accuracy of similar video segment extraction and has broad applicability, suitable not only for single videos but also for spliced ​​and cut videos. Attached Figure Description

[0038] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0039] Figure 1 This is a flowchart illustrating the implementation of a method for determining similar video segments provided in an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of training images for the contrastive learning model provided in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the structure of a device for determining similar video segments provided in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0045] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0046] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0047] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0048] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0049] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0050] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0051] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0052] In one embodiment, such as Figure 1 As shown, a method for determining similar video clips is provided, including the following steps:

[0053] Step S101: Calculate the video similarity between the first video and the second video;

[0054] Step S102: If the video similarity meets the preset video similarity, use the first preset algorithm to determine the similar images corresponding to the first video and the second video;

[0055] Step S103: Determine similar video segments corresponding to the first video and the second video based on similar images.

[0056] The similarity of images is determined by calculating the image similarity between frames in the first video and frames in the second video. The image similarity is determined by extracting image features corresponding to frames in the first video and frames in the second video using a contrastive learning model.

[0057] Contrastive learning is a cutting-edge method in machine learning that aims to automatically learn representations of data from large datasets without requiring cumbersome manual annotation. In the field of computer video, a crucial step in contrastive learning is to apply rich data augmentation and perturbation to images, such as randomly cropping images or changing their brightness, while still extracting the inherent features of objects within the image. Figure 2 For example, the common feature of the following 10 images is dogs. The contrastive learning model can easily learn this common feature from the 10 images. The other parts of the images are some unnatural factors added in the later editing. Therefore, by using the above 10 images to train the contrastive learning model, the contrastive learning model can directly extract this common feature, that is, the common features of multiple images.

[0058] Specifically, this invention first acquires two videos, a first video and a second video; it calculates the similarity between the two videos using their similarity matrices; if the video similarity satisfies a preset video similarity (a preset value within the system, not specifically limited), it extracts image features corresponding to frames in the first video and frames in the second video using a contrastive learning model; then, it calculates the image similarity between frames in the first video and frames in the second video using a greedy algorithm, thus obtaining similar images corresponding to the first video and the second video; finally, it determines similar video segments corresponding to the first video and the second video based on these similar images.

[0059] This invention provides a method for determining similar video segments, comprising: calculating the video similarity between a first video and a second video; if the video similarity satisfies a preset video similarity, determining similar images corresponding to the first video and the second video using a first preset algorithm; and then determining similar video segments corresponding to the first video and the second video based on the similar images. This invention calculates the similarity between two videos by using a similarity matrix composed of the first video and the second video, and then uses a greedy algorithm to extract similar video segments corresponding to the first video and the second video, thus improving the accuracy of similar video segment determination. It also has broad applicability, suitable not only for single videos but also for spliced ​​and cut videos.

[0060] In one embodiment, step S101 includes:

[0061] Step S201: Determine the similarity matrix of the first video and the second video.

[0062] Specifically, determining the similarity matrix includes: extracting m frames from the first video and n frames from the second video; and performing an inner product operation on each frame in the m frames and each frame in the n frames to form a similarity matrix.

[0063] Furthermore, assuming a first video ft and a second video frame fq, extract m frames from the first video ft and n frames from the second video frame fq. Perform an inner product operation on the m and n frames to construct the following similarity matrix S:

[0064]

[0065] Step S202: Determine the video similarity using the second preset algorithm and the similarity matrix.

[0066] Specifically, determining video similarity includes: extracting the maximum value of each column vector and the maximum value of each row vector in the similarity matrix; summing the maximum values ​​of each column vector and dividing the result by the number of column vectors to obtain a first quotient, which is then used as the maximum value of the similarity matrix in the column direction; summing the maximum values ​​of each row vector and dividing the result by the number of row vectors to obtain a second quotient, which is then used as the maximum value of the similarity matrix in the row direction; and selecting the maximum value from the maximum values ​​in the column direction and the maximum value in the row direction as the video similarity.

[0067] To clearly illustrate the calculation process of video similarity, suppose the first video ft in the similarity matrix S contains 4 images, namely... and The second frame of the video contains two images, which are... and

[0068] First, perform an inner product operation on the images from the two videos to obtain the following similarity matrix fqft:

[0069]

[0070] Secondly, the maximum value of the similarity matrix in the column direction and the maximum value in the row direction is calculated using the following formula, which is the video similarity.

[0071]

[0072] Specifically, substituting the similarity matrix fqft (i.e., the vector in formula (2)) into formula (3) yields formula (4), which is:

[0073]

[0074] According to formula (4), the similarity between the two videos (i.e., video similarity) is 0.8.

[0075] In one embodiment, step S102 includes:

[0076] Step S301: If the video similarity meets the preset video similarity, select one frame image at a time, starting from the first frame image in the m frame images;

[0077] Step S302: Search for a frame image in n frames that has an image similarity greater than or equal to a preset image similarity with a frame image, until no frame image in n frames can be found that has an image similarity greater than or equal to a preset image similarity with a frame image;

[0078] Step S303: Select the frame images with image similarity greater than or equal to the preset image similarity as similar images.

[0079] In the above embodiment, the similarity between the two videos is 0.8. If the preset video similarity is less than or equal to 0.8, it means that the two videos meet the similarity requirement, and the next step of similar image extraction can be carried out; if the preset video similarity is greater than 0.8, it means that the two videos do not meet the similarity requirement, and similar image extraction cannot be carried out.

[0080] Assuming the first video and the second video meet the similarity requirements, similar images are extracted from both. This invention extracts similar images by first calculating the similarity between frames in the first video and their corresponding frames in the second video, and then identifying frames with similarity scores greater than or equal to a preset similarity score as similar images. Specifically, the first frame in the first video is compared with all frames in the second video. If the similarity score of the resulting frame (the i-th frame) is greater than or equal to the preset similarity score, then the second frame in the first video is compared with all frames after the i-th frame. If the similarity score of the resulting j-th frame is greater than or equal to the preset similarity score, then the third frame in the first video is compared with all frames after the j-th frame, and so on, until no frame in the second video has a similarity score greater than or equal to the preset similarity score.

[0081] Taking the second iteration of the second video and the third iteration of the second video as examples, the specific explanations of steps S302 and S302 are as follows:

[0082] The second traversal of the second video ends the traversal. Specifically: if there is an i-th frame image in the n-frame images whose image similarity to the first frame image in the m-frame images is greater than or equal to the preset image similarity, continue to select the second frame image in the m-frame images and traverse the frames after the i-th frame image in the n-frame images, where i is greater than 0 and less than n; until there is no j-th frame image in the frames after the i-th frame image whose image similarity to the second frame image is greater than or equal to the preset image similarity, where j is greater than i and less than n; extract the first frame image and the i-th frame image as similar images.

[0083] The third iteration of the second video ends the iteration. Specifically: if there is an i-th frame in the n-frame images whose image similarity to the first frame in the m-frame images is greater than or equal to a preset image similarity, continue to select the second frame in the m-frame images and iterate through the frames after the i-th frame in the n-frame images, where i is greater than 0 and less than n; if there is a j-th frame in the frames after the i-th frame images whose image similarity to the second frame images is greater than or equal to a preset image similarity, continue to select the third frame images and iterate through the frames after the j-th frame in the n-frame images, until there is no k-th frame in the frames after the j-th frame images whose image similarity to the third frame images is greater than or equal to a preset image similarity, where j is greater than i and less than n, and k is greater than j and less than n; extract the first frame image, the second frame image, the i-th frame image, and the j-th frame image as similar images.

[0084] Furthermore, the traversal ends on the fourth traversal of the second video, the fifth traversal of the second video, and so on until the z-th traversal of the second video. The traversal process always follows the above rules, where z is a positive integer greater than 5.

[0085] It should be noted that during the above image similarity calculation process, if a frame image with an image similarity greater than or equal to a frame image in the first video is found in the second video, then when calculating the image similarity next time, it is necessary to traverse all frame images after the frame image in the second video obtained in the previous calculation.

[0086] Based on short video search scenarios, we simulated eleven batch video editing methods, including video splicing, picture-in-picture, border addition, brightness variation, video time trimming, aspect ratio cropping, flipping, rotation, video acceleration, subtitle addition, and watermark addition. Modifying 500 videos using these eleven methods yielded a total of 6,000 videos. The method of this invention was then tested on these 6,000 videos, with the goal of detecting eleven videos from the same source for each of the twelve videos in each group. The video splicing and video time trimming methods were also used to test the accuracy of extracting similar video segments.

[0087] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0088] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0089] Figure 3The diagram illustrates the structure of a device for determining similar video segments according to an embodiment of the present invention. For ease of explanation, only the parts relevant to the embodiment are shown. The device for determining similar video segments includes a video similarity calculation module 31, a similar image determination module 32, and a similar video segment determination module 33, as detailed below:

[0090] Video similarity calculation module 31 is used to calculate the video similarity between the first video and the second video;

[0091] The similar image determination module 32 is used to determine similar images corresponding to the first video and the second video using a first preset algorithm if the video similarity meets the preset video similarity. The similar images are determined by calculating the image similarity between the frame images in the first video and the frame images in the second video. The image similarity is determined by the image features corresponding to the frame images in the first video and the frame images in the second video extracted by the comparison model.

[0092] The similar video segment determination module 33 is used to determine similar video segments corresponding to the first video and the second video based on similar images.

[0093] In one possible implementation, the video similarity calculation module 31 includes:

[0094] The similarity matrix determination submodule is used to determine the similarity matrix between the first video and the second video;

[0095] The video similarity determination submodule is used to determine video similarity using a second preset algorithm and a similarity matrix.

[0096] In one possible implementation, the similarity matrix determination submodule includes:

[0097] The frame video extraction unit is used to extract m frames from the first video and n frames from the second video, respectively.

[0098] The inner product operation unit is used to perform inner product operations on each frame of m frames and each frame of n frames to form a similarity matrix.

[0099] In one possible implementation, the video similarity determination submodule includes:

[0100] The maximum value extraction unit is used to extract the maximum value of each column vector and the maximum value of each row vector in the similarity matrix.

[0101] The first maximum value determination unit is used to sum the maximum values ​​of each column vector in the column vector and divide the result by the number of column vectors to obtain the first quotient value, and use the first quotient value as the maximum value of the similarity matrix in the column direction;

[0102] The second maximum value determination unit is used to sum the maximum value of each row vector in the row vector and divide the result by the number of row vectors to obtain the second quotient value, and use the second quotient value as the maximum value of the similarity matrix in the row direction;

[0103] The video similarity determination unit is used to select the maximum value among the maximum values ​​in the listed directions and the maximum values ​​in the row direction as the video similarity.

[0104] In one possible implementation, the similar image determination module 32 includes:

[0105] The frame image selection submodule is used to select one frame image at a time, starting from the first frame image in m frames, if the video similarity meets the preset video similarity.

[0106] The frame image search submodule is used to search for a frame image in n frames that has an image similarity greater than or equal to a preset image similarity with a given frame image, until no frame image with an image similarity greater than or equal to the preset image similarity can be found in the n frames.

[0107] The similar image determination submodule is used to identify frame images with similarity scores greater than or equal to a preset image similarity score as similar images.

[0108] In one possible implementation, the frame image lookup submodule includes:

[0109] The first search unit is used to select the second frame image in the m frames and traverse the frames after the i-th frame image in the n frames if there is an i-th frame image in the n frames that has an image similarity greater than or equal to a preset image similarity with the first frame image in the m frames, where i is greater than 0 and less than n.

[0110] The first termination unit is used until there is no frame j in the frame images after the i-th frame image where the image similarity to the second frame image is greater than or equal to the preset image similarity, where j is greater than i and less than n.

[0111] In one possible implementation, the frame image lookup submodule includes:

[0112] The first search unit is used to select the second frame image in the m frames and traverse the frames after the i-th frame image in the n frames if there is an i-th frame image in the n frames that has an image similarity greater than or equal to a preset image similarity with the first frame image in the m frames, where i is greater than 0 and less than n.

[0113] The second termination unit is used to continue selecting the third frame image and traversing the frames after the j-th frame image in the n-th frame images if there is a j-th frame image whose image similarity to the second frame image is greater than or equal to a preset image similarity, until there is no k-th frame image whose image similarity to the third frame image is greater than or equal to a preset image similarity in the frames after the j-th frame image, where j is greater than i and less than n, and k is greater than j and less than n.

[0114] Figure 4 This is a schematic diagram of a terminal provided in an embodiment of the present invention. Figure 4 As shown, the terminal 4 in this embodiment includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, it implements the steps in the above-described methods for determining similar video segments, for example... Figure 1 Steps 101 to 103 are shown. Alternatively, when processor 40 executes computer program 42, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of modules / units 31 to 33 shown.

[0115] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0116] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0117] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0118] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for determining similar video clips, characterized in that, include: Calculating the video similarity between the first video and the second video includes: determining the similarity matrix between the first video and the second video; and determining the video similarity using a second preset algorithm and the similarity matrix. If the video similarity satisfies a preset video similarity, the first preset algorithm, i.e., the greedy algorithm, is used to determine the similar images corresponding to the first video and the second video. The similar images are determined by calculating the image similarity between the frame images in the first video and the frame images in the second video. The image similarity is determined by the image features corresponding to the frame images in the first video and the frame images in the second video extracted by the contrastive learning model. Based on the similar images, similar video segments corresponding to the first video and the second video are determined; Wherein, if the video similarity satisfies a preset video similarity, determining the similar images corresponding to the first video and the second video using a first preset algorithm includes: If the video similarity satisfies the preset video similarity, one frame image is selected each time, starting from the first frame image in the m-frame images; Search for a frame image in n frames that has an image similarity greater than or equal to a preset image similarity with the first frame image, until no frame image with an image similarity greater than or equal to the preset image similarity can be found in the n frames; The frame image with a similarity greater than or equal to a preset image similarity is taken as the similar image; If a frame image with an image similarity greater than or equal to a preset image similarity is searched among the n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes: If there is an i-th frame image in the n-frame images whose image similarity to the first frame image in the m-frame images is greater than or equal to a preset image similarity, then continue to select the second frame image in the m-frame images and traverse the frame images after the i-th frame image in the n-frame images, where i is greater than 0 and less than n; Until there is no frame image in the i-th frame that has an image similarity greater than or equal to the preset image similarity with the second frame, where j is greater than i and less than n; If a frame image with an image similarity greater than or equal to a preset image similarity is searched among the n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes: If there is an i-th frame image in the n-frame images whose image similarity to the first frame image in the m-frame images is greater than or equal to a preset image similarity, then continue to select the second frame image in the m-frame images and traverse the frame images after the i-th frame image in the n-frame images, where i is greater than 0 and less than n; If there is a frame image j in the frame images following the i-th frame image that has an image similarity greater than or equal to a preset image similarity with the second frame image, continue to select the third frame image and traverse the frame images following the j-th frame image in the n-th frame images until there is no frame image k in the frame images following the j-th frame image that has an image similarity greater than or equal to a preset image similarity with the third frame image, where j is greater than i and less than n, and k is greater than j and less than n.

2. The method for determining similar video segments as described in claim 1, characterized in that, Determining the similarity matrix between the first video and the second video includes: Extract m frames from the first video and n frames from the second video respectively; The similarity matrix is ​​formed by performing an inner product operation on each frame of the m-frame images and each frame of the n-frame images.

3. The method for determining similar video segments as described in claim 2, characterized in that, The step of determining the video similarity using the second preset algorithm and the similarity matrix includes: Extract the maximum value of each column vector and the maximum value of each row vector in the similarity matrix; The sum of the maximum values ​​of each column vector in the column vector is divided by the number of column vectors to obtain a first quotient value, which is then used as the maximum value of the similarity matrix in the column direction. The result of summing the maximum value of each row vector in the row vector is divided by the number of row vectors to obtain a second quotient value, and the second quotient value is used as the maximum value of the similarity matrix in the row direction; The maximum value between the maximum value in the listed directions and the maximum value in the row direction is selected as the video similarity.

4. A device for determining similar video clips, characterized in that, include: A video similarity calculation module is used to calculate the video similarity between a first video and a second video, including: determining a similarity matrix between the first video and the second video; and determining the video similarity using a second preset algorithm and the similarity matrix. A similar image determination module is used to determine similar images corresponding to the first video and the second video using a first preset algorithm, i.e., a greedy algorithm, if the video similarity satisfies a preset video similarity. The similar images are determined by calculating the image similarity between frame images in the first video and frame images in the second video. The image similarity is determined by the image features corresponding to frame images in the first video and frame images in the second video extracted by a contrastive learning model. A similar video clip determination module is used to determine similar video clips corresponding to the first video and the second video based on the similar images; Wherein, if the video similarity satisfies a preset video similarity, determining the similar images corresponding to the first video and the second video using a first preset algorithm includes: If the video similarity satisfies the preset video similarity, one frame image is selected each time, starting from the first frame image in the m-frame images; Search for a frame image in n frames that has an image similarity greater than or equal to a preset image similarity with the first frame image, until no frame image with an image similarity greater than or equal to the preset image similarity can be found in the n frames; The frame image with a similarity greater than or equal to a preset image similarity is taken as the similar image; If a frame image with an image similarity greater than or equal to a preset image similarity is searched among the n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes: If there is an i-th frame image in the n-frame images whose image similarity to the first frame image in the m-frame images is greater than or equal to a preset image similarity, then continue to select the second frame image in the m-frame images and traverse the frame images after the i-th frame image in the n-frame images, where i is greater than 0 and less than n; Until there is no frame image in the i-th frame that has an image similarity greater than or equal to the preset image similarity with the second frame, where j is greater than i and less than n; If a frame image with an image similarity greater than or equal to a preset image similarity is searched among the n frames, until no frame image with an image similarity greater than or equal to the preset image similarity is found among the n frames, this includes: If there is an i-th frame image in the n-frame images whose image similarity to the first frame image in the m-frame images is greater than or equal to a preset image similarity, then continue to select the second frame image in the m-frame images and traverse the frame images after the i-th frame image in the n-frame images, where i is greater than 0 and less than n; If there is a frame image j in the frame images following the i-th frame image that has an image similarity greater than or equal to a preset image similarity with the second frame image, continue to select the third frame image and traverse the frame images following the j-th frame image in the n-th frame images until there is no frame image k in the frame images following the j-th frame image that has an image similarity greater than or equal to a preset image similarity with the third frame image, where j is greater than i and less than n, and k is greater than j and less than n.

5. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for determining similar video segments as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for determining similar video segments as described in any one of claims 1 to 3.

Images