Video association method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a video association method and apparatus, an electronic device, a storage medium and a program product. The method comprises the steps of obtaining a first role feature in a first video; determining the similarity between the first role feature and a plurality of second role features, wherein the plurality of second role features are role features of a plurality of second videos; determining a plurality of candidate videos of which the similarity is greater than or equal to a first similarity threshold value from the plurality of second videos; determining a plurality of video albums respectively corresponding to the plurality of candidate videos based on a video album database; determining the confidence score of each candidate video based on the weighting of the proportion of the video album corresponding to each candidate video in the plurality of video album and the similarity corresponding to each candidate video; determining at least one third video of which the confidence score is greater than or equal to a confidence threshold from the plurality of candidate videos; and generating an association relationship between the first video and the at least one third video.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video processing, and particularly to a video association method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] Short video creators often intercept segments from long videos, and after various edits, publish them to various platforms, which generates a large amount of video data. For video platforms, how to associate these short videos with the corresponding long videos is a problem that needs to be solved.

[0003] In order to achieve effective long-short video association, it is necessary to analyze the video content by means of technology to determine the source of the video segment. Therefore, the long-short association matching method based on frame matching can solve the association of most edited short videos and long videos, but when dealing with heavily edited videos, due to reasons such as excessive color editing of the filter and deformation of the character's expression, the association often fails.

[0004] Therefore, there is an urgent need for a method that can effectively determine the long video associated with the short video and establish the association relationship between the long and short videos. Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video association method, apparatus, electronic device, storage medium, and program product.

[0006] In a first aspect of an embodiment of the present disclosure, a video association method is provided. The method includes: obtaining a first character feature in a first video, where the first video is obtained by editing at least one video; determining the similarity between the first character feature and a plurality of second character features, where the plurality of second character features are respectively the character features of a plurality of second videos; determining a plurality of candidate videos from the plurality of second videos whose similarity is greater than or equal to a first similarity threshold; based on a video album database, determining a plurality of video albums corresponding to the plurality of candidate videos respectively, where the video album database includes a plurality of video albums and the videos included in each video album; determining the confidence score of each candidate video based on the proportion of the video album corresponding to each candidate video in the plurality of video albums and the weighting of the similarity corresponding to each candidate video; determining at least one third video from the plurality of candidate videos whose confidence score is greater than or equal to a confidence threshold; generating an association relationship between the first video and the at least one third video respectively.

[0007] In some embodiments of the present disclosure, the first character feature includes a plurality of character features; the determining the similarity between the first character feature and a plurality of second character features includes: determining the similarity between each character feature in the plurality of character features and the plurality of second character features respectively.

[0008] In some embodiments of the present disclosure, determining the confidence score of each candidate video based on the weighted ratio of the video album corresponding to each candidate video in the multiple video albums and the similarity corresponding to each candidate video includes: determining the weighted ratio of the video album corresponding to each candidate video in the multiple video albums, the similarity corresponding to each candidate video, and the ratio of each candidate video in the multiple candidate videos as the confidence score of each candidate video.

[0009] In some embodiments of the present disclosure, the multiple candidate videos include the same multiple first candidate videos and at least one second candidate video, and the at least one second candidate video is different; determining the confidence score of each candidate video based on the weighted ratio of the video album corresponding to each candidate video in the multiple video albums and the similarity corresponding to each candidate video includes: determining the weighted ratio of the video album corresponding to the first candidate video in the multiple video albums and the similarities corresponding to the multiple first candidate videos as the confidence score of the multiple first candidate videos; determining the weighted ratio of the video album corresponding to each second candidate video in the multiple video albums and the similarity corresponding to each second candidate video as the confidence score of each second candidate video.

[0010] In some embodiments of the present disclosure, before determining multiple candidate videos with similarities greater than or equal to a first similarity threshold from the multiple second videos, the method further includes: determining whether there are videos with similarities greater than or equal to a second similarity threshold in the multiple second videos, where the second similarity threshold is greater than the first similarity threshold; determining multiple candidate videos with similarities greater than or equal to the first similarity threshold from the multiple second videos includes: in the case where there are no videos with similarities greater than or equal to the second similarity threshold in the multiple second videos, determining the multiple candidate videos with similarities greater than or equal to the first similarity threshold from the multiple second videos.

[0011] In some embodiments of the present disclosure, before determining multiple candidate videos with similarities greater than or equal to a first similarity threshold from the multiple second videos, the method further includes: determining whether there are videos with similarities greater than or equal to a second similarity threshold in the multiple second videos, where the second similarity threshold is greater than the first similarity threshold; in the case where there is at least one fourth video with a similarity greater than or equal to the second similarity threshold in the multiple second videos, generating an association relationship between the first video and the at least one fourth video.

[0012] In a second aspect of the embodiments of the present disclosure, a video association device is provided. The device includes: an acquisition module, configured to acquire a first character feature in a first video, where the first video is obtained by clipping at least one video; a determination module, configured to determine the similarity between the first character feature and multiple second character features, where the multiple second character features are respectively character features of multiple second videos; determine multiple candidate videos from the multiple second videos whose similarity is greater than or equal to a first similarity threshold; based on a video album database, determine multiple video albums corresponding to the multiple candidate videos respectively, where the video album database includes multiple video albums and videos included in each video album; based on the proportion of the video album corresponding to each candidate video in the multiple video albums and the weighting of the similarity corresponding to each candidate video, determine the confidence score of each candidate video; determine at least one third video from the multiple candidate videos whose confidence score is greater than or equal to a confidence threshold; a generation module, configured to generate an association relationship between the first video and the at least one third video.

[0013] In some embodiments of the present disclosure, the first character feature includes multiple character features; the determination module is specifically configured to determine the similarity between each character feature in the multiple character features and multiple second character features respectively.

[0014] In some embodiments of the present disclosure, the determination module is specifically configured to determine the proportion of the video album corresponding to each candidate video in the multiple video albums, the similarity corresponding to each candidate video, and the weighting of the proportion of each candidate video in the multiple candidate videos, and determine the confidence score of each candidate video.

[0015] In some embodiments of the present disclosure, the multiple candidate videos include the same multiple first candidate videos and at least one second candidate video, and the at least one second candidate video is different respectively; the determination module is specifically configured to determine the proportion of the video album corresponding to the first candidate video in the multiple video albums and the weighting of the similarity corresponding to the multiple first candidate videos respectively, and determine the confidence score of the multiple first candidate videos; determine the proportion of the video album corresponding to each second candidate video in the multiple video albums and the weighting of the similarity corresponding to each second candidate video, and determine the confidence score of each second candidate video.

[0016] In some embodiments of the present disclosure, the determining module is further configured to determine whether there is a video in the multiple second videos whose similarity is greater than or equal to a second similarity threshold before determining multiple candidate videos whose similarity is greater than or equal to a first similarity threshold from the multiple second videos, where the second similarity threshold is greater than the first similarity threshold; specifically, the determining module is configured to determine the multiple candidate videos whose similarity is greater than or equal to the first similarity threshold from the multiple second videos when there is no video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold.

[0017] In some embodiments of the present disclosure, the determining module is further configured to determine whether there is a video in the multiple second videos whose similarity is greater than or equal to a second similarity threshold before determining multiple candidate videos whose similarity is greater than or equal to a first similarity threshold from the multiple second videos, where the second similarity threshold is greater than the first similarity threshold; when there is at least one fourth video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, generate an association relationship between the first video and the at least one fourth video.

[0018] In a third aspect of the embodiments of the present disclosure, there is provided an electronic device, which includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the video association method described in the first aspect is implemented.

[0019] In a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the video association method described in the first aspect is implemented.

[0020] In a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, where the computer program product includes a computer program. When the computer program product runs on a processor, the processor is caused to execute the computer program to implement the video association method described in the first aspect.

[0021] In a sixth aspect of the embodiments of the present disclosure, there is provided a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run program instructions to implement the video association method described in the first aspect.

[0022] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art: obtaining a first character feature in a first video, where the first video is obtained by splicing at least one video clip (i.e., a short video); determining the similarity between the first character feature and multiple second character features, where the multiple second character features are respectively the character features of multiple second videos (i.e., long videos); determining multiple candidate videos from the multiple second videos whose similarity is greater than or equal to a first similarity threshold; based on a video album database, determining multiple video albums corresponding to the multiple candidate videos, where the video album database includes multiple video albums and the videos included in each video album; determining the confidence score of each candidate video based on the proportion of the video album corresponding to each candidate video in the multiple video albums and the weighting of the similarity corresponding to each candidate video; determining at least one third video from the multiple candidate videos whose confidence score is greater than or equal to a confidence threshold; generating an association relationship between the first video and the at least one third video. In the embodiments of the present disclosure, after performing similarity matching on the character features of a short video (the first video) and the character features of a long video (the second video) to obtain candidate videos, the album information corresponding to the candidate videos is combined to determine the confidence score of each candidate video, and then the candidate videos whose confidence score is greater than or equal to the confidence threshold are determined as the target long videos (the third videos), so as to filter the candidate videos obtained by character feature matching, screen out the candidate videos without matching, and finally associate the short video with the target long video. The association accuracy between the short video and the long video can be improved, and the association effect between the long video and the short video can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0024] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is one of the flow diagrams of the video association method provided by the embodiments of the present disclosure;

[0026] Figure 2 It is another flow diagram of the video association method provided by the embodiments of the present disclosure;

[0027] Figure 3 It is yet another flow diagram of the video association method provided by the embodiments of the present disclosure;

[0028] Figure 4 Structural block diagram of a video association device provided by an embodiment of the present disclosure;

[0029] Figure 5 Structural block diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0030] In order to more clearly understand the above objects, features, and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0031] Many specific details are set forth in the following description to facilitate a thorough understanding of the present disclosure, but the present disclosure may be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] The terms "first", "second", etc. in the specification and claims of the present disclosure are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such used data may be interchanged under appropriate circumstances so that the embodiments of the present disclosure can be implemented in an order different from those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object may be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally represents an "or" relationship between the associated objects before and after.

[0033] Currently, the association between short videos and long videos mainly relies on the selection made by short video authors during upload or matching by analyzing the short video titles. These methods have many defects. For example, the uploader may wrongly select tags; or deliberately generate videos with titles that do not match the content to attract traffic; or many users are reluctant to cooperate with adding tags when uploading short videos, resulting in a large number of videos that cannot be accurately associated with long videos. In particular, the existing methods are inefficient in processing a large number of edited short videos, especially ineffective for editing methods such as large-area cropping, landscape-to-portrait rotation, and filter processing.

[0034] In order to achieve effective long-short video association, it is necessary to analyze the video content by means of technology to determine the source of video clips. This also applies to the copyright holders of long videos to detect whether there are copyright infringement acts. Therefore, there is a practical need for long-short association based on frame matching. The long-short association matching method based on character matching can solve the situation of associating most edited short videos with long videos, but when dealing with severely edited videos, due to reasons such as excessive color change of the filter-edited frames and deformation of character expressions, the association often fails.

[0035] Among the video resources provided by video-playing applications, there are some video resources clipped from content in the form of videos including characters, such as TV dramas, movies, mini-dramas, episodes of variety shows, or advertising videos, etc., which are called short videos. For example, a clip of a certain TV drama, movie, or variety show, or a video spliced from different clips.

[0036] Among them, the content in the form of videos including characters, such as TV dramas, movies, mini-dramas, episodes of variety shows, or advertising videos, etc., is called long videos.

[0037] In some embodiments of the present disclosure, the video lengths of short videos and long videos are not limited. Generally, the duration of a short video is less than that of a long video. However, in some cases where a short video is clipped from multiple long videos, the duration of the short video may be less than that of the long video, or may be greater than or equal to that of the long video, which is not limited herein.

[0038] The server in the embodiments of the present disclosure may be a single server, a server cluster, or multiple server sets, which is not limited herein. Among them, the server cluster may include one or more types of servers.

[0039] The execution subject of the video association method provided by the embodiments of the present disclosure may be the above-mentioned server, or a functional module and / or functional entity in the server that can implement the video association method, which can be specifically determined according to actual usage requirements and is not limited in the embodiments of the present disclosure.

[0040] Next, with reference to the accompanying drawings, the video association method provided by the embodiments of the present disclosure will be described in detail through specific embodiments and their application scenarios.

[0041] As Figure 1 shown, the embodiments of the present disclosure provide a video association method, which may include the following steps 101 to 107.

[0042] 101. Obtain the first character feature in the first video.

[0043] Among them, the first video is clipped from at least one video. Therefore, the first video can also be called a short video.

[0044] Among them, the character feature may be a human feature, an animal feature, an anime character feature, etc., which is not limited herein.

[0045] Among them, the first character feature may be pre-stored, or may be obtained by extracting the character feature from the first video in real time.

[0046] Among them, extracting the character features of the first video includes: obtaining at least two first video frames included in the first video based on the first frame extraction density; obtaining the first character features based on the at least two first video frames.

[0047] Exemplarily, the process of obtaining the first character features based on at least two first video frames may be: performing human body region position detection on at least two first video frames, and filtering out the first video frames in which no detection frames are recognized in the at least two first video frames to obtain a set of portrait frames. Cropping each first video frame in the set of portrait frames based on the detection frame to obtain a set of portrait blocks. Based on a pre-trained feature extraction model, performing feature extraction on the portrait blocks in the set of portrait blocks to obtain the first character features.

[0048] Exemplarily, obtaining the first character features in the first video includes: obtaining a plurality of character features included in the first video, and the occurrence frequency of each character feature in the first video; filtering out the character features with an occurrence frequency less than the frequency threshold in the plurality of character features to obtain a set of high-frequency features; performing aggregation processing on the set of high-frequency features to obtain the first character features.

[0049] Exemplarily, the first video includes a protagonist and passers-by. In the foregoing feature extraction process, the character features of both the protagonist and the passers-by will be extracted. In this case, by setting the frequency threshold, the character features of the passers-by are filtered to reduce the noise in the obtained set of high-frequency features and improve the data accuracy of the obtained first character features; after obtaining the set of high-frequency features, through the foregoing feature clustering and aggregation processing, the first character features can be obtained.

[0050] Exemplarily, the foregoing frequency threshold may be 50, 100, 150, etc. The user can adaptively adjust the specific value of the frequency threshold based on requirements, and the embodiments of the present application do not limit this.

[0051] Furthermore, the frequency threshold can be adaptively adjusted based on the video duration of each first video to ensure the reliability of the foregoing filtering operation and reduce the probability of the character features of the protagonist being filtered out or the character features of the passers-by being retained.

[0052] 102. Determine the similarity between the first character features and a plurality of second character features respectively.

[0053] Among them, the plurality of second character features are respectively the character features of a plurality of second videos. Therefore, the second video can also be called a long video.

[0054] Among them, the video source of the first video is unknown, and the video source of the second video is known.

[0055] Among them, the second character feature can be pre-stored or obtained by extracting the character feature of the second video in real time.

[0056] Among them, extracting the character feature of the second video includes: obtaining at least two second video frames included in the second video based on the second frame extraction density; obtaining the second character feature based on the at least two second video frames; wherein, the first frame extraction density is greater than the second frame extraction density. For details, reference can be made to the relevant description of extracting the character feature of the first video by the business capital, which will not be elaborated here.

[0057] It can be understood that in the case where the first video (i.e., the short video) is highly edited, the image jitter amplitude of the first video is large (compared with the second video). Therefore, it is necessary to increase the first frame extraction density corresponding to the first video to adapt to the large image jitter amplitude, reduce the probability of blurred images in the first video frames, thereby facilitating the subsequent image processing process of the first video frames and improving the accuracy of the obtained first character features.

[0058] For example, the above first frame extraction density can be three frames per second, and the above second frame extraction density can be one frame per second. The user can also adaptively adjust the above first frame extraction density and second frame extraction density according to actual needs, which is not limited here.

[0059] Among them, to obtain the similarity between the first character feature and the second character feature, the similarity score between the first character feature and the second character feature can be directly calculated, or the feature difference score between the first character feature and the second character feature (such as the Euclidean distance between the first character feature and the second character feature) can be calculated first, and then the difference between 1 and the feature difference score is determined as the similarity; it can be determined according to the actual situation specifically, which is not limited here.

[0060] 103. Determine multiple candidate videos from the multiple second videos whose similarity is greater than or equal to the first similarity threshold.

[0061] Among them, the first similarity threshold can be determined according to the actual situation, which is not limited here.

[0062] Among them, the multiple second videos can include the same video, and the multiple second videos can respectively correspond to different second character features. Furthermore, the second videos whose similarity is greater than or equal to the first similarity threshold can be the same or different, that is, the multiple candidate videos can include the same video or different videos.

[0063] 104. Based on the video album database, determine multiple video albums corresponding to the multiple candidate videos respectively.

[0064] Among them, the video album database includes multiple video albums and the videos included in each video album. Each video album can correspond to one or more videos. Different video albums correspond to different videos. For example, a video album can be a collection of videos, and the collection of videos includes one or more videos.

[0065] The video album can be a TV drama, a movie, or a variety show, and the video album can include at least one episode. For a TV drama, the identifier of the video album can be the name of the TV drama, and the number of episodes included in the video album is the number of episodes included in the TV drama; for a movie, the identifier of the video album can be the name of the movie, and the video album includes one episode, which is the movie itself; for a variety show, the video album can be the name of the variety show, and the number of episodes included in the video album is the number of episodes of each program included in the variety show.

[0066] 105. Determine the confidence score of each candidate video based on the weighted ratio of the video album corresponding to each candidate video among the multiple video albums and the similarity corresponding to each candidate video.

[0067] In some embodiments of the present disclosure, the weighted ratio of the video album corresponding to each candidate video among the multiple video albums and the similarity corresponding to each candidate video can be determined as the confidence score of each candidate video.

[0068] Among them, the weight of the ratio of the video album corresponding to each candidate video among the multiple video albums and the weight of the similarity corresponding to each candidate video can be determined according to the actual situation and are not limited herein.

[0069] Example 1, there are 4 candidate videos, and the 4 candidate videos are all different. The 4 candidate videos correspond to 4 video albums. If the 4 video albums are all different, the ratio of each video album among the 4 video albums is one-fourth; if two of the 4 video albums, the first video albums, are the same and the others are all different, the ratio of the first video album among the 4 video albums is one-half, and the ratio of the other video albums among the 4 video albums is one-fourth.

[0070] 106. Determine at least one third video from the multiple candidate videos whose confidence score is greater than or equal to the confidence threshold.

[0071] Among them, the confidence threshold can be determined according to the actual situation and is not limited herein.

[0072] 107. Generate the association relationship between the first video and the at least one third video.

[0073] Exemplarily, a video association database can be set up on the server. This video association database is used to store the identification information corresponding to the short videos and the identification information corresponding to the long videos that have an association relationship. After determining the long video (the third video) associated with the short video (the first video), the identification information corresponding to the long video and the identification information corresponding to the short video are respectively obtained, and these two pieces of identification information are stored in the video association database.

[0074] Among them, the association relationship can also include the playing timestamp information of some or all segments of the short video in the corresponding long video. The association relationship can also include other information, which is not limited here.

[0075] It can be understood that in the problem of associating short videos and long videos, the traditional method has a relatively high failure rate when dealing with heavily edited short videos. This is mainly because the obvious differences in the pictures between a large number of edited short videos and the original long videos result in a relatively low matching score based on picture similarity, thus unable to accurately associate.

[0076] However, no matter how the short video is edited, the character information in the long video is usually retained. Therefore, by matching the similarity of characters, the short video can be more accurately associated with its source long video. Therefore, in the embodiments of the present disclosure, after obtaining candidate videos by performing similarity matching based on the character features of the short video (the first video) and the character features of the long video (the second video), the confidence score of each candidate video is determined by combining the album information corresponding to the candidate video. Then, the candidate videos with a confidence score greater than or equal to the confidence threshold are determined as the target long video (the third video), realizing the filtering of the candidate videos obtained by character feature matching, screening out the unmatched candidate videos, and finally associating the short video with the target long video. This can improve the association accuracy between the short video and the long video and improve the association effect between the long video and the short video.

[0077] In the embodiments of the present disclosure, for short videos of unknown origin existing in a video platform, by applying the video association method provided in the embodiments of the present application, the mutual association between long videos and short videos in the video platform can be completed conveniently and accurately. On the one hand, this enables the video platform to complete the label setting of the short videos associated with the long videos based on the labels included in the long videos, regularize the label management of the short videos by the video platform, and reduce the adverse effects brought by the chaotic label setting of the short videos (such as the incorrect label setting by the short video publisher or the intentional setting of labels unrelated to the short video content for the purpose of attracting traffic), creating a good video browsing environment for users and enabling users to obtain a better user experience. On the other hand, it also facilitates the video platform and the copyright holders of the long videos to monitor the possible infringement problems of the short videos. That is, by applying the video association method provided in the present application, the short videos of unknown origin uploaded to the video platform can be identified to timely know the long video sources of such short videos of unknown origin, and then the short videos can be marked through the aforementioned label setting method, so that the video platform (based on a preset retrieval program) and the copyright holders of the long videos can (based on a preset interface program for data interaction with the video platform) analyze the possible infringement behaviors of the short videos in the video platform.

[0078] In some embodiments of the present disclosure, in combination with Figure 1 , such as Figure 2 or Figure 3 shown, the first role feature includes multiple role features; step 102 can be specifically implemented by the following step 102a.

[0079] 102a. Determine the similarity between each role feature in the multiple role features and multiple second role features respectively.

[0080] In some embodiments of the present disclosure, multiple similarities can be obtained according to step 102a. The multiple similarities correspond to the multiple second role features one by one. If the multiple second videos correspond to the multiple second role features one by one, then the multiple similarities correspond to the multiple second videos one by one (at this time, the multiple second videos may include the same second video); if the multiple second videos and the multiple second role features are in a one-to-many correspondence relationship, then the multiple similarities and the multiple second videos may include a one-to-one correspondence relationship or a many-to-one relationship. Therefore, the multiple similarities may include the similarities between different role features in the second video and different role features in the first video.

[0081] In the embodiments of the present disclosure, by respectively matching a plurality of character features in the first video with the second character feature in the second video, more candidate videos matching the first video can be obtained, thereby increasing the possibility of associating the second video with the first video, avoiding filtering out the second video actually associated with the first video by only matching one character feature in the first video with the second character feature in the second video, improving the association accuracy between short videos and long videos, and improving the association effect between long videos and short videos.

[0082] In some embodiments of the present disclosure, as Figure 2 shown, based on the above step 102a, the above step 105 can be specifically implemented through the following step 105a.

[0083] 105a. Determine the confidence score of each candidate video by weighting the proportion of the video album corresponding to each candidate video in the plurality of video albums, the similarity corresponding to each candidate video, and the proportion of each candidate video in the plurality of candidate videos.

[0084] Among them, the weights of the proportion of the video album corresponding to each candidate video in the plurality of video albums, the weight of the similarity corresponding to each candidate video, and the weight of the proportion of each candidate video in the plurality of candidate videos can be determined according to the actual situation and are not limited herein.

[0085] Example 2, if there are 4 candidate videos and all 4 candidate videos are different, then the proportion of each candidate video in the 4 candidate videos is one-fourth; if two of the 4 candidate videos are the same (indicating that the similarities between two character features in the first video and two character features in candidate video 1 are both greater than or equal to the first similarity threshold), and the other candidate videos are all different, then the proportion of candidate video 1 in the 4 candidate videos is one-half, and the proportion of the other candidate videos in the 4 candidate videos is one-fourth.

[0086] In some embodiments of the present disclosure, by weighting the proportion of the video album corresponding to each candidate video in the plurality of video albums, the similarity corresponding to each candidate video, and the proportion of each candidate video in the plurality of candidate videos, and determining it as the confidence score of each candidate video, the accuracy of the confidence score of the candidate video can be improved. While ensuring a high accuracy rate of the long videos associated with the short videos determined, the recall rate of the long videos is improved, that is, more potential long video results with an association relationship can be found, and at the same time, the long video results without an association relationship are filtered out, effectively avoiding mis-matching.

[0087] In some embodiments of the present disclosure, the multiple candidate videos include the same multiple first candidate videos and at least one second candidate video, and the at least one second candidate video is different respectively; as Figure 3 shown, based on the above step 102a, the above step 105 can be specifically implemented through the following steps 105b and 105c.

[0088] 105b. Determine the weighted sum of the proportion of the video album corresponding to the first candidate video in the multiple video albums and the similarity corresponding to each of the multiple first candidate videos as the confidence score of the multiple first candidate videos.

[0089] 105c. Determine the weighted sum of the proportion of the video album corresponding to each second candidate video in the multiple video albums and the similarity corresponding to each second candidate video as the confidence score of each second candidate video.

[0090] Example 3, the multiple candidate videos are 4 candidate videos. If there are two first candidate videos among the 4 candidate videos (it is described that the similarity between the two character features in the first video and the two character features in the first candidate video is greater than or equal to the first similarity threshold) and two second candidate videos, and 3 of the 4 video albums corresponding to the 4 candidate videos are the same (corresponding to two first candidate videos and one second candidate video (denoted as second candidate video 1)), and one video album (corresponding to the other second candidate video (denoted as second candidate video 2)), the similarities corresponding to the two first candidate videos are 0.4 and 0.45 respectively, the similarity corresponding to the second candidate video 1 is 0.39, and the similarity corresponding to the second candidate video 2 is 0.42; then the confidence score of the first candidate video is (3 / 4)a + 0.4b + 0.45b, the confidence score of the second candidate video 1 is (3 / 4)a + 0.39b, and the confidence score of the second candidate video 2 is (1 / 4)a + 0.42b, where a and b are the corresponding weights respectively.

[0091] In the embodiments of the present disclosure, calculating the confidence scores of the same candidate videos and different candidate videos among the multiple candidate videos respectively can ensure the accuracy of the confidence score calculation, thereby improving the accuracy of the associated long videos and effectively avoiding mis-matching.

[0092] In some embodiments of the present disclosure, before the above step 103, the video association method provided by the embodiments of the present disclosure may further include the following step 108, and the above step 103 can be specifically implemented through the following step 103a.

[0093] 108. Determine whether there is a video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold.

[0094] Among them, the second similarity threshold is greater than the first similarity threshold.

[0095] 103a. In the case where there is no video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, determine the multiple candidate videos whose similarity is greater than or equal to the first similarity threshold from the multiple second videos.

[0096] In some embodiments of the present disclosure, before the above step 103, the video association method provided by the embodiments of the present disclosure may further include the following steps 108 and 109.

[0097] 108. Determine whether there is a video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold.

[0098] Among them, the second similarity threshold is greater than the first similarity threshold.

[0099] 109. In the case where there is at least one fourth video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, generate an association relationship between the first video and each of the at least one fourth video.

[0100] Among them, the second similarity threshold can be determined according to the actual situation and is not limited here.

[0101] For example, the second similarity threshold can be 0.75, and the first similarity threshold can be 0.5.

[0102] In the embodiments of the present disclosure, first determine whether there is a video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold. In the case where there is no video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, determine the multiple candidate videos whose similarity is greater than or equal to the first similarity threshold from the multiple second videos. In the case where there is at least one fourth video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, generate an association relationship between the first video and each of the at least one fourth video. Since the second similarity threshold is greater than the first similarity threshold, based on the second similarity threshold, at least one fourth video associated with the first video can be accurately determined. When there is no video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold, reducing the similarity threshold may result in no match. Therefore, by combining the video albums of the candidate videos to determine the confidence of the candidate videos, candidate videos with mis-matches can be filtered out, which can improve the matching accuracy and the recall rate.

[0103] In the embodiments of the present disclosure, the dual-threshold matching mechanism designs two matching strategies, namely, a low similarity threshold and a high similarity threshold. First, the high similarity threshold matching is performed. If it is not successful, the matching is relaxed to the low similarity threshold. While ensuring high accuracy, the recall rate is improved, that is, more potential matching results can be found, and inaccurate matches are filtered out, effectively avoiding false matches.

[0104] In the embodiments of the present disclosure, based on video album-assisted filtering, on the basis of low similarity threshold matching, by checking the album information of the matching results, the confidence score of the candidate video is calculated based on the weighted calculation of the proportion and similarity of the video album. By introducing the proportion of the video album to assist in filtering, false matches can be effectively reduced, and the overall matching accuracy can be improved, especially in situations where low similarity threshold matching is prone to introduce errors.

[0105] In the embodiments of the present disclosure, through automated and precise video content recognition and association, the capabilities of the video platform in content management, recommendation, review, and data analysis are improved, enabling it to operate and maintain the platform more intelligently and efficiently.

[0106] Figure 4 The following is a structural block diagram of a video association device shown in the embodiments of the present disclosure. As Figure 4 shown, it includes: an acquisition module 401, configured to acquire the first character feature in the first video, where the first video is obtained by splicing at least one video clip; a determination module 402, configured to determine the similarity between the first character feature and multiple second character features, where the multiple second character features are respectively the character features of multiple second videos; determine multiple candidate videos from the multiple second videos whose similarity is greater than or equal to the first similarity threshold; based on the video album database, determine multiple video albums corresponding to the multiple candidate videos respectively, where the video album database includes multiple video albums and the videos included in each video album; based on the proportion of the video album corresponding to each candidate video in the multiple video albums and the weighting of the similarity corresponding to each candidate video, determine the confidence score of each candidate video; determine at least one third video from the multiple candidate videos whose confidence score is greater than or equal to the confidence threshold; a generation module 403, configured to generate the association relationship between the first video and the at least one third video.

[0107] In some embodiments of the present disclosure, the first character feature includes multiple character features; the determination module 402 is specifically configured to determine the similarity between each character feature in the multiple character features and the multiple second character features.

[0108] In some embodiments of the present disclosure, the determining module 402 is specifically configured to determine the confidence score of each candidate video by weighting the proportion of the video album corresponding to each candidate video in the multiple video albums, the similarity corresponding to each candidate video, and the proportion of each candidate video in the multiple candidate videos.

[0109] In some embodiments of the present disclosure, the multiple candidate videos include the same multiple first candidate videos and at least one second candidate video, and the at least one second candidate video is different respectively; the determining module 402 is specifically configured to determine the confidence score of the multiple first candidate videos by weighting the proportion of the video album corresponding to the first candidate video in the multiple video albums and the similarities corresponding to the multiple first candidate videos respectively; and determine the confidence score of each second candidate video by weighting the proportion of the video album corresponding to each second candidate video in the multiple video albums and the similarity corresponding to each second candidate video.

[0110] In some embodiments of the present disclosure, the determining module 402 is further configured to determine whether there is a video in the multiple second videos whose similarity is greater than or equal to a second similarity threshold before determining the multiple candidate videos whose similarity is greater than or equal to a first similarity threshold from the multiple second videos, and the second similarity threshold is greater than the first similarity threshold; the determining module 402 is specifically configured to determine the multiple candidate videos whose similarity is greater than or equal to the first similarity threshold from the multiple second videos when there is no video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold.

[0111] In some embodiments of the present disclosure, the determining module 402 is further configured to determine whether there is a video in the multiple second videos whose similarity is greater than or equal to a second similarity threshold before determining the multiple candidate videos whose similarity is greater than or equal to a first similarity threshold from the multiple second videos, and the second similarity threshold is greater than the first similarity threshold; and generate an association relationship between the first video and each of the at least one fourth video when there is at least one fourth video in the multiple second videos whose similarity is greater than or equal to the second similarity threshold.

[0112] In the embodiments of the present disclosure, each module can implement the video association method provided in the above method embodiments and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0113] Figure 5 It is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure, which is used to exemplarily illustrate the electronic device for implementing any video association method in the embodiments of the present disclosure, and should not be construed as a specific limitation to the embodiments of the present disclosure.

[0114] As Figure 5As shown, the electronic device 500 may include a processor (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0115] Generally, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. Although the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be implemented or had alternatively.

[0116] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processor 501, the functions defined in any video association method provided by the embodiments of the present disclosure may be executed.

[0117] It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0118] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (for example, a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0119] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist separately without being assembled into the electronic device.

[0120] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: obtain first character features in a first video, the first video being based on at least one video clip; determine the similarities between the first character features and a plurality of second character features, the plurality of second character features being the character features of a plurality of second videos respectively; determine a plurality of candidate videos from the plurality of second videos whose similarities are greater than or equal to a first similarity threshold; based on a video album database, determine a plurality of video albums corresponding to the plurality of candidate videos respectively, the video album database including a plurality of video albums and the videos included in each video album; determine the confidence score of each candidate video based on the proportion of the video album corresponding to each candidate video in the plurality of video albums and the weighting of the similarity corresponding to each candidate video; determine at least one third video from the plurality of candidate videos whose confidence score is greater than or equal to a confidence threshold; generate an association relationship between the first video and the at least one third video respectively.

[0121] In the embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on a computer, partially on a computer, executed as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0123] The units described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0124] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), Systems on Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0125] In the context of the present disclosure, a computer-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a computer-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0127] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0128] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. A video association method, characterized in that: The method comprises: Obtaining a first character feature in a first video, where the first video is obtained based on at least one video clip; Determine similarities between the first character feature and a plurality of second character features, respectively, where the plurality of second character features are character features of a plurality of second videos; Determine, from the plurality of second videos, a plurality of candidate videos whose similarity is greater than or equal to a first similarity threshold; Determine, based on a video album database, a plurality of video albums corresponding to the plurality of candidate videos, respectively, wherein the video album database includes a plurality of video albums and videos included in each video album; Determine a confidence score for each candidate video based on a proportion of the video album corresponding to each candidate video in the multiple video albums and a weighted similarity with each candidate video; Determine at least one third video having a confidence score greater than or equal to a confidence threshold from the plurality of candidate videos; Generate association relationships between the first video and the at least one third video.

2. The method according to claim 1, characterized in that The first character feature includes a plurality of character features; The determining of the similarity between the first character feature and a plurality of second character features respectively includes: Determine the similarity between each of the plurality of character features and the plurality of second character features.

3. The method according to claim 2, characterized in that The determining of the confidence score of each candidate video based on the proportion of the video album corresponding to each candidate video in the multiple video albums and the weighting of the similarity corresponding to each candidate video includes: The confidence score of each candidate video is determined by weighting the proportion of the video album corresponding to each candidate video in the multiple video albums, the similarity corresponding to each candidate video, and the proportion of each candidate video in the multiple candidate videos.

4. The method according to claim 2, characterized in that: The multiple candidate videos include the same multiple first candidate videos and at least one second candidate video, and the at least one second candidate video is different from each other; The determining of the confidence score of each candidate video based on the proportion of the video album corresponding to each candidate video in the multiple video albums and the weighting of the similarity corresponding to each candidate video includes: Determine the confidence scores of the multiple first candidate videos by weighting the proportion of the video album corresponding to the first candidate video in the multiple video albums and the similarities corresponding to the multiple first candidate videos respectively; The confidence score of each second candidate video is determined by weighting the proportion of the video album corresponding to each second candidate video in the multiple video albums and the similarity corresponding to each second candidate video.

5. The method according to any one of claims 1 to 4, characterized in that Before determining a plurality of candidate videos having a similarity greater than or equal to a first similarity threshold from the plurality of second videos, the method further includes: Determine whether there is a video among the plurality of second videos whose similarity is greater than or equal to a second similarity threshold, where the second similarity threshold is greater than the first similarity threshold; Determining a plurality of candidate videos having a similarity greater than or equal to a first similarity threshold from the plurality of second videos comprises: When there is no video whose similarity is greater than or equal to the second similarity threshold among the plurality of second videos, the plurality of candidate videos whose similarity is greater than or equal to the first similarity threshold are determined from the plurality of second videos.

6. The method according to any one of claims 1 to 4, characterized in that Before determining a plurality of candidate videos having a similarity greater than or equal to a first similarity threshold from the plurality of second videos, the method further includes: Determine whether there is a video among the plurality of second videos whose similarity is greater than or equal to a second similarity threshold, where the second similarity threshold is greater than the first similarity threshold; When there is at least one fourth video whose similarity is greater than or equal to the second similarity threshold among the plurality of second videos, an association relationship between the first video and the at least one fourth video is generated.

7. A video association device, characterized in that: include: An acquisition module, configured to acquire a first character feature in a first video, wherein the first video is obtained based on at least one video clip; A determination module, configured to determine similarities between the first character feature and a plurality of second character features, wherein the plurality of second character features are character features of a plurality of second videos; Determine, from the plurality of second videos, a plurality of candidate videos whose similarity is greater than or equal to a first similarity threshold; Determine, based on a video album database, a plurality of video albums corresponding to the plurality of candidate videos, respectively, wherein the video album database includes a plurality of video albums and videos included in each video album; Determine a confidence score for each candidate video based on a proportion of the video album corresponding to each candidate video in the multiple video albums and a weighted similarity with each candidate video; Determine at least one third video having a confidence score greater than or equal to a confidence threshold from the plurality of candidate videos; A generation module is used to generate an association relationship between the first video and the at least one third video.

8. An electronic device, characterized in that: include: A memory and a processor, the memory is used to store a computer program; the processor is used to execute the video association method according to any one of claims 1 to 6 when calling the computer program.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the video association method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the video association method according to any one of claims 1 to 6 is implemented.

Images