Video tag mining method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and storage medium for mining video tags, wherein the tags currently tagged with the video are obtained, and for each tag currently tagged with the video, similar tags of the tags are screened out respectively, and the confidence of the similar tags of the tags is calculated, and the confidence of the similar tags of each tag currently tagged with the video is used to screen out similar tags with high confidence as alternative tagged tags for the video. Using the method of the present application, similar tags close to existing tags can be obtained based on the similarity between tags on the basis of manually tagged tags, and then these similar tags are screened to obtain similar tags with high confidence as alternative tagged tags for the video, which are used to expand the tags of the video. The method, device, electronic device and storage medium for mining video tags provided by the present application can also be applied to computer vision technology.

Description

Technical Field

[0001] The present application relates to the field of data mining technology, and in particular to a method, device, electronic device and storage medium for mining video tags. Background Art

[0002] Tags are often used to describe or categorize products. For example, video tags describe the semantics of each video. Each tag is a word or phrase, and a video can usually be described with one or more tags. In video applications, video tags become an important feature to characterize video content, helping users better understand and recommend videos. Therefore, to provide users with a better user experience, videos need to be labeled based on their content.

[0003] In the prior art, video labels are usually manually annotated by specialized staff. The staff interprets the video based on the video content they see, and then selects the corresponding labels to annotate the video. However, different staff members interpret the video content from different perspectives and may mark different labels. For example, for a food teaching video, staff member A may label the video "Cantonese cuisine" from the perspective of cuisine; staff member B may label the video "braised dishes" from the perspective of cooking methods. Therefore, if you rely solely on manual labeling of videos, there is a high probability that the labels annotated on the video will not fully cover the video content, resulting in the problem of missing labels in the video, affecting the user experience. Summary of the Invention

[0004] In view of this, the present application provides a method, device, electronic device and storage medium for mining video tags to solve the problem in the prior art that the tags marked on the video do not fully cover the video content, resulting in missing tags in the video.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] The first aspect of the present application discloses a method for mining video tags, comprising:

[0007] Get the label currently marked on the video;

[0008] For each tag currently tagged with the video, similar tags of the tag are screened out respectively, and the confidence of the similar tags of the tag is calculated; wherein the similar tags of the tag have a high similarity to the tag; the confidence of the similar tags of the tag is used to indicate the probability that the video is tagged with the similar tags of the tag;

[0009] By using the confidence of similar tags of each tag currently tagged with the video, similar tags with high confidence are screened out from the similar tags of each tag currently tagged with the video as candidate tags for the video.

[0010] Optionally, the above method, for each tag currently tagged in the video, filters out tags similar to the tag, including:

[0011] For each tag currently tagged with the video, the similarity between the tag and the target tag is calculated using a video set of the tag and a video set of a target tag; wherein the video set of the tag includes videos tagged with the tag; the video set of the target tag includes videos tagged with the target tag; and the target tag belongs to a tag whose video set and the video set of the tag have a non-empty intersection;

[0012] For each tag currently tagged with the video, the first K target tags in the target tag sequence sorted by similarity are selected as similar tags of the tag; K is a preset positive integer.

[0013] Optionally, the above method, using the confidence of similar tags of each tag currently tagged with the video, and screening out similar tags with high confidence as candidate tags for the video from similar tags of each tag currently tagged with the video, includes:

[0014] For each label currently tagged with the video, the first N similar labels in a similar label sequence sorted by the confidence level of the similar labels are selected as candidate labeling labels for the video; N is a preset positive integer.

[0015] Optionally, the above method further includes:

[0016] Determining whether the confidence level of each candidate label for the video is greater than a preset value;

[0017] For candidate labeling tags whose confidence is greater than a preset value, the display probability of the candidate labeling tags is calculated using the confidence of the candidate labeling tags;

[0018] For an alternative labeling tag whose confidence is not greater than a preset value, the display probability of the alternative labeling tag is set to zero; wherein the display probability of the alternative labeling tag is used to illustrate the probability of the alternative labeling tag being added to the labeling tag of the video.

[0019] The second aspect of the present application discloses a video tag mining device, comprising:

[0020] An acquisition unit, used to obtain the label currently marked on the video;

[0021] A first screening unit is configured to screen out similar tags of each tag currently tagged with the video, and calculate the confidence of the similar tags of the tag; wherein the similar tags of the tag have a high similarity to the tag; and the confidence of the similar tags of the tag is used to indicate the probability that the video is tagged with the similar tags of the tag;

[0022] The second screening unit is used to use the confidence of the similar tags of each tag currently marked on the video to screen out the similar tags with high confidence from the similar tags of each tag currently marked on the video as the alternative marking tags for the video.

[0023] Optionally, in the above device, the first screening unit includes:

[0024] a calculation subunit, configured to calculate, for each tag currently tagged on the video, a similarity between the tag and the target tag using a video set of the tag and a video set of a target tag; wherein the video set of the tag includes videos tagged with the tag; the video set of the target tag includes videos tagged with the target tag; and the target tag belongs to a tag whose video set and the video set of the tag have a non-empty intersection;

[0025] The first screening subunit is used to select, for each tag currently tagged with the video, the first K target tags in the target tag sequence sorted by similarity as similar tags of the tag; K is a preset positive integer.

[0026] Optionally, in the above device, the second screening unit includes:

[0027] The second screening subunit is used to select, for each label currently tagged with the video, the first N similar labels in a similar label sequence sorted by the confidence level of the similar labels as candidate labeling labels for the video; N is a preset positive integer.

[0028] Optionally, the above device further includes:

[0029] A judging unit, configured to judge whether a confidence level of each candidate annotation label of the video is greater than a preset value;

[0030] a calculation unit, configured to calculate, for each candidate labeling tag having a confidence level greater than a preset value, a display probability of the candidate labeling tag using the confidence level of the candidate labeling tag;

[0031] The setting unit is configured to set the display probability of an alternative labeling label whose confidence is not greater than a preset value to zero; wherein the display probability of the alternative labeling label is used to illustrate the probability of the alternative labeling label being added to the labeling label of the video.

[0032] A third aspect of the present application discloses an electronic device, comprising:

[0033] one or more processors;

[0034] a storage device having one or more programs stored thereon;

[0035] When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of the first and second aspects of the present application.

[0036] A fourth aspect of the present application discloses a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of the first and second aspects of the present application is implemented.

[0037] As can be seen from the above technical solution, in a method for mining video tags provided by the present application, the tags currently tagged with the video are first obtained, and then, for each tag currently tagged with the video, similar tags of the tags are screened out respectively, and the confidence of the similar tags of the tags is calculated. Among them, the similar tags of the tags have a high similarity with the tags. The confidence of the similar tags of the tags is used to illustrate the probability that the video is tagged with the similar tags of the tags. Finally, using the confidence of the similar tags of each tag currently tagged with the video, similar tags with high confidence are screened out from the similar tags of each tag currently tagged with the video, as candidate tagged tags for the video. It can be seen that, using the method of the present application, similar tags close to the existing tags can be obtained based on the similarity between the tags on the basis of manually tagged tags, and then these similar tags are screened to obtain similar tags with high confidence as candidate tagged tags for the video, which are used to expand the tags of the video. This solves the problem in the prior art that the tags tagged with the videos do not fully cover the video content, resulting in missing tags for the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0039] Figure 1a A schematic diagram of an execution module of a video tag mining method disclosed in an embodiment of the present application;

[0040] Figure 1b A diagram illustrating an application scenario of the video tag mining method disclosed in an embodiment of the present application;

[0041] Figure 1c This is a flowchart of a video tag mining method disclosed in an embodiment of the present application;

[0042] Figure 2 This is a flowchart of an implementation method of filtering out similar tags for each tag currently tagged on the video in step S102 disclosed in another embodiment of the present application;

[0043] Figure 3 This is a flowchart of another video tag mining method disclosed in another embodiment of the present application;

[0044] Figure 4 A schematic diagram of a video tag mining device disclosed in another embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of an electronic device disclosed in another embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0048] Furthermore, in this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any actual relationship or order between these entities or operations.

[0049] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0050] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0051] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in target identification, tracking, and measurement. Further image processing is performed to transform the computer-generated images into images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0052] The solutions provided in the embodiments of this application involve computer vision technology based on artificial intelligence, which is specifically illustrated by the following embodiments.

[0053] The embodiments of the present application provide a method, device, electronic device and storage medium for mining video tags to solve the problem in the prior art that the tags marked on the video do not fully cover the video content, resulting in the video missing tags. In addition, the method, device, electronic device and storage medium for mining video tags disclosed in the embodiments of the present application can also be applied to a data sharing system. The data sharing system refers to a system for sharing data between nodes. The data sharing system may include multiple nodes. The multiple nodes may refer to various clients in the data sharing system. Each node in the data sharing system stores an identical blockchain. Each node can receive input information when performing normal work and maintain the shared data in the data sharing system based on the received input information. The method for mining video tags can be executed by the nodes in the data sharing system. After the node executes the method for mining video tags, the execution result can be stored in the blockchain to realize data sharing between nodes.

[0054] First of all, it should be noted that Figure 1a The collaborative filtering module 101 and the post-processing module 102 jointly perform the video tag mining method provided in the embodiment of the present application. Specifically, the collaborative filtering module 101 receives a set of video and tag annotations, predicts the tags that are missed for each video, and then the post-processing module 102 calculates the probability of each tag being added.

[0055] The video tag mining method provided in the embodiment of the present application can be applied to multiple projects including video information flow recommendation and video retrieval, and can mine the missing tags of the video to improve the user experience. Figure 1b As shown, each video has multiple tags, such as "fishing" and "wild fishing." After watching the current content, users can click on the corresponding tag to return to more similar content. Video tags may be missing. By mining missing tags, the tags can enrich the description of the video content, providing users with more choices and allowing them to select tags based on their interests, improving the user experience.

[0056] In video retrieval, for cross-modal searches, for example, where the search content is text descriptions but the search results are videos, a common search strategy is to use video tags. For example, when searching for "wild fishing," videos tagged with "wild fishing" will be returned to the user. By discovering missing tags in videos, the recall range of videos can be improved, the coverage of search results can be enhanced, and the user experience can be improved.

[0057] The embodiment of the present application provides a method for mining video tags, such as Figure 1c As shown, specifically including:

[0058] S101: Obtain the label currently marked on the video.

[0059] The video and tag annotation set includes multiple videos and the tags currently tagged for each video. When collaborative filtering module 101 receives the video and tag annotation set and predicts the missing tags for each video, the video and tag annotation set is input into collaborative filtering module 101. Collaborative filtering module 101 first obtains the currently tagged tags for each video and then completes the prediction of the missing tags for the video using the content of subsequent steps. This embodiment uses a video as an example to illustrate video tag mining.

[0060] It's important to note that to help users better understand and recommend videos, videos are typically annotated with tags corresponding to their content. Tags can be words or phrases. While some videos have already been annotated with tags, due to limitations in existing technology, these tags are often incomplete, with some tags that match the video's content omitted. Therefore, to expand the video's potential missing tags, we first retrieve the video's currently annotated tags, i.

[0061] S102: For each tag currently tagged in the video, similar tags of the tag are screened out respectively, and the confidence of the similar tags of the tag is calculated.

[0062] Among them, for each tag currently marked with the video, multiple similar tags can be screened out, and each similar tag of the tag has a high similarity with the tag; the confidence of the similar tag of the tag is used to illustrate the probability that the video is marked with a similar tag of the tag.

[0063] It should be noted that after obtaining the label that the video is currently tagged with, similar labels of the label are screened out for each label that the video is currently tagged with, wherein similar labels of a label are labels that have a high similarity with the label. If the similarity between two labels is high, then these two labels may be used to label the same video at the same time. Therefore, after filtering out the similar labels corresponding to each label, the confidence of each similar label is calculated separately, wherein the confidence of the similar label is used to illustrate the probability that the video is tagged with a similar label of the label, that is, the higher the confidence of the similar label, the greater the probability that the similar label can be used to label the corresponding video.

[0064] Optionally, in another embodiment of the present application, in step S102, for each tag currently tagged in the video, similar tags are screened out respectively, such as Figure 2 As shown, this may include:

[0065] S201. For each tag currently tagged with the video, use the video set of the tag and the video set of the target tag to calculate the similarity between the tag and the target tag; wherein, the video set of the tag includes videos tagged with the tag; the video set of the target tag includes videos tagged with the target tag; the target tag belongs to a tag whose video set and the video set of the tag intersect and are not empty.

[0066] It should be noted that in order to calculate the similarity between tags, we can first define a tag vector for the tag, for example, the tag vector of tag i is W i , the dimension d of the label vector is equal to the number of all videos obtained, that is, the number of all videos in the annotated set of videos and labels received by the collaborative filtering module 101. The value of the label vector is related to the relationship between the label and the video. For example, when video u is annotated with label i, the label vector W of label i is i W in iu =1; when video u is not labeled with label i, the label vector W of label i i W in iu = 0. Since the number of labels annotated for each video is limited, the label vector of a label is also extremely sparse.

[0067] Define the label similarity matrix S, whose dimension is d*d. The cosine similarity can be used to calculate the similarity between label i and label i′. The specific formula is as follows:

[0068]

[0069] In the above formula, W i T Refers to W i The transposed matrix.

[0070] Because the label vector for a tag is very high-dimensional and sparse, it's unnecessary to construct the label vector and then calculate the inner product to compute the similarity between tag i and tag i'. We can also transform the above formula using a set of videos with the tag to compute the similarity between tags, where the set of videos with the tag includes the videos tagged with the tag. Let g(i) be the set of videos tagged with tag i.

[0071] g(i)={u|w iu =1}

[0072] At this time, the specific conversion formula for the similarity between labels is as follows:

[0073]

[0074] In the above formula, g(i) represents the video set of videos labeled with label i, and g(i′) represents the video set of videos labeled with label i′.

[0075] After obtaining the tags currently tagged with the video, for each tag currently tagged with the video, the similarity between the tag and the target tag is calculated using the above conversion formula using the video set with the tag and the video set with the target tag. The target tag is a tag whose video set and the video set with the tag do not intersect. Before calculating the similarity between tags, a posting list can be constructed to record the correspondence between tags and videos. The posting list can then be used to find the target tag whose video set intersects with the video set with the tagged tag.

[0076] S202: For each tag currently tagged in the video, select the first K target tags in the target tag sequence sorted by similarity as similar tags; K is a preset positive integer.

[0077] It should be noted that for each label currently marked in the video, after calculating the similarity between each target label and the current label, the target labels are sorted from large to small according to the similarity, and the first K target labels in the target label sequence sorted by similarity are selected as the similar labels of the label, where K is a preset positive integer and can be set according to actual conditions, for example, 5.

[0078] Assuming that the similarity between tag i and tag i′ is high, it means that tag i′ is a similar tag to tag i, and the confidence of tag i′ needs to be calculated. For each similar tag, the confidence can be calculated based on the tag set corresponding to the similar tag and combined with the preset confidence formula. The elements in the tag set corresponding to the similar tag are K similar tags to the tag currently annotated in the video. For example, the confidence of similar tag i′ can be calculated using the following confidence formula:

[0079]

[0080] Among them, y ui ′ is the confidence of similar label i′, w i″u The label vector w representing label i″ i″ The element corresponding to the video u in s i′i″ is the similarity between label i′ and label i″, I i′ is a tag set consisting of similar tags of tag i′, i″ is a tag set I i′ An element in .

[0081] S103 , using the confidence of similar tags of each tag currently tagged in the video, filter out similar tags with high confidence from similar tags of each tag currently tagged in the video, and use them as candidate tags for the video.

[0082] It should be noted that after calculating the confidence of the similar tags corresponding to each tag currently marked in the video, we can filter out similar tags with high confidence priority based on the similarity of each similar tag, that is, select similar tags with higher confidence from all similar tags as alternative tagging tags for the video. After that, we can use these alternative tagging tags of the video to expand the tags of the video to solve the problem of missing tagging tags for the video.

[0083] Optionally, in another embodiment of the present application, an implementation of step S103 may include:

[0084] For each label currently tagged in the video, the first N similar labels in the similar label sequence sorted by the confidence level of the similar labels are selected as candidate labeling labels for the video; N is a preset positive integer.

[0085] It should be noted that after calculating the confidence level of the similar tags corresponding to each tag currently annotated in the video, the corresponding similar tags are sorted from highest to lowest according to their confidence level to construct a similar tag sequence. During the sorting process, if some similar tags are already annotated with the current video, these similar tags are removed and not included in the similar tag sequence. The top N similar tags are then selected from the similar tag sequence as candidate tags for the video. K is a preset positive integer that can be set according to actual conditions, for example, 3.

[0086] In the method for mining video tags provided in the embodiment of the present application, the tags currently tagged with the video are first obtained, and then for each tag currently tagged with the video, similar tags of the tags are screened out respectively, and the confidence of the similar tags of the tags is calculated. Among them, the similar tags of the tags have a high similarity with the tags; the confidence of the similar tags of the tags is used to illustrate the probability that the video is tagged with the similar tags of the tags. Finally, using the confidence of the similar tags of each tag currently tagged with the video, similar tags with high confidence are screened out from the similar tags of each tag currently tagged with the video, as candidate tagged tags for the video. It can be seen that using the method of the present application, based on the manual tagged tags, similar tags close to the existing tags can be obtained according to the similarity between the tags, and then these similar tags are screened to obtain similar tags with high confidence as candidate tagged tags for the video, which are used to expand the tags of the video. This solves the problem in the prior art that the tags tagged with the videos do not fully cover the video content, resulting in missing tags for the video.

[0087] It should also be noted that there are some existing solutions that use models to complete the mining of missing labels in videos, such as graph-based models or low-rank models. However, the algorithms are usually complex and the model training takes a long time. In today's video field, especially in the short video field, the video scale is usually tens of millions or even hundreds of millions, and hot spots are updated quickly and for a short time, so new labels are constantly generated. Therefore, this places high demands on the training speed of the model. Once the training cost is relatively high, resulting in insufficient processing capabilities for new labels and failure to keep up with video hot spots, it will affect the user experience.

[0088] The video tag mining method provided in the embodiments of this application searches for similar tags based on the video tags and supplements the tags that are missing from the video. The solution is simple and feasible, with fast updates and can keep up with video hot spots. For example, processing the annotation results of hundreds of millions of videos can be recalculated from scratch in just a few hours, which is a significant time advantage compared to the model training cycle of existing deep learning methods, which can take days or even weeks.

[0089] In addition, for newly generated videos, as long as the video has been tagged with some tags, similar tags can be recommended based on the tagged tags, which is simple and convenient. In addition, the similarity between tags is generally stable and generally not easy to change, and the requirements for real-time model updates are not high. For newly generated tags, only the tag similarity matrix S needs to be updated, and the similarity matrix does not need to be completely recalculated. It only needs to calculate the similarity between the new tags and the existing tags, and add a row and a column to the similarity matrix S. The update burden of the similarity matrix is very light, usually updated once a day, which enables the solution to effectively monitor real-time video hotspots.

[0090] Optionally, in another embodiment of the application, as Figure 3 As shown, the video tag mining method may include:

[0091] S301: Obtain the label currently marked on the video.

[0092] S302. For each tag currently tagged with the video, similar tags of the tag are screened out respectively, and the confidence of the similar tags of the tag is calculated; wherein, the similar tags of the tag have a high similarity with the tag; the confidence of the similar tags of the tag is used to indicate the probability that the video is tagged with the similar tags of the tag.

[0093] S303: Using the confidence of similar tags of each tag currently tagged in the video, similar tags with high confidence are screened out from the similar tags of each tag currently tagged in the video as candidate tags for the video.

[0094] It should be noted that steps S301 to S303 can refer to the above embodiment and are not described in detail here. Furthermore, after the collaborative filtering module 101 executes steps S301 to S303 to obtain candidate label tags for the video, the candidate label tags are input to the post-processing module 102, which then executes the following steps.

[0095] S304: Determine whether the confidence level of each candidate annotation label of the video is greater than a preset value.

[0096] It should be noted that after the alternative annotation labels are screened out in step S103, in some cases it may not be necessary to add all the alternative annotation labels as labels for video display. In this case, the corresponding display probability of each alternative annotation label can be calculated, and the annotation labels of the video can be added according to the display probability of each alternative label.

[0097] Specifically, the collaborative filtering module 101 obtains candidate labels for the video. Because the collaborative filtering algorithm used by the collaborative filtering module 101 is not 100% reliable, the candidate labels for the video need to be added to the manually annotated labels with a certain probability. For example, in a video information stream recommendation scenario, each time a video's labels are displayed, the manually annotated labels will always be displayed, while the candidate labels obtained by the collaborative filtering module 101 will also be displayed with a certain probability.

[0098] When calculating the display probability of each candidate label, it is first determined whether the confidence of each candidate label of the video is greater than a preset value, and the preset value can be set according to the actual situation.

[0099] S305 : For candidate annotation tags whose confidences are greater than a preset value, calculate the display probability of the candidate annotation tags using the confidences of the candidate annotation tags.

[0100] It should be noted that when the confidence of an alternative label is greater than a preset value, the confidence of the alternative label is substituted into the display probability calculation formula to calculate the display probability of each alternative label with a confidence greater than the preset value. The specific formula is as follows:

[0101]

[0102] Among them, P ui is the display probability of the alternative annotation label, y ui is the confidence of the candidate label, y max is the maximum confidence value of all candidate annotation labels, and τ is the preset value mentioned above.

[0103] After calculating the display probability of the candidate annotation label whose confidence is greater than the preset value, the candidate annotation label is added to the annotation label of the video according to the display probability.

[0104] S306 : For candidate annotation labels whose confidence is not greater than a preset value, set the display probability of the candidate annotation label to zero.

[0105] In steps S305 and S306 , the display probability of the candidate annotation tags is used to illustrate the probability of the candidate annotation tags being added to the annotation tags of the video.

[0106] It should be noted that when the confidence of the candidate annotation label is not greater than the preset value, it means that the confidence of the current candidate annotation label is low, so the display probability of the current candidate annotation label is set to zero, that is, it will not be added to the annotation label of the video.

[0107] The method for mining video tags disclosed in the embodiment of the present application can be understood as adopting a collaborative filtering algorithm, the basic idea of which is that if users have had the same preferences in the past (such as having purchased the same items), then they will have similar preferences in the future. Recommendations are made by finding neighbors, and similar items to items that have already been purchased are recommended to users, or items that are liked by people similar to the user are recommended. The basic assumption of this type of method is that if two items are similar, then it is easy for the same user to generate behavior; if two users are similar, then it is easy for them to generate behavior on the same item. In the video tagging scenario, different videos are analogized to different users, and different tags are analogized to different commodities. The present application creatively transforms the problem of supplementing missing tags for videos into the problem of "recommending other tags to videos".

[0108] Moreover, the collaborative filtering algorithm used in the method for mining video tags disclosed in this application is based on collaborative filtering of video tags, which is actually collaborative filtering based on items. In addition, there are also user-based collaborative filtering algorithms, which essentially make recommendations based on the user's historical purchasing behavior. However, the user-based collaborative filtering algorithm needs to calculate the similarity between videos, and videos usually have tens of millions or even hundreds of millions of scales, so calculating the similarity between videos is very difficult. The item-based collaborative filtering used in this application only needs to calculate the similarity between tags, which usually have only hundreds of thousands of scales, so it is relatively easy to calculate.

[0109] In addition, in addition to using the most basic item-based collaborative filtering algorithm, this application can also use improved algorithms based on item-based collaborative filtering, including SlopeOne, penalizing popular users, normalizing the item similarity matrix to the maximum value to improve recommendation diversity, time context recommendation, etc.

[0110] Another embodiment of the present application also discloses a video tag mining device, such as Figure 4 Shown, including:

[0111] The acquisition unit 401 is used to acquire the label currently marked on the video.

[0112] The first screening unit 402 is used to screen out similar tags of each tag currently tagged with the video, and calculate the confidence of the similar tags of the tag; wherein the similar tags of the tag have a high similarity with the tag; the confidence of the similar tags of the tag is used to indicate the probability that the video is tagged with the similar tags of the tag.

[0113] The second screening unit 403 is used to use the confidence of similar tags of each tag currently tagged in the video to screen out similar tags with high confidence as candidate tags for the video.

[0114] In this embodiment, the specific execution process of the acquisition unit 401, the first screening unit 402 and the second screening unit 403 can be found in the corresponding Figure 1c The content of the method embodiment will not be repeated here.

[0115] In a video tag mining device provided by an embodiment of the present application, an acquisition unit 401 first acquires the tags currently tagged with the video. Then, a first screening unit 402 filters out similar tags for each tag currently tagged with the video and calculates the confidence of the similar tags. A similar tag has a high degree of similarity to the tag; the confidence of the similar tag indicates the probability that the video is tagged with the tag's similar tag. Finally, a second screening unit 403 uses the confidence of the similar tags of each tag currently tagged with the video to filter out similar tags with high confidence levels from the similar tags of each tag currently tagged with the video, and uses these as candidate tags for the video. Thus, using the method of the present application, based on manually tagged tags, similar tags that are close to existing tags can be obtained based on the similarity between tags. These similar tags can then be filtered to obtain similar tags with high confidence levels as candidate tags for the video, which can be used to expand the video's tags. This solves the problem in the prior art where the tags tagged with videos do not fully cover the video content, resulting in missing tags in the video.

[0116] Optionally, in another embodiment of the present application, an implementation of the first screening unit 402 includes:

[0117] The calculation subunit is used to calculate the similarity between the label and the target label for each label currently tagged with the video, using the label's video set and the target label's video set; wherein the label's video set includes videos tagged with the label; the target label's video set includes videos tagged with the target label; the target label belongs to a label whose video set and the label's video set have a non-empty intersection.

[0118] The first screening subunit is used to select the first K target tags in the target tag sequence sorted by similarity as similar tags for each tag currently marked in the video; K is a preset positive integer.

[0119] In this embodiment, the specific execution process of the calculation subunit and the first screening subunit can be found in Figure 2 The corresponding method embodiment content will not be repeated here.

[0120] Optionally, in another embodiment of the present application, an implementation of the second screening unit 403 includes:

[0121] The second screening subunit is used to select the first N similar tags in the similar tag sequence sorted by the confidence level of the similar tags for each tag currently tagged in the video as the candidate tagging tags for the video; N is a preset positive integer.

[0122] In this embodiment, the specific execution process of the second screening sub-unit can be found in the corresponding method embodiment described above, and will not be repeated here.

[0123] Optionally, in another embodiment of the present application, the video tag mining device includes Figure 4 In addition to the units, it can also include:

[0124] The judging unit is configured to judge whether the confidence level of each candidate annotation label of the video is greater than a preset value.

[0125] The calculation unit is configured to calculate, for the candidate labeling tags whose confidences are greater than a preset value, a display probability of the candidate labeling tags using the confidences of the candidate labeling tags.

[0126] The setting unit is used to set the display probability of the alternative labeling label to zero for the alternative labeling label whose confidence is not greater than the preset value; wherein the display probability of the alternative labeling label is used to indicate the probability of the alternative labeling label being added to the labeling label of the video.

[0127] In this embodiment, the specific execution process of the judgment unit, the calculation unit and the setting unit can be found in Figure 3 The corresponding method embodiment content will not be repeated here.

[0128] It should also be noted that the acquisition unit, the first screening unit and the second screening unit belong to Figure 1a The execution unit, judgment unit, calculation unit, and setting unit in the collaborative filtering module 101 belong to Figure 1a The execution unit in the post-processing module 102.

[0129] Another embodiment of the present application further provides an electronic device, such as Figure 5 As shown, specifically including:

[0130] One or more processors 501 .

[0131] The storage device 502 stores one or more programs.

[0132] When one or more programs are executed by one or more processors 501 , the one or more processors 501 implement any one of the methods in the above embodiments.

[0133] Another embodiment of the present application further provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, any one of the methods in the above embodiments is implemented.

[0134] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0135] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0136] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for mining video tags, characterized in that: include: Get the label currently marked on the video; For each tag currently tagged with the video, the similarity between the tag and the target tag is calculated based on the number of videos tagged with both the tag and the target tag using the video set of the tag and the video set of the target tag; wherein the video set of the tag includes videos tagged with the tag; the video set of the target tag includes videos tagged with the target tag; and the target tag belongs to a tag whose video set and the video set of the tag have a non-empty intersection; For each tag currently tagged in the video, select the first K target tags in the target tag sequence sorted by similarity as similar tags of the tag; K is a preset positive integer; the similar tags of the tag have a high similarity with the tag; For each similar tag, the confidence of the similar tag is calculated based on the similarity between the similar tag and each tag in the similar tag set, as well as the tag vector of each tag in the similar tag set; the similar tag set is a tag set consisting of similar tags of the similar tag; the dimension of the tag vector is equal to the number of all acquired videos, and the value of the element corresponding to the video in the tag vector of each tag is related to the relationship between the tag and the video; the confidence of the similar tag of the tag is used to indicate the probability that the video is labeled with a similar tag of the tag; By using the confidence of similar tags of each tag currently tagged with the video, similar tags with high confidence are screened out from the similar tags of each tag currently tagged with the video as candidate tags for the video.

2. The method according to claim 1, characterized in that The method of using the confidence of similar tags of each tag currently tagged with the video to filter out similar tags with high confidence as candidate tags for the video includes: For each label currently tagged with the video, the first N similar labels in a similar label sequence sorted by the confidence level of the similar labels are selected as candidate labeling labels for the video; N is a preset positive integer.

3. The method according to claim 1, characterized in that Also includes: Determining whether the confidence level of each candidate label for the video is greater than a preset value; For candidate labeling tags whose confidence is greater than a preset value, the display probability of the candidate labeling tags is calculated using the confidence of the candidate labeling tags; For an alternative labeling tag whose confidence is not greater than a preset value, the display probability of the alternative labeling tag is set to zero; wherein the display probability of the alternative labeling tag is used to illustrate the probability of the alternative labeling tag being added to the labeling tag of the video.

4. A video tag mining device, characterized in that: include: An acquisition unit, used to obtain the label currently marked on the video; A first screening unit is configured to screen out similar tags of each tag currently tagged with the video, and calculate the confidence of the similar tags of the tag; wherein the similar tags of the tag have a high similarity to the tag; and the confidence of the similar tags of the tag is used to indicate the probability that the video is tagged with the similar tags of the tag; A second screening unit is configured to use the confidence of similar tags of each tag currently tagged with the video to screen out similar tags with high confidence as candidate tags for the video from similar tags of each tag currently tagged with the video; The first screening unit comprises: A calculation subunit is configured to calculate, for each tag currently tagged with the video, the similarity between the tag and the target tag based on the number of videos tagged with both the tag and the target tag using a video set of the tag and a video set of the target tag; wherein the video set of the tag includes videos tagged with the tag; the video set of the target tag includes videos tagged with the target tag; and the target tag belongs to a tag whose video set and the video set of the tag have a non-empty intersection; The first screening subunit is configured to select, for each label currently tagged on the video, the first K target labels in the target label sequence sorted by similarity as similar labels of the label; K is a preset positive integer; The first screening unit is further specifically used to: for each similar tag, calculate the confidence of the similar tag based on the similarity between the similar tag and each tag in the similar tag set, and the tag vector of each tag in the similar tag set; the similar tag set is a tag set composed of similar tags of the similar tags; the dimension of the tag vector is equal to the number of all videos obtained, and the value of the element corresponding to the video in the tag vector of each tag is related to the relationship between the tag and the video.

5. The device according to claim 4, characterized in that The second screening unit comprises: The second screening subunit is used to select, for each label currently tagged with the video, the first N similar labels in a similar label sequence sorted by the confidence level of the similar labels as candidate labeling labels for the video; N is a preset positive integer.

6. The device according to claim 4, characterized in that Also includes: A judging unit, configured to judge whether a confidence level of each candidate annotation label of the video is greater than a preset value; a calculation unit, configured to calculate, for each candidate labeling tag having a confidence level greater than a preset value, a display probability of the candidate labeling tag using the confidence level of the candidate labeling tag; The setting unit is configured to set the display probability of an alternative labeling label whose confidence is not greater than a preset value to zero; wherein the display probability of the alternative labeling label is used to illustrate the probability of the alternative labeling label being added to the labeling label of the video.

7. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 3.

8. A computer storage medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images