Video tag recognition method, device, equipment and storage medium

Abstract

The present application provides a video tag recognition method, apparatus, device and storage medium, which relates to the field of artificial intelligence technology. In this method, by determining the text association relationship between at least one candidate video tag contained in the candidate video tag set and the video text content in the target video, the interaction characteristics between the target video and each candidate video tag are obtained. Based on the historical video tags corresponding to each historical video uploaded by the target object, the distribution of historical video tags used to characterize the video publishing preference of the target object is obtained. Compared with the solution of identifying video tags based only on video content, the method of tag identification based on the interaction characteristics between video content and each candidate video tag and the video publishing preference of the target object has a wider and more comprehensive feature dimension, thereby effectively improving the accuracy of tag identification, providing more accurate and comprehensive tag information for video recommendation distribution, and thus improving the effect of video distribution.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a video tag recognition method, apparatus, device, and storage medium. Background Art

[0002] With the development of internet technology, various video applications have emerged one after another. Target users can obtain video files from these applications, as well as upload and share them to video platforms through these applications. After a target user uploads a video file to a video platform, the platform needs to identify and record the video tags of the uploaded video file for subsequent use in video distribution processes such as video recommendation and video retrieval.

[0003] Under related technologies, video platforms mainly obtain video tags by analyzing the content of video files. However, the amount of information carried by video files is limited. Therefore, the video tags obtained in this way are less accurate, which in turn affects the effect of subsequent video distribution. Summary of the Invention

[0004] The present application relates to a video tag recognition method, apparatus, device and storage medium for improving the accuracy of identifying video tags, thereby improving the effect of video distribution.

[0005] In one aspect, an embodiment of the present application provides a method for identifying video tags, the method comprising the following steps:

[0006] Obtaining video text content contained in a target video uploaded by a target object, and obtaining a set of candidate video tags, wherein the set of candidate video tags includes at least one candidate video tag, and each candidate video tag is used to represent a video attribute;

[0007] Determining a text association relationship between each of the at least one candidate video tag and the video text content;

[0008] Determining a historical video tag distribution of the target object based on a historical video tag corresponding to each of at least one historical video uploaded by the target object;

[0009] Based on the respective text association relationships and the historical video tag distribution, a target video tag of the target video is determined from the candidate video tag set.

[0010] In one aspect, an embodiment of the present application provides a video tag recognition device, the device comprising:

[0011] an acquisition module, configured to acquire video text content contained in a target video uploaded by a target object, and to acquire a set of candidate video tags, wherein the set of candidate video tags includes at least one candidate video tag, and each candidate video tag is used to represent a video attribute;

[0012] a matching module, configured to respectively determine a text association relationship between the at least one candidate video tag and the video text content;

[0013] A statistics module, configured to determine a distribution of historical video tags of the target object based on a historical video tag corresponding to at least one historical video uploaded by the target object;

[0014] A prediction module is configured to determine a target video tag of the target video from the candidate video tag set based on the respective text association relationships and the historical video tag distribution.

[0015] Optionally, the text association relationship includes a first association relationship, wherein the first association relationship represents a degree of text overlap between a candidate video tag and the video text content;

[0016] The matching module is specifically used for:

[0017] For the at least one candidate video tag, perform the following steps respectively:

[0018] Converting a candidate video tag into at least one tag text of a preset granularity, and converting the video text content into at least one video text of the preset granularity;

[0019] respectively determining the degree of overlap between each of the label texts in the at least one label text and the at least one video text;

[0020] Based on the respective overlap degrees, a first association relationship corresponding to the one candidate video tag is determined.

[0021] Optionally, the text association relationship further includes a second association relationship, wherein the second association relationship represents the text feature similarity between a candidate video tag and the video text content;

[0022] The matching module is further configured to:

[0023] Performing feature extraction on the at least one label text to obtain at least one label text feature, and performing feature extraction on the at least one video text to obtain at least one video text feature;

[0024] For the at least one video text feature, the following steps are respectively performed: determining the feature similarity between each label text feature in the at least one label text feature and a video text feature;

[0025] Based on the similarities of the respective features, a second association relationship corresponding to the candidate video tag is determined.

[0026] Optionally, the historical video tag distribution includes: at least one historical video tag corresponding to the target object, and a target probability that the target object uploads a video associated with the at least one historical video tag;

[0027] The statistics module is specifically used for:

[0028] Obtaining at least one historical video tag corresponding to the target object based on the historical video tag corresponding to each of the at least one historical video uploaded by the target object;

[0029] For the at least one historical video tag, the following steps are respectively performed: based on the historical playback attribute of at least one historical video associated with the historical video tag, a target probability of the target object uploading the video associated with the historical video tag is determined.

[0030] Optionally, the statistics module is specifically used to:

[0031] For the at least one historical video, the following steps are respectively performed: based on the historical playback attributes of the historical video and the number of days the historical video was uploaded, determining the weight of the historical video relative to the historical video label;

[0032] Based on the weight corresponding to each of the at least one historical video, a target probability is determined that the target object uploads a video associated with the historical video tag.

[0033] Optionally, the prediction module is specifically configured to:

[0034] Merging each text association relationship with the historical video tag distribution to obtain joint attributes of the target video;

[0035] Performing feature extraction on the joint attributes of the target video to obtain label interaction features of the target video;

[0036] Based on the label interaction feature of the target video, a target video label of the target video is predicted from the candidate video label set.

[0037] Optionally, the prediction module is specifically configured to:

[0038] Extracting features of the target video text content, audio frame sequence, and image frame sequence to obtain corresponding text features, audio features, and image features;

[0039] Obtaining video features of the target video based on the text features, the audio features, and the image features;

[0040] The tag interaction feature is fused with the video feature to obtain a fused feature, and based on the fused feature, a target video tag of the target video is predicted from the candidate video tag set.

[0041] Optionally, the prediction module is specifically configured to:

[0042] Merging each first association relationship with the historical video tag distribution to obtain a first joint attribute of the target video;

[0043] Each second association relationship is fused with the historical video tag distribution to obtain a second joint attribute of the target video.

[0044] Optionally, the prediction module is specifically configured to:

[0045] Performing feature extraction on the first joint attribute to obtain a first joint attribute feature;

[0046] performing feature extraction on the second joint attribute to obtain a second joint attribute feature;

[0047] The first joint attribute feature and the second joint attribute feature are fused to obtain a label interaction feature of the target video.

[0048] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned video tag recognition method when executing the program.

[0049] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of the above-mentioned video tag identification method.

[0050] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs the steps of the above-mentioned video tag recognition method.

[0051] In an embodiment of the present application, by determining the textual association relationship between at least one candidate video tag included in the candidate video tag set and the video text content in the target video, the interaction features between the video content and each candidate video tag are obtained. Based on the historical video tags corresponding to each historical video uploaded by the target object, the distribution of historical video tags used to characterize the video publishing preferences of the target object is obtained. Since the interaction features between the video content and each candidate video tag can characterize the range of video tags roughly corresponding to the video content, and the target object is likely to upload videos associated with its own video publishing preferences, compared to the solution of identifying video tags based only on video content, when the interaction features between the video content and each candidate video tag are used as a reference and the video tag is identified in conjunction with the video publishing preferences of the target object, the dimension of the features used is wider and the correlation with the video tags actually corresponding to the target video is higher, thereby effectively improving the accuracy and recall rate of tag identification, providing more accurate and comprehensive tag information for later video recommendation distribution, and thus improving the effect of subsequent video distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application;

[0054] Figure 2 A flowchart of a video tag recognition method provided in an embodiment of the present application;

[0055] Figure 3a A schematic diagram of the structure of the first interaction matrix provided in an embodiment of the present application;

[0056] Figure 3b A schematic diagram of the structure of the second interaction matrix provided in an embodiment of the present application;

[0057] Figure 4 A flowchart of a video tag recognition method provided in an embodiment of the present application;

[0058] Figure 5 A flowchart of a method for determining tag interaction features provided in an embodiment of the present application;

[0059] Figure 6 A flowchart of a method for determining tag interaction features provided in an embodiment of the present application;

[0060] Figure 7 A flowchart of a method for determining tag interaction features provided in an embodiment of the present application;

[0061] Figure 8 A flowchart of a video tag recognition method provided in an embodiment of the present application;

[0062] Figure 9 A flowchart of a video tag recognition method provided in an embodiment of the present application;

[0063] Figure 10 A schematic diagram of the structure of a video tag recognition device provided in an embodiment of the present application;

[0064] Figure 11 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] For ease of understanding, the terms involved in the embodiments of the present invention are explained below.

[0067] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0068] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0069] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language people use in daily life, so it has a close connection with the research of linguistics. Natural language processing technologies usually include text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies. For example, in the embodiments of this application, NLP technology is used to identify video tags.

[0070] BERT: (Bidirectional Encoder Representations from Transformers), that is, the Encoder of the bidirectional Transformer, describes character-level, word-level, sentence-level and even inter-sentence relationship features. ALBERT is a lightweight BERT.

[0071] The embodiments of this application will be introduced below.

[0072] After the video creator uploads a video file to the video platform through a video application, the video platform needs to identify and record the video tags of the uploaded video file for subsequent application in video distribution processes such as video recommendation and video retrieval.

[0073] Under the related technology, the video platform mainly obtains video tags by parsing the content of the video file. However, the amount of information carried by the video file is limited. Therefore, the accuracy of the video tags obtained by this method is relatively low, which in turn affects the video distribution effect in subsequent processes such as video recommendation and video retrieval.

[0074] Through analysis, it is found that there is often an obvious interaction between the video content and the candidate video tags and the video tags of the historical videos released by the target object. For example, in the target video released by the video creator, the video title includes the text "The baby is so cute", and the candidate video tag set includes the "cute baby" tag, and there are many videos related to "cute baby" in the historical videos released by the video creator.

[0075] Since the interaction features between the video content and the candidate video tags can often characterize the range of the label distribution to which the video content roughly belongs, and the interaction features between the video content and the historical video tags can capture the video release preferences of the video creator, therefore, when identifying the tags of the uploaded target video by combining the interaction features between the video content and the candidate video tags and the historical video tags released by the video creator, the accuracy of tag identification can be effectively improved, and进而提高视频分发的效果。 It should be noted that the last sentence in the original Chinese seems to be incomplete in English translation. You may want to check and complete it for a more accurate translation. The part "进而提高视频分发的效果" is translated as "and then improve the video distribution effect" in a more complete sense.

[0076] In view of this, an embodiment of the present application provides a video tag identification method, which specifically includes the following steps: obtaining the video text content contained in the target video uploaded by the target object, and obtaining a candidate video tag set, wherein the candidate video tag set contains at least one candidate video tag, and each candidate video tag is used to characterize a video attribute. Determine the text association relationship between at least one candidate video tag and the video text content respectively, and then determine the historical video tag distribution of the target object based on the historical video tags corresponding to at least one historical video uploaded by the target object. Finally, based on each text association relationship and the historical video tag distribution, determine the target video tag from the candidate video tag set.

[0077] In an embodiment of the present application, by determining the textual association relationship between at least one candidate video tag included in the candidate video tag set and the video text content in the target video, the interaction features between the video content and each candidate video tag are obtained. Based on the historical video tags corresponding to each historical video uploaded by the target object, the distribution of historical video tags used to characterize the video publishing preferences of the target object is obtained. Since the interaction features between the video content and each candidate video tag can characterize the range of video tags roughly corresponding to the video content, and the target object is likely to upload videos associated with its own video publishing preferences, compared to the solution of identifying video tags based only on video content, when the interaction features between the video content and each candidate video tag are used as a reference and the video tag is identified in conjunction with the video publishing preferences of the target object, the dimension of the features used is wider and the correlation with the video tags actually corresponding to the target video is higher, thereby effectively improving the accuracy and recall rate of tag identification, providing more accurate and comprehensive tag information for later video recommendation distribution, and thus improving the effect of subsequent video distribution.

[0078] refer to Figure 1 , which is a system architecture diagram applicable to an embodiment of the present application. The system architecture includes at least a terminal device 101 and a server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.

[0079] Video applications are pre-installed in the terminal device 101. Video applications include video playback applications, short video applications, live broadcast applications, etc. Video application types include client applications, web applications, mini-program applications, etc. The terminal device 101 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart TV, smart car device, etc., but is not limited to these.

[0080] Server 102 is the background server of the video application. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0081] The video tag recognition method in the embodiment of the present application can be executed by the terminal device 101, can be executed by the server 102, or can be executed interactively by the terminal device 101 and the server 102.

[0082] For example, the server 102 executes the video tag identification method in the embodiment of the present application, which includes the following steps:

[0083] In response to the target object uploading the target video, the terminal device 101 sends the target video to the server 102. The server 102 obtains the video text content contained in the target video uploaded by the target object, and obtains a set of candidate video tags, wherein the candidate video tag set contains at least one candidate video tag, and each candidate video tag is used to represent a video attribute. The text association relationship between at least one candidate video tag and the video text content is determined respectively, and then the historical video tag distribution of the target object is determined based on the historical video tag corresponding to at least one historical video uploaded by the target object. Finally, based on each text association relationship and the historical video tag distribution, the target video tag of the target video is determined from the candidate video tag set.

[0084] based on Figure 1 As shown in the system architecture diagram, the embodiment of the present application provides a process of a video tag recognition method, such as Figure 2 As shown, the process of the method is executed by a computer device, which may be Figure 1 The terminal device 101 and / or server 102 shown includes the following steps:

[0085] Step S201: obtaining the video text content contained in the target video uploaded by the target object, and obtaining a set of candidate video tags.

[0086] Specifically, the target object can be a user account, device ID, application ID, etc., and the target video contains one or more of the following video text content: video title, subtitles, dialogue, barrage, commentary text, etc.

[0087] The video text content contained in the target video can be obtained through voice recognition or image recognition. If the target video file contains text subfiles such as subtitles, the video text content contained in the target video can be directly obtained from the text subfiles.

[0088] The candidate video tag set is a preset tag set, wherein the candidate video tag set includes at least one candidate video tag, each candidate video tag is used to represent a video attribute, and the candidate video tag set is updated in real time or periodically.

[0089] Step S202: determining a text association relationship between at least one candidate video tag and the video text content.

[0090] Specifically, the text association relationship includes a first association relationship and / or a second association relationship, wherein the first association relationship represents the degree of text overlap between a candidate video tag and the video text content, that is, the interaction characteristics of the candidate video tag and the video text content in plain text. The second association relationship represents the text feature similarity between a candidate video tag and the text content, that is, the implicit interaction characteristics of the candidate video tag and the video text content after deep representation. The text association relationship between each candidate video tag and the video text content is determined separately.

[0091] Step S203: determining the historical video tag distribution of the target object based on the historical video tags corresponding to at least one historical video uploaded by the target object.

[0092] Specifically, if the video tags of the target subject's previously released videos are concentrated within a certain video tag range, then it can be inferred that the video tags of the target subject's currently uploaded target video are likely to fall within that video tag range. Therefore, before performing video tag prediction, you can first obtain the historical video tags corresponding to all historical videos uploaded by the target subject, then deduplicate each of the obtained historical video tags. Based on the deduplicated historical video tags, you can obtain the target subject's historical video tag distribution. Subsequently, you can combine the target subject's historical video tag distribution to perform video tag recognition.

[0093] Step S204 : determining a target video tag of the target video from the candidate video tag set based on the respective text association relationships and the historical video tag distribution.

[0094] Specifically, the obtained text association relationships are fused with the historical video tag distribution to obtain the joint features of the target video, and then the target video tag of the target video is predicted from the candidate video tag set based on the joint features. The target video tag of the target video can be one or more.

[0095] In an embodiment of the present application, by determining the textual association relationship between at least one candidate video tag included in the candidate video tag set and the video text content in the target video, the interaction features between the video content and each candidate video tag are obtained. Based on the historical video tags corresponding to each historical video uploaded by the target object, the distribution of historical video tags used to characterize the video publishing preferences of the target object is obtained. Since the interaction features between the video content and each candidate video tag can characterize the range of video tags roughly corresponding to the video content, and the target object is likely to upload videos associated with its own video publishing preferences, compared to the solution of identifying video tags based only on video content, when the interaction features between the video content and each candidate video tag are used as a reference and the video tag is identified in conjunction with the video publishing preferences of the target object, the dimension of the features used is wider and the correlation with the video tags actually corresponding to the target video is higher, thereby effectively improving the accuracy and recall rate of tag identification, providing more accurate and comprehensive tag information for later video recommendation distribution, and thus improving the effect of subsequent video distribution.

[0096] Optionally, in the above step S202, the embodiment of the present application provides at least the following implementation methods for determining the text association relationship between the candidate video tags and the video text content:

[0097] Implementation method 1: The text association relationship includes a first association relationship, and the first association relationship represents the degree of text overlap between a candidate video tag and the video text content.

[0098] For at least one candidate video tag, perform the following steps:

[0099] A candidate video tag is converted into at least one tag text of a preset granularity, and a video text content is converted into at least one video text of a preset granularity.

[0100] The preset granularity can be character granularity, word granularity, or other granularity. For example, if the candidate video label is set to "Football World Cup", when the candidate video label is converted into a label text with word granularity, the obtained label text includes: "football" and "World Cup".

[0101] When the candidate video labels are converted into word-granular label texts, the obtained label texts include: "football", "ball", "world", "world", and "cup".

[0102] The method of converting the video text content into at least one video text of a preset granularity is the same as the method of converting the candidate video tag into at least one tag text of a preset granularity, which will not be repeated here.

[0103] Determine the overlap degree between each tag text in at least one tag text and at least one video text respectively, and then determine the first association relationship corresponding to a candidate video tag based on each overlap degree.

[0104] Specifically, for each tag text in at least one tag text, determine the overlap degree between this tag text and each video text in at least one video text. Among them, if the tag text and the video text are the same, the corresponding overlap degree is 1; if the tag text and the video text are different, the corresponding overlap degree is 0.

[0105] Based on each overlap degree, determine a first interaction matrix with M rows and N columns corresponding to a candidate video tag, where M rows correspond to M tag texts in the candidate video tag, N columns correspond to N video texts in the video text content, and M and N are positive integers greater than 1.

[0106] Among them, the element in the m-th row and n-th column of the first interaction matrix represents whether the m-th tag text and the n-th video text are the same, 0 < m ≤ M, 0 < n ≤ N. Use the first interaction matrix corresponding to a candidate video tag to represent the text overlap degree between this candidate video tag and the video text content, that is, the first association relationship.

[0107] For example, set the candidate video tag as "football", and the video title of the target video is "Highlights of a Football Game". When converting the candidate video tag into a character-level tag text, the obtained tag texts include: "foot" and "ball". When converting the video title into a character-level tag text, the obtained video texts include: "foot", "ball", "game", "match", "wonderful", "brilliant", "highlights", "collection".

[0108] Determine the overlap degree between the tag text "foot" and the video texts "foot", "ball", "game", "match", "wonderful", "brilliant", "highlights", "collection" respectively, and the obtained overlap degree set is {1, 0, 0, 0, 0, 0, 0, 0}.

[0109] Determine the overlap degree between the tag text "ball" and the video texts "foot", "ball", "game", "match", "wonderful", "brilliant", "highlights", "collection" respectively, and the obtained overlap degree set is {0, 1, 0, 0, 0, 0, 0, 0}.

[0110] Based on the obtained each overlap degree, determine the first interaction matrix between the candidate video tag "football" and the video title "Highlights of a Football Game", specifically as Figure 3aAs shown in the figure, among them, the elements in the first row and the first column of the first interaction matrix are 1, indicating that the label text "foot" is consistent with the video text "foot"; the elements in the first row and the second column of the first interaction matrix are 0, indicating that the label text "foot" is inconsistent with the video text "ball". The meanings represented by other elements can be referred to the elements in the first row and the first column and the elements in the first row and the second column described above, which will not be elaborated here.

[0111] In the embodiments of the present application, since the higher the degree of overlap between two texts, the higher the correlation between the two texts. Therefore, by determining the text overlap degree between the candidate video label and the video text content, the interaction characteristics of the candidate video label and the video text content in plain text can be effectively captured, providing more-dimensional features for subsequent video label recognition, thereby improving the accuracy of video label recognition.

[0112] Embodiment 2: The text association relationship includes a second association relationship, and the second association relationship characterizes the text feature similarity between a candidate video label and the video text content. <COUNTER>For at least one candidate video label, the following steps are respectively executed:

[0114] Feature extraction is respectively performed on at least one label text to obtain at least one label text feature, and feature extraction is respectively performed on at least one video text to obtain at least one video text feature.

[0115] Specifically, a text feature extraction model is used to perform feature extraction on each label text to obtain a label text feature, and a text feature extraction model is used to perform feature extraction on each video text to obtain a video text feature. Among them, the text feature extraction model can be ALBERT, BERT, Transformer-Encoder, etc. It should be noted that the neural network structures such as the text feature extraction model involved in the embodiments of the present application can be stored on the blockchain, which will not be elaborated later.

[0116] For at least one video text feature, the following steps are respectively executed: respectively determine the feature similarity between each label text feature in at least one label text feature and a video text feature. Based on each feature similarity, determine the second association relationship corresponding to a candidate video label.

[0117] Specifically, the feature similarity between the label text feature and the video text feature can be Euclidean distance, cosine similarity, Manhattan distance, Chebyshev distance, etc.

[0118] Based on the obtained feature similarities for each, determine a second interaction matrix of X rows and Y columns corresponding to a candidate video tag, where the X rows correspond to X tag texts in the candidate video tag, the Y columns correspond to Y video texts in the video text content, and X and Y are positive integers greater than 1.

[0119] The element in the x-th row and y-th column of the second interaction matrix represents the feature similarity between the x-th tag text and the y-th video text, where 0 < x ≤ X and 0 < y ≤ Y. Use the second interaction matrix corresponding to a candidate video tag to characterize the text feature similarity, i.e., the second association relationship, between the candidate video tag and the video text content.

[0120] For example, assume the candidate video tag is "football" and the video title of the target video is "Highlights of a Football Game". When converting the candidate video tag into tag texts at the character level, the obtained tag texts include: "foot" and "ball". When converting the video title into tag texts at the character level, the obtained video texts include: "foot", "ball", "game", "match", "wonderful", "brilliant", "highlights", "collection".

[0121] Use ALBERT to extract features from the tag texts "foot" and "ball" respectively to obtain tag text feature 1 and tag text feature 2. Use ALBERT to extract features from the video texts "foot", "ball", "game", "match", "wonderful", "brilliant", "highlights", "collection" respectively to obtain video text feature 1, video text feature 2, video text feature 3, video text feature 4, video text feature 5, video text feature 6, video text feature 7, video text feature 8.

[0122] Determine the feature similarities between tag text feature 1 and video text features 1 to 8 respectively, and the obtained set of feature similarities is {0.9, 0.1, 0.1, 0.2, 0.2, 0.1, 0.1, 0.1}.

[0123] Determine the feature similarities between tag text feature 2 and video text features 1 to 8 respectively, and the obtained set of feature similarities is {0.1, 0.9, 0.1, 0.1, 0.2, 0.2, 0.1, 0.1}.

[0124] Based on the obtained feature similarities for each, determine the second interaction matrix between the candidate video tag "football" and the video title "Highlights of a Football Game", specifically as Figure 3bAs shown, the element in the first row and the first column of the second interaction matrix is ​​0.9, indicating that the feature similarity between the label text "foot" and the video text "foot" is 0.9; the element in the first row and the second column of the second interaction matrix is ​​0.1, indicating that the feature similarity between the label text "foot" and the video text "ball" is 0.1. The meanings represented by other elements can refer to the elements in the first row and the first column and the elements in the first row and the second column described above, which will not be repeated here.

[0125] In the embodiment of the present application, since the closer the text features corresponding to two texts are, the higher the correlation between the two texts is, therefore, by determining the feature similarity between the candidate video tag and the video text content, the implicit interaction features of the candidate video tag and the video text after deep representation can be effectively captured, providing more dimensional features for subsequent video tag recognition, thereby improving the accuracy of video tag recognition.

[0126] In the third embodiment, the text association relationship includes a first association relationship and a second association relationship, wherein the first association relationship represents the degree of text overlap between a candidate video tag and the video text content, and the second association relationship represents the text feature similarity between a candidate video tag and the video text content.

[0127] For at least one candidate video tag, perform the following steps:

[0128] A candidate video tag is converted into at least one tag text of a preset granularity, and the video text content is converted into at least one video text of a preset granularity. A degree of overlap between each tag text in the at least one tag text and the at least one video text is determined. Based on each degree of overlap, a first association relationship corresponding to the candidate video tag is determined.

[0129] Feature extraction is performed on each of the at least one tag text to obtain at least one tag text feature, and feature extraction is performed on each of the at least one video text to obtain at least one video text feature. For each of the at least one video text features, the following steps are performed: Feature similarity is determined between each of the at least one tag text features and a video text feature. Based on each feature similarity, a second association relationship corresponding to a candidate video tag is determined.

[0130] The above has provided examples for the process of determining the first association relationship and the second association relationship, which will not be repeated here.

[0131] In an embodiment of the present application, by capturing the interaction features between candidate video tags and video text content in plain text, as well as the implicit interaction features between candidate video tags and video text content after deep representation, multi-dimensional interaction features are provided for subsequent video tag recognition, thereby effectively improving the accuracy of tag recognition and further improving the effect of subsequent video distribution.

[0132] Optionally, in the above step S203, the historical video tag distribution includes: at least one historical video tag corresponding to the target object, and a target probability that the target object uploads a video associated with the at least one historical video tag.

[0133] Based on the historical video tags corresponding to the at least one historical video uploaded by the target object, at least one historical video tag corresponding to the target object is obtained.

[0134] Specifically, first determine at least one historical video uploaded by the target object within a preset time period, then obtain the historical video tag corresponding to each historical video based on the identification query of each historical video, deduplicate all the obtained historical video tags, and obtain at least one historical video tag corresponding to the target object.

[0135] The at least one historical video may correspond to the same historical video tag or may correspond to different historical video tags. The historical video tags may be obtained through tag recognition or manually marked.

[0136] For example, user account A uploaded three videos in the past month, namely historical video 1, historical video 2 and historical video 3. Among them, the historical video tags of historical video 1 are "mother and baby" and "cute baby", the historical video tags of historical video 2 are "mother and baby" and "parenting", and the historical video tag of historical video 3 is "pet".

[0137] After deduplication of the historical video labels corresponding to the three historical videos, the historical video labels corresponding to user account A are obtained as "mother and baby", "cute baby", "parenting" and "pet".

[0138] For at least one historical video tag, the following steps are respectively performed: based on the historical playback attribute of at least one historical video associated with the historical video tag, determining a target probability that the target object uploads a video associated with the historical video tag.

[0139] Specifically, the historical playback attributes can be one or more of the following: number of valid playbacks, number of clicks, number of exposures, number of shares, number of collections, etc., where an effective playback indicates that the ratio of the video playback time to the total video time is greater than a preset threshold, where the preset threshold is associated with the total video time.

[0140] In an embodiment of the present application, by determining at least one historical video tag corresponding to the target object and the target probability of the target object uploading a video associated with at least one historical video tag, the target object's video publishing preference can be captured. Since the target object is more likely to upload videos related to its own video publishing preference, when the target object's video publishing preference is applied to video tag recognition, the accuracy of video tag recognition can be effectively improved.

[0141] In one possible implementation, for a historical video tag and a historical video associated with the tag, a weight of the historical video relative to the tag is determined based on the video's historical playback attributes and the number of days since it was uploaded. Then, based on the weight corresponding to at least one historical video, a target probability is determined for the target subject to upload the video associated with the tag.

[0142] Specifically, when the historical playback attribute of a historical video is the number of valid playbacks of the historical video, the weight of the historical video relative to the historical video label is determined based on the historical playback attribute of the historical video and the number of days the historical video was uploaded, specifically satisfying the following formula (1):

[0143]

[0144] Among them, V ki represents the weight of historical video k relative to historical video label i, s represents the number of valid playback times of historical video k, and t represents the number of days since historical video k was uploaded.

[0145] The weights corresponding to at least one historical video associated with the historical video tag i are summed or weighted summed to obtain the total weight value of the historical video tag i, which specifically satisfies the following formula (2):

[0146]

[0147] Among them, Q i represents the total weight value of historical video label i, and j represents the number of historical videos associated with historical video label i.

[0148] After obtaining the total weight value of each historical video tag, the total weight value of each historical video tag is subjected to probability normalization processing to obtain the target probability of the target object uploading the video associated with each historical video tag. The process of performing probability normalization processing on the total weight value of the historical video tag i is specifically shown in the following formula (3):

[0149]

[0150] Among them, W irepresents the target probability of the target object uploading the video associated with the historical video tag i, n represents the number of historical video tags corresponding to the target object, Q i Represents the total weight value of historical video label i.

[0151] Based on the obtained target probabilities, a historical video tag distribution array CU_tag_dist is constructed. The array is a floating-point array, where CU_tag_dist[i] represents the target probability of the video associated with the historical video tag i uploaded by the target object.

[0152] In the embodiment of the present application, the higher the number of valid playback times of the historical video, the more likely the target object is to upload a video with the same type of tag again. At the same time, the video tags of the historical videos recently released by the target object can better represent the target object's recent video release preferences. Therefore, when determining the target object's video release preferences by combining the number of valid playback times of the historical video and the number of days the historical video was uploaded, the accuracy of the obtained video release preferences can be effectively improved.

[0153] Optionally, in the above step S204, the embodiment of the present application adopts at least one of the following possible implementation methods to predict the target video label of the target video, such as Figure 4 As shown, the following steps are included:

[0154] Step S401: fuse the text association relationships with the historical video tag distribution to obtain the joint attributes of the target video.

[0155] Specifically, when the text association relationship is a first association relationship, each first association relationship is fused with the historical video tag distribution to obtain the joint attribute of the target video.

[0156] When the text association relationship is a second association relationship, each second association relationship is fused with the historical video tag distribution to obtain the joint attributes of the target video.

[0157] When the text association relationship includes a first association relationship and a second association relationship, each first association relationship is fused with the historical video label distribution to obtain the first joint attribute of the target video. At the same time, each second association relationship is fused with the historical video label distribution to obtain the second joint attribute of the target video.

[0158] The interaction features between candidate video tags and video text content in plain text are combined with the video release preferences of the target object to obtain the joint attributes of the target video in the plain text dimension; the implicit interaction features between candidate video tags and video text content after deep representation are combined with the video release preferences of the target object to obtain the joint attributes of the target video in the latent dimension. The subsequent integration of multi-dimensional joint attribute features for video tag recognition can improve the accuracy of video tag recognition.

[0159] Step S402: extract features of the joint attributes of the target video to obtain label interaction features of the target video.

[0160] Specifically, a neural network model is used to extract features of the joint attributes of the target video, extract interactions in a larger context, and obtain label interaction features of the target video, that is, the interaction representation of the video label and the target video. The neural network model includes but is not limited to convolutional neural networks (CNN), recurrent neural networks (RNN), and long short-term memory networks (LSTM).

[0161] In one possible implementation, when the text association relationship is a first association relationship, each first association relationship is fused with the historical video tag distribution to obtain the joint attributes of the target video, and then feature extraction is performed on the joint attributes to obtain the tag interaction features of the target video.

[0162] For example, if Figure 5 As shown, a candidate video tag set and a target video uploaded by user account A are obtained. The target video's video text content includes the title, dialogue, and subtitles. Each candidate video tag in the candidate video tag set is converted into a word-granular tag text, and the video text content is converted into a word-granular video text.

[0163] For each candidate video tag, the overlap between each tag text corresponding to the candidate video tag and each video text in the video text content is determined. If the tag text and the video text are consistent, the corresponding overlap is 1; if the tag text and the video text are inconsistent, the corresponding overlap is 0. Based on the obtained overlaps, the first interaction matrix corresponding to the candidate video tag is determined. Each candidate video tag corresponds to a first interaction matrix.

[0164] Based on the historical video tags corresponding to each historical video uploaded by user account A in the past period of time, the historical video tags corresponding to user account A are obtained, and the historical video tag distribution array CU_tag_dist corresponding to user account A is constructed, where CU_tag_dist[i] represents the target probability of videos associated with historical video tag i uploaded by user account A.

[0165] Each first interaction matrix is ​​multiplied by the historical video label distribution array to obtain joint attributes. Then, a convolutional neural network is used to extract features from the joint attributes to obtain the label interaction features of the target video.

[0166] In one possible implementation, when the text association relationship is the second association relationship, each second association relationship is fused with the historical video tag distribution to obtain the joint attributes of the target video, and features are extracted from the joint attributes to obtain tag interaction features of the target video.

[0167] For example, if Figure 6 As shown, a candidate video tag set and a target video uploaded by user account A are obtained. The target video's video text content includes the title, dialogue, and subtitles. Each candidate video tag in the candidate video tag set is converted into a word-granular tag text, and the video text content is converted into a word-granular video text.

[0168] The text feature extraction model (ALBERT) is used to extract features from each label text and obtain the label text features corresponding to each label text. ALBERT is also used to extract features from each video text and obtain the video text features corresponding to each video text.

[0169] For each candidate video tag, determine the feature similarity between each tag text feature corresponding to the candidate video tag and each video text feature corresponding to the video text content. Based on the obtained feature similarities, determine a second interaction matrix corresponding to the candidate video tag. Each candidate video tag corresponds to a second interaction matrix.

[0170] Based on the historical video tags corresponding to each historical video uploaded by user account A in the past period of time, the historical video tags corresponding to user account A are obtained, and the historical video tag distribution array CU_tag_dist corresponding to user account A is constructed. CU_tag_dist[i] represents the target probability of videos associated with historical video tag i uploaded by user account A.

[0171] Each second interaction matrix is ​​multiplied by the historical video label distribution array to obtain joint attributes. Then, a convolutional neural network is used to extract features from the joint attributes to obtain the label interaction features of the target video.

[0172] In one possible implementation, when the text association relationship includes a first association relationship and a second association relationship, feature extraction is performed on the first joint attribute to obtain a first joint attribute feature. Feature extraction is performed on the second joint attribute to obtain a second joint attribute feature. The first joint attribute feature and the second joint attribute feature are then fused to obtain a label interaction feature for the target video.

[0173] For example, if Figure 7As shown, a candidate video tag set and a target video uploaded by user account A are obtained. The target video's video text content includes the title, dialogue, and subtitles. Each candidate video tag in the candidate video tag set is converted into a word-granular tag text, and the video text content is converted into a word-granular video text.

[0174] For each candidate video tag, the overlap between each tag text corresponding to the candidate video tag and each video text in the video text content is determined. If the tag text and the video text are consistent, the corresponding overlap is 1; if the tag text and the video text are inconsistent, the corresponding overlap is 0. Based on the obtained overlaps, the first interaction matrix corresponding to the candidate video tag is determined. Each candidate video tag corresponds to a first interaction matrix.

[0175] The text feature extraction model (ALBERT) is used to extract features from each label text and obtain the label text features corresponding to each label text. ALBERT is also used to extract features from each video text and obtain the video text features corresponding to each video text.

[0176] For each candidate video tag, determine the feature similarities between each tag text feature corresponding to the candidate video tag and each video text feature corresponding to the video text content. Based on the obtained feature similarities, determine the second interaction matrix corresponding to the candidate video tag. Each candidate video tag corresponds to a second interaction matrix.

[0177] Based on the historical video tags corresponding to each historical video uploaded by user account A in the past period of time, the historical video tags corresponding to user account A are obtained, and the historical video tag distribution array CU_tag_dist corresponding to user account A is constructed. CU_tag_dist[i] represents the target probability of videos associated with historical video tag i uploaded by user account A.

[0178] Each first interaction matrix is ​​multiplied by the historical video label distribution array to obtain a first joint attribute. Then, a convolutional neural network is used to extract features of the first joint attribute to obtain first joint attribute features.

[0179] Multiply each second interaction matrix by the historical video label distribution array to obtain the second joint attribute. Then, use a convolutional neural network to extract features from the second joint attribute to obtain the second joint attribute feature. The first joint attribute feature and the second joint attribute feature are fused to obtain the label interaction feature of the target video.

[0180] The first joint attribute is obtained by combining the plaintext interaction features between candidate video tags and video text content with the historical video tag distribution of the target object. The second joint attribute is obtained by combining the implicit interaction features between candidate video tags and video text content after deep representation with the historical video tag distribution of the target object. The first and second joint attributes are then fused to obtain the tag interaction feature, which fully explores the interaction clues between video tags and video content.

[0181] Step S403 : predicting a target video tag of the target video from the candidate video tag set based on the tag interaction feature of the target video.

[0182] Specifically, after obtaining the label interaction features of the target video, the target video label of the target video can be predicted from the candidate video label set directly based on the label interaction features of the target video, or the target video label can be predicted from the candidate video label set based on the label interaction features of the target video and the video content.

[0183] In an embodiment of the present application, the explicit interaction features and implicit interaction features of the candidate video tags and the video text content are combined with the video publishing preferences of the target object to obtain multi-dimensional joint attributes. By extracting interaction features of the joint attributes in a larger context range, label interaction features for characterizing deep-level interaction clues are obtained. Therefore, when predicting the target video tag of the target video based on the label interaction features, the accuracy of video label recognition can be effectively improved.

[0184] In one possible implementation, the video content includes video text, audio frame sequences, and image frame sequences. Feature extraction is performed on the video text, audio frame sequences, and image frame sequences contained in the target video to obtain corresponding text features, audio features, and image features. Video features of the target video are then obtained based on the text features, audio features, and image features. The label interaction features are then fused with the video features to obtain fused features. Based on the fused features, a target video label for the target video is predicted from a set of candidate video labels.

[0185] Specifically, text feature extraction models such as ALBERT, BERT, and Transformer-Encode can be used to extract text features corresponding to the video text content. Audio feature extraction models such as VGGish and recurrent neural networks can be used to extract audio features corresponding to the audio frame sequence. Image feature extraction models such as EfficientNet and convolutional neural networks can be used to extract image features corresponding to the image frame sequence. The text features, audio features, and image features are fused to obtain the video features of the target video, that is, a multi-dimensional representation of the target video. The label interaction features are then fused with the video features to obtain fused features. Based on these fused features, the predicted probability of the target video under each candidate video label is predicted from the candidate video label set.

[0186] The obtained prediction probabilities can be sorted from largest to smallest, and the candidate video labels corresponding to the first R predicted probabilities are used as the target video label of the target video, where R is a preset positive integer. Alternatively, a target prediction probability greater than a preset threshold can be screened out from the predicted probabilities, and the candidate video labels corresponding to the target prediction probabilities are used as the target video label of the target video.

[0187] In addition, before performing video label recognition, the various networks involved above can be jointly trained, wherein the training sample data includes sample videos pre-labeled with video labels, a candidate video label set, and a historical video label distribution of sample objects.

[0188] For example, if Figure 8 As shown, the video content of the target video uploaded by user account A includes video text content, audio frame sequence and image frame sequence, wherein the video text content includes title, dialogue and subtitles, the audio frame sequence includes audio frame 1, audio frame 2, ..., audio frame f, and the image frame sequence includes image frame 1, image frame 2, ..., image frame g.

[0189] A text feature extraction model (ALBERT) is used to extract features from the target video's text content, obtaining the corresponding text features. An audio feature extraction model (VGGish) is used to extract features from the target video's audio frame sequence, obtaining the corresponding audio features. An image feature extraction model (EfficientNet) is used to extract features from the target video's image frame sequence, obtaining the corresponding image features.

[0190] The text, audio, and image features are fused to obtain a multi-dimensional representation of the target video. The tag interaction features are then fused with the multi-dimensional representation of the target video to obtain a fused feature, which represents the combined tag interaction and multi-dimensional content of the video. Based on the fused feature, the target video tag is then predicted from a set of candidate video tags.

[0191] In an embodiment of the present application, the interaction features between the candidate video tags and the video text content in plain text are combined with the video release preference of the target object to obtain a first joint attribute; the implicit interaction features between the candidate video tags and the video text content after deep representation are combined with the video release preference of the target object to obtain a second joint attribute, and then the first joint attribute and the second joint attribute are fused to obtain the tag interaction feature, which fully explores the interaction clues between the video tags and the video content. Then, the tag interaction feature is combined with the multi-dimensional representation of the video content to identify the video tag of the target video, which can greatly improve the video tag recognition ability, provide more complete and accurate tag features for later video distribution, improve the video distribution effect of the video platform, and promote a virtuous cycle of video release, identification, and distribution.

[0192] In order to better explain the embodiment of the present application, the following describes a video tag recognition method provided by the embodiment of the present application in combination with a specific implementation scenario. The process of the method can be as follows: Figure 1 The terminal device 101 shown in FIG. 1 may also be executed by the server 102, or may be executed interactively by the terminal device 101 and the server 102, including the following steps: Figure 9 As shown:

[0193] Assume that user account A has uploaded target video D through a video app. The video text content in target video D consists of the title, dialogue, and subtitles. Convert the video text content into word-granular video text. Convert each candidate video tag in the candidate video tag set into word-granular tag text.

[0194] For each candidate video tag, the overlap between each tag text corresponding to the candidate video tag and each video text in the video text content is determined. If the tag text and the video text are consistent, the corresponding overlap is 1; if the tag text and the video text are inconsistent, the corresponding overlap is 0. Based on the obtained overlaps, the first interaction matrix corresponding to the candidate video tag is determined.

[0195] The text feature extraction model (ALBERT) is used to extract features from each label text and obtain the label text features corresponding to each label text. ALBERT is also used to extract features from each video text and obtain the video text features corresponding to each video text.

[0196] For each candidate video tag, determine the feature similarities between each tag text feature corresponding to the candidate video tag and each video text feature corresponding to the video text content, and determine the second interaction matrix corresponding to the candidate video tag based on the obtained feature similarities.

[0197] Based on the historical video tags corresponding to each historical video uploaded by user account A in the past period of time, the historical video tags corresponding to user account A are obtained, and the historical video tag distribution array CU_tag_dist corresponding to user account A is constructed. CU_tag_dist[i] represents the target probability of videos associated with historical video tag i uploaded by user account A.

[0198] Each first interaction matrix is ​​multiplied by the historical video label distribution array to obtain a first joint attribute. Then, a convolutional neural network is used to extract features of the first joint attribute to obtain first joint attribute features.

[0199] Multiply each second interaction matrix by the historical video label distribution array to obtain the second joint attribute. Then, use a convolutional neural network to extract features from the second joint attribute to obtain the second joint attribute feature. The first joint attribute feature and the second joint attribute feature are fused to obtain the label interaction feature of the target video.

[0200] A text feature extraction model (ALBERT) is used to extract features from the target video's text content, obtaining the corresponding text features. An audio feature extraction model (VGGish) is used to extract features from the target video's audio frame sequence, obtaining the corresponding audio features. An image feature extraction model (EfficientNet) is used to extract features from the target video's image frame sequence, obtaining the corresponding image features.

[0201] The text features, audio features, and image features are fused to obtain the video features of the target video. The tag interaction features are then fused with the video features to obtain the fused features. Based on the fused features, the target video label is predicted from the candidate video label set.

[0202] In an embodiment of the present application, by determining the textual association relationship between at least one candidate video tag included in the candidate video tag set and the video text content, the interaction features between the target video and each candidate video tag in plain text and the implicit interaction features after deep representation are obtained. Based on the historical video tags corresponding to each of the historical videos uploaded by the target object, the historical video tag distribution used to characterize the video release preference of the target object is obtained. The interaction features between the candidate video tags and the video text content in plain text are combined with the historical video tag distribution of the target object to obtain a first joint attribute. The implicit interaction features between the candidate video tags and the video text content after deep representation are combined with the historical video tag distribution of the target object to obtain a second joint attribute, and then the first joint attribute and the second joint attribute are fused to obtain the tag interaction feature, which fully explores the interaction clues between the video tag and the video content. Then, the tag interaction feature is combined with the multi-dimensional representation of the video content to identify the video tag of the target video, which can greatly improve the video tag recognition capability, provide more complete and accurate tag features for later video distribution, and improve the video distribution effect of the video platform.

[0203] Based on the same technical concept, the embodiment of the present application provides a structural diagram of a video tag recognition device, such as Figure 10 As shown, the device 1000 includes:

[0204] An acquisition module 1001 is configured to acquire video text content contained in a target video uploaded by a target object, and to acquire a set of candidate video tags, wherein the set of candidate video tags includes at least one candidate video tag, and each candidate video tag is used to represent a video attribute;

[0205] A matching module 1002 is configured to determine a text association relationship between each of the at least one candidate video tag and the video text content;

[0206] A statistics module 1003 is configured to determine a distribution of historical video tags of the target object based on the historical video tags corresponding to at least one historical video uploaded by the target object;

[0207] The prediction module 1004 is configured to determine a target video tag of the target video from the candidate video tag set based on the respective text association relationships and the historical video tag distribution.

[0208] Optionally, the text association relationship includes a first association relationship, wherein the first association relationship represents a degree of text overlap between a candidate video tag and the video text content;

[0209] The matching module 1002 is specifically configured to:

[0210] For the at least one candidate video tag, perform the following steps respectively:

[0211] Converting a candidate video tag into at least one tag text of a preset granularity, and converting the video text content into at least one video text of the preset granularity;

[0212] respectively determining the degree of overlap between each of the label texts in the at least one label text and the at least one video text;

[0213] Based on the respective overlap degrees, a first association relationship corresponding to the one candidate video tag is determined.

[0214] Optionally, the text association relationship further includes a second association relationship, wherein the second association relationship represents the text feature similarity between a candidate video tag and the video text content;

[0215] The matching module 1002 is further configured to:

[0216] Performing feature extraction on the at least one label text to obtain at least one label text feature, and performing feature extraction on the at least one video text to obtain at least one video text feature;

[0217] For the at least one video text feature, the following steps are respectively performed: determining the feature similarity between each label text feature in the at least one label text feature and a video text feature;

[0218] Based on the similarities of the respective features, a second association relationship corresponding to the candidate video tag is determined.

[0219] Optionally, the historical video tag distribution includes: at least one historical video tag corresponding to the target object, and a target probability that the target object uploads a video associated with the at least one historical video tag;

[0220] The statistics module 1003 is specifically used for:

[0221] Obtaining at least one historical video tag corresponding to the target object based on the historical video tag corresponding to each of the at least one historical video uploaded by the target object;

[0222] For the at least one historical video tag, the following steps are respectively performed: based on the historical playback attribute of at least one historical video associated with the historical video tag, a target probability of the target object uploading the video associated with the historical video tag is determined.

[0223] Optionally, the statistics module 1003 is specifically configured to:

[0224] For the at least one historical video, the following steps are respectively performed: based on the historical playback attributes of the historical video and the number of days the historical video was uploaded, determining the weight of the historical video relative to the historical video label;

[0225] Based on the weight corresponding to each of the at least one historical video, a target probability is determined that the target object uploads a video associated with the historical video tag.

[0226] Optionally, the prediction module 1004 is specifically configured to:

[0227] Merging each text association relationship with the historical video tag distribution to obtain joint attributes of the target video;

[0228] Performing feature extraction on the joint attributes of the target video to obtain label interaction features of the target video;

[0229] Based on the label interaction feature of the target video, a target video label of the target video is predicted from the candidate video label set.

[0230] Optionally, the prediction module 1004 is specifically configured to:

[0231] Extracting features of the target video text content, audio frame sequence, and image frame sequence to obtain corresponding text features, audio features, and image features;

[0232] Obtaining video features of the target video based on the text features, the audio features, and the image features;

[0233] The tag interaction feature is fused with the video feature to obtain a fused feature, and based on the fused feature, a target video tag of the target video is predicted from the candidate video tag set.

[0234] Optionally, the prediction module 1004 is specifically configured to:

[0235] Merging each first association relationship with the historical video tag distribution to obtain a first joint attribute of the target video;

[0236] Each second association relationship is fused with the historical video tag distribution to obtain a second joint attribute of the target video.

[0237] Optionally, the prediction module 1004 is specifically configured to:

[0238] Performing feature extraction on the first joint attribute to obtain a first joint attribute feature;

[0239] performing feature extraction on the second joint attribute to obtain a second joint attribute feature;

[0240] The first joint attribute feature and the second joint attribute feature are fused to obtain a label interaction feature of the target video.

[0241] In an embodiment of the present application, by determining the textual association relationship between at least one candidate video tag included in the candidate video tag set and the video text content in the target video, the interaction characteristics between the target video and each candidate video tag are obtained. Based on the historical video tags corresponding to each historical video uploaded by the target object, the distribution of historical video tags used to characterize the video publishing preference of the target object is obtained. Compared with the solution of identifying video tags based only on video content, the method of tag identification based on the interaction characteristics between the target video and each candidate video tag and the video publishing preference of the target object has a wider and more comprehensive feature dimension, thereby effectively improving the accuracy of tag identification and thereby improving the effect of subsequent video distribution.

[0242] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be Figure 1 The terminal device and / or server shown, such as Figure 11 As shown, it includes at least one processor 1101 and a memory 1102 connected to the at least one processor. The specific connection medium between the processor 1101 and the memory 1102 is not limited in the embodiment of the present application. Figure 11 For example, the processor 1101 and the memory 1102 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0243] In the embodiment of the present application, the memory 1102 stores instructions that can be executed by at least one processor 1101. The at least one processor 1101 can perform the steps of the above-mentioned video tag recognition method by executing the instructions stored in the memory 1102.

[0244] The processor 1101 is the control center of the computer device. It can connect various parts of the computer device using various interfaces and lines. It can identify the video tag of the target video by running or executing instructions stored in the memory 1102 and calling data stored in the memory 1102. Optionally, the processor 1101 may include one or more processing units. The processor 1101 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1101. In some embodiments, the processor 1101 and the memory 1102 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0245] The processor 1101 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0246] Memory 1102 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1102 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. Memory 1102 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1102 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0247] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned video tag identification method.

[0248] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs the steps of the above-mentioned video tag recognition method.

[0249] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0250] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0251] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0252] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0253] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0254] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A video tag recognition method, characterized in that: include: Obtaining video text content contained in a target video uploaded by a target object, and obtaining a set of candidate video tags, wherein the set of candidate video tags includes at least one candidate video tag, and each candidate video tag is used to represent a video attribute; Determining a text association relationship between each of the at least one candidate video tag and the video text content; Determining a historical video tag distribution of the target object based on a historical video tag corresponding to at least one historical video uploaded by the target object; Based on the respective text association relationships and the historical video tag distribution, a target video tag of the target video is determined from the candidate video tag set.

2. The method according to claim 1, wherein The text association relationship includes a first association relationship, wherein the first association relationship represents a degree of text overlap between a candidate video tag and the video text content; The determining of the text association relationship between each of the at least one candidate video tag and the video text content includes: For the at least one candidate video tag, perform the following steps respectively: Converting a candidate video tag into at least one tag text of a preset granularity, and converting the video text content into at least one video text of the preset granularity; respectively determining the degree of overlap between each of the label texts in the at least one label text and the at least one video text; Based on the respective overlap degrees, a first association relationship corresponding to the one candidate video tag is determined.

3. The method according to claim 2, wherein The text association relationship further includes a second association relationship, wherein the second association relationship represents the text feature similarity between a candidate video tag and the video text content; The determining of the text association relationship between the at least one candidate video tag and the video text content respectively further includes: Performing feature extraction on the at least one label text to obtain at least one label text feature, and performing feature extraction on the at least one video text to obtain at least one video text feature; For the at least one video text feature, the following steps are respectively performed: determining the feature similarity between each label text feature in the at least one label text feature and a video text feature; Based on the similarities of the respective features, a second association relationship corresponding to the candidate video tag is determined.

4. The method according to claim 1, wherein The historical video tag distribution includes: at least one historical video tag corresponding to the target object, and a target probability that the target object uploads a video associated with the at least one historical video tag; The determining of the historical video tag distribution of the target object based on the historical video tag corresponding to each of the at least one historical video uploaded by the target object includes: Obtaining at least one historical video tag corresponding to the target object based on the historical video tag corresponding to each of the at least one historical video uploaded by the target object; For the at least one historical video tag, perform the following steps respectively: Based on a historical playback attribute of at least one historical video associated with a historical video tag, a target probability of the target object uploading the video associated with the historical video tag is determined.

5. The method according to claim 4, wherein The determining, based on the historical playback attribute of at least one historical video associated with a historical video tag, the target probability of the target object uploading the video associated with the historical video tag includes: For the at least one historical video, the following steps are respectively performed: based on the historical playback attributes of the historical video and the number of days the historical video was uploaded, determining the weight of the historical video relative to the historical video label; Based on the weight corresponding to each of the at least one historical video, a target probability is determined that the target object uploads a video associated with the historical video tag.

6. The method according to claim 3, wherein The determining of the target video tag of the target video from the candidate video tag set based on the respective text association relationships and the historical video tag distribution includes: Merging each text association relationship with the historical video tag distribution to obtain joint attributes of the target video; Performing feature extraction on the joint attributes of the target video to obtain label interaction features of the target video; Based on the label interaction feature of the target video, a target video label of the target video is predicted from the candidate video label set.

7. The method according to claim 6, wherein The step of predicting a target video tag of the target video from the candidate video tag set based on the tag interaction feature of the target video includes: Extracting features of the target video text content, audio frame sequence, and image frame sequence to obtain corresponding text features, audio features, and image features; Obtaining video features of the target video based on the text features, the audio features, and the image features; The tag interaction feature is fused with the video feature to obtain a fused feature, and based on the fused feature, a target video tag of the target video is predicted from the candidate video tag set.

8. The method according to claim 6, wherein The step of fusing the text association relationships with the historical video tag distribution to obtain the joint attributes of the target video includes: Merging each first association relationship with the historical video tag distribution to obtain a first joint attribute of the target video; Each second association relationship is fused with the historical video tag distribution to obtain a second joint attribute of the target video.

9. The method according to claim 8, wherein The extracting features of the joint attributes of the target video to obtain the label interaction features of the target video includes: Performing feature extraction on the first joint attribute to obtain a first joint attribute feature; performing feature extraction on the second joint attribute to obtain a second joint attribute feature; The first joint attribute feature and the second joint attribute feature are fused to obtain a label interaction feature of the target video.

10. A video tag recognition device, characterized in that: include: an acquisition module, configured to acquire video text content contained in a target video uploaded by a target object, and to acquire a set of candidate video tags, wherein the set of candidate video tags includes at least one candidate video tag, and each candidate video tag is used to represent a video attribute; a matching module, configured to respectively determine a text association relationship between the at least one candidate video tag and the video text content; A statistics module, configured to determine a distribution of historical video tags of the target object based on a historical video tag corresponding to at least one historical video uploaded by the target object; A prediction module is configured to determine a target video tag of the target video from the candidate video tag set based on the respective text association relationships and the historical video tag distribution.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that It stores a computer program that can be executed by a computer device. When the program is run on the computer device, the computer device executes the steps of any one of the methods according to claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is caused to perform the steps of the method according to any one of claims 1 to 9.

Images