Video labeling method, system, equipment and device

Abstract

The invention relates to a video labeling method, system and device and a medium, and the method comprises the steps: inputting a target video into a multi-mode large model, and obtaining a target video description of the target video; determining a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multi-modal large model; and marking the target video based on the target classification label. According to the method, the corresponding relation between the video and the label is established by utilizing the understanding ability of the multi-modal large model to the video content and the understanding ability of the pre-trained classification model to the text content, and the video is intelligently labeled by using the corresponding label. Manual processing is not needed, and the problems that in a traditional labeling mode, efficiency is low, cost is high, and labeling is not uniform due to manual personal subjective understanding are effectively solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a video annotation method, system, device and medium. Background Art

[0002] With the development of the Internet, video has gradually become a popular form of online activity. Video playback service providers need to classify and manage the videos they serve. The usual method is video annotation, which is to label the actions and expressions of the characters in the video. This is an important part of video understanding. The traditional video annotation method usually requires manual viewing of the corresponding video, and then the video is labeled according to the supervisor's understanding of the video content and the pre-set label classification. On the one hand, this implementation method requires a large amount of manpower to watch the video content, and the video content uploaded by users on the Internet increases dramatically every day. In the long run, more human resources will be consumed to label the video. On the other hand, due to the different cognition of each person, there must be personal subjective factors in the understanding of the content in the video, which makes it difficult for the video annotation labels to converge, making it difficult to reach a consensus on videos under the same label classification.

[0003] Therefore, a solution is urgently needed to solve the above problems. Summary of the invention

[0004] The present application provides a video annotation method to solve the problems of high labor cost and difficulty in unifying annotation results in traditional video annotation implementation.

[0005] In a first aspect, the present application provides a video annotation method, comprising:

[0006] Inputting a target video into a multimodal large model to obtain a target video description of the target video;

[0007] Determining a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0008] The target video is labeled based on the target classification label.

[0009] In some embodiments of the present application, the classification model is trained in the following manner:

[0010] Determine sample videos and their corresponding classification labels;

[0011] Inputting the sample video into a multimodal large model to obtain a sample description of the sample video output by the multimodal large model;

[0012] Constructing training data based on the sample description and labels corresponding to the sample video;

[0013] The classification model is trained based on the training data.

[0014] In some embodiments of the present application, labeling the target video based on the target classification label includes:

[0015] Determine multiple related videos corresponding to the target classification label;

[0016] Obtaining user interaction information corresponding to the related video, and determining an associated tag according to the user interaction information and the target video description content;

[0017] The target video is labeled by the associated tag.

[0018] In some embodiments of the present application, obtaining user interaction information corresponding to a plurality of videos, and determining associated tags according to the user interaction information and the target video description content, includes:

[0019] Acquire user interaction information corresponding to the relevant video, and perform a word segmentation operation on the user interaction information to obtain corresponding word segments;

[0020] Count the number of occurrences of each word, and take the words that appear a predetermined number of times as high-frequency keywords;

[0021] Generate associated tags according to the high-frequency keywords.

[0022] In some embodiments of the present application, generating associated tags according to the high-frequency keywords includes:

[0023] Obtain the corresponding user interaction information timestamp of the relevant video;

[0024] Determine, according to the timestamp and the content of the user interaction information, high-frequency keywords corresponding to the user interaction information in different time ranges;

[0025] The associated tags are generated by selecting high-frequency keywords corresponding to a time range closest to the target video.

[0026] In some embodiments of the present application, determining high-frequency keywords corresponding to user interaction information in different time ranges according to the timestamp and the content of the user interaction information includes:

[0027] Dividing the interaction information into multiple time intervals at predetermined time intervals according to the timestamps of all user interaction information;

[0028] Count the number of high-frequency keywords in each time interval to determine the highest number of high-frequency keywords in each time interval;

[0029] The high-frequency keyword with the highest number is used as the high-frequency keyword corresponding to each time interval.

[0030] In some embodiments of the present application, the video annotation method further includes:

[0031] Based on the number of the high-frequency keywords in a plurality of consecutive time intervals, the change rate of the high-frequency keywords is determined, and the high-frequency keywords whose change rate exceeds a predetermined value are eliminated.

[0032] The second aspect of the present application further proposes a video annotation system, comprising:

[0033] A video description generation module, used for inputting a target video into a multimodal large model to obtain a target video description of the target video;

[0034] A classification label confirmation module, used to determine a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0035] A video annotation module is used to annotate the target video based on the target classification label.

[0036] The third aspect of the present application also proposes a computer device: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to execute the video annotation method described in any one of the above embodiments.

[0037] The fourth aspect of the present application further proposes a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the video annotation method of any one of the above-mentioned embodiments.

[0038] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains the target video description of the target video by inputting the target video to be labeled into the multimodal large model, and then trains the corresponding classification model in advance according to the video description output by the multimodal large model for the labeled video, and establishes the mapping relationship between the video description output by the multimodal large model and the label. When classifying the description sentence of the target video to be labeled, the classification model can be used to give the classification result for the target video. This solves the problems of slow efficiency, high cost and inconsistency caused by human subjective understanding in the traditional labeling method. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0042] Figure 1 A flowchart of a video annotation method provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the classification model training process provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of annotating a video to be annotated by associating tags provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a process for generating associated tags based on high-frequency keywords provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a method flow for determining high-frequency keywords provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a method flow for determining high-frequency keywords based on a time range provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of a video annotation system provided in an embodiment of the present application;

[0049] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0052] In order to solve the problems of high cost, low efficiency and lack of unified standards in the prior art, the present application provides a video annotation method, which can achieve efficient, fast and accurate annotation of videos to be annotated.

[0053] Figure 1 A schematic diagram of a process flow of a video annotation method provided in an embodiment of the present application includes:

[0054] Step S1, inputting a target video into a multimodal large model to obtain a target video description of the target video;

[0055] Step S2: determining a target classification label corresponding to the target video description by a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0056] Step S3: labeling the target video based on the target classification label.

[0057] In an embodiment of the present application, the target video is a video to be annotated, and the target video description is a text description output by the multimodal large model for the input video to be annotated, that is, the text description of the video elements in the video to be annotated by the multimodal large model; the multimodal large model includes "videochat, videollama, videochatgpt, videollava, videochat2, stllm" and the like.

[0058] The classification model is a pre-trained artificial intelligence model, whose input is a text description and whose output is a label corresponding to the training sample during training. The target classification label is a preset label for classifying the video to be labeled. The range of the target classification label or the type of label corresponds to the labeled label in the sample data when the classification model was pre-trained. The sample data during classification model training is the video description output by the corresponding multimodal large model for the labeled video. In other words, what kind of multimodal large model is used to obtain the target video description of the video to be labeled, and the corresponding multimodal large model is selected to generate the video description of the labeled video, and the training sample data is used to train the classification model.

[0059] In some embodiments of the present application, the target classification label used to annotate the video to be annotated also includes a label with an associated relationship formed by extracting keywords from user interaction information on the classified video playback page after the corresponding classified video is made public on the Internet.

[0060] In an embodiment of the present application, in step S1, the video to be annotated is input into the multimodal large model, and the large model is allowed to output a text description of the video to be annotated after understanding, that is, to describe the content in the video to be annotated. Specifically, the corresponding prompt words can be used to allow the multimodal large model to output a description of the corresponding scene in the video or the characters, scenes, etc. in the picture. For example, the corresponding prompt words include: "Describe the following video concisely", "Provide a brief description of the given video clip", "Offer a succinct explanation of the footage presented", etc. In some embodiments of the present application, multiple prompt words of the corresponding prompt word project can be used to guide the multimodal large model to output the corresponding target video description step by step.

[0061] In step S2, the target video description of the video to be labeled obtained in step S1 is input into a pre-trained classification model, and the corresponding target classification label is obtained through the classification model.

[0062] In some embodiments of the present application, classification can also be performed through a large language model, that is, based on the label and the video description corresponding to the label-labeled video by the multimodal large model as a reference, the target video description corresponding to the video to be labeled is input into the large language model, and the large language model is allowed to determine which of the target video description and the label-labeled video description used as the reference is closest. After determining the closest label-labeled video description, the label corresponding to the label-labeled video description is used as the target classification label.

[0063] In some embodiments of the present application, some open source large language models can be fine-tuned based on the labels and the video descriptions corresponding to the videos annotated with the labels by the multimodal large model as sample data, so that the fine-tuned large language models can be directly used to classify the target video descriptions of the videos to be annotated.

[0064] In step S3, the target classification label determined in step S2 is used as the classification label of the video to be labeled, and the video to be labeled is labeled.

[0065] like Figure 2 As shown, in some embodiments of the present application, the classification model is trained in the following manner:

[0066] Step S21, determining sample videos and their corresponding classification labels;

[0067] Step S22: input the sample video into the multimodal large model to obtain a sample description of the sample video output by the multimodal large model;

[0068] Step S23: constructing training data based on the sample description and labels corresponding to the sample video;

[0069] Step S24: training the classification model based on the training data.

[0070] In some embodiments of the present application, in order to make the corresponding classification model more adaptable to or more matching the video description of the video to be labeled, the video description corresponding to the video labeled by the multimodal large model based on the label and the label correspondence is preferably used as the sample data, so that the trained classification model can better classify the target video description.

[0071] Specifically, we first prepare sample videos, which are videos that have been labeled with classification labels. Then, we input the sample videos into the corresponding multimodal large model, and use the multimodal large model to obtain the description corresponding to the sample videos, i.e., the sample descriptions. We further associate the sample descriptions with the labels of the sample videos, i.e., we associate the sample descriptions as feature data with the labels as classification results, to form training data.

[0072] Furthermore, if a dedicated model is to be trained, only the corresponding algorithm needs to be selected to train the corresponding dedicated classification model based on the training data. For example, the training data is divided into a training set and a test set. Taking BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language model based on the Transformer architecture) as an example, the appropriate model structure (the number of hidden layers and the number of hidden units in each layer) is determined, and training is performed based on the training set. After the training is completed, the test set is used to test whether the predetermined classification performance is met, so as to build a dedicated classification model.

[0073] In some instances of the present application, the open source large language model can also be fine-tuned in a supervised fine-tuning manner based on the above-mentioned training data, that is, the video description of the training data is used as a feature, and the label corresponding to the video description is used as the result to fine-tune the corresponding large language model, so as to generate a corresponding classification model based on the large language model.

[0074] In some embodiments of the present application, a corresponding classification model may also be trained based on the base language model pair and using the above training data.

[0075] like Figure 3 As shown, in some embodiments of the present application, labeling the target video based on the target classification label includes:

[0076] Step S31, determining multiple related videos corresponding to the target classification label;

[0077] Step S32: obtaining user interaction information corresponding to the related video, and determining a related tag according to the user interaction information and the target video description content;

[0078] Step S33: label the target video using the associated tag.

[0079] In some scenarios, the application scenarios of video annotation include video content search, that is, users enter the corresponding keywords, and the video playback platform provides the corresponding video based on the keywords. However, due to the development of Internet terms on the Internet, Internet hot words or Internet hot memes continue to appear. When searching, users may enter new Internet hot words that appeared in a certain period, and the pre-set video classification tags may be difficult to keep up with the development of Internet terms. Therefore, it is difficult to match the corresponding video content based on the Internet hot words entered by the user.

[0080] To this end, in this embodiment, the present application proposes to form new associated tags to mark the video by binding corresponding tags to network hot words.

[0081] In this embodiment, the related video refers to a video that has been classified or labeled and has the same label as the video to be labeled. The user interaction information refers to the text information input by the user in the comment area or bullet screen corresponding to the related video.

[0082] Specifically, first determine multiple related videos corresponding to known classification tags, then obtain the bullet screen or comment content on the playback page of the related video, find representative words related to the target classification tag from the bullet screen or comment content as the associated tag, and then use the associated tag to annotate the video. For example, in a war drama, the villain intended to poison the decent characters with chicken soup, but was discovered by the decent characters and did not succeed. Combined with the meaning of the word "chicken soup" in the current culture, which is strong on the outside but weak on the inside, a large number of netizens input "chicken soup is coming" in the corresponding screen to mock the villain. For such video content, when other users upload the corresponding clips after the secondary creation, when the video description output by the multimodal large model is combined with the corresponding classification model to obtain the corresponding label, it is inevitable that it is difficult to output a label that conforms to the current user's cognition. Therefore, the representative words with network user consensus that appear in the related video are used as associated tags. When the user searches for the corresponding content, the corresponding video can be directly presented to the user. Thereby realizing the intelligent extension of the classification tag.

[0083] like Figure 4 As shown, in some embodiments of the present application, obtaining user interaction information corresponding to multiple videos, and determining associated tags according to the user interaction information and the target video description content include:

[0084] Step S41, obtaining user interaction information corresponding to the relevant video, and performing a word segmentation operation on the user interaction information to obtain corresponding word segments;

[0085] Step S42, counting the number of occurrences of each participle, and taking the participles that appear a predetermined number of times as high-frequency keywords;

[0086] Step S43: Generate associated tags according to the high-frequency keywords.

[0087] In this embodiment, as described above, when the video to be annotated is annotated by determining the corresponding associated tags through the user's interactive information, since the content input by the user may not be an ideal tag word, for example, it may be some descriptive sentences, etc. Therefore, it is necessary to first obtain the text content of the user's comments in the barrage and / or comment area of ​​the relevant video, and then segment the text content through the corresponding word segmentation tool to obtain the corresponding shortened keywords.

[0088] Furthermore, after obtaining the corresponding keywords, the number of occurrences of each keyword is counted, and the keywords with a predetermined number are regarded as high-frequency keywords. The predetermined number for determining high-frequency keywords can be determined based on user interaction information of different periods or different videos.

[0089] After the high-frequency keywords are determined, the high-frequency keywords can be used as associated tags related to the target classification tags to annotate the video to be annotated.

[0090] It should be noted that the associated tags are determined based on the interactive information of the labeled videos corresponding to the target classification tags determined by the video to be labeled. Of course, the labeled videos can also be labeled according to the associated tags. In actual labeling, the associated tags can be determined dynamically based on the real-time interactive information of the related videos under the corresponding tags. When a new video to be labeled appears, the corresponding associated tags can be directly obtained. There is no need to reprocess each video to be labeled.

[0091] In some embodiments of the present application, when counting the number of corresponding keywords, the act of liking the interactive information is also regarded as one interactive information for statistics.

[0092] like Figure 5 As shown, in some embodiments of the present application, generating associated tags according to the high-frequency keywords includes:

[0093] Step S51, obtaining the corresponding user interaction information timestamp of the relevant video;

[0094] Step S52: determining high-frequency keywords corresponding to the user interaction information in different time ranges according to the timestamp and the content of the user interaction information;

[0095] Step S53: Select high-frequency keywords corresponding to the time range closest to the target video to generate the associated tags.

[0096] In this embodiment, for the same video content, the interactive information output by users watching at different times is also time-related or epoch-related due to the changes in cultural activities in reality. For the same video content, the comments of users corresponding to the content in different time ranges are also different. Measuring the associated tags by the number of keywords alone may not be applicable to the changes in users' views on the same video content over time, especially some video contents are advanced in conception and difficult for the general public to understand in time.

[0097] Therefore, when generating corresponding associated tags based on high-frequency keywords, it is also necessary to consider the time factor, determine the corresponding high-frequency keywords in different time ranges, and further generate corresponding associated tags for different time ranges.

[0098] Specifically, obtain the timestamp of the barrage or comment content sent by the corresponding user under the relevant video corresponding to the tag, that is, the time when the barrage or comment is sent, and the time when the corresponding like behavior occurs.

[0099] Furthermore, the interactive information is divided into different time ranges based on a month, a day, or an hour, and then the number of keywords corresponding to each time range is counted, and the keywords whose number reaches the corresponding predetermined number are regarded as high-frequency keywords in each time range. When annotating the video to be annotated, the high-frequency keywords of the time range closest to the current time are regarded as associated tags.

[0100] like Figure 6 As shown, in some embodiments of the present application, according to the timestamp and the content of the user interaction information, determining the high-frequency keywords corresponding to the user interaction information in different time ranges respectively includes:

[0101] Step S61: Divide the interaction information into multiple time intervals at predetermined time intervals according to the timestamps of all user interaction information;

[0102] Step S62: Count the number of each high-frequency keyword in each time interval to determine the high-frequency keyword with the highest number in each time interval;

[0103] Step S63: Use the high-frequency keywords with the highest number as the high-frequency keywords corresponding to each time interval.

[0104] In this embodiment, the time range of high-frequency keywords can be determined according to the number of keywords within a period of time, that is, the keyword with the largest number of keywords within a period of time is regarded as the high-frequency keyword, and the corresponding time is regarded as the time range to which the high-frequency keyword belongs.

[0105] Specifically, the interactive information can also be divided into different time intervals according to a predetermined time, such as one month, one day, or one hour. The number of corresponding keywords in each time interval is further counted, and the keyword with the highest number of keywords in each time interval is taken as a high-frequency keyword.

[0106] Furthermore, in some embodiments, due to the different granularity of the time interval, there may be corresponding high-frequency keywords that are the most numerous keywords in consecutive time intervals. And these consecutive time intervals are the leading time ranges of the corresponding high-frequency keywords. Therefore, under this time division method, the time range closest to the current time may include multiple consecutive time intervals. This indicates that the corresponding high-frequency keywords have a high popularity at the current stage.

[0107] In some embodiments of the present application, the video annotation method further includes:

[0108] Based on the number of the high-frequency keywords in a plurality of consecutive time intervals, the change rate of the high-frequency keywords is determined, and the high-frequency keywords whose change rate exceeds a predetermined value are eliminated.

[0109] In this embodiment, since the corresponding associated tags are determined by the user's interactive information, and the user's interactive information is text content input by the user, there may be some potential malicious comments. The content of malicious comments often praises the content in the video, such as the actors of certain roles; or because different video contents are online at the same time, the corresponding video producers slander other video contents, or some "fan circle" cultural reasons slander other video contents. In most cases, this is an organized and planned behavior, such as automatically initiating comments through some automated tools. Or it is the behavior of a certain group of people swiping barrages or swiping the same comments under certain video content.

[0110] To solve this problem, when determining high-frequency keywords, the present application calculates the change rate based on the number of high-frequency keywords in different time intervals after counting the number of high-frequency keywords in multiple consecutive time intervals. If the change rate exceeds a predetermined value, the keyword is considered to be an abnormal keyword. The high-frequency keyword can be eliminated, and then the keyword with the highest number is searched again as the high-frequency keyword of the corresponding time range while ignoring the high-frequency keyword.

[0111] In some embodiments of the present application, since normal high-frequency keywords must have a certain spreading trend from their first appearance to their development to a certain height, the setting of the predetermined value corresponding to the change rate can refer to the change rates of other keywords in multiple time intervals in the interactive information of the corresponding user under the relevant video during the same period, and the change rates of multiple other keywords are averaged and then multiplied by the corresponding coefficient to determine the predetermined value for screening abnormal high-frequency keywords.

[0112] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains the target video description of the target video by inputting the target video to be labeled into the multimodal large model, and then trains the corresponding classification model in advance according to the video description output by the multimodal large model for the labeled video, and establishes the mapping relationship between the video description output by the multimodal large model and the label. When classifying the description sentence of the target video to be labeled, the classification model can be used to give the classification result for the target video. This solves the problems of slow efficiency, high cost and inconsistency caused by human subjective understanding in the traditional labeling method.

[0113] Furthermore, by generating relevant tags that keep up with the times through the interactive information of existing videos corresponding to the target classification tags, and annotating the videos to be annotated through relevant tags, it can adapt to the mainstream understanding of users in different periods, making the tags annotated for videos more vital under the ever-changing network development. On the other hand, it can reduce the repeated training caused by the update of the classification model for the ever-changing video content, greatly reducing the cost of video annotation.

[0114] like Figure 7 As shown, the second aspect of the present application further proposes a video annotation system, comprising:

[0115] A video description generating module 1 is used to input a target video into a multimodal large model to obtain a target video description of the target video;

[0116] A classification label confirmation module 2 is used to determine the target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0117] The video labeling module 3 is used to label the target video based on the target classification label.

[0118] like Figure 8 As shown, the third aspect of the present application further proposes a computer device: at least one communication interface 812; at least one bus 814 connected to the at least one communication interface 812; at least one processor 811 connected to the at least one bus 814; at least one memory 813 connected to the at least one bus 814, wherein the processor 811 is configured to execute the video annotation method described in any one of the above embodiments, including:

[0119] Inputting a target video into a multimodal large model to obtain a target video description of the target video;

[0120] Determining a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0121] The target video is labeled based on the target classification label.

[0122] In some embodiments of the present application, the classification model is trained in the following manner:

[0123] Determine sample videos and their corresponding classification labels;

[0124] Inputting the sample video into a multimodal large model to obtain a sample description of the sample video output by the multimodal large model;

[0125] Constructing training data based on the sample description and labels corresponding to the sample video;

[0126] The classification model is trained based on the training data.

[0127] In some embodiments of the present application, labeling the target video based on the target classification label includes:

[0128] Determine multiple related videos corresponding to the target classification label;

[0129] Obtaining user interaction information corresponding to the related video, and determining an associated tag according to the user interaction information and the target video description content;

[0130] The target video is labeled by the associated tag.

[0131] In some embodiments of the present application, obtaining user interaction information corresponding to a plurality of videos, and determining associated tags according to the user interaction information and the target video description content, includes:

[0132] Acquire user interaction information corresponding to the relevant video, and perform a word segmentation operation on the user interaction information to obtain corresponding word segments;

[0133] Count the number of occurrences of each word, and take the words that appear a predetermined number of times as high-frequency keywords;

[0134] Generate associated tags according to the high-frequency keywords.

[0135] In some embodiments of the present application, generating associated tags according to the high-frequency keywords includes:

[0136] Obtain the corresponding user interaction information timestamp of the relevant video;

[0137] Determine, according to the timestamp and the content of the user interaction information, high-frequency keywords corresponding to the user interaction information in different time ranges;

[0138] The associated tags are generated by selecting high-frequency keywords corresponding to a time range closest to the target video.

[0139] In some embodiments of the present application, determining high-frequency keywords corresponding to user interaction information in different time ranges according to the timestamp and the content of the user interaction information includes:

[0140] Dividing the interaction information into multiple time intervals at predetermined time intervals according to the timestamps of all user interaction information;

[0141] Count the number of high-frequency keywords in each time interval to determine the highest number of high-frequency keywords in each time interval;

[0142] The high-frequency keyword with the highest number is used as the high-frequency keyword corresponding to each time interval.

[0143] In some embodiments of the present application, the video annotation method further includes:

[0144] Based on the number of the high-frequency keywords in a plurality of consecutive time intervals, the change rate of the high-frequency keywords is determined, and the high-frequency keywords whose change rate exceeds a predetermined value are eliminated.

[0145] The fourth aspect of the present application further proposes a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the video annotation method of any one of the above embodiments, including:

[0146] Inputting a target video into a multimodal large model to obtain a target video description of the target video;

[0147] Determining a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model;

[0148] The target video is labeled based on the target classification label.

[0149] In some embodiments of the present application, the classification model is trained in the following manner:

[0150] Determine sample videos and their corresponding classification labels;

[0151] Inputting the sample video into a multimodal large model to obtain a sample description of the sample video output by the multimodal large model;

[0152] Constructing training data based on the sample description and labels corresponding to the sample video;

[0153] The classification model is trained based on the training data.

[0154] In some embodiments of the present application, labeling the target video based on the target classification label includes:

[0155] Determine multiple related videos corresponding to the target classification label;

[0156] Obtaining user interaction information corresponding to the related video, and determining an associated tag according to the user interaction information and the target video description content;

[0157] The target video is labeled by the associated tag.

[0158] In some embodiments of the present application, obtaining user interaction information corresponding to a plurality of videos, and determining associated tags according to the user interaction information and the target video description content, includes:

[0159] Acquire user interaction information corresponding to the relevant video, and perform a word segmentation operation on the user interaction information to obtain corresponding word segments;

[0160] Count the number of occurrences of each word, and take the words that appear a predetermined number of times as high-frequency keywords;

[0161] Generate associated tags according to the high-frequency keywords.

[0162] In some embodiments of the present application, generating associated tags according to the high-frequency keywords includes:

[0163] Obtain the corresponding user interaction information timestamp of the relevant video;

[0164] Determine, according to the timestamp and the content of the user interaction information, high-frequency keywords corresponding to the user interaction information in different time ranges;

[0165] The associated tags are generated by selecting high-frequency keywords corresponding to a time range closest to the target video.

[0166] In some embodiments of the present application, determining high-frequency keywords corresponding to user interaction information in different time ranges according to the timestamp and the content of the user interaction information includes:

[0167] Dividing the interaction information into multiple time intervals at predetermined time intervals according to the timestamps of all user interaction information;

[0168] Count the number of high-frequency keywords in each time interval to determine the highest number of high-frequency keywords in each time interval;

[0169] The high-frequency keyword with the highest number is used as the high-frequency keyword corresponding to each time interval.

[0170] In some embodiments of the present application, the video annotation method further includes:

[0171] Based on the number of the high-frequency keywords in a plurality of consecutive time intervals, the change rate of the high-frequency keywords is determined, and the high-frequency keywords whose change rate exceeds a predetermined value are eliminated.

[0172] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0174] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0175] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A video annotation method, characterized in that: include: Inputting a target video into a multimodal large model to obtain a target video description of the target video; Determining a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model; The target video is labeled based on the target classification label.

2. The method according to claim 1, characterized in that: The classification model is trained in the following way: Determine sample videos and their corresponding classification labels; Inputting the sample video into a multimodal large model to obtain a sample description of the sample video output by the multimodal large model; Constructing training data based on the sample description and labels corresponding to the sample video; The classification model is trained based on the training data.

3. The method according to claim 1, characterized in that: The labeling the target video based on the target classification label includes: Determine multiple related videos corresponding to the target classification label; Obtaining user interaction information corresponding to the related video, and determining an associated tag according to the user interaction information and the target video description content; The target video is labeled by the associated tag.

4. The method according to claim 3, characterized in that The obtaining of user interaction information corresponding to the plurality of videos and determining associated tags according to the user interaction information and the target video description content includes: Acquire user interaction information corresponding to the relevant video, and perform a word segmentation operation on the user interaction information to obtain corresponding word segments; Count the number of occurrences of each word, and take the words that appear a predetermined number of times as high-frequency keywords; Generate associated tags according to the high-frequency keywords.

5. The method according to claim 4, characterized in that Generating associated tags according to the high-frequency keywords includes: Obtain the corresponding user interaction information timestamp of the relevant video; Determine, according to the timestamp and the content of the user interaction information, high-frequency keywords corresponding to the user interaction information in different time ranges; The associated tags are generated by selecting high-frequency keywords corresponding to the time range closest to the target video.

6. The method according to claim 5, characterized in that The determining, according to the timestamp and the content of the user interaction information, high-frequency keywords corresponding to the user interaction information in different time ranges respectively includes: Dividing the interaction information into multiple time intervals at predetermined time intervals according to the timestamps of all user interaction information; Count the number of high-frequency keywords in each time interval to determine the highest number of high-frequency keywords in each time interval; The high-frequency keyword with the highest number is used as the high-frequency keyword corresponding to each time interval.

7. The method according to claim 6, characterized in that The method further comprises: Based on the number of the high-frequency keywords in a plurality of consecutive time intervals, the change rate of the high-frequency keywords is determined, and the high-frequency keywords whose change rate exceeds a predetermined value are eliminated.

8. A video annotation system, characterized in that: include: A video description generation module, used for inputting a target video into a multimodal large model to obtain a target video description of the target video; A classification label confirmation module, used to determine a target classification label corresponding to the target video description through a pre-trained classification model, wherein the classification model is generated based on the video description output by the multimodal large model; A video annotation module is used to annotate the target video based on the target classification label.

9. A computer device: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein: The processor is configured to execute the video annotation method according to any one of claims 1 to 7.

10. A computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the video annotation method according to any one of claims 1 to 7.

Images