Video tag generation method and device, model training method and device, equipment and medium

Abstract

The invention provides a video tag generation method and device, a model training method and device, equipment and a medium. The video tag generation method comprises the following steps: acquiring representation information of a target video, and processing the representation information by using a video classification model to obtain a target video content type of the target video; determining a target cue word template corresponding to the target video content type according to a pre-constructed corresponding relationship between the video content type and the cue word template; wherein the cue word templates are constructed based on information extraction requirements of the corresponding video types, and different cue word templates respectively point to structured texts of exclusive semantic contents of the corresponding video types; according to the representation information and a target cue word template, generating a target cue word corresponding to the target video; and taking the target prompt word and the video material information of the target video as input of a large language model, and enabling the large language model to output a video tag of the target video. According to the invention, the video tag generation accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video tag generation method, model training method, apparatus, device and medium. Background Technology

[0002] Video content tags (such as short videos, TV series, and movies) are widely used in recommendation systems as a fine-grained representation of video display content. For example, a video with the title "So-and-so responds to pilot program where delivery riders help throw away trash: Voluntary participation with a platform subsidy of 0.5 yuan per order" would be tagged with: "So-and-so," "platform subsidy," and "delivery rider." These tags are used in various stages of the recommendation system, including user profiling, user recall, and ranking.

[0003] Previous methods primarily learned manually labeled content tags from end-to-end generative models (such as transformers) to give them a certain generalization ability before generating video content tags. The main problem with this method is: 1. Manually labeling data is time-consuming and labor-intensive. Due to the large number and rich content of videos, labeling data that allows the model to learn fully requires a significant investment of human and material resources.

[0004] 2. Content tags are an open set, and determining which words can serve as content tags is quite difficult. Because different annotators have varying levels of understanding, the quality of the final annotations and the learning performance of the model are difficult to guarantee. Summary of the Invention

[0005] The purpose of this application is to provide a video tag generation method, model training method, apparatus, device, and medium to ensure the accuracy of generated video tags. The specific technical solution is as follows: In a first aspect of this application, a video tag generation method is provided, comprising: Obtain the representation information of the target video, and process the representation information using a video classification model to obtain the target video content type of the target video; Based on the pre-built correspondence between video content types and prompt word templates, the target prompt word template corresponding to the target video content type is determined; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; Based on the representation information and the target prompt word template, generate the target prompt word corresponding to the target video; The target prompt words and the video material information of the target video are used as input to the large language model, so that the large language model outputs the video tag of the target video.

[0006] A second aspect of this application provides a model training method, comprising: Obtain the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video; Based on the correspondence between video content types and prompt word templates, determine the sample prompt word templates corresponding to the sample video content types; Based on the sample prompt word template and the representation information, the sample prompt words for the sample video are determined; The sample prompt words and the video material information of the sample video are input into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video; The loss value of the initial large language model is calculated based on the labeled video tags and the predicted video tags; The model parameters of the initial large language model are updated based on the loss value, and the initial large language model is iteratively trained until it converges, thus obtaining the final large language model for generating video tags.

[0007] In a third aspect of this application, a video tag generation apparatus is provided, comprising: The video information acquisition module is used to acquire the representation information of the target video and process the representation information using a video classification model to obtain the target video content type of the target video. The target template determination module is used to determine the target prompt word template corresponding to the target video content type based on the pre-built correspondence between video content types and prompt word templates; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; The target prompt word generation module is used to generate target prompt words corresponding to the target video based on the representation information and the target prompt word template; The video tag acquisition module is used to take the target prompt words and the video material information of the target video as input to the large language model, so that the large language model outputs the video tags of the target video.

[0008] In a fourth aspect of this application, a model training apparatus is provided, comprising: The basic information acquisition module is used to acquire the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video. The sample template determination module is used to determine the sample prompt word template corresponding to the sample video content type based on the correspondence between the video content type and the prompt word template. The sample prompt word determination module is used to determine the sample prompt words of the sample video based on the sample prompt word template and the representation information; The predicted tag acquisition module is used to input the sample prompt words and the video material information of the sample video into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video. The loss value calculation module is used to calculate the loss value of the initial large language model based on the labeled video tags and the predicted video tags. The model acquisition module is used to update the model parameters of the initial large language model based on the loss value, and to iteratively train the initial large language model until the initial large language model converges, thereby obtaining the final large language model for generating video tags.

[0009] In another aspect of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements any of the video tag generation methods or model training methods described above.

[0010] In another aspect of this application, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform any of the video tag generation methods or model training methods described above.

[0011] In another aspect of this application, a computer program product containing instructions is also provided, on which a computer program is stored, which, when run on a computer, causes the computer to execute any of the video tag generation methods or model training methods described above.

[0012] The solution provided in this application generates video prompts by pre-constructing prompt word templates for different video types, and then processes these prompts using a large-scale model to generate video tags. This process leverages the powerful content understanding capabilities of the large-scale model to generate video content tags, ensuring the accuracy of the generated video tags. Furthermore, by combining a pre-built video tag library and retaining specific tags as needed, the accuracy of the generated video tags is further improved. Compared to manual annotation, this method saves labor costs and reduces the time required for video tag generation. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0014] Figure 1 A flowchart illustrating the steps of a video tag generation method provided in this application embodiment; Figure 2 A flowchart illustrating the steps of a method for obtaining text description information provided in this application embodiment; Figure 3 A flowchart illustrating the steps of a method for obtaining target prompt words provided in an embodiment of this application; Figure 4 A flowchart illustrating the steps of a method for obtaining target video tags provided in this application embodiment; Figure 5 A flowchart illustrating the steps of another method for obtaining target video tags provided in this application embodiment; Figure 6 A flowchart illustrating the steps of another method for obtaining target video tags provided in this application embodiment; Figure 7 A schematic diagram illustrating a video content tag generation process based on a large model, provided for an embodiment of this application; Figure 8 A flowchart illustrating the steps of a model training method provided in this application embodiment; Figure 9 This is a schematic diagram of the structure of a video tag generation device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0016] Figure 1A flowchart illustrating the steps of a video tag generation method provided in this application embodiment is shown below. Figure 1 As shown, the video tag generation method may include steps 101, 102, 103, and 104.

[0017] Step 101: Obtain the representation information of the target video and the target video content type of the target video obtained by processing the representation information by the video classification model.

[0018] In this embodiment, the target video refers to the specific video content for which tags need to be generated, which can originate from various platforms (such as video platforms, local storage, etc.) and includes visual images, audio (optional), subtitles (optional), and other information. In this example, the target video is preferably a video.

[0019] Representational information refers to information that reflects the core features of a target video and serves as the foundational data for determining video content type and generating prompts. In this example, representational information may include, but is not limited to, textual descriptions and video frame elements.

[0020] The text description information refers to a textual summary of the core content of the target video, which is a set of key information extracted from the video. In this example, the text description information may include: title, description (such as introduction, tags, etc.), and OCR (Optical Character Recognition) text in the video frames (such as subtitles, text labels in the frame), etc.

[0021] The target video content type refers to the standardized classification of the category to which the target video content belongs. In this example, the target video type can be, but is not limited to, sports, games, etc.

[0022] When generating video tags for a target video, representational information about the target video can be obtained. For example, for the text description information of the target video: video metadata can be obtained, such as directly reading the title, description, and original tags provided by the platform. OCR recognition is performed on video frames to obtain text information (such as billboards, title text). Title, description, and other information are extracted from the video metadata and combined with the OCR-recognized information to obtain the text description information of the target video. For the video image information of the target video: keyframe information can be extracted to obtain video images, etc.

[0023] The process of obtaining text description information will be combined in the following embodiments. Figure 2 A detailed description will not be repeated here.

[0024] Regarding the target video content type: a video classification model can be used to process the representation information to obtain the target video content type. Specifically, a pre-trained video classification model (such as a BERT-based classifier) ​​can be used as input, taking the representation information of the target video as input and outputting the most probable video content type (e.g., the model predicts the probability of "food preparation" to be 0.92, thus determining the type as "food preparation"), which can then be used as the target video content type.

[0025] Step 102: Based on the pre-built correspondence between video content types and prompt word templates, determine the target prompt word template corresponding to the target video content type; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type.

[0026] A cue word template is a structured text framework designed for a specific video type to standardize the generation logic of target cue words. The template contains fixed fields (type-related guidance information) and variable fields (dynamic information to be extracted from the text description), and its function is to transform video content into text that meets the input requirements of the tag generation model.

[0027] In this example, the prompt word template can be constructed based on the information extraction requirements of the video type. Different prompt word templates point to the exclusive semantic content of the corresponding video type (i.e., the set of semantic information unique to a specific video content type that reflects the core attributes and information transmission focus of that type).

[0028] In practical implementation, due to the significant differences in video content, the content can be pre-categorized into 22 categories: food, music, sports, news, games, military, children's content, automobiles, etc. Different prompt templates can be built based on different content categories, allowing for more targeted extraction of content tags for the corresponding videos. An example is shown below: 1. For sports categories, the main content to be extracted in the prompt is: event name (XXX football match, XX basketball match), people, brands, product names (such as XX shoes, table tennis rackets, badminton rackets, tennis rackets), themes (such as own goal, Grand Slam, retirement), and sports type (basketball, football, table tennis).

[0029] 2. For the game category, the prompt specifies that the main content to be extracted is: game IP name, event name, game characters, well-known figures or teams, video highlights (such as key plot points, game item-related content, famous scenes, funny commentary, etc.).

[0030] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0031] In this embodiment, a correspondence between prompt word templates and video content types can be established in advance. After obtaining the target video content type of the target video, the target prompt word template corresponding to the target video content type can be identified based on the correspondence.

[0032] Step 103: Generate target prompts corresponding to the target video based on the representation information and the target prompt template.

[0033] Target cue words refer to the complete text generated after filling the target cue word template with representational information, and they serve as input to the tag generation model. Their function is to organize the core information of the video according to the template structure, helping the model generate tags more accurately. For example, if the representational information is extracted as "Topic: Rice Cooker Cake Making, Steps: 5 steps, Ingredients: Eggs, Flour," after filling the "Food Making" template, the target cue words might be: "Video Theme: Rice Cooker Cake Making, Core Content: Making a Cake in 5 Steps, Key Elements: Eggs, Flour, Tag Direction: Food, Tutorial," etc.

[0034] After obtaining the representation information and target cue word template of the target video, target cue words for the target video can be generated based on these information and template. Specifically, the video content to be extracted can be determined from the cue word template, and the corresponding target cue words can be obtained by filling the template with the cue word information. The process of generating target cue words will be discussed in the following embodiments. Figure 3 A detailed description will not be repeated here.

[0035] Step 104: Use the target prompt word and the video material information of the target video as input to the large language model, so that the large language model outputs the video tag of the target video.

[0036] Large language models are fine-tuned models that can be used to generate video tags, typically built using natural language processing techniques. The model input is target prompt words, and the output is a set of tags related to the video content (video tags).

[0037] The fine-tuning process for the large language model will be described in detail in the following embodiments, and will not be repeated here.

[0038] Video tags refer to the direct output of a large language model; they are tags generated by the model after processing target prompt words.

[0039] After obtaining the target keywords for the target video, the target keywords and video footage information can be input into a large language model. The large language model then processes the target keywords and video footage information to output video tags for the target video. Specifically, the large language model first parses the semantic guidance of the target keywords and the video footage information, deeply integrates the two to establish semantic relationships, extracts core information that fits the specific semantics, generates standardized tags according to a preset format, and performs deduplication, normalization, and verification before finally outputting the video tags.

[0040] The video tag generation method provided in this application generates video prompts by pre-constructing prompt word templates for different video types, and then processes these prompts using a large-scale model to generate video tags. This process leverages the powerful content understanding capabilities of the large-scale model to generate video content tags, ensuring the accuracy of the generated video tags. Furthermore, by combining a pre-constructed video tag library and retaining specified tags as needed, the accuracy of the generated video tags is further improved. Compared to manual annotation, this method saves labor costs and reduces the time required for video tag generation.

[0041] Next, combined Figure 2 The process of obtaining text description information of the target video is described in detail.

[0042] Reference Figure 2 The diagram illustrates a flowchart of a method for obtaining text description information according to an embodiment of this application. Figure 2 As shown, the method for obtaining text description information may include steps 201, 202, and 203.

[0043] Step 201: Extract the title text and description text of the target video from the metadata of the video library corresponding to the target video.

[0044] In this embodiment, the video library refers to a structured database or file system that stores video resources, including the video files themselves and associated metadata (such as title, description, duration, etc.), and is the data source for extracting title and description text.

[0045] Metadata refers to structured information related to videos recorded in a video library, used to describe the basic attributes of the videos. In this example, the core metadata includes title text and description text.

[0046] Title text refers to a brief summary of the core content of a video in the metadata, which can be used to quickly identify the theme of the video.

[0047] Description text refers to the detailed text in the metadata that describes the video content. It usually supplements information not covered in the title (such as content highlights, background, applicable scenarios, etc.).

[0048] When generating video tags for a target video, the target video's metadata in the video library can be located. Specifically, the corresponding metadata record can be retrieved from the video library using the target video's unique identifier (such as video ID or storage path). The video library can be built based on a database (such as MySQL or MongoDB) or a file system (such as a JSON file), and metadata is typically stored in association with the video file. Then, the content of the "Title" field can be read from the metadata entry. If the metadata has formatting issues (such as empty values ​​or garbled characters), it needs to be cleaned: empty values ​​are filled with "Unknown Title," and garbled characters are corrected through encoding conversion (such as UTF-8 correction). Additionally, the content of the "Description" field can be read from the metadata entries. If the description text is too long (e.g., exceeding 500 characters), it can be truncated or summarized; if there is duplicate content (e.g., keywords appearing multiple times), it can be optimized using deduplication algorithms (e.g., hash-based text deduplication).

[0049] Step 202: Extract text from the video frames of the target video using optical character recognition technology to obtain the video frame text.

[0050] Optical Character Recognition (OCR) is a technology that uses computers to recognize text in images and convert it into editable text. In this example, it can be used to extract text (such as subtitles, logos, titles, etc.) appearing in video frames.

[0051] Video frame text refers to the text content extracted from video frames using OCR technology, including subtitles and handwritten text in the video.

[0052] When extracting text from video frames of a target video, frame extraction can be performed first to ensure coverage of key content. Sampling strategies can include: 1. Fixed-interval sampling: Extracting frames at time intervals (e.g., 1 frame every 2 seconds) or frame intervals (e.g., 1 frame every 10 frames), suitable for videos with evenly distributed text. 2. Keyframe sampling: Extracting frames with significant content changes (e.g., transition frames, subtitle appearance frames) using video frame change detection algorithms (e.g., threshold judgment based on inter-frame differences), suitable for videos with text concentrated in specific segments (e.g., opening and closing credits, key explanatory frames).

[0053] Then, the extracted video frames can be optimized to improve the accuracy of OCR recognition. Optimization methods can include, but are not limited to, image noise reduction, contrast enhancement, and text region localization.

[0054] Finally, an OCR tool can be used to perform text recognition on the preprocessed video frame images to obtain the recognized video frame text.

[0055] Step 203: Use the title text, the description text, and the video frame text as the text description information.

[0056] After obtaining the title text, description text, and video frame text, these can be used as text description information.

[0057] This application embodiment extracts title and description text from video library metadata, combines OCR technology to recognize text in video frames, integrates them to form comprehensive text description information, generates standardized input based on video type matching exclusive prompt word templates, generates video tags with the help of pre-trained models, and optimizes them through standardized tag library filtering. Ultimately, it realizes the automated generation of accurate and standardized target tags from video content, effectively improving the coverage, accuracy and standardization of video tags, and providing high-quality structured data support for downstream applications such as video retrieval and recommendation.

[0058] Next, combined Figure 3 The process of generating target prompts is described in detail.

[0059] Reference Figure 3 The diagram illustrates a flowchart of a target prompt word acquisition method provided in an embodiment of this application. Figure 3 As shown, the method for obtaining target prompt words may include steps 301 and 302.

[0060] Step 301: Obtain wildcards for the specified topic content on the target prompt word template.

[0061] In this embodiment, the wildcard for specifying the topic content refers to the placeholder symbol in the target prompt word template used to refer to dynamic information related to the core topic of the video (usually marked in a specific format, such as {topic}{core content}{key element} etc.), corresponding to the key information items (such as topic, steps, ingredients, etc.) that need to be extracted from the representation information in the video content.

[0062] After obtaining the target prompt word template corresponding to the target video type, the target prompt word template can be structured and parsed. Wildcards can be identified through preset rules (such as wildcard format conventions, such as text enclosed in {}). That is, wildcards for the specified topic content on the target prompt word template can be obtained.

[0063] Step 302: Fill the corresponding position of the target prompt word template with the representation information, replace the corresponding wildcard, and obtain the target prompt word.

[0064] Next, the representation information can be filled into the corresponding positions in the target prompt word template, replacing the corresponding wildcards to obtain the target prompt words. Specifically, key information can be extracted from the representation information, and natural language processing tools (such as entity recognition and keyword matching) can be used to locate the content corresponding to the wildcards. For example, from the text description "Title: Making Cake in a Rice Cooker, Ingredients: Eggs, Flour, Steps: 5 steps", extract "Topic = Making Cake in a Rice Cooker", "Ingredients = Eggs, Flour", "Number of Steps = 5 steps", etc.

[0065] Then, the extracted information can be filled into the wildcard positions of the target prompt template according to the corresponding relationships. If a wildcard has no corresponding information (such as no ingredient mentioned), it is filled with "unknown" or an empty value to ensure the integrity of the template structure. For example, after filling the template with the above information, we get: "Video theme: Rice cooker cake making, key elements: eggs, flour, steps: 5 steps", etc. Finally, we can check whether there are any formatting errors in the filled text (such as wildcards not being completely replaced or content being repeated), and generate the final target prompt by deduplication and completion processing to ensure that the text logic is coherent and the information is complete.

[0066] This application embodiment achieves automated generation of standardized target prompt words from unstructured video text by accurately identifying topic wildcards in the prompt word template and dynamically filling and replacing them with representation information. This effectively ensures the semantic matching degree and format standardization of the prompt words and video content, provides high-quality input for subsequent tag generation models, and improves the accuracy and efficiency of the overall process.

[0067] Next, combined Figure 4 The process of obtaining target video tags is described in detail.

[0068] Reference Figure 4 The diagram illustrates a flowchart of a target video tag acquisition method provided in an embodiment of this application. Figure 4 As shown, the method for obtaining the target video tag may include steps 401 and 402.

[0069] Step 401: Match the video tag with the tags in the video tag library to obtain the video tags to be deleted that are not included in the video tag library.

[0070] In this embodiment, the video tag library refers to a standardized set of structured tags, which is a "dictionary" or "specification table" of tags.

[0071] In this example, the video tag library for the videos can be pre-built by business personnel, and mainly consists of the following parts: 1. Named Entities: Long video album names such as movies, TV series, variety shows, and animations; actor names; director names; film, television, animation, and variety show character names; game names; song names; well-known figures in various fields; brand names; place names; dish names, etc.

[0072] 2. Terminology from various professional fields, such as: artificial intelligence, etc.

[0073] 3. Various themes / hot keywords, such as: custom home furnishings, whole-house customization, etc.

[0074] Target video tags refer to the standardized tags that are finally output after filtering. They must meet the following requirements: they must be valid tags in the video tag library, be free of redundancy, and accurately reflect the core content of the video.

[0075] After obtaining the video tags, they can be filtered based on a pre-built video tag library to obtain the target video tags. Specifically, the video tags can be matched with tags in the video tag library; those that do not match are deleted, and the remaining video tags are the target video tags.

[0076] In practical applications, the content tags generated by large language models often differ from business requirements. Therefore, a pre-built content tag library based on business needs is used to ensure that only tags from this library are output. For example: Video title: [Literacy Class] Basic Literacy 16.

[0077] Video description: Kids, let's learn the Chinese characters for "brother, younger brother, older sister, younger sister, uncle, and elder brother" together.

[0078] The tags output by the large model are: literacy class, literacy basics, brother, younger brother, older sister, younger sister, uncle, elder brother.

[0079] Among them, the semantics of tags such as "brother, younger brother, older sister, younger sister, uncle, and elder brother" are too general and not required by the business, so they will not be included in the content tag library and will be filtered out.

[0080] After obtaining the video tags, the video tags can be matched with tags in the video tag library to obtain the video tags that are not included in the video tag library and are to be deleted.

[0081] Step 402: Delete the video tag to be deleted from the video tag to obtain the target video tag of the target video.

[0082] Then, the video tag to be deleted can be removed from the video tags to obtain the target video tag. That is, tags that are not in the tag library are filtered.

[0083] This application embodiment accurately matches video tags with a standardized tag library, eliminates non-standard tags not in the library, effectively filters redundant and invalid content, and finally outputs target video tags that conform to business standards, thereby improving the standardization and reliability of tags and providing high-quality structured data support for video classification, retrieval and other applications.

[0084] In this embodiment, after obtaining the video tags, the video tags can first be filtered using a video tag library, and then a semantic aggregation algorithm can be used to aggregate the filtered video tags to obtain the final target video tags. This implementation process will be discussed in conjunction with the following embodiments. Figure 5 Provide a detailed description.

[0085] Reference Figure 5 The diagram illustrates a flowchart of another method for obtaining target video tags provided in an embodiment of this application. Figure 5 As shown, the method for obtaining the target video tag may include steps 501 and 502.

[0086] Step 501: Use a semantic aggregation algorithm to perform synonym tag aggregation on the video tags to obtain the target video tags for the target video.

[0087] Semantic aggregation algorithms are algorithms based on natural language processing technology that identify synonymous or near-synonymous tags and perform clustering by calculating the semantic similarity between tags (such as word vector similarity and cosine similarity).

[0088] After obtaining the video tags of the target video, a semantic aggregation algorithm can be used to perform synonym aggregation processing on the video tags to obtain the target video tags. That is, merging synonym or near-synonymous tags in the video tags reduces redundancy and enhances the accuracy and conciseness of the tags. The specific implementation process of the semantic aggregation algorithm will be combined in the following embodiments. Figure 6 A detailed description will not be repeated here.

[0089] Reference Figure 6 This illustrates a flowchart of another target video tag acquisition method provided in an embodiment of this application. Figure 6 As shown, the method for obtaining the target video tag may include steps 601, 602, 603, 604, 605, 606, and 607.

[0090] Step 601: Obtain the video tag vector corresponding to the video tag, and the lexical information of the video tag.

[0091] In this embodiment, lexical information refers to the surface morphology and structural features of the tag text. In this example, lexical information may include: the co-occurrence probability of characters in the video tag, part of speech (noun, verb, etc.), thesaurus, and other information.

[0092] After obtaining the video tags of the target video, the corresponding video tag vectors can be acquired. Specifically, the video tags can be standardized (e.g., removing special characters and unifying capitalization) to ensure the text format is consistent with the model input requirements. Then, each preprocessed video tag is input into a pre-trained model, which outputs a corresponding high-dimensional vector (e.g., Word2Vec generates a 300-dimensional vector), forming a "tag-vector" mapping table. Simultaneously, lexical information of the video tags can be obtained, such as by analyzing the co-occurrence probability of characters in the video tags, parts of speech (nouns, verbs, etc.), and a thesaurus, to analyze the lexical information between video tags.

[0093] Step 602: Calculate the semantic similarity between each video tag based on the video tag vector.

[0094] Semantic similarity refers to the degree of semantic similarity between two video tags. It is calculated by comparing the similarity of their corresponding video tag vectors (the value range is usually 0-1, and the higher the value, the closer the semantics).

[0095] After obtaining the video tag vectors of the video tags, the semantic similarity between each video tag can be calculated based on the video tag vectors. For example, cosine similarity can be used to calculate the cosine similarity between pairwise video tag vectors, which can then be used as the semantic similarity.

[0096] Step 603: Calculate the lexical similarity between each video tag based on the lexical information.

[0097] After obtaining the lexical information of each video tag, the video tags can be combined in pairs, and multi-dimensional lexical similarity (such as edit distance similarity, stemming degree, thesaurus matching degree, etc.) can be calculated based on the lexical information, and then fused into a comprehensive lexical similarity.

[0098] Step 604: The semantic similarity and the lexical similarity are fused according to preset weights to obtain the tag similarity.

[0099] After obtaining semantic similarity and lexical similarity, the semantic similarity and lexical similarity can be fused according to preset weights to obtain tag similarity. Specifically, the semantic similarity and lexical similarity corresponding to each pair of tags can be weighted and summed to obtain a comprehensive tag similarity.

[0100] Step 605: Filter out synonymous video tags from the video tags whose tag similarity is greater than the similarity threshold.

[0101] The similarity threshold is a pre-set critical value (such as 0.7) used to determine whether two tags are synonymous. When the similarity between two tags is greater than this threshold, they are determined to be synonymous tags.

[0102] Synonymous video tags refer to combinations of tags in videos whose tag similarity is greater than a similarity threshold, that is, tags that express similar meanings.

[0103] After obtaining the tag similarity, synonymous video tags with a similarity greater than the similarity threshold can be filtered out from the video tags.

[0104] Step 606: Aggregate the synonymous video tags to obtain aggregated video tags.

[0105] Aggregated video tags refer to the representative tags retained after aggregating synonymous video tags. Typically, the most standardized and highest-priority tag is selected from the synonym tags as the representative.

[0106] After obtaining synonymous video tags, these tags can be aggregated to obtain aggregated video tags. This involves merging synonymous tags, retaining representative tags, and reducing redundancy. For example, synonymous tags such as "International Workers' Day," "May Day Labor Day," "Labor Day," and "May Day" have the same literal meaning. If semantic aggregation can be performed—that is, if synonymous tags are identified and normalized, such as using "May Day Labor Day" in the previous example—then the business side will benefit from better usage.

[0107] Step 607: Take the aggregated video tag and other video tags in the video tags other than the synonymous video tag as the target video tag.

[0108] After obtaining the aggregated video tags, the aggregated video tags and other video tags (excluding synonymous video tags) can be used as target video tags.

[0109] This application embodiment converts video tags into vectors and calculates semantic similarity and lexical similarity based on the tag vectors and lexical information, respectively. It then calculates the comprehensive tag similarity through a weighted method, accurately identifies synonymous tags, and aggregates them into representative tags. Finally, it outputs target video tags without redundancy and with accurate semantics, which effectively improves the simplicity and standardization of tags and provides higher quality structured data support for applications such as video retrieval and recommendation.

[0110] Next, combined Figure 7 A detailed description of the complete video tag generation process is provided. For example... Figure 7As shown, the video tag generation process may include: 1. Basic data acquisition stage: Extracting video media: Extracting video media from the video metadata database as the objects of processing. These media will be used for further operations such as extracting video frames.

[0111] Video frame extraction: Based on the acquired video medium, video frames are extracted to prepare for subsequent image-related processing (such as combining OCR to extract text information).

[0112] Extracting video metadata: Extracting video metadata from the video metadata database. This metadata is broken down into data from different dimensions such as cover image, title, and description, providing a basis for generating content tags from multiple perspectives.

[0113] 2. Information-assisted processing and classification preparation stage: OCR processing: Using a video metadata database, OCR (Optical Character Recognition) operations are performed on extracted video frames (or related image content) to extract text information from the images, enriching the data dimensions used for analyzing video content.

[0114] Video content classification model application: The acquired and processed information is input into the video content classification model, which determines the content category of the video based on the built-in algorithm and classification logic.

[0115] Output content category: After the video content classification model runs, it outputs the specific content category of the video obtained by the model. Subsequently, the appropriate prompt will be selected from the prompt set based on this category.

[0116] 3. Prompt generation and large model invocation stage: Generate corresponding category prompts: Based on the content category output by the video content classification model, select corresponding templates from the prompt set, and generate corresponding prompts by combining the text description information of the video. This prompt will be used as an instruction to input into the large model to guide the large model to generate content tags.

[0117] Large model service call: Input the generated corresponding category prompt into the large model service, and use the natural language processing and generation capabilities of the large model to generate content tags about the video based on the prompt and various video information obtained in the early stage.

[0118] 4. Tag optimization and output stage: Content tag filtering: The content tags generated by the large model are compared and filtered with a pre-defined set of content tags. Tags that do not meet business requirements (such as not being in the tag set or being semantically irrelevant) are filtered out, while valid tags are retained.

[0119] Content tag aggregation: Aggregate the effective content tags obtained after filtering, such as merging tags with similar semantics or repetitive meanings, to make the tag system more concise and accurate.

[0120] Output: After completing all the above processing, the final optimized video content tags are output. These tags can be used for various business scenarios such as video classification, retrieval, and recommendation.

[0121] This application can improve the quantity and quality of video content tags, thereby increasing business metrics such as average viewing time per user in video distribution scenarios. Typical scenarios include: 1. In video services, optimizing video content tags can further improve the quality of user profiles, expand recommendation recall, and add more features to the ranking model to enhance recommendation performance. 2. In video search scenarios, optimizing video content tags can further improve the relevance of queried videos, thereby increasing search accuracy.

[0122] This application's embodiments utilize a large model to understand the text content of videos, saving the manpower and resources required for data annotation in traditional methods, while generating high-quality content tags. By optimizing the prompt, more targeted content tags can be extracted for different content. Building a content tag library and filtering the tags generated by the large model can improve the accuracy of the large model's tags. Semantic aggregation of tags can further improve the quality of the tags and the efficiency for business users.

[0123] Reference Figure 8 The diagram illustrates a flowchart of a model training method provided in an embodiment of this application. Figure 8 As shown, the model training method includes steps 801 to 806.

[0124] Step 801: Obtain the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video.

[0125] In this embodiment, when training a large language model, basic sample data required for model training can be collected, including input features (video representation information, such as text descriptions and video content types) and annotation labels. Specifically, representative sample videos can be selected from the business video library, covering different scenes, types (such as food, technology, and lifestyle) and lengths to ensure sample diversity.

[0126] Then, through the text extraction process described above (metadata title / description + OCR video frame text), a structured text description of each sample video is obtained.

[0127] The matching prompt word templates are based on the content type of each sample video (such as "food preparation" or "technology tutorial"), either manually labeled or through a mature video classification model.

[0128] Business experts or labeling teams create accurate, standardized labels based on sample video content and a business tag library.

[0129] Step 802: Determine the sample prompt word template corresponding to the sample video content type based on the correspondence between the video content type and the prompt word template.

[0130] After obtaining the sample video content type, the corresponding sample prompt word template can be determined based on the correspondence between the video content type and the prompt word template. This involves matching the sample video with the corresponding prompt word template, ensuring that the subsequently generated sample prompt words have a consistent format and conform to the model input specifications.

[0131] Step 803: Determine the sample prompt words for the sample video based on the sample prompt word template and the representation information.

[0132] After obtaining the sample prompt word template, the sample prompt words for the sample video can be determined based on the template and the representation information. This involves filling the corresponding positions in the sample prompt word template with the representation information of the sample prompt words to obtain the sample prompt words.

[0133] Step 804: Input the sample prompt words and the video material information of the sample video into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video.

[0134] After obtaining the sample prompts, the sample prompts and video footage information from the sample video can be input into the initial large language model. The implementation process can be as follows: 1. Text preprocessing: Use the model's matching tokenizer (such as BERT's tokenizer) to segment the sample prompt words, map them to word list IDs, and add special symbols (such as [CLS] to indicate the beginning of the sentence and [SEP] to indicate the end of the sentence).

[0135] 2. Sequence length processing: Truncate or pad the preprocessed token sequence with zeros to the fixed length specified by the model (e.g., 512 tokens) to ensure consistent input dimensions.

[0136] 3. Input Model: Convert the processed token ID sequence, attention mask (marking the position of valid tokens), etc. into tensors and input them into the model to be trained (such as a label generation model based on GPT or BERT). The model generates the probability distribution of predicted labels through an encoder-decoder structure.

[0137] After inputting the sample prompts and video clips into the initial large language model, the model processes the information to obtain the predicted video tags corresponding to the sample videos. Specifically, the initial large language model outputs the predicted probabilities of all tags in the tag library (e.g., for 1000 tags in the tag library, each tag corresponds to a probability value of 0-1, representing the model's confidence that the tag is related to the video).

[0138] Predicted labels can be selected from the output probabilities by setting a probability threshold (e.g., 0.5) or by taking the Top-K (e.g., the top 5 with the highest probabilities). Then, the predicted labels can be converted into a format consistent with the labeled labels (e.g., a list format) for easier subsequent loss calculation.

[0139] Step 805: Calculate the loss value of the initial large language model based on the labeled video tags and the predicted video tags.

[0140] After obtaining the predicted video labels, the loss value of the initial large language model can be calculated based on the labeled video labels and the predicted video labels. This quantifies the difference between the model's predictions and the actual labels, providing a basis for updating the model parameters. The specific implementation process may include: 1. Tag Vectorization: Convert the labeled video tags and predicted video tags into binary vectors of "tag library dimension" (e.g., if the tag library has 1000 tags, the vector length is 1000; if a tag exists, the corresponding position is 1, otherwise it is 0).

[0141] 2. Select a loss function: Choose an appropriate loss function based on the task type, such as binary cross-entropy loss.

[0142] 3. Calculate the loss value: Substitute the labeled vector and the predicted probability vector into the loss function to obtain the loss value of a single sample; average the loss values ​​of all samples in the batch to obtain the total loss value of the current training batch.

[0143] Step 806: Update the model parameters of the initial large language model based on the loss value, and iteratively train the initial large language model until it converges to obtain the final large language model.

[0144] After calculating the loss value, the model parameters of the initial large language model can be updated based on the loss value, and the initial large language model can be iteratively trained until the initial large language model converges, thus obtaining the final large language model.

[0145] In practice, optimizers (such as Adam or SGD) can be used to calculate the gradient of the model parameters based on the loss value (using the backpropagation algorithm), and the parameters are updated according to the gradient direction. Steps 801-806 are repeated, inputting training samples in batches to continuously update the model parameters. Simultaneously, the model performance (such as label prediction accuracy and F1 score) is periodically evaluated on the validation set (samples independent of the training set).

[0146] Training will stop when one of the following conditions is met: 1. The training loss and validation loss no longer decrease for several consecutive rounds (e.g., 10 rounds) (or the decrease is less than the threshold, e.g., 0.001). 2. Reach the preset maximum number of training rounds (e.g., 100 rounds); 3. The validation set performance (such as F1 score) meets the business expectations.

[0147] The converged model parameters, tokenizer, etc., are saved as a deployable model file, which serves as the final large language model for generating the video Bai Qian.

[0148] The model training method provided in this application generates accurate prompt words by combining the representation information of the video with prompt word templates of the corresponding video content type, drives model learning, and then iteratively optimizes the model parameters with the loss values ​​of labeled and predicted labels. In the end, it can efficiently train a convergent and accurate large language model, and improve the accuracy and adaptability of automatic video label generation.

[0149] Reference Figure 9 The diagram shows a structural schematic of a video tag generation device provided in an embodiment of this application. Figure 9 As shown, the video tag generating device 900 may include the following modules: The video information acquisition module 910 is used to acquire the representation information of the target video and process the representation information using a video classification model to obtain the target video content type of the target video. The target template determination module 920 is used to determine the target prompt word template corresponding to the target video content type based on the pre-built correspondence between video content types and prompt word templates; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; The target prompt word generation module 930 is used to generate target prompt words corresponding to the target video based on the representation information and the target prompt word template; The video tag acquisition module 940 is used to take the target prompt word and the video material information of the target video as input to the large language model, so that the large language model outputs the video tag of the target video.

[0150] Optionally, the representation information includes at least the textual description information of the target video. The video information acquisition module includes: The text extraction unit is used to extract the title text and description text of the target video from the metadata of the video library corresponding to the target video; The video text acquisition unit is used to extract text from video frames of the target video based on optical character recognition technology to obtain video frame text. The text description acquisition unit is used to use the title text, the description text, and the video frame text as the text description information.

[0151] Optionally, the target prompt word generation module includes: Wildcard acquisition unit, used to acquire wildcards for the specified topic content on the target prompt word template; The target prompt acquisition unit is used to fill the representation information into the corresponding position of the target prompt word template, replace the corresponding wildcard, and obtain the target prompt word.

[0152] Optionally, the device further includes: The tag matching module is used to match the video tags with tags in the video tag library to obtain video tags that are not included in the video tag library and are to be deleted. The first tag acquisition module is used to delete the video tag to be deleted from the video tags to obtain the target video tag of the target video.

[0153] Optionally, the device further includes: The second tag acquisition module is used to perform synonym tag aggregation processing on the video tags using a semantic aggregation algorithm to obtain the target video tags of the target video.

[0154] Optionally, the second tag acquisition module includes: The lexical information acquisition unit is used to acquire the video tag vector corresponding to the video tag, and the lexical information of the video tag; A semantic similarity calculation unit is used to calculate the semantic similarity between each video tag based on the video tag vector; The lexical similarity calculation unit is used to calculate the lexical similarity between each of the video tags based on the lexical information. The tag similarity acquisition unit is used to fuse the semantic similarity and the lexical similarity according to a preset weight to obtain the tag similarity; The synonym tag filtering unit is used to filter out synonymous video tags from the video tags whose tag similarity is greater than a similarity threshold; The aggregation tag acquisition unit is used to aggregate the synonymous video tags to obtain aggregated video tags; The target tag acquisition unit is used to take the aggregated video tag and other video tags in the video tags other than the synonymous video tag as the target video tag.

[0155] The video tag generation device provided in this application generates video prompts by pre-constructing prompt word templates for different video types, and then processes these prompts using a large-scale model to generate video tags. This process leverages the powerful content understanding capabilities of the large-scale model to generate video content tags, ensuring the accuracy of the generated video tags. Furthermore, by combining a pre-constructed video tag library and retaining specified tags as needed, the accuracy of the generated video tags is further improved. Compared to manual tagging, this method saves labor costs and reduces the time required for video tag generation.

[0156] Reference Figure 10 The diagram shows a schematic representation of a model training apparatus provided in an embodiment of this application. Figure 10 As shown, the model training device 1000 may include the following modules: The basic information acquisition module 1010 is used to acquire basic video information of the sample video, including: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tag corresponding to the sample video. The sample template determination module 1020 is used to determine the sample prompt word template corresponding to the sample video content type based on the correspondence between the video content type and the prompt word template. The sample prompt word determination module 1030 is used to determine the sample prompt words of the sample video based on the sample prompt word template and the representation information; The prediction tag acquisition module 1040 is used to input the sample prompt words and the video material information of the sample video into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video. The loss value calculation module 1050 is used to calculate the loss value of the initial large language model based on the labeled video tags and the predicted video tags. The model acquisition module 1060 is used to update the model parameters of the initial large language model based on the loss value, and to iteratively train the initial large language model until the initial large language model converges, thereby obtaining the final large language model for generating video tags.

[0157] The model training device provided in this application generates accurate prompt words by combining the representation information of the video with the prompt word template of the corresponding video content type, drives the model to learn, and then iteratively optimizes the model parameters with the loss values ​​of the labeled and predicted labels. Finally, it can efficiently train a convergent and accurate large language model, and improve the accuracy and adaptability of automatic video label generation.

[0158] This application also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104. Memory 1103 is used to store computer programs; When processor 1101 executes the program stored in memory 1103, it performs the following steps: Obtain the representation information of the target video, and process the representation information using a video classification model to obtain the target video content type of the target video; Based on the pre-built correspondence between video content types and prompt word templates, the target prompt word template corresponding to the target video content type is determined; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; Based on the representation information and the target prompt word template, generate the target prompt word corresponding to the target video; The target prompt words and the video material information of the target video are used as input to the large language model, so that the large language model outputs the video tag of the target video.

[0159] When the processor 1101 executes the program stored in the memory 1103, it performs the following steps: Obtain the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video; Based on the correspondence between video content types and prompt word templates, determine the sample prompt word templates corresponding to the sample video content types; Based on the sample prompt word template and the representation information, the sample prompt words for the sample video are determined; The sample prompt words and the video material information of the sample video are input into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video; The loss value of the initial large language model is calculated based on the labeled video tags and the predicted video tags; The model parameters of the initial large language model are updated based on the loss value, and the initial large language model is iteratively trained until it converges, thus obtaining the final large language model for generating video tags.

[0160] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0161] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0162] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the video tag generation methods or model training methods described in the above embodiments.

[0163] In another embodiment provided in this application, a computer program product containing instructions is also provided, on which a computer program is stored, which, when run on a computer, causes the computer to execute any of the video tag generation methods or model training methods described above.

[0164] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0165] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0167] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for generating video tags, characterized in that, include: Obtain the representation information of the target video, and process the representation information using a video classification model to obtain the target video content type of the target video; Based on the pre-built correspondence between video content types and prompt word templates, the target prompt word template corresponding to the target video content type is determined; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; Based on the representation information and the target prompt word template, generate the target prompt word corresponding to the target video; The target prompt words and the video material information of the target video are used as input to the large language model, so that the large language model outputs the video tag of the target video.

2. The method according to claim 1, characterized in that, The representation information includes at least the textual description information of the target video. The acquisition of the representation information of the target video includes: Extract the title and description text of the target video from the metadata of the video library corresponding to the target video; Text is extracted from video frames of the target video based on optical character recognition technology to obtain video frame text. The title text, the description text, and the video frame text are used as the text description information.

3. The method according to claim 1, characterized in that, The step of generating target prompts corresponding to the target video based on the representation information and the target prompt template includes: Obtain wildcards for the specified topic content on the target prompt word template; The representation information is filled into the corresponding position of the target prompt word template, and the corresponding wildcard is replaced to obtain the target prompt word.

4. The method according to claim 1, characterized in that, After the large language model outputs the video tags for the target video, the method further includes: The video tags are matched with the tags in the video tag library to obtain the video tags that are not included in the video tag library and are to be deleted. The target video tag is obtained by deleting the video tag to be deleted from the video tag.

5. The method according to claim 1, characterized in that, After the large language model outputs the video tags for the target video, the method further includes: The video tags are aggregated using a semantic aggregation algorithm to obtain the target video tags for the target video.

6. The method according to claim 5, characterized in that, The step of using a semantic aggregation algorithm to perform synonym tag aggregation on the video tags to obtain the target video tags for the target video includes: Obtain the video tag vector corresponding to the video tag, and the lexical information of the video tag; Based on the video tag vectors, the semantic similarity between each video tag is calculated; Based on the lexical information, the lexical similarity between each video tag is calculated; The semantic similarity and the lexical similarity are fused according to preset weights to obtain the tag similarity; Filter out synonymous video tags from the video tags whose tag similarity is greater than a similarity threshold; The synonymous video tags are aggregated to obtain aggregated video tags; The aggregated video tag and other video tags in the video tags, excluding the synonymous video tag, are used as the target video tag.

7. A model training method, characterized in that, include: Obtain the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video; Based on the correspondence between video content types and prompt word templates, determine the sample prompt word templates corresponding to the sample video content types; Based on the sample prompt word template and the representation information, the sample prompt words for the sample video are determined; The sample prompt words and the video material information of the sample video are input into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video; The loss value of the initial large language model is calculated based on the labeled video tags and the predicted video tags; The model parameters of the initial large language model are updated based on the loss value, and the initial large language model is iteratively trained until it converges, thus obtaining the final large language model for generating video tags.

8. A video tag generation device, characterized in that, include: The video information acquisition module is used to acquire the representation information of the target video and process the representation information using a video classification model to obtain the target video content type of the target video. The target template determination module is used to determine the target prompt word template corresponding to the target video content type based on the pre-built correspondence between video content types and prompt word templates; wherein, the prompt word template is constructed based on the information extraction requirements of the corresponding video type, and different prompt word templates point to the structured text of the exclusive semantic content of the corresponding video type; The target prompt word generation module is used to generate target prompt words corresponding to the target video based on the representation information and the target prompt word template; The video tag acquisition module is used to take the target prompt words and the video material information of the target video as input to the large language model, so that the large language model outputs the video tags of the target video.

9. A model training device, characterized in that, include: The basic information acquisition module is used to acquire the basic video information of the sample video, which includes: the representation information corresponding to the sample video, the sample video content type of the sample video, and the labeled video tags corresponding to the sample video. The sample template determination module is used to determine the sample prompt word template corresponding to the sample video content type based on the correspondence between the video content type and the prompt word template. The sample prompt word determination module is used to determine the sample prompt words of the sample video based on the sample prompt word template and the representation information; The predicted tag acquisition module is used to input the sample prompt words and the video material information of the sample video into the initial large language model, so that the initial large language model outputs the predicted video tag corresponding to the sample video. The loss value calculation module is used to calculate the loss value of the initial large language model based on the labeled video tags and the predicted video tags. The model acquisition module is used to update the model parameters of the initial large language model based on the loss value, and to iteratively train the initial large language model until the initial large language model converges, thereby obtaining the final large language model for generating video tags.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

12. A computer program product containing instructions, wherein a computer program is stored thereon, characterized in that, When the computer program is run on a computer, it causes the computer to perform the method as described in any one of claims 1-7.

Images