Tag generation method and apparatus

Abstract

The present application provides a tag generation method, comprising: acquiring first content and supplementary data; determining, from the supplementary data, second content related to the first content; determining, from a tag set, a first tag related to the first content and the second content; and inputting the first content, the second content, and the first tag into a language model to obtain a second tag. By means of the tag generation method of the present application, a tag related to content can be selected from a tag set by means of a language model while both a tag recall rate and an accuracy rate are guaranteed.

Description

A label generation method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202411606881.1, filed on November 11, 2024, entitled "A Label Generation Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence, and more particularly to a label generation method and apparatus. Background Technology

[0003] Tags are a common technique used in search engines or content recommendation systems. Tags help search engines or content recommendation systems quickly and accurately understand user interests, search intent, and product characteristics, thereby simplifying the search or recommendation process.

[0004] Referring to Figure 1, a tagging system in the prior art uses a tag selector to filter corresponding tags for information such as news or videos from a tag set, and applies the tags to downstream tasks such as search or recommendation. Referring to Figure 2, in a search scenario for game applications, the tagging system generates a corresponding tag "King of Glory" for newly launched game applications, allowing users to quickly and accurately find the corresponding game application using the keyword "King of Glory".

[0005] Currently, tag selectors can be implemented through manual retrieval or small language models. However, the results of manual retrieval are related to the keywords entered by the user, which is subjective and costly. Small language models are limited by their limited understanding capabilities, resulting in unreliable results. In addition, they require a batch of labeled data for training, which is also costly.

[0006] With the rise of language models, how to use language models to select suitable tags from massive amounts of tags while taking into account both tag recall and accuracy is an urgent problem to be solved. Summary of the Invention

[0007] This application provides a tag generation method and apparatus that can filter content-related tags from a tag set using a language model while taking into account both recall and precision.

[0008] In a first aspect, this application provides a tag generation method, comprising: obtaining first content and supplementary data; determining second content related to the first content from the supplementary data; determining a first tag related to the first content and the second content from a tag set; inputting the first content, the second content and the first tag into a language model to obtain a second tag; and using the second tag as a tag for the first content.

[0009] In the tag generation method provided in this application, second content related to the first content is determined from supplementary data, and first tags related to the first and second content are determined from a tag set. These first tags are used as input tags for a language model. This method ensures the recall rate of the first tags input to the language model by introducing supplementary data, and ensures the accuracy rate of the first tags input to the language model by determining the first tags related to the first and second content from the tag set. This helps improve the recall and accuracy rate of the second tags generated by the language model based on the input tags. Furthermore, this application alleviates the illusion of the language model by injecting second content related to the first content determined from supplementary data, thereby improving the accuracy rate of the generated second tags. Therefore, the tag generation method provided in this application can filter content-related tags from the tag set through a language model while taking into account both tag recall and accuracy.

[0010] The first tag is used to provide candidate tags for the second tag, which is the final result tag. The second tag can be used in downstream tasks such as search engines or content recommendation to tag or update the tags of the first content in the downstream task.

[0011] Supplementary data comprises one or more of external knowledge, manually generated rules, and historical samples. External knowledge refers to information derived from non-label sets, including tags and website resources. Manually generated rules refer to manually specified tag generation rules. Historical samples refer to the historical input of the first content and the output of the second tag. In this application, supplementary data is used to first retrieve the first tag related to the first content from the tag set, and then use the first tag to guide the language model to generate the second tag related to the first content, thereby improving the recall and accuracy of the generated second tag.

[0012] As one possible implementation, determining a first tag related to the first and second content from the tag set specifically includes: determining a third tag related to the first content and a fourth tag related to the second content from the tag set; and determining the first tag based on the third and fourth tags. In this approach, since the second content originates from supplementary data, using the fourth tag related to the second content to determine the first tag helps improve the recall rate of the first tag. Furthermore, determining the first tag related to the first and second content from the tag set ensures the accuracy of the input tags for the language model.

[0013] As one possible implementation, determining third tags related to the first content from a tag set includes: retrieving third tags related to the first content from the tag set based on the type of the first content. In this approach, retrieving third tags related to the first content based on its type helps improve the accuracy of the retrieved third tags. Specifically, the type of the first content is first determined, then a first encoding model is determined based on the type of the first content, and finally, the third tags related to the first content are determined from the tag set using the first encoding model. The first encoding model can be one or more of a text encoding model, a keyword encoding model, a graph encoding model, and a speech encoding model.

[0014] As one possible implementation, determining a fourth tag related to the second content from a tag set includes: retrieving a fourth tag related to the second content from the tag set based on the type of the second content. In this approach, retrieving the fourth tag related to the second content by its type helps improve the accuracy of the retrieved fourth tag. Specifically, first, the type of the second content is determined; then, a second encoding model is determined based on the type of the second content; finally, the fourth tag related to the second content is determined from the tag set using the second encoding model. The second encoding model can be one or more of a text encoding model, a keyword encoding model, a graph encoding model, and a speech encoding model.

[0015] By using one or more of the text encoding model, keyword encoding model, graph encoding model, and speech encoding model related to the types of the first content and the second content, information matching is performed to determine the first tag related to the first content and the first content from the first tag set. The appropriate encoding model can be selected to generate the first tag according to the type of the first content and the type of the second content, which helps to improve the recall and accuracy of the first tag.

[0016] One possible implementation involves determining the first tag based on the third and fourth tags, including merging the third and fourth tags to generate the first tag. The third and fourth tags may contain duplicate tags; during merging, duplicate tags are deduplicated, and the first tag is generated based on the deduplicated third and fourth tags.

[0017] As one possible implementation, the first content, the second content, and the first tag are input into the language model to obtain the second tag. Specifically, this includes: obtaining a prompt word template; filling the prompt word template with the first content, the second content, and the first tag to obtain the corresponding prompt word; and inputting the prompt word into the language model to obtain the second tag.

[0018] One possible implementation involves inputting prompt words into a language model to obtain second labels. Specifically, this includes: inputting the first content, second content, and first label into the language model; outputting multiple fifth labels and a first confidence score corresponding to each fifth label; filtering fifth labels with a first confidence score less than a first threshold to obtain second labels; and ensuring that each fifth label belongs to the first label. This method can filter out second labels with higher confidence scores, improving the accuracy of the final generated second labels. The first confidence score can be transmitted along with the second label to downstream tasks, allowing them to use the second label appropriately based on the first confidence score.

[0019] One possible implementation involves inputting prompt words into a language model to obtain second labels. Specifically, this includes: inputting the first content, second content, and first label into the language model; outputting multiple fifth labels and a first confidence level corresponding to each fifth label; and multiple sixth labels and a second confidence level corresponding to each sixth label; filtering out fifth labels with a first confidence level less than a first threshold and sixth labels with a second confidence level less than a second threshold to obtain the second labels; fifth labels belong to the first labels, while sixth labels do not. In this way, second labels with higher confidence levels can be selected from the first labels, and new labels with higher confidence levels can be added to supplement the second labels, thus helping to improve the accuracy and recall of the final generated second labels. The confidence levels can be transmitted along with the second labels to downstream tasks so that downstream tasks can effectively use the second labels based on their confidence levels.

[0020] One possible implementation involves inputting prompt words into a language model to obtain the second label. Specifically, this includes: inputting the first content, the second content, and the first label into the language model; outputting multiple sixth labels and a second confidence score corresponding to each sixth label; filtering out sixth labels with a second confidence score less than a second threshold to obtain the second label; and ensuring that the sixth label does not belong to the first label. This method allows for the generation of second labels by selecting sixth labels with higher confidence scores. This approach is suitable for cases where the accuracy of the first label is low, improving the accuracy and recall of the final generated second label. The confidence score can be transmitted along with the second label to downstream tasks, enabling them to effectively utilize the second label based on the confidence score.

[0021] One possible implementation is to update the sixth tag to the tag set, thereby achieving timely updates to the tag set.

[0022] As one possible implementation, the primary content is one or more of text, images, audio, and video.

[0023] Secondly, this application provides a tag generation apparatus for performing the method in the first aspect or any possible implementation thereof. Specifically, the apparatus includes modules for performing the method in the first aspect or any possible implementation thereof.

[0024] Thirdly, this application provides a computing device, comprising: at least one memory for storing a program; at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method of the first aspect or a possible implementation thereof.

[0025] Fourthly, this application provides a computer storage medium having a computer program stored thereon, which, when run on a processor, causes the processor to perform the method of the first aspect or a possible implementation thereof.

[0026] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method of the first aspect or a possible implementation thereof. Attached Figure Description

[0027] Figure 1 is an example diagram of a labeling system;

[0028] Figure 2 is an example diagram of a label application scenario;

[0029] Figure 3 is an example diagram of the application architecture provided in this application;

[0030] Figure 4 is an example diagram of the system framework provided in this application;

[0031] Figure 5 is a flowchart of the label generation method provided in this application;

[0032] Figure 6 is a flowchart of the first label determination method provided in this application;

[0033] Figure 7 is a block diagram of the label generation device provided in this application;

[0034] Figure 8 is a submodule diagram of the first label determination module provided in this application;

[0035] Figure 9 is an example diagram of the computing device provided in this application. Detailed Implementation

[0036] The embodiments of this application will now be described with reference to the accompanying drawings. As will be apparent to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0037] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects of the same type in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0038] To facilitate understanding of this application, the subject matter of this application will be explained first.

[0039] (1) Tag

[0040] Tags are used to categorize and organize information. They can be any form of word or phrase used to describe the characteristics or type of content, helping users find information of interest more easily. In search engine and content recommendation systems, tags help the system quickly and accurately understand user interests, search intent, and product characteristics, simplifying the search and recommendation process.

[0041] (2) Language Model

[0042] A language model is an abstract mathematical model of language based on objective linguistic facts; it represents a correspondence. The relationship between a language model and objective linguistic facts is analogous to the relationship between an abstract line and a concrete line in mathematics.

[0043] (3) Tag recall

[0044] In this application, tag recall refers to the ratio of the number of relevant tags retrieved from the tag set to the total number of tags in the tag set.

[0045] (4) Label accuracy

[0046] In this application, the label accuracy rate refers to the ratio of the number of relevant labels retrieved from the label set to the total number of labels retrieved.

[0047] Before providing a detailed description of the technical solutions provided in the embodiments of this application, the application scenarios and system framework involved in the embodiments of this application will be introduced first.

[0048] Figure 3 is an example diagram of the application architecture provided in the embodiments of this application. The tag generation device provided in this application can be applied to scenarios such as search engines, content recommendation, or advertising platforms. Specifically, the tag generation device provided in this application can be used to generate corresponding tags for the content in the scenario, and the generated tags can be applied to downstream tasks such as search, recommendation, or advertising. The content in the scenario can be data of any object, such as data of news, products, users, software applications, photo albums, music, movies, or advertisements. The data type of the content in the scenario can be one or more of text, images, audio, and video. The content can be new content appearing in the scenario or historical content existing in the scenario. Those skilled in the art should understand that any object in the scenario that can be tagged falls within the technical scope of this application, and will not be exhaustively listed here.

[0049] Since the tag generation method provided in this application can be applied to both network-side and terminal-side scenarios, the execution subject of this application can be a server or a user terminal. For ease of description, the execution subject of this application is collectively referred to as an electronic device.

[0050] When the electronic device is a server, it can be a standalone server, a server cluster or distributed system consisting of multiple physical servers, a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or a cloud computing service center.

[0051] When the electronic device is a user terminal, it can be any electronic product that can interact with the user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as personal computer (PC), mobile phone, smartphone, personal digital assistant (PDA), wearable device, pocket PC (PPC), tablet computer, smart car system, etc.

[0052] Those skilled in the art should understand that the above application scenarios and electronic devices are merely examples. Other existing or future application scenarios and electronic devices that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference.

[0053] Figure 4 is an example diagram of the system framework provided in an embodiment of this application. This system is used to generate tags for news information, which includes one or more of the following: title, summary, category, keywords, image, audio, and video. The system provided in this embodiment includes a recall stage and a sorting stage. The recall stage is used to generate candidate tags for the news information, and the sorting stage is used to generate final tags. It should be noted that the news in Figure 4 is only an example; the system framework provided in this embodiment can also be applied to other objects, such as goods, users, software applications, photo albums, music, movies, or advertisements.

[0054] Supplementary data comprises one or more of external knowledge, manually generated rules, and historical samples. This supplementary data is used to provide additional knowledge to the news information, while the tag set provides candidate tags for the news information. During the recall phase, the system retrieves supplementary data related to the news information from the supplementary data and uses a multi-path recall coding model to determine candidate tags corresponding to the news information and related supplementary data from the tag set. Specifically, the multi-path recall coding model in this application includes a text coding model, a graph coding model, a speech coding model, and a video coding model. These models are used to select the appropriate coding model based on the data type of the news information and related supplementary data to generate candidate tags. The data type can be one or more of text, images, speech, and video.

[0055] During the recall phase, the recall rate of candidate tags was ensured by introducing supplementary data, and the accuracy of candidate tags was ensured by identifying candidate tags corresponding to news information and relevant supplementary data from the tag set.

[0056] In the ranking stage, candidate labels and relevant supplementary data are input into the language model, which outputs closed-domain labels and open-domain labels. Closed-domain labels are those from the label set, while open-domain labels are those generated by the language model that do not belong to the label set. The final labels are generated based on these closed-domain and open-domain labels. When there are multiple closed-domain and open-domain labels, the language model can output a confidence score for each label and filter them based on these scores. The final labels are then generated using the filtered closed-domain and open-domain labels. In the ranking stage, this application alleviates the illusion of language model performance by injecting relevant supplementary data, thereby improving the accuracy of the final labels.

[0057] The technical solution provided in this application will be described below.

[0058] Referring to Figure 5, this application embodiment provides a tag generation method, which is applied in an electronic device and includes the following steps:

[0059] S501, Obtain the first content and supplementary data.

[0060] The primary content can be one or more of text, images, audio, and video. It can be new content appearing in the scene or historical content existing within the scene. The primary content can be unimodal or multimodal data. Text can be any tagged text, such as keywords, categories, titles, summaries, or body text; no specific limitations are imposed here.

[0061] When new content appears in scenarios such as search engines, content recommendation, or advertising platforms, it can be tagged. For example, in a search engine for game applications, when "XX game" is a newly launched game application, content related to "XX game" is new content. A corresponding tag is generated for this new content, and tagged based on the generated tag. In this case, the first content appearing is the new content, specifically text composed of one or more of keywords, categories, and titles. When it is necessary to tag or update tags for historical content in scenarios such as search engines, content recommendation, and advertising platforms, tags are generated based on the historical content, and the historical content is tagged or its tags are updated based on the generated tags. The triggering conditions for updates can be fixed periods, popularity, etc.

[0062] Supplementary data comprises one or more of external knowledge, manually defined rules, and historical samples. External knowledge refers to information derived from non-labeled sets, including tags and website resources. Manually defined rules refer to manually prescribed tag generation rules. Historical samples refer to the historical input of the first content and the output of the second tag. In this application, supplementary data is used to supplement the first content with knowledge, resulting in second content. This second content is then used to guide the language model in generating second tags related to the first content, thereby improving the recall and accuracy of the generated second tags.

[0063] S502. Determine the second content related to the first content from the supplementary data.

[0064] The supplementary data can be one or more of text, images, audio, and video. In this step, the supplementary data is used to supplement the first content with knowledge, and the result of the supplementation is the second content. A text encoding model can be used to determine the second content related to the first content from the supplementary data. The text encoding model is a text similarity matching model in the prior art, which will not be elaborated on here. In this step, the text encoding model is used to retrieve the second content related to the first content.

[0065] S503. Determine the first tag related to the first content and the second content from the tag set.

[0066] The tag set is a predefined set. One or more models, including text encoding, keyword encoding, graph encoding, and speech encoding, can be used to determine the first tag related to the first and second content from the tag set. The number of first tags can be one or more.

[0067] Using the second content determined from the supplementary data to generate the first tag can supplement the knowledge for the generation of the first tag, improve the recall rate of the first tag, and thus improve the recall rate of the second tag generated using the first tag.

[0068] Specifically, referring to Figure 6, step S503 may include the following sub-steps:

[0069] S5031. Determine a third tag related to the first content and a fourth tag related to the second content from the tag set;

[0070] Based on the type of the first content, a third tag related to the first content is retrieved from the tag set. Specifically, the type of the first content is first determined, then a first encoding model is determined based on the type of the first content, and finally, the third tag related to the first content is determined from the tag set using the first encoding model.

[0071] The first content type can be one or more of text, images, audio, and video; the first encoding model is one or more of text encoding model, graph encoding model, audio encoding model, and video encoding model. The text encoding model can be used to retrieve tags related to the text in the first and / or second content; the graph encoding model can be used to retrieve tags related to the images in the first and / or second content; the audio encoding model can be used to retrieve tags related to the audio in the first and / or second content; and the video encoding model can be used to retrieve tags related to the video in the first and / or second content. The video encoding model can include both graph encoding model and audio encoding model.

[0072] When there are multiple first-level encoding models, each model is used to process the corresponding type of first content. The third label is generated using the first-level encoding model most relevant to the first content, thus improving the accuracy of the third label. For example, if the first content is text and an image, the corresponding first-level encoding models are a text encoding model and a graph encoding model. The text encoding model processes the text in the first content, and the graph encoding model processes the image. The outputs of the text encoding model and the graph encoding model together constitute the third label. There can be one or more third labels.

[0073] Based on the type of the second content, a fourth tag related to the second content is retrieved from the tag set. Specifically, the type of the second content is first determined, then a second encoding model is determined based on the type of the second content, and finally, the fourth tag related to the second content is determined from the tag set using the second encoding model.

[0074] The second content type can be one or more of text, images, audio, and video; the second encoding model can be one or more of text encoding, graph encoding, audio encoding, and video encoding. When there are multiple second encoding models, each second encoding model is used to process the corresponding type of second content, so that the second encoding model most relevant to the second content is used to generate the fourth tag, improving the accuracy of the fourth tag. For example, if the second content type is text and images, the corresponding second encoding models are a text encoding model and a graph encoding model. The text encoding model is used to process the text in the second content, and the graph encoding model is used to process the images in the second content. The outputs of the text encoding model and the graph encoding model together constitute the fourth tag. The number of fourth tags can be one or more.

[0075] S5032. Determine the first label based on the third and fourth labels.

[0076] The third and fourth tags are merged to generate the first tag. The third and fourth tags may contain duplicate tags; during the merging process, duplicate tags are removed, and the first tag is generated based on the deduplicated third and fourth tags.

[0077] In step S503, information matching is performed using one or more of the text encoding model, keyword encoding model, graph encoding model, and speech encoding model that are related to the types of the first content and the second content, so as to determine the first tag related to the first content and the first content from the first tag set. The appropriate encoding model can be selected to generate the first tag according to the type of the first content and the type of the second content, which helps to improve the recall and accuracy of the first tag.

[0078] It should be noted that the text encoding model in step S503 is the same as the text encoding model in step S502.

[0079] S504. Input the first content, the second content, and the first label into the language model to obtain the second label.

[0080] The first tag is used to provide candidate tags for the second tag, which is the final result tag. The second tag can be used in downstream tasks such as search engines or content recommendation to tag or update the tags of the first content in the downstream task. For example, when the first content is new content appearing in the scene, the second tag can be used to tag the new content; when the first content is historical content existing in the scene, the second tag can be used to tag or update the tags of the historical content.

[0081] Even when generating coherent and fluent text, language models can produce content that is inconsistent with reality; this is known as the "language model illusion problem." In this step, the second content generated based on supplementary data is used to inject information into the generation of the second label, improving the language model's understanding and mitigating the language model illusion problem, thus increasing the accuracy of the generated second label. The number of second labels can be one or multiple.

[0082] The language model is used to output tags that are highly relevant to the first and second content. The language model in this embodiment is a common model in the prior art, such as natural language processing models, Pangu model, Wenxin model, ChatGPT, etc., which will not be listed exhaustively here.

[0083] In this application, a prompt word template is pre-set. The prompt word template includes input fields for first content, second content, and a first label. Filling the input fields of the prompt word template with the first content, second content, and first label generates a prompt word for generating a second label. Inputting this prompt word into a language model yields the corresponding second label. Regarding the output of the second label, the prompt word template can be configured to instruct the language model to output only the label, or it can be configured to output the label and its corresponding confidence score. When inputting the first content, second content, and first label into the language model, the prompt word template can be obtained first, then the first content, supplementary data, and first label can be filled into the prompt word template to obtain the corresponding prompt word, which is then input into the language model.

[0084] The second label can come entirely from the first label, entirely from new labels generated by the language model, or partly from the first label and partly from new labels generated by the language model. New labels generated by the language model refer to labels that do not belong to the first label. These new labels can be added to the label set to update the label set. It should be noted that new labels generated by the language model may already be part of the label set. Therefore, when adding new labels to the label set, it is necessary to filter the new labels to update the label set only using new labels that do not belong to the label set.

[0085] As one implementation method, the first content, the second content, and the first tag are filled into the prompt word template to generate the corresponding prompt word. The prompt word is then input into the language model to output the second tag.

[0086] As one implementation method, the first content, second content, and first tag are filled into the prompt word template to generate corresponding prompt words. These prompt words are then input into a language model, which outputs a fifth tag and a first confidence score corresponding to that fifth tag. Fifth tags with a first confidence score less than a first threshold are filtered out to obtain the second tag. Here, the fifth tag is entirely derived from the first tag; there can be one or more fifth tags, each corresponding to a first confidence score. Fifth tags with a first confidence score greater than or equal to the first threshold constitute the second tag. This method allows for the selection of second tags with higher confidence scores, improving the accuracy of the final generated second tag. The first confidence score can be transmitted along with the second tag to downstream tasks, enabling them to use the second tag appropriately based on the first confidence score.

[0087] In one implementation, the first content, the second content, and the first tag are filled into the prompt word template to generate corresponding prompt words. The prompt words are input into a language model, which outputs a fifth tag and its corresponding first confidence level, as well as a sixth tag and its corresponding second confidence level. Fifth tags with a first confidence level less than a first threshold and sixth tags with a second confidence level less than a second threshold are filtered out to obtain the second tag. Here, the fifth tag is entirely derived from the first tag, and the sixth tag is a new tag generated by the language model based on factors such as popularity that does not belong to the first tag. The sixth tag is used to expand the second tag. Each fifth tag corresponds to a first confidence level, and each sixth tag corresponds to a second confidence level. Fifth tags with a first confidence level greater than or equal to the first threshold and second tags with a second confidence level greater than or equal to the second threshold together constitute the second tag. The number of fifth and sixth tags can be one or more. The first and second thresholds are pre-set thresholds, which can be the same or different. In this way, second labels with high confidence can be selected from the first labels, and a sixth label with high confidence can be selected to supplement the second labels, which helps to improve the accuracy and recall of the final generated second labels. The confidence level can be transmitted along with the second labels to downstream tasks so that downstream tasks can effectively use the second labels based on the confidence level.

[0088] As one implementation method, the first content, second content, and first tag are filled into the prompt word template to generate corresponding prompt words. These prompt words are then input into a language model, which outputs a sixth tag and a second confidence score corresponding to that sixth tag. Sixth tags with a second confidence score less than a second threshold are filtered out to obtain the second tag. Here, the sixth tag is a new tag generated by the language model based on factors such as popularity, and it does not belong to the first tag. Each sixth tag corresponds to a second confidence score. Sixth tags with a second confidence score greater than or equal to the second threshold constitute the second tag. The number of sixth tags can be one or more. This method allows for the generation of second tags by selecting sixth tags with higher confidence scores. This approach is suitable for situations where the accuracy of the first tag is low, improving the accuracy and recall of the final generated second tag. The confidence score can be transmitted along with the second tag to downstream tasks so that downstream tasks can effectively use the second tag based on the confidence score.

[0089] By using frequency to remove some tags from the tag set and adding a sixth tag with a second confidence level higher than a third threshold to the tag set, timely updates to the tag set can be achieved. Optionally, when adding a sixth tag to the tag set, all sixth tags can be added. It should be noted that the sixth tag may belong to the tag set. When adding a sixth tag to the tag set to update the tag set, the sixth tag needs to be filtered to update the tag set only using sixth tags that do not belong to the tag set. The third threshold is a pre-set threshold, which can be the same as or different from the second threshold.

[0090] In the tag generation method provided in this application, second content related to the first content is determined from supplementary data, and first tags related to the first and second content are determined from a tag set. These first tags are used as input tags for a language model. This method ensures the recall rate of the first tags input to the language model by introducing supplementary data, and ensures the accuracy rate of the first tags input to the language model by determining the first tags related to the first and second content from the tag set. This helps improve the recall and accuracy rate of the second tags generated by the language model based on the input tags. Furthermore, this application alleviates the illusion of the language model by injecting second content related to the first content determined from supplementary data, thereby improving the accuracy rate of the generated second tags. Therefore, the tag generation method provided in this application can filter content-related tags from the tag set through a language model while taking into account both tag recall and accuracy.

[0091] Referring to Figure 7, an embodiment of this application provides a label generation apparatus 700, including:

[0092] The acquisition module 701 is used to acquire the first content and supplementary data.

[0093] The primary content can be one or more of text, images, audio, and video. It can be new content appearing in the scene or historical content existing within the scene. The primary content can be unimodal or multimodal data. Text can be any tagged text, such as keywords, categories, titles, summaries, or body text; no specific limitations are imposed here.

[0094] Supplementary data can be one or more of external knowledge, manually generated rules, and historical samples. In this application, supplementary data is used to supplement the first content with knowledge, and the result of the supplementation is the second content. The second content is then used to guide the language model to generate second tags related to the first content, thereby improving the recall and accuracy of the generated second tags.

[0095] The second content determination module 702 is used to determine the second content related to the first content from the supplementary data.

[0096] Supplementary data comprises one or more of external knowledge, manually generated rules, and historical samples. External knowledge refers to information derived from non-label sets, including tags and website resources. Manually generated rules refer to manually specified tag generation rules. Historical samples refer to the historical input of the first content and the output of the second tag. In this application, supplementary data is used to first retrieve the first tag related to the first content from the tag set, and then use the first tag to guide the language model to generate the second tag related to the first content, thereby improving the recall and accuracy of the generated second tag.

[0097] The first tag determination module 703 is used to determine the first tag related to the first content and the second content from the tag set.

[0098] Referring to Figure 8, the first label determination module 703 includes the following sub-modules.

[0099] The first determining module 7031 is used to determine a third tag related to the first content and a fourth tag related to the second content from the tag set.

[0100] Based on the type of the first content, third tags related to the first content are retrieved from the tag set. Specifically, the type of the first content is first determined, then a first encoding model is determined based on the type of the first content, and finally, third tags related to the first content are determined from the tag set using the first encoding model. The type of the first content can be one or more of text, image, audio, and video; the first encoding model can be one or more of text encoding model, graph encoding model, audio encoding model, and video encoding model.

[0101] Based on the type of the second content, a fourth tag related to the second content is retrieved from the tag set. Specifically, first, the type of the second content is determined; then, a second encoding model is determined based on the type of the second content; finally, the fourth tag related to the second content is determined from the tag set using the second encoding model. The type of the second content can be one or more of text, image, audio, and video; the second encoding model can be one or more of text encoding model, graph encoding model, audio encoding model, and video encoding model.

[0102] The second determining module 7032 is used to determine the first label based on the third label and the fourth label.

[0103] The third and fourth tags are merged to generate the first tag. The third and fourth tags may contain duplicate tags; during the merging process, duplicate tags are removed, and the first tag is generated based on the deduplicated third and fourth tags.

[0104] For detailed implementation processes of the various modules included in Figure 8, please refer to the relevant content of each step of the method shown in Figure 6.

[0105] The second tag generation module 704 is used to input the first content, the second content, and the first tag into the language model to obtain the second tag.

[0106] The first tag is used to provide candidate tags for the second tag, which is the final result tag. The second tag can be used in downstream tasks such as search engines or content recommendation to tag or update the tags of the first content in the downstream task.

[0107] The second label can come entirely from the first label, entirely from new labels generated by the language model, or partly from the first label and partly from new labels generated by the language model. New labels generated by the language model refer to labels that do not belong to the first label.

[0108] For detailed implementation processes of the various modules included in the embodiments of this application, please refer to the relevant content of each step of the method shown in Figure 5.

[0109] In the tag generation method provided in this application, second content related to the first content is determined from supplementary data, and first tags related to the first and second content are determined from a tag set. These first tags are used as input tags for a language model. This method ensures the recall rate of the first tags input to the language model by introducing supplementary data, and ensures the accuracy rate of the first tags input to the language model by determining the first tags related to the first and second content from the tag set. This helps improve the recall and accuracy rate of the second tags generated by the language model based on the input tags. Furthermore, this application alleviates the illusion of the language model by injecting second content related to the first content determined from supplementary data, thereby improving the accuracy rate of the generated second tags. Therefore, the tag generation method provided in this application can filter content-related tags from the tag set through a language model while taking into account both tag recall and accuracy.

[0110] Please refer to Figure 9, which is an example diagram of an embodiment of the computing device in this application.

[0111] The computing device provided in this embodiment can be a processor, a server, or a dedicated data processing device, etc. The specific form of the device is not limited in this embodiment.

[0112] The computing device 900 can vary considerably due to different configurations or performance, and may include one or more processors 901 and memory 902, in which programs or data are stored.

[0113] The memory 902 can be volatile or non-volatile. Optionally, the processor 901 is one or more central processing units (CPUs), graphics processing units (GPUs), or other dedicated processors, such as Ascend. The CPU can be a single-core CPU or a multi-core CPU. The processor 901 can communicate with the memory 902 and execute a series of instructions stored in the memory 902 on the computing device 900.

[0114] The computing device 900 also includes one or more wired or wireless network interfaces 903, such as Ethernet interfaces.

[0115] Optionally, although not shown in Figure 9, the computing device 900 may also include one or more power supplies; one or more input / output interfaces, which can be used to connect to a display, mouse, keyboard, touch screen device or sensing device, etc. The input / output interfaces are optional components and may or may not be present, and are not limited here.

[0116] In this embodiment, the memory 902 in the computing device 900 stores a computer program. When the processor 901 executes the computer program, the execution flow can refer to the method flow described in the foregoing method embodiment, and will not be repeated here.

[0117] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, they can be implemented in whole or in part in the form of a computer program product.

[0118] A computer program product includes one or more computer instructions. When the computer program product runs on a processor, the computer loads and executes the computer execution instructions, producing all or part of the processes or functions of the embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0119] Computer-readable storage media can be any usable medium that a computer can store, or a data storage device such as a server or data center that integrates one or more usable media. The usable medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)). The computer-readable storage medium stores a computer program that, when executed on a processor, produces all or part of the processes or functions described in the embodiments of this application.

[0120] The technical solutions provided in this application have been described in detail above. Specific examples have been used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A label generation method, characterized in that, include: Obtain first content and supplementary data; the supplementary data is used to supplement the first content with knowledge. Determine the second content related to the first content from the supplementary data; Determine a first tag related to the first content and the second content from the tag set; The first content, the second content, and the first tag are input into the language model to obtain the second tag; the second tag is used as the tag for the first content.

2. The method as described in claim 1, characterized in that, The step of determining the first tag related to the first content and the second content from the tag set specifically includes: Determine a third tag related to the first content and a fourth tag related to the second content from the tag set; The first label is determined based on the third label and the fourth label.

3. The method as described in claim 2, characterized in that, The step of determining a third tag related to the first content from the tag set includes: Retrieve a third tag related to the first content from the tag set based on the type of the first content.

4. The method as described in claim 2, characterized in that, The step of determining a fourth tag related to the second content from the tag set includes: A fourth tag related to the second content is retrieved from the tag set based on the type of the second content.

5. The method according to any one of claims 2-4, characterized in that, Determining the first tag based on the third tag and the fourth tag includes: The third tag and the fourth tag are merged to generate the first tag.

6. The method according to any one of claims 1-5, characterized in that, The step of inputting the first content, the second content, and the first tag into a language model to obtain the second tag specifically includes: Get the prompt word template; Fill the prompt word template with the first content, the second content, and the first tag to obtain the corresponding prompt word; The prompt word is input into the language model to obtain the second label.

7. The method as described in claim 6, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The prompt word is input into the language model, which outputs multiple fifth labels and a first confidence score corresponding to each fifth label; the fifth label belongs to the first label. The fifth label is filtered out if the first confidence level is less than the first threshold to obtain the second label.

8. The method as described in claim 6, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The first content, the second content, and the first tag are input into the language model, which outputs multiple fifth tags and a first confidence level corresponding to each fifth tag, as well as multiple sixth tags and a second confidence level corresponding to each sixth tag; the fifth tags belong to the first tag, and the sixth tags do not belong to the first tag; The second label is obtained by filtering out the fifth label whose first confidence level is less than the first threshold and the sixth label whose second confidence level is less than the second threshold.

9. The method as described in claim 6, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The first content, the second content, and the first tag are input into the language model, and multiple sixth tags and a second confidence level corresponding to each sixth tag are output; the sixth tag does not belong to the first tag. The sixth label is obtained by filtering out the sixth labels whose second confidence level is less than the second threshold.

10. The method as described in claim 8 or 9, characterized in that, After outputting the sixth tag, the following is also included: Update the sixth tag to the tag set.

11. The method according to any one of claims 1-10, characterized in that, The first content is one or more of text, images, audio, and video.

12. A label generating apparatus, characterized in that, include: The acquisition module is used to acquire the initial content and supplementary data; The second content determination module is used to determine second content related to the first content from the supplementary data; The first tag determination module is used to determine a first tag related to the first content and the second content from the tag set; The second tag generation module is used to input the first content, the second content, and the first tag into a language model to obtain a second tag; the second tag is used as a tag for the first content.

13. The apparatus as claimed in claim 12, characterized in that, The first tag determination module specifically includes: The first determining module is used to determine a third tag related to the first content and a fourth tag related to the second content from the tag set; The second determining module is used to determine the first tag based on the third tag and the fourth tag.

14. The apparatus as claimed in claim 13, characterized in that, The step of determining a third tag related to the first content from the tag set includes: Retrieve a third tag related to the first content from the tag set based on the type of the first content.

15. The apparatus as claimed in claim 13, characterized in that, The step of determining a fourth tag related to the second content from the tag set includes: A fourth tag related to the second content is retrieved from the tag set based on the type of the second content.

16. The apparatus according to any one of claims 13-15, characterized in that, Determining the first tag based on the third tag and the fourth tag includes: The third tag and the fourth tag are merged to generate the first tag.

17. The apparatus according to any one of claims 12-16, characterized in that, The step of inputting the first content, the second content, and the first tag into a language model to obtain the second tag specifically includes: Get the prompt word template; Fill the prompt word template with the first content, the second content, and the first tag to obtain the corresponding prompt word; The prompt word is input into the language model to obtain the second label.

18. The apparatus as claimed in claim 17, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The prompt word is input into the language model, which outputs multiple fifth labels and a first confidence score corresponding to each fifth label; the fifth label belongs to the first label. The fifth label is filtered out if the first confidence level is less than the first threshold to obtain the second label.

19. The apparatus as claimed in claim 17, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The first content, the second content, and the first tag are input into the language model, which outputs multiple fifth tags and a first confidence level corresponding to each fifth tag, as well as multiple sixth tags and a second confidence level corresponding to each sixth tag; the fifth tags belong to the first tag, and the sixth tags do not belong to the first tag; The second label is obtained by filtering out the fifth label whose first confidence level is less than the first threshold and the sixth label whose second confidence level is less than the second threshold.

20. The apparatus as claimed in claim 17, characterized in that, The step of inputting the prompt word into the language model to obtain the second label specifically includes: The first content, the second content, and the first tag are input into the language model, and multiple sixth tags and a second confidence level corresponding to each sixth tag are output; the sixth tag does not belong to the first tag. The sixth label is obtained by filtering out the sixth labels whose second confidence level is less than the second threshold.

21. The apparatus as claimed in claim 19 or 20, characterized in that, Also includes: Update the sixth tag to the tag set.

22. The apparatus according to any one of claims 12-21, characterized in that, The first content is one or more of text, images, audio, and video.

23. A computing device, characterized in that, include: At least one memory for storing programs; At least one processor for executing the program stored in the memory; When the program stored in the memory is executed, the processor is used to execute the method as described in any one of claims 1-11.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, causes the processor to perform the method as described in any one of claims 1-11.

25. A computer program product, characterized in that, When the computer program product is run on a processor, the processor causes the processor to perform the method as described in any one of claims 1-11.

Images