Open domain label system construction method and device based on large language model

Abstract

The invention provides an open domain label system construction method and device based on a large language model, and relates to the technical field of data processing. The method comprises the following steps: constructing a label association pool according to different to-be-labeled contents in a first time period and initial labels output by labeling the to-be-labeled contents by using a first large language model; determining an initial tag system and a tag cluster of any initial tag based on the tag occurrence frequency of each initial tag; based on a preset semantic understanding task description, determining a normalized tag of each initial tag by using the zero sample capability of the second large language model; constructing a label directed graph set; and if it is detected that the first tag in the first tag directed graph is a normalized tag of the second tag in the directed graph and the normalized tag of the first tag in the second tag directed graph is a third tag, constructing a target tag system. According to the method, the time from emerging topic appearance to label system response is shortened, and the semantic accuracy of the label system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a large language model-based open domain label system construction method and device. BACKGROUND

[0002] Text classification, as a core basic task of natural language processing, is essentially to realize accurate mapping from high-dimensional semantic space to discrete label space. Under the background of the increasing complexity of Internet content ecology, traditional text classification has evolved from simple category attribution judgment to multi-dimensional and fine-grained content semantic understanding task. Modern content platforms need to build a precise label system to realize the deep semantic representation of content. This system not only carries the basic function of content understanding, but also becomes the key technology base for driving personalized recommendation, precise search, user portrait construction and other core businesses.

[0003] However, the existing label system has the following defects in the operation and maintenance mechanism: Label evolution lags behind: the traditional manually driven label construction and maintenance mode cannot match the exponential growth speed of Internet content and the rapid iteration characteristics of semantic concepts. Emerging topics often have a significant time delay from their appearance to the response of the label system. This time sequence mismatch causes the label system to always be in a "catching up" state, and it cannot realize the forward-looking coverage of the changes in the content ecology. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a large language model-based open domain label system construction method and device, which does not require human intervention from label generation, system initialization to normalization optimization, greatly shortens the time period from the appearance of emerging topics to the response of the label system, and enables the label system to synchronously perceive the changes in the enterprise content ecology, no longer in the "catching up" state, and realizes timely response to the iteration of content semantics.

[0005] In a first aspect, a method for constructing an open-domain label system based on a large language model is provided. The method can include: constructing a label association pool based on different content to be labeled of a target enterprise in a first time period and initial labels output by labeling the different content to be labeled using a trained first large language model, the label association pool including an association relationship between the different content to be labeled and corresponding initial labels; the first large language model is trained based on a constructed supervised data set; determining an initial label system based on a label occurrence frequency of each initial label in the content to be labeled; determining a label clustering cluster of any initial label in the initial label system based on a similarity between the initial label and any initial label other than the initial label; determining a normalized label of each initial label by processing each label in the label clustering cluster of the different initial labels using a zero-shot capability of a second large language model based on a pre-set semantic understanding task description; constructing a label directed graph set with each initial label in the initial label system as a node and a mapping relationship between the initial label and the corresponding normalized label as an edge, the label directed graph set including a label directed graph constructed by different initial labels and corresponding normalized labels; if it is detected that a first label in a first label directed graph is a normalized label of a second label in a corresponding label directed graph, and the normalized label of the first label in the second label directed graph is a third label, then constructing a target label system, the target label system including a mapping relationship between the first label and the third label.

[0006] In one possible implementation, before constructing the label association pool, the method further includes: configuring Prompt template information for a third large language model having Few-shot learning capability, the Prompt template information including Prompt templates of different content types and pre-set label definition rules; generating a Few-shot learning example set based on different content of the target enterprise in a second time period and artificial labels of corresponding content satisfying the pre-set label definition rules; and labeling the different content based on the learned third large language model to generate a supervised data set; the supervised data set including different content and corresponding original labels; using each content in the supervised data set as an input training sample and using the corresponding original label as a real label to iteratively train the pre-trained first large language model to obtain the trained first large language model.

[0007] In one possible implementation, the label definition rules include a label broadness determination rule, a label fineness determination rule, and a label format normalization requirement.

[0008] In one possible implementation, an initial labeling system is determined based on the frequency of occurrence of each output initial label in the content to be labeled, including: obtaining the frequency of occurrence of each output label in different contents to be labeled within a first time period, the content coverage of each initial label in the corresponding content to be labeled, and the number of labels in the label association pool; based on the number of labels, the content coverage of each initial label, and the frequency of occurrence of each initial label, determining a distribution curve between the number of labels and the content coverage under different label occurrence frequencies; according to the distribution curve, determining the frequency of occurrence of labels corresponding to the minimum content coverage among the content coverages greater than a preset content coverage threshold as a first frequency threshold; and determining a set of initial labels corresponding to the frequency of occurrence of labels greater than the first frequency threshold as the initial labeling system.

[0009] In one possible implementation, based on the similarity between any initial tag in the initial tag system and any initial tag other than the initial tag, determining the tag cluster of the initial tag includes: based on any initial tag in the initial tag system, calculating the similarity between the initial tag and any initial tag other than the initial tag; and selecting initial tags whose similarity to the initial tag is higher than a preset similarity threshold to form the tag cluster of the initial tag.

[0010] In one possible implementation, based on a preset semantic understanding task description, the zero-sample capability of the second largest language model is used to process each label in the label clustering clusters of different initial labels respectively, and determine the normalized label of each initial label, including: based on a preset semantic understanding task description, the zero-sample capability of the second largest language model is used to process each label in the label clustering clusters of different initial labels respectively, and obtain the central label of each label in the corresponding label clustering cluster; if any initial label in the initial label system corresponds to a label clustering cluster, then the central label of the label clustering cluster is determined as the normalized label of the initial label; if any initial label in the initial label system corresponds to a label clustering cluster, then the central label of the label clustering cluster is determined as the normalized label of the initial label; If the initial label corresponds to multiple label clusters, and there is a central label with the highest label occurrence frequency among the central labels of each label cluster, then the central label with the highest label occurrence frequency is determined as the normalized label of the initial label; if any initial label in the initial label system corresponds to multiple label clusters, and there are multiple central labels with the highest label occurrence frequency among the central labels of each label cluster, then the configured comprehensive quality score algorithm is used to process the multiple central labels with the highest label occurrence frequency respectively, obtain the comprehensive quality scores of the corresponding central labels, and determine the central label with the highest comprehensive score as the normalized label of the initial label.

[0011] In one possible implementation, after the target label system is constructed, the method further includes: obtaining a plurality of to-be-added labels, and continuously counting the label occurrence frequency of each to-be-added label every day, taking the to-be-added label with a label occurrence frequency greater than a second frequency threshold as a candidate label, and constructing a candidate set; calculating the semantic similarity between any candidate label in the candidate set and each initial label in the initial label system; determining the initial label in the initial label system corresponding to the highest semantic similarity as the matching label that best matches the candidate label; if the highest semantic similarity is not less than a preset semantic similarity threshold, adding the candidate label as the synonym label of the matching label to the initial label system, and establishing a mapping relationship between the candidate label and the matching label corresponding to the normalization label in the target label system; if the highest semantic similarity is less than the preset semantic similarity threshold, adding the candidate label to the initial label system and the target label system.

[0012] In a second aspect, an open domain label system construction device based on a large language model is provided, which can include: A construction unit is configured to construct a label association pool according to different to-be-labeled content of a target enterprise in a first time period and initial labels output by labeling the different to-be-labeled content using a trained first large language model, the label association pool including an association relationship between the different to-be-labeled content and the corresponding initial labels; the first large language model is trained based on a constructed supervised data set; A determination unit is configured to determine an initial label system based on the label occurrence frequency of each initial label in the to-be-labeled content, and determine a label clustering cluster of an initial label based on the similarity between the initial label and any initial label other than the initial label in the initial label system; based on a preset semantic understanding task description, the determination unit is configured to use the zero-shot capability of a second large language model to process each label in the label clustering cluster of a different initial label, and determine a normalization label of the initial label. The construction unit is further configured to construct a label directed graph set with each initial label in the initial label system as a node and the mapping relationship between the initial label and the corresponding normalization label as an edge, the label directed graph set including a label directed graph constructed by different initial labels and corresponding normalization labels; and if it is detected that a first label in a first label directed graph is a normalization label of a second label in a corresponding label directed graph, and the normalization label of the first label in a second label directed graph is a third label, a target label system is constructed, the target label system including the mapping relationship between the first label and the third label.

[0013] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps of any one of the first aspect.

[0014] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the method steps of any one of the first aspect.

[0015] The application provides a large language model-based open domain label system construction method and device. The method constructs a label association pool according to different to-be-labeled contents of a target enterprise in a first time period and initial labels output by labeling different to-be-labeled contents by using a trained first large language model. The label association pool includes the association relationship between different to-be-labeled contents and corresponding initial labels. The first large language model is obtained by training a constructed supervised data set. An initial label system is determined based on the label occurrence frequency of each initial label output in the to-be-labeled content. A label clustering cluster of any initial label in the initial label system is determined based on the similarity between the initial label and any initial label other than the initial label. The normalized labels of different initial labels are determined by processing each label in the label clustering cluster of each initial label based on a preset semantic understanding task description and using the zero-shot capability of a second large language model. A label directed graph set is constructed with each initial label in the initial label system as a node and the mapping relationship between each initial label and the corresponding normalized label as an edge. The label directed graph set includes different initial labels and corresponding normalized labels to construct a label directed graph. If it is detected that a first label in the first label directed graph is a normalized label of a second label in the corresponding label directed graph and the normalized label of the first label in the second label directed graph is a third label, a target label system is constructed, and the target label system includes the mapping relationship between the first label and the third label. This method does not require human intervention from label generation, system initialization to normalization optimization, greatly shortens the response time from the emergence of a new topic to the label system, enables the label system to synchronously perceive changes in the enterprise content ecosystem, and is no longer in a "catching up" state, enabling timely response to content semantic iteration. Through semantic similarity clustering, zero-shot semantic understanding and multi-hop mapping optimization, the semantic precision of the label system is improved, redundancy and ambiguity are reduced, and precise and efficient label support is provided for intelligent scenarios such as downstream content recommendation, accurate retrieval and customer profiling. In addition, the model can be trained and adapted to the personalized business needs of enterprises through customized model training and task description, balancing generalization and customization value. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced as follows. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 A flowchart of a large language model-based open domain label system construction method provided by the embodiments of the present application is shown in the figure. Figure 2 A schematic diagram of a statistical chart provided by the embodiments of the present application is shown in the figure. Figure 3 A schematic diagram of a label directed graph set provided by the embodiments of the present application is shown in the figure. Figure 4 A schematic diagram of a target label system construction process provided by the embodiments of the present application is shown in the figure. Figure 5 A flowchart of a daily automatic updating mechanism provided by the embodiments of the present application is shown in the figure. Figure 6 A structural schematic diagram of a large language model-based open domain label system construction device provided by the embodiments of the present application is shown in the figure. Figure 7 A structural schematic diagram of an electronic device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, and not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application. Unless otherwise defined, the technical terms or scientific terms used in the present application should be understood as the general meaning understood by those skilled in the art. The words "first", "second", and similar words used in the present application do not represent any order, number, or importance, but are only used to distinguish different components. The words "include" or "contain" and similar words mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, and do not exclude other elements or objects. The words "connect", "couple", or "connect" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0019] The open domain label system construction in the prior art follows the traditional paradigm of static predefinition-manual maintenance, and has a fundamental timing adaptation defect. The core problem of this mechanism is that the structural mismatch between the evolution speed of semantic concepts and the response ability of the label system causes timing delay: the traditional manually driven label construction and maintenance process includes multiple serial links such as "concept recognition → semantic analysis → label design → review and release", and the overall response period is usually measured by weeks or months. The emerging semantic concepts in the Internet content ecosystem grow exponentially, and the time window from the appearance of a hot topic to its peak is usually only hours or days. This order of magnitude difference in time scale causes the label system to always be in a "catching up" state, and cannot achieve synchronous perception and forward-looking coverage of semantic changes in content. At the same time, the current mainstream large language model application paradigm has the dual technical defects of capability constraints and paradigm limitations: (1) the existing scheme forcibly constrains the large language model with open language generation capability in a pre-defined discrete label space, which essentially degenerates the unbounded generation problem of continuous semantic space into a bounded decision problem of discrete selection space. This results in a loss of several orders of magnitude in expression capability. (2) The zero-shot and few-shot capabilities of large language models obtained through large-scale pre-training cannot be effectively utilized under the closed set paradigm. The model is forced to compress and map the rich semantic knowledge learned in the pre-training phase to a limited pre-defined concept space, resulting in approximate matching rather than precise generation when facing emerging semantic concepts, which severely restricts the sensitivity and adaptability of the label system to semantic changes.

[0020] To solve the above problems, the present application provides a large language model-based open domain label system construction method and device, which discards the limitations of traditional artificial pre-defined closed label set and breaks through the traditional practice of allowing large language models to make multiple-choice classification within a fixed label range. With a large language model as the driving engine, an intelligent label system with open domain characteristics, fine-grained recognition capability, and self-adaptive updating mechanism is constructed. Compared with the traditional pre-defined fixed label set, the label system constructed by the present application has openness and stronger semantic expression capability. That is, the present application fundamentally breaks through the paradigm constraints of traditional closed set classification, and constructs an intelligent label system that can autonomously perceive semantic changes, respond to content evolution in real time, and dynamically adjust label granularity through generative semantic modeling, dynamic label evolution, and multi-granularity adaptive normalization. The core innovation of the present application is to transform the large language model from a "multiple-choice judge" to a "semantic creator", realizing a fundamental technological leap from static predefinition to dynamic generation, from fixed granularity to adaptive granularity, and from manual maintenance to machine autonomy, and providing a new technical path for solving the label bottleneck problem in large-scale content understanding.

[0021] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings of the specification. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0022] Figure 1 A flowchart of a large language model-based open domain label system construction method provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the method can include the following steps. Figure 1 Step S110, constructing a label association pool according to different to-be-labeled content of the target enterprise in a first time period and initial labels output by labeling the different to-be-labeled content using a trained first large language model.

[0023] Before performing this step, to ensure that the first large language model can stably output labels according to the specified format, the first large language model can be trained in the following manner, including: Step A, configuring Prompt template information for a third large language model with Few-shot learning capability. The Prompt template information can include Prompt templates of different content types (including event class content that tells a story around an event and object class content that describes or introduces a related topic around an object) and preset label definition rules. The preset label definition rules can include label broad determination rules, label fine determination rules, and label format standardization requirements.

[0024] (1) Label broad determination rule: the label cannot accurately understand the original content theme after the label is separated from the original content. For example, the original content is an article introducing the TV series of Journey to the West, and the output label is “TV series”. When looking at this label alone, it cannot be known that the content theme is Journey to the West, and the correct label should be a label of “TV series Journey to the West” class.

[0025] (2) Label fine determination rule: the label is too fine and long. For example, the content is a technical sharing article that mainly introduces the blogger's experience of learning python in 30 days. If the output label is “how to quickly learn python in 30 days”, this label is too fine. Although the label can well reflect the content theme after being separated from the original content, when the label is used in downstream tasks, the content that can be corresponded to the label will be very small. Therefore, the correct label can be “python learning”. At the same time, to prevent the generation of such fine labels, it can be specified in the Prompt template for generating labels that the number of characters in the label is within 15 characters to a certain extent to avoid such problems.

[0026] ​(3) Tag format standardization requirement: In order to facilitate the automatic processing and analysis of downstream systems, all generated tags must follow a unified format specification for standardized output, such as all tags outputting structured information with “,” as the separator, and all letters remaining in lowercase.

[0027] Step B, based on the obtained different content (including high-quality Q&A content and original articles) of the target enterprise in the second time period (such as the last 3 years) and the corresponding content manually labeled to meet the preset tag definition rules, generate a Few-shot learning example set; Specifically, select 3-5 typical samples from different content, manually label their original tags as Few-shot learning examples, and form a “input content-output corresponding manual label” mapping pair to learn the tag generation rule that meets the preset tag definition rule. After that, based on the learned third large language model, label the different content in the second time period to generate a supervised data set. The supervised data set can include different content and corresponding original labels.

[0028] Further, the automatically generated tags of the third large language model can be comprehensively manually reviewed, and the irrelevant, low-quality or format-inconsistent labeling entries are strictly removed, thereby obtaining a supervised data set.

[0029] Step C, taking each content in the supervised data set as an input training sample and the corresponding original label as a true label, iteratively training the pre-trained first large language model to obtain a trained first large language model. Specifically, from the supervised data set, about a preset number of labeled data, such as 12,000, are selected by random sampling method to constitute the training data set required for fine-tuning of the pre-trained first large language model. This data set is carefully designed to ensure that it covers a variety of label types and content scenarios, providing sufficient semantic coverage for model training. Based on the pre-trained first large language model (such as MiniCPM-2B model), implement full-parameter supervised fine-tuning on the constructed training data set, focusing on optimizing the following two core objectives: enhance the model's ability to recognize and understand fine-grained semantic features, improve the semantic accuracy and content relevance of generated labels, and through repeated training, strengthen the model's strict adherence to the pre-defined tag format specification, ensuring that the output results meet the unified format requirements, such as comma separation, label quantity limitation, etc.

[0030] Further, the first large language model that has been supervised fine-tuned is deployed into the content automatic labeling task to generate batch labels for the actual questions and article contents involved in the task. Representative content samples are randomly selected from the labeling results to construct an independent test evaluation dataset. The automatically generated labels in the test set are professionally evaluated, and comprehensive scores are given from multiple dimensions such as label semantic accuracy, content relevance, and format specification. Finally, the quantitative evaluation results of the model label accuracy are obtained, and the specific label accuracy evaluation results are shown in Table 1: Table 1

[0031] Optionally, the supervised dataset can be constructed in the above manner or can be constructed by humans in advance.

[0032] Returning to step S110, in a specific implementation, the target enterprise's to-be-labeled content in a first time period is obtained. The first time period is a recent time period ending at the current time, such as the last 2 years. The to-be-labeled content can be high-quality question and answer content and original articles. "High quality" means that the question and answer content and original articles can accurately solve user needs and have outstanding performance in content value, credibility, and expression logic. It can be understood that the high-quality question and answer content and original articles can be selected by humans, or the computer can use recommendation algorithms to recommend content based on user reading behavior, interest preferences, etc., or use a web crawler to obtain article resources on the Internet, and then use information filtering techniques (such as setting keyword filtering, content duplication detection, etc.) to filter out original articles that meet the quality requirements. The present application does not limit this. The first time period and the second time period can be the same, or the second time period can be greater than the first time period.

[0033] Then, the to-be-labeled content is input into the trained first large language model to label the input to-be-labeled content using the trained first large language model to obtain initial labels output by the first large language model; and based on the association relationship between different to-be-labeled content and corresponding initial labels, a label association pool is constructed, that is, the label association pool can include the association relationship between different to-be-labeled content and corresponding initial labels. The first large language model is trained based on the constructed supervised dataset, and the supervised dataset is determined based on different content in the second time period and a pre-set label definition rule.

[0034] Step S120, determining an initial label system based on the label occurrence frequency of each initial label in the to-be-labeled content.

[0035] In a specific implementation, if the first frequency threshold is a pre-configured frequency threshold, the frequency of occurrence of each output initial label in different to-be-annotated contents within the first time period can be obtained, and an initial label system can be generated based on each initial label whose frequency of occurrence is greater than the first frequency threshold.

[0036] If the first frequency threshold is not a pre-configured frequency threshold, the output frequency of occurrence of each initial tag in different content to be annotated within the first time period (or "number of tag occurrences"), the content coverage of each initial tag in the corresponding content to be annotated (or "content coverage ratio"), and the number of tags in the tag association pool can be obtained. Based on the number of tags, the content coverage of each initial tag, and the frequency of occurrence of each initial tag, a distribution curve between the number of tags and the content coverage at different tag occurrence frequencies can be determined. Then, according to the distribution curve, the first frequency threshold is determined based on the tag occurrence frequency and the content coverage. The set of tags corresponding to the tag occurrence frequencies greater than the first frequency threshold is determined as the initial tag system.

[0037] In an example, the relationship between the number of occurrences of each initial tag, the proportion of covered content, and the number of tags can be referred to as follows: Figure 2 The statistical chart shown shows a typical long-tail distribution between initial tags and content coverage. The top characteristic is that a small number of high-frequency "head" tags cover the vast majority of content, demonstrating strong representativeness and universality. The tail characteristic is that a large number of low-frequency "tail" tags cover only a very small amount of specific content, resulting in relatively limited application value. Based on the above statistical analysis results, and considering the balance between tag set size (i.e., number of tags) and content coverage, this application has developed the following tag screening strategy: 1) High-frequency tags (or "head tag set") with a frequency of occurrence greater than 15 times (i.e., the first frequency threshold) are selected as the core tag set. This threshold effectively balances the manageability of the tag count with sufficient content coverage. 2) The filtered head tag set covers over 94% of the content on the site, totaling approximately 3.6 million valid tags, constituting this application's initial tag system, or "Top360w tag set." In other words, while ensuring a coverage of at least 94%, the frequency threshold with the lowest number of tags is selected. As can be seen from the figure, when the frequency of a tag appearance is greater than or equal to 15 times, the content coverage is close to 94%, and the number of tags is relatively small, achieving a balance between the number of tags and content coverage. Therefore, 15 times is determined as the first frequency threshold.

[0038] Step S130 : determining a tag cluster of the initial tag based on the similarity between any initial tag and any initial tag other than the initial tag in the initial tag system.

[0039] In a specific implementation, based on any initial label in the initial label system, the similarity between the initial label and any initial label other than the initial label is calculated; and the initial labels with a similarity higher than a preset similarity threshold value are selected to form a label clustering cluster of the label, thereby obtaining the label clustering cluster of each initial label in the initial label system.

[0040] Specifically, the Embedding model is used to vectorize and encode each initial label in the initial label system to generate a high-dimensional semantic vector representation, i.e., a label vector of each initial label; based on cosine similarity measurement, the most similar semantic label vectors are retrieved for the label vector of each initial label in the vector space of the initial label system, which can be set to the top 50 labels greater than a specific threshold value, to construct a label clustering cluster corresponding to each initial label in the Top360w label set.

[0041] In step S140, based on the preset semantic understanding task description, the zero-shot capability of the second large language model is used to process each label in the label clustering cluster of different initial labels to determine the normalized label of each label.

[0042] Based on the preset semantic understanding task description, the zero-shot capability of the second large language model is used to process each label in the label clustering cluster of different initial labels to determine the normalized label of each label. Specifically, the determined labels in the label clustering cluster of different initial labels are input into a large language model (such as GPT-4) with zero-shot capability in the format of python list, and the large language model determines the semantic representativeness and universality of each label in the label clustering cluster according to the preset semantic understanding task description through the zero-shot capability, and outputs the most representative center label of the label clustering cluster. The output format is python dict nested in python list. Further, the center label of the label clustering cluster can be regarded as the key of the label clustering cluster, and each label in the label clustering cluster can be regarded as the value corresponding to the key.

[0043] If any initial label in the initial label system corresponds to a label clustering cluster, the center label of the label clustering cluster of the initial label is determined as the normalized label of the initial label. If any initial label in the initial label system corresponds to multiple label clustering clusters, and there is one center label with the highest label occurrence frequency among the center labels of the label clustering clusters, the center label with the highest label occurrence frequency is determined as the normalized label of the initial label. At this time, a certain target initial label in the initial label system corresponds to multiple label clustering clusters, that is, a certain target initial label corresponds to multiple keys. The key with the highest label occurrence frequency is determined as the normalized label of the target initial label in the initial label system.

[0044] If any initial label in the initial label system corresponds to multiple label clustering clusters, and there are multiple center labels with the highest label occurrence frequency among the center labels of the label clustering clusters, the configured comprehensive quality score algorithm is used to process the multiple center labels with the highest label occurrence frequency respectively, to obtain the comprehensive quality scores of the corresponding center labels, and the center label with the highest comprehensive score is determined as the normalized label of the initial label. At this time, a certain target initial label in the initial label system corresponds to multiple label clustering clusters, that is, a certain target label corresponds to multiple keys, and the label occurrence frequencies of at least two keys in the multiple keys are equal and highest. At this time, the comprehensive quality scores of the keys are calculated, and the key with the highest comprehensive score is determined as the normalized label of the target initial label.

[0045] The above embodiment can effectively eliminate the multiple selection ambiguity problem.

[0046] The configured comprehensive quality score algorithm is a linear weighted scoring function, which combines the "literal similarity" (number of repeated characters) and "commonality" (label occurrence frequency) two dimensions and gives different weights to solve the case that the frequencies of multiple center labels are equal in the semantic aggregation process.

[0047] In one example, a certain target initial label in the initial label system corresponds to multiple keys, and the label occurrence frequencies of at least two target keys in the multiple keys are equal and highest. At this time, the comprehensive quality score algorithm is used to calculate the comprehensive quality scores of the keys, and the key with the highest comprehensive score is determined as the normalized label of the target initial label.

[0048] In one example, the comprehensive quality score algorithm can be expressed as: ; In the formula, i is the i-th target key in the target keys with equal and highest label occurrence frequencies, is the comprehensive quality score of the i-th target key, is the number of repeated characters of the i-th key and each value of the label clustering cluster corresponding to the i-th target key, the frequency of occurrence of the label for the i-th target key, the character length of the i-th target key, the control repetition word weight, the label frequency weight, the value range of the two weights is [0, 1], and .

[0049] It can be understood that the number of repeated characters refers to the number of characters shared by the i-th target key and each value of the corresponding label cluster, which does not necessarily have to be continuous in position.

[0050] Step S150, taking any initial label in the initial label system as a node, and the mapping relationship between the initial label and the corresponding normalized label as an edge, a label directed graph set is constructed, wherein the label directed graph set includes different initial labels and the corresponding normalized labels to construct a label directed graph.

[0051] Specifically, taking each initial label in the initial label system (i.e., the Top360w label set) as a node, and the mapping relationship between the corresponding initial label and the corresponding normalized label as an edge, and arranging them in order from low to high according to the label frequency of occurrence, to ensure that there is no loop structure in the mapping relationship, forming a standard directed acyclic graph topology, i.e., constructing a label directed graph, which can avoid the problem of circular dependency.

[0052] In one example, taking the label "Master's graduate student enrollment matters needing attention" with a label frequency of 11, the label "graduate student enrollment" with a label frequency of 120, and the label "graduate student enrollment" with a label frequency of 560 as examples, wherein the normalized label of the label "Master's graduate student enrollment matters needing attention" is the label "graduate student enrollment", and the normalized label of the label "graduate student enrollment" is the label "graduate student enrollment", therefore, in order from low to high according to the label frequency of occurrence, as shown in Figure 3 the connection arrow from the label "Master's graduate student enrollment matters needing attention" to the label "graduate student enrollment", and the connection arrow from the label "graduate student enrollment" to the label "graduate student enrollment", thereby obtaining a label directed graph set constructed by the label directed graph between each label and the corresponding normalized label.

[0053] Step S160, constructing a target label system based on the detection results of the mapping relationship between the labels involved in different label directed graphs in the label directed graph set.

[0054] Detecting the mapping relationship between the labels involved in the first label directed graph and the second label directed graph of the first label directed graph and the second label directed graph in the label directed graph set. If it is detected that the first label in the first label directed graph is the normalized label of the second label in the corresponding label directed graph, and that the normalized label of the first label in the second label directed graph is the third label, a mapping relationship between the first label and the third label is established, and a target label system is constructed, which includes the mapping relationship between the first label and the third label. This involves performing a multi-hop merge of the "original label → intermediate label → normalized label" path in the label directed graph, establishing a direct connection between the source label and the final target label, and achieving efficient "multi-hop merging" to obtain the "Top 100w label set." This "multi-hop path compression" strategy not only maintains the interpretability of the label mapping process but also significantly reduces the overhead of downstream calls.

[0055] Continuing from the previous example, combined Figure 3 , the label "Master's degree graduate enrollment registration notes" and the label "Graduate enrollment" can establish a mapping relationship through the label "Graduate enrollment registration", then the mapping relationship between the label "Master's degree graduate enrollment registration notes" and the label "Graduate enrollment" is connected with a connecting arrow, and the mapping relationship between the label "Master's degree graduate enrollment registration notes" and the label "Graduate enrollment" is stored in the constructed target label system, wherein the construction process of the target label system is as follows: Figure 4 shown.

[0056] Through the above implementation, the initial Top360w tag set has been significantly refined and optimized. 1) Processing Results: 3.6 million tags were efficiently merged and compressed into approximately 1 million normalized tags, forming a refined tag system (hereinafter referred to as the "Top100w tag set"). 2) Mapping Completeness: Each original tag is fully mapped to the Top100w normalized tag set, ensuring full coverage and traceability of the tag conversion process. 3) Optimization Effect: While fully preserving the fine-grained semantic characteristics of the tags, the content coverage of individual tags has been significantly improved, effectively improving the practicality and application efficiency of the tag system.

[0057] In some embodiments, after the target tag system is constructed, in order to achieve dynamic updates of the tag system in a T+1 cycle and ensure that the tag set can respond to content changes and hot trends in a timely manner, the following daily automatic update mechanism can also be implemented. Specifically: A plurality of tags to be added are obtained, and the frequency of occurrence of each tag to be added is continuously counted daily. Tags to be added whose frequency of occurrence is greater than a second frequency threshold are taken as candidate tags, and a candidate set C is constructed.

[0058] Specifically: All tags not included in the Top360w tag set are comprehensively counted and defined as "non-Top tags" (i.e., tags to be added). These include: newly generated tags on the day (such as daily new tags) and historically accumulated but low-frequency stock tags (such as old tags with a cumulative frequency <15 / historical long-tail tags); a second frequency threshold is set (such as ≥20 times / day), and tags to be added whose frequency exceeds the second frequency threshold are selected as candidate tags to construct candidate set C.

[0059] Afterwards, the semantic similarity between any candidate tag in the candidate set C and each tag in the initial tag system is calculated; the tag in the initial tag system corresponding to the highest semantic similarity is determined as the matching tag that best matches the candidate tag; If the highest semantic similarity is not less than the preset semantic similarity threshold (e.g., 0.8), the candidate tag is added to the initial tag system as a synonym of the matching tag, and a mapping relationship between the candidate tag and the normalized tag corresponding to the matching tag is established in the target tag system; If the highest semantic similarity is less than the preset semantic similarity threshold, the candidate tag is added to the initial tag system and the target tag system.

[0060] like Figure 5 As shown in the figure, the process of the daily automatic update mechanism includes: finding the difference between the tags corresponding to all content related to the target enterprise and the tags in the Top360w tag set, that is, finding tags not included in the Top360w tag set and treating them as non-Top tags; then, selecting non-Top tags whose frequency of appearance is greater than a second frequency threshold as tags to be added to the Top (or "tags to be added"); Get the embedding index of the Top360w tag set, that is, get the label vector of each label in the Top360w tag set, calculate the similarity (or "semantic similarity") between the label vector of each label and the label vector of the tag to be added to the top, and get the T1 label with the highest similarity, that is, get the T1 label with the highest similarity to the tag to be added to the top (T1 label is the label in the Top360w tag set); Determine whether the highest similarity is greater than the similarity threshold Y; If so, the top tag to be added is added to the Top360w tag set as a synonymous mapping and merged into the same Top100w tag set as T1 (that is, a mapping relationship is established between the top tag to be added and the normalized tag corresponding to T1).

[0061] If not, the top tag to be added is added as a new tag to the Top360w tag set and is also added as a new tag to the Top100w tag set.

[0062] The application realizes full-automatic expansion and maintenance of the label system through the above T+1 dynamic updating process. The label system can maintain a highly sensitive real-time perception ability to content changes, timely capture emerging topics and hot trends, and automatically identify, filter and integrate new labels according to content evolution to ensure the completeness and timeliness of label coverage. The entire updating process does not require human operation intervention and maintenance, significantly reducing the labor cost and operation complexity of label system management, and effectively solving the core pain point that the traditional static label system is difficult to adapt to rapid content changes.

[0063] In some embodiments, whether the label broadness determination rule is satisfied can be determined by whether the information entropy loss rate of the output label L after being separated from the content to be labeled C is greater than τ1. If the label L cannot uniquely reconstruct the core theme of C after being separated from the content C (i.e., the information entropy loss rate ΔH>τ1), it is considered to be too wide.

[0064] Whether the label fineness determination rule is satisfied can be determined by whether the description length of the output label L exceeds 15 characters and whether the expected content coverage P(C|L) of the output label L is less than τ2 (based on historical statistical prior estimation). If the description length of the output label L exceeds 15 characters and the expected content coverage P(C|L)<τ2, it is considered to be too fine.

[0065] Whether the label format standardization requirement is satisfied can be determined by whether the output format is in lowercase and uses a comma-separated string sequence (such as tag1,tag2).

[0066] In some embodiments, to meet the demand for semantic relevance and generality, avoid the dominance of a single factor, and balance the influence of a single factor while strengthening the contribution of both, the product term can be used to enhance the contribution of both, while the basic linear term can be used to balance the influence of a single factor. The specific calculation logic can include: First, the ratio of the number of repeated characters to the character length is denoted as A, which is normalized to the interval [0, 1] to eliminate the influence of label length difference; Second, the ratio of the label occurrence frequency to the highest label occurrence frequency in the label clustering cluster corresponding to the i-th target key is denoted as B, which is normalized to the interval [0, 1] to avoid excessive monopoly of high-frequency labels; Finally, the comprehensive quality score algorithm can be expressed as: ; wherein, and is the balance coefficient (both sums are 1) of the synergy item (product item) and the base item (sum item), the product item can highlight the labels that are both semantically related and high-frequency, and the sum item can avoid the misselection of labels that are extremely high in a single factor but extremely low in another factor (for example, when A = 0.9, B = 0.1, the product item = 0.09, and the sum item = 0.5, the comprehensive score = 0.6 x 0.09 + 0.4 x 0.5 = 0.254, which is not too high). = 0.6, = 0.4, the comprehensive score = 0.6 x 0.09 + 0.4 x 0.5 = 0.254, which is not too high).

[0067] In some embodiments, the weight and ratio can be inversely deduced by the degree of semantic difference within the label clustering cluster, and the core logic is: the greater the semantic difference, the more it needs to rely on semantic relevance, and the more concentrated the semantics, the more it needs to rely on frequency universality. The specific operation is as follows: First, calculate the semantic dispersion of each label clustering cluster, specifically: calculate the standard deviation (or average cosine distance) through the Embedding vectors of all labels in the cluster, the higher the dispersion, the greater the semantic difference of the labels in the cluster (for example, when a label clustering cluster with "Amazon after-sales" as the key mixes in a label of the "Apple repair" type, the dispersion will increase significantly); Subsequently, normalize the semantic dispersion to the [0, 1] interval, and directly take the normalized dispersion value as , , and take 1- . For example, the semantic dispersion of a certain cluster is normalized to 0.7, indicating that the semantic difference in the cluster is large, and it is necessary to focus on screening semantically matched labels through the number of repeated characters, so = 0.7, = 0.3; if the dispersion of another cluster is normalized to 0.2, the semantics are highly concentrated, so the frequency can better reflect the universality, so = 0.2, = 0.8.

[0068] Corresponding to the above method, the embodiments of the present application also provide an open domain label system construction device based on a large language model, as shown in Figure 6 , the device comprises: A construction unit 610 is configured to construct a label association pool according to different to-be-labeled contents of a target enterprise in a first time period and initial labels output by labeling the different to-be-labeled contents using a trained first large language model, wherein the label association pool comprises an association relationship between the different to-be-labeled contents and the corresponding initial labels; and the first large language model is trained based on a constructed supervised data set. The determining unit 620 is configured to determine an initial label system based on label occurrence frequencies of the output initial labels in the to-be-labeled content, and determine a label clustering cluster of each initial label in the initial label system based on a similarity between any initial label and any initial label other than the initial label. The constructing unit 610 is further configured to construct a label directed graph set with each initial label in the initial label system as a node and a mapping relationship between the initial label and a corresponding normalized label as an edge, the label directed graph set including a label directed graph constructed by different initial labels and corresponding normalized labels, and construct a target label system if it is detected that a first label in a first label directed graph is a normalized label of a second label in a corresponding label directed graph and a normalized label of the first label in the second label directed graph is a third label, the target label system including a mapping relationship between the first label and the third label.

[0069] The functions of each functional unit of the label system construction device based on a large language model provided in the above embodiments can be realized through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the label system construction device based on a large language model provided in the embodiments of the present application are not repeated here.

[0070] The embodiments of the present application further provide an electronic device, as shown in the accompanying drawings, which comprises a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 complete mutual communication through the communication bus 740. Figure 7 The processor 710 is configured to implement the following steps when executing the program stored in the memory 730.

[0071] The memory 730 is configured to store a computer program. The processor 710 is configured to implement the following steps when executing the program stored in the memory 730. The label association pool includes an association relationship between the different to-be-labeled content and the corresponding initial label; and the first large language model is trained based on the constructed supervised data set; The determining unit 620 is configured to determine an initial label system based on label occurrence frequencies of the output initial labels in the to-be-labeled content, and determine a label clustering cluster of each initial label in the initial label system based on a similarity between any initial label and any initial label other than the initial label. The determining unit 620 is configured to determine an initial label system based on label occurrence frequencies of the output initial labels in the to-be-labeled content, and determine a label clustering cluster of each initial label in the initial label system based on a similarity between any initial label and any initial label other than the initial label. Based on the preset semantic understanding task description, the zero-shot capability of the second large language model is used to process each label in the label clustering cluster of different initial labels respectively, and the normalized label of each initial label is determined. Taking each initial label in the initial label system as a node and the mapping relationship between the initial label and the corresponding normalized label as an edge, a label directed graph set is constructed, and the label directed graph set includes different initial labels and corresponding normalized labels constructing a label directed graph. If it is detected that the first label in the first label directed graph is the normalized label of the second label in the corresponding label directed graph, and the normalized label of the first label in the second label directed graph is a third label, a target label system is constructed, and the target label system includes the mapping relationship between the first label and the third label.

[0072] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0073] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0074] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0075] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0076] The implementation manners and beneficial effects of the electronic device in the above-mentioned embodiments can be known from the above description of the problems solved by the various components of the electronic device Figure 1 The specific working process and beneficial effects of the electronic device provided by the embodiments of the present application are not repeated here, as they can be known from the implementation of each step in the above-mentioned embodiments.

[0077] In another embodiment provided by the present application, a computer readable storage medium is also provided, which stores instructions, when the instructions are run on a computer, causing the computer to execute the method for constructing an open domain label system based on a large language model according to any one of the above-mentioned embodiments.

[0078] In another embodiment provided by the present application, a computer program product containing instructions is also provided, when the instructions are run on a computer, causing the computer to execute the method for constructing an open domain label system based on a large language model according to any one of the above-mentioned embodiments.

[0079] Those skilled in the art should understand that the embodiments in the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application can be in the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of computer program products implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0080] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0081] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0082] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide a process for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or blocks ​ one block or multiple blocks.

[0083] Although the preferred embodiments in the application have been described, those skilled in the art who know the basic inventive concept once they know this application can make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the application.

[0084] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the application without departing from the spirit and scope of the embodiments of the application. Thus, if these modifications and variations of the embodiments of the application fall within the scope of the claims of the embodiments of the application and their equivalent technologies, the embodiments of the application also intend to include these modifications and variations.

Claims

1. A method for constructing an open domain labeling system based on a large language model, characterized in that: The method comprises: Constructing a label association pool based on different content to be annotated for the target enterprise within a first time period and initial labels output by annotating the different content to be annotated using a trained first language model, wherein the label association pool includes associations between the different content to be annotated and the corresponding initial labels; the first language model is trained based on the constructed supervised dataset; Based on the frequency of occurrence of each output initial label in the content to be annotated, an initial label system is determined: Determining a tag cluster of the initial tag based on a similarity between any initial tag in the initial tag system and any initial tag other than the initial tag; Based on a preset semantic understanding task description, the zero-shot capability of the second largest language model is used to process each label in the label clusters of different initial labels to determine the normalized label of each initial label; Taking each initial label in the initial label system as a node and the mapping relationship between each initial label and the corresponding normalized label as an edge, a label directed graph is constructed, wherein the label directed graph includes different initial labels and corresponding normalized labels to construct a label directed graph; If it is detected that the first label in the first label directed graph is the normalized label of the second label in the corresponding label directed graph, and the normalized label of the first label in the second label directed graph is the third label, then a target label system is constructed, and the target label system includes a mapping relationship between the first label and the third label.

2. The method according to claim 1, wherein Before building the tag association pool, the method further includes: Configuring prompt template information for the third language model with few-shot learning capability, wherein the prompt template information includes prompt templates for different content types and preset tag definition rules; generating a few-shot learning example set based on the obtained different contents of the target enterprise within the second time period and the manual labels of the corresponding contents that meet the preset label definition rules; and after obtaining a learned third language model based on the few-shot learning example set, labeling the different contents using the learned third language model to generate a supervised dataset; the supervised dataset includes the different contents and the corresponding original labels; Each content in the supervised data set is used as an input training sample, and the corresponding original label is used as a true label. The pre-trained first language model is iteratively trained to obtain a trained first language model.

3. The method according to claim 2, wherein The label definition rules include broad label determination rules, detailed label determination rules and label format standardization requirements.

4. The method according to claim 1, wherein Determining an initial labeling system based on the output frequency of occurrence of each initial label in the content to be labeled includes: Obtaining the output frequency of occurrence of each tag in different to-be-annotated contents within the first time period, the content coverage of each initial tag in the corresponding to-be-annotated contents, and the number of tags in the tag association pool; Determining a distribution curve between the number of tags and the content coverage under different tag occurrence frequencies based on the number of tags, the content coverage of each initial tag, and the tag occurrence frequency of each initial tag; According to the distribution curve, a first frequency threshold is determined based on the tag occurrence frequency and content coverage; A set of initial tags corresponding to tag occurrence frequencies greater than the first frequency threshold is determined as an initial tag system.

5. The method according to claim 1, wherein Determining a tag cluster of the initial tag based on a similarity between any initial tag in the initial tag system and any initial tag other than the initial tag includes: Based on any initial tag in the initial tag system, calculating the similarity between the initial tag and any initial tag other than the initial tag; Initial tags whose similarity to the initial tags is higher than a preset similarity threshold are selected to form a tag cluster of the initial tags.

6. The method according to claim 1, wherein Based on the preset semantic understanding task description, the zero-shot capability of the second largest language model is used to process each label in the label clusters of different initial labels and determine the normalized label of each initial label, including: Based on the preset semantic understanding task description, the zero-shot capability of the second largest language model is used to process each label in the label clusters with different initial labels, and obtain the central label of each label in the corresponding label cluster; If any initial label in the initial label system corresponds to a label cluster, the central label of the label cluster is determined as the normalized label of the initial label; If any initial label in the initial label system corresponds to multiple label clusters, and there is a central label with the highest label occurrence frequency among the central labels of each label cluster, then the central label with the highest label occurrence frequency is determined as the normalized label of the initial label; If any initial label in the initial label system corresponds to multiple label clusters, and there are multiple central labels with the highest label occurrence frequency among the central labels of each label cluster, the configured comprehensive quality score algorithm is used to process the multiple central labels with the highest label occurrence frequency respectively to obtain the comprehensive quality scores of the corresponding central labels, and the central label with the highest comprehensive score is determined as the normalized label of the initial label.

7. The method according to claim 1, wherein After constructing the target label system, the method further includes: Acquire multiple tags to be added, and continuously count the frequency of occurrence of each tag to be added every day, and use the tags to be added whose frequency of occurrence is greater than a second frequency threshold as candidate tags to construct a candidate set; Calculating the semantic similarity between any candidate tag in the candidate set and each initial tag in the initial tag system; Determine the initial tag in the initial tag system corresponding to the highest semantic similarity as the matching tag that best matches the candidate tag; If the highest semantic similarity is not less than a preset semantic similarity threshold, the candidate tag is added to the initial tag system as a synonym of the matching tag, and a mapping relationship between the candidate tag and the normalized tag corresponding to the matching tag is established in the target tag system; If the highest semantic similarity is less than a preset semantic similarity threshold, the candidate tag is added to the initial tag system and the target tag system.

8. A device for constructing an open domain labeling system based on a large language model, characterized in that: The device comprises: a construction unit, configured to construct a label association pool based on different to-be-annotated contents of a target enterprise within a first time period and initial labels output by annotating the different to-be-annotated contents using a trained first language model, wherein the label association pool includes associations between the different to-be-annotated contents and the corresponding initial labels; the first language model is trained based on a constructed supervised dataset; A determination unit is configured to determine an initial labeling system based on the frequency of occurrence of each output initial label in the content to be annotated; and determine a label cluster of the initial label based on the similarity between any initial label in the initial labeling system and any initial label other than the initial label; based on a preset semantic understanding task description, utilize the zero-shot capability of the second largest language model to process each label in the label cluster of different initial labels to determine a normalized label of each initial label; The construction unit is also used to construct a label directed graph with each initial label in the initial label system as a node and the mapping relationship between each initial label and the corresponding normalized label as an edge, wherein the label directed graph includes a label directed graph constructed by different initial labels and corresponding normalized labels; and if it is detected that the first label in the first label directed graph is the normalized label of the second label in the corresponding label directed graph, and the normalized label of the first label in the second label directed graph is the third label, then a target label system is constructed, wherein the target label system includes the mapping relationship between the first label and the third label.

9. An electronic device, characterized in that: The electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images