Label recognition model training method and apparatus, and computer readable storage medium

Abstract

A label identification model training method and device and a computer readable storage medium, the method comprising: collecting field text, and generating pseudo labels corresponding to the field text; constructing pseudo label samples according to the field text and the corresponding pseudo labels, pre-training a label identification model using the pseudo label samples, and obtaining a pre-trained label identification model; receiving user input text content and an original label set corresponding to the text content, expanding the original label set corresponding to the text content to obtain an expanded label set, and constructing training samples according to the text content and the corresponding expanded label set; and fine-tuning the pre-trained label identification model using the training samples to obtain a final label identification model. The present application can make up for the problem of a small number of samples and reduce the dependence on manual annotation. In addition, the embodiments of the present application can improve the label quality and the accuracy of the label identification result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a training method, apparatus, and computer-readable storage medium for a label recognition model. Background Technology

[0002] With the continuous upgrading of online learning resource platforms and the increasing number of uploaded learning resources, the problems of incomplete and poor-quality content tags added by uploaders are becoming increasingly prominent. This is especially true for training and education platforms, which require extensive and accurate tagging of content to enable machines to better understand its characteristics and make targeted recommendations for user learning content. Therefore, it is necessary to research and develop automatic tagging for online course learning resources to address the issue of insufficient and incomplete manually labeled data for user learning content recommendations and searches on existing online learning resource platforms.

[0003] Text content tag recognition technology solutions mainly consist of traditional methods, deep learning methods, and a combination of both. Traditional methods use feature analysis methods such as Term Frequency-Inverse Document Frequency (TF-IDF) to select the most representative words in the original text. However, due to the limited size of the model, the extraction results are often inaccurate. Deep learning methods use large models such as multi-label prediction models to select the words in the candidate tag library that best match the original text. This is also a commonly used method. However, most existing technical solutions in this category adopt end-to-end methods, which usually suffer from problems such as small sample size for tag recognition and high manual annotation costs. Summary of the Invention

[0004] At least one embodiment of this application provides a training method, apparatus, and computer-readable storage medium for a label recognition model, which addresses the problems of small sample size and high manual annotation cost in the prior art for label recognition.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a method for training a label recognition model, including:

[0007] Collect domain text and generate pseudo-tags corresponding to the domain text;

[0008] Based on the domain text and its corresponding pseudo-labels, pseudo-label samples are constructed, and the pseudo-label samples are used to pre-train the label recognition model to obtain the pre-trained label recognition model.

[0009] Receive text content input by the user and the original tag set corresponding to the text content, expand the original tag set corresponding to the text content to obtain an expanded tag set, and construct training samples based on the text content and its corresponding expanded tag set;

[0010] Using the training samples, the pre-trained label recognition model is fine-tuned to obtain the final label recognition model.

[0011] Optionally, pseudo-tags corresponding to the domain text are generated, including at least one of the following:

[0012] Calculate the importance index of each word in the domain text, and extract the first number of words with the highest importance index as pseudo-tags corresponding to the domain text;

[0013] A pseudo-summary of the domain text is generated using a preset summarization algorithm; the importance index of each word in the pseudo-summary is calculated, and the second number of words with the highest importance index are extracted as pseudo-tags corresponding to the domain text.

[0014] Optionally, the label recognition model is pre-trained using the pseudo-label samples to obtain a pre-trained label recognition model, including:

[0015] The label recognition model is trained simultaneously through a first training task and a second training task to obtain a pre-trained label recognition model; wherein...

[0016] The first training task is to input the domain text of the pseudo-label sample into the label recognition model, and train the model with the pseudo-labels generated by the label recognition model corresponding to the domain text as the optimization objective. The second training task is to randomly replace the words in the domain text of the pseudo-label sample with mask labels to generate new domain text; input the new domain text into the label recognition model, and train the model with the reconstruction of the words replaced by the mask labels as the optimization objective.

[0017] Optionally, the original tag set corresponding to the text content is expanded to obtain an expanded tag set, including:

[0018] For the current text content, a similar text search algorithm is used to search for at least one similar text content that is closest to the current text content from all text content.

[0019] From the original tag set corresponding to the similar text content, identify tags that match the current text content and add them to the original tag set corresponding to the current text content to obtain the extended tag set corresponding to the current text content.

[0020] Optionally, from the original tag set corresponding to the similar text content, tags that match the current text content are identified, including:

[0021] Each tag in the original tag set corresponding to the similar text content is input together with the current text content into a pre-trained matching model, wherein the matching model is used to identify whether the tag matches the text;

[0022] Based on the recognition results output by the matching model, a tag matching the current text content is obtained.

[0023] Optional, also includes:

[0024] The matching model is trained by following these steps:

[0025] Construct positive and negative samples, wherein positive samples include text content and positive tags, and the positive tags are tags in the original tag set corresponding to the text content; negative samples include text content and negative tags, and the negative tags are tags that do not belong to the original tag set corresponding to the text content.

[0026] The text content from the positive and negative samples is input into the matching model to generate an encoded representation of the text content. The model is then iteratively trained with the optimization objective of whether the current label matches the current text content, thus obtaining the matching model.

[0027] Optionally, the encoded representation of the generated text content includes:

[0028] The text content is divided into multiple segments, an encoded representation of each segment is generated, and the encoded representations of the multiple segments are subjected to average pooling to obtain the encoded representation of the text content.

[0029] or,

[0030] The text content is divided into multiple segments. If the number of segments obtained is less than a preset number, padding is performed to obtain the preset number of segments. An encoded representation of each segment is generated and input into a fully connected layer to obtain the encoded representation of the text content.

[0031] Optionally, the step of fine-tuning the pre-trained label recognition model using the training samples includes:

[0032] The text content in the training samples is input into the pre-trained label recognition model to generate a generated label set text obtained by concatenating at least one label. The model is iteratively trained with the goal of optimizing the generated label set text to be close to the extended label set text corresponding to the text content, so as to obtain the final label recognition model. The extended label set text is a string formed by concatenating each label in the extended label set.

[0033] Optionally, when generating a text set of generated tags obtained by concatenating at least one tag, the method further includes:

[0034] Determine whether the currently generated tag is a tag in the tag library, which includes official tags in the tag system tree and temporary tags added by the user that are outside the tag system tree;

[0035] If the currently generated tag is not a tag in the tag library, replace the currently generated tag with the closest tag in the tag library;

[0036] If the currently generated tag is a tag in the tag library, the currently generated tag is normalized at the character level according to the corresponding tag in the tag library.

[0037] Optionally, when generating a text set of generated tags by concatenating at least one tag, the method further includes:

[0038] If the currently generated tag is an official tag in the tag library, the parent tag of the currently generated tag is completed by tracing the path in the tag system tree.

[0039] Optionally, after obtaining the final label recognition model, the method further includes:

[0040] The label recognition model is used to infer the text to be inferred, and a set of inference labels corresponding to the text to be inferred is generated.

[0041] Optionally, when generating the set of inference tags corresponding to the text to be inferred, the method further includes:

[0042] Determine whether the tags in the inference tag set are tags in the tag library, the tag library including official tags in the tag system tree and temporary tags added by the user outside the tag system tree;

[0043] If a tag in the inference tag set is not a tag in the tag library, the tag in the inference tag set is replaced with the closest tag in the tag library;

[0044] If the tags in the inference tag set are tags in the tag library, then the tags in the inference tag set are normalized at the character level according to the corresponding tags in the tag library.

[0045] Optionally, when generating the set of inference tags corresponding to the text to be inferred, the method further includes:

[0046] If a tag in the inference tag set is an official tag in the tag library, the parent tags of the tags in the inference tag set are supplemented by tracing the path in the tag system tree.

[0047] Secondly, embodiments of this application provide a training apparatus for a label recognition model, comprising:

[0048] The generation module is used to collect domain text and generate pseudo-tags corresponding to the domain text.

[0049] The first training module is used to construct pseudo-label samples based on the domain text and its corresponding pseudo-labels, and to pre-train the label recognition model using the pseudo-label samples to obtain the pre-trained label recognition model.

[0050] The construction module is used to receive the text content input by the user and the original tag set corresponding to the text content, expand the original tag set corresponding to the text content to obtain an expanded tag set, and construct training samples based on the text content and its corresponding expanded tag set.

[0051] The second training module is used to fine-tune the pre-trained label recognition model using the training samples to obtain the final label recognition model.

[0052] Thirdly, embodiments of this application provide a training apparatus for a tag recognition model, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method described in the first aspect.

[0053] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the method described above.

[0054] Compared with existing technologies, the training method, apparatus, and computer-readable storage medium for the label recognition model provided in this application can, to some extent, compensate for the problem of insufficient samples, introduce domain information, and reduce reliance on manual annotation. Furthermore, this application's embodiments expand the labels of the training text through a matching model, mitigating label quality issues to some extent. The label recognition model in this application employs a generative model, which is more flexible, eliminates the need to construct label hierarchy relationships, and does not limit the number or level of labels in the inference results. Supplemented by label post-processing, it further improves label accuracy. Attached Figure Description

[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0056] Figure 1 This is a flowchart illustrating a training method for a label recognition model according to an embodiment of this application.

[0057] Figure 2 An example diagram of an extended tag set provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the structure of the training device for the label recognition model in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the structure of a training device for a label recognition model according to another embodiment of this application. Detailed Implementation

[0060] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0061] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The terms “and / or” in the specification and claims indicate at least one of the connected objects.

[0062] The following description provides examples and is not intended to limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0063] As described in the background section, existing text content tag recognition solutions do not specifically address the problems of course tag recognition tasks due to the lack of end-to-end deep learning models and their limited utilization of additional task information. Taking online learning course tag recognition as an example, the following technical problems typically exist:

[0064] 1) Existing course tagging methods suffer from limited sample size and high manual annotation costs: Since a single video course segment is often close to one hour long, annotators need to watch the entire video course before they can accurately annotate the course tags. Furthermore, the number of candidate tags for a course can often reach tens of thousands (including temporary tags), further increasing the difficulty of manual annotation. End-to-end deep learning models, due to their large number of parameters, typically require a sufficient amount of data to avoid overfitting.

[0065] 2) Poor quality course tags: Course creators are required to create content tags when uploading resources. Due to individual differences and the course creators' unfamiliarity with the tagging system, problems arise with poor quality and incomplete tag coverage. End-to-end deep learning models heavily rely on tag quality; if the tag quality is insufficient, the quality of the tags obtained by the model's inference will often be unsatisfactory.

[0066] 3) The number of tags is not fixed and lacks hierarchical structure information: Because a single online course is long and covers multiple independent learning contents, course creators often configure multiple tags for a course. The number of tags is not fixed and includes a large number of temporary tags (tags that have not yet been manually classified into the tag system tree), lacking usable tag hierarchical structure information. End-to-end deep learning models often use classification models (selecting target tags from a candidate tag library) and rely on tag hierarchical structure information, which is not flexible enough and is not naturally suitable for this task.

[0067] To address at least one of the above problems, this application provides a method for training a label recognition model, such as... Figure 1 As shown, the method includes:

[0068] Step 11: Collect domain text and generate pseudo-tags corresponding to the domain text.

[0069] Here, this application can collect domain text from domain data. Domain data is often substantial, providing the model with domain knowledge information that a limited amount of task corpus cannot offer, thus helping the model to make more accurate inferences. Taking online courses as an example, available domain data includes, but is not limited to, video subtitles, courseware, and training materials.

[0070] The embodiments of this application can generate pseudo-tags in at least one of the following ways:

[0071] (1) Calculate the importance index (e.g., TF-IDF value) of each word in the domain text, and extract the first number of words with the highest importance index as pseudo-labels corresponding to the domain text. Here, the first number is a pre-set positive integer.

[0072] (2) Using a preset summary generation algorithm, generate a pseudo-summary of the domain text; calculate the importance index (e.g., TF-IDF value) of each word in the pseudo-summary, and extract the second number of words with the highest importance index as pseudo-tags corresponding to the domain text. Here, the second number is also a preset positive integer.

[0073] Using the above methods, the embodiments of this application can automatically generate pseudo-labels, eliminating the need for manual labeling and reducing the workload of labeling sample samples.

[0074] Step 12: Based on the domain text and its corresponding pseudo-labels, construct pseudo-label samples, and use the pseudo-label samples to pre-train the label recognition model to obtain the pre-trained label recognition model.

[0075] Here, each pseudo-label sample includes a domain text and a corresponding pseudo-label. After obtaining multiple pseudo-label samples, the label recognition model can be pre-trained using these samples. In this embodiment, the label recognition model can employ an encoder-decoder model, such as the T5 model. During pre-training, the label recognition model can be trained simultaneously using a first training task and a second training task to obtain a pre-trained label recognition model. The first training task involves inputting the domain text of the pseudo-label sample into the label recognition model and training it with the goal of generating the pseudo-label corresponding to the domain text. The second training task involves randomly replacing words in the domain text of the pseudo-label sample with mask labels to generate new domain text. The new domain text is then input into the label recognition model, and training it with the goal of reconstructing the words replaced by the mask labels.

[0076] Step 13: Receive the text content input by the user and the original tag set corresponding to the text content, expand the original tag set corresponding to the text content to obtain an expanded tag set, and construct training samples based on the text content and its corresponding expanded tag set.

[0077] Here, this embodiment constructs training samples to fine-tune the label recognition model obtained in step 12 after pre-training. Specifically, it can receive multiple text contents input by the user and the original label set corresponding to each text content. The original label set includes at least one label (original label, i.e., positive label in the following text). To address the issue of insufficient quality of user-provided labels, this embodiment further expands the original label set by introducing new labels, thereby obtaining an expanded label set. Then, using each text content and its corresponding expanded label set, a training sample is constructed, thus obtaining multiple training samples.

[0078] Step 14: Using the training samples, fine-tune the pre-trained label recognition model to obtain the final label recognition model.

[0079] Here, the training samples obtained in step 13 are used to fine-tune the pre-trained label recognition model to obtain the final label recognition model. Specifically, the text content in the training samples can be input into the pre-trained label recognition model to generate at least one label, and then generate a generated label set text obtained by concatenating the at least one label. Iterative training is performed with the optimization objective of the generated label set text being close to the extended label set text corresponding to the text content, to obtain the final label recognition model. The extended label set text is a string formed by concatenating the labels in the extended label set.

[0080] Furthermore, when inputting the text content from the training samples into the pre-trained label recognition model to generate a set of generated labels concatenated from at least one label, this embodiment of the application can also perform label post-processing on the generated labels to standardize their representation. For example, in some cases, the generated labels are not conventional representations in natural language, or they differ from typical labels at the character level, such as in case normalization or the presence of typos. This embodiment of the application can further post-process the generated labels to ensure a uniform representation that conforms to the conventional representations of natural language.

[0081] Specifically, it can be determined whether the currently generated tag is a tag in the tag library. Here, the tag library includes official tags in the tag system tree and temporary tags added by the user outside the tag system tree. If the currently generated tag is not in the tag library, it is replaced with the closest tag in the tag library, which can be done using a tag similarity matching algorithm to search for the closest tag. If the currently generated tag is in the tag library, it undergoes character-level normalization processing according to the corresponding tag in the tag library, such as case normalization and typo correction.

[0082] Furthermore, if the currently generated tag is an official tag in the tag library, this embodiment of the application can also complete the parent tag of the currently generated tag by tracing the path in the tag system tree.

[0083] Through the above steps, this application embodiment obtains a large amount of domain text from domain data, pre-trains the label recognition model based on the domain text, expands the original label set input by the user, constructs training text using the expanded label set, and fine-tunes the pre-trained label recognition model, thereby improving the label recognition performance of the finally trained label recognition model.

[0084] The following is a detailed explanation of one implementation method for expanding the original tag set corresponding to the text content to obtain an expanded tag set in step 13 above.

[0085] For a given text content (referred to as the current text content for ease of description), a similar text search algorithm, such as K-Nearest Neighbor (KNN), is used to search for at least one similar text content that is closest to the current text content from all text content. Then, tags that match the current text content are identified from the original tag set corresponding to the similar text content and added to the original tag set corresponding to the current text content, resulting in an expanded tag set corresponding to the current text content.

[0086] Here, in this embodiment of the application, when identifying tags matching the current text content from the original tag set corresponding to similar text content, a pre-trained matching model can be used for identification. Specifically, each tag in the original tag set corresponding to the similar text content is input together with the current text content into the pre-trained matching model, wherein the matching model is used to identify whether the tag matches the text. Then, based on the identification result output by the matching model, the tag matching the current text content is obtained.

[0087] The aforementioned matching model can specifically be a pre-trained language model, such as the BERT (Bidirectional Encoder Representation from Transformer) model. The matching model can be trained according to the following steps in this embodiment:

[0088] (a) Construct positive samples and negative samples, wherein positive samples include text content and positive tags, the positive tags being tags in the original tag set corresponding to the text content, and negative samples include text content and negative tags, the negative tags being tags that do not belong to the original tag set corresponding to the text content.

[0089] Here, all the labels for the text content can be considered as a complete label set. For a specific piece of text content, labels in this complete label set that do not belong to the original label set corresponding to that text content can be used as negative labels. The specific number of positive and negative labels can be selected based on the actual training environment and computing power.

[0090] (b) Input the text content in the positive and negative samples into the matching model to generate the encoded representation of the text content, and perform iterative training with the optimization objective of whether the current label matches the current text content to obtain the matching model.

[0091] There are several ways to generate the encoded representation of the text content. For example, the text content can be divided into multiple segments, each segment can be encoded, and average pooling can be applied to these segments to obtain the encoded representation of the text content. Alternatively, the text content can be divided into multiple segments, where if the number of segments is less than a preset number, padding is used to obtain the preset number of segments; an encoded representation for each segment can be generated, and this representation can be input into a fully connected layer to obtain the encoded representation of the text content. Furthermore, during iterative training, an objective function can be set that is positively correlated with a first matching metric and negatively correlated with a second matching metric. Iterative training can then be performed with the optimization objective of maximizing the value of the objective function. Here, the first matching metric is the matching degree between the positive label of a positive sample and the text content of that positive sample, and the second matching metric is the matching degree between the positive label of a negative sample and the text content of that negative sample.

[0092] After obtaining the final trained label recognition model, this embodiment of the application can use the model for reasoning. Specifically, the text to be reasoned is input into the label recognition model, and the label recognition model is used to reason about the text to be reasoned, generating a set of reasoning labels corresponding to the text to be reasoned. The set of reasoning labels includes at least one reasoning label recognized by the model.

[0093] Similarly, after obtaining the set of inference tags corresponding to the text to be inferred, this embodiment of the application can also perform tag post-processing. For example, it can determine whether the tags in the inference tag set are tags in a tag library, which includes official tags in the tag system tree and temporary tags added by the user outside the tag system tree. If the tags in the inference tag set are not tags in the tag library, the tags in the inference tag set are replaced with the closest tags in the tag library; if the tags in the inference tag set are tags in the tag library, the tags in the inference tag set are normalized at the character level according to the corresponding tags in the tag library. In addition, if the tags in the inference tag set are official tags in the tag library, the parent tags of the tags in the inference tag set can be supplemented by path tracing in the tag system tree.

[0094] As can be seen from the above embodiments, the present application adopts a pseudo-label-based domain text pre-trained label recognition model, which can, to some extent, compensate for the problem of insufficient samples, introduce domain information, and reduce the reliance on manual annotation. Furthermore, the present application expands the labels of the training text through a matching model, thus alleviating the label quality problem to some extent. The label recognition model of the present application adopts a generative model, which is more flexible, does not require the construction of label hierarchy relationships, and does not limit the number and level of labels in the inference results. Supplemented by label post-processing, it further improves the accuracy of the labels.

[0095] The training method of the tag recognition model in the embodiments of this application has been described above. The following example of generating online course tags will provide a more detailed explanation of the above embodiments of this application.

[0096] This example mainly includes three parts: pre-training of the pseudo-labeled domain model (i.e., the label recognition model mentioned above, which is an encoder-decoder model in this example), fine-tuning training of the generative domain model (i.e., the label recognition model mentioned above), and prediction and inference of the generative domain model.

[0097] (1) The pretraining of pseudo-labeled generative domain models consists of the following two steps:

[0098] ① Constructing pseudo-labeled samples: Domain data is often abundant, providing the model with domain knowledge information that the limited task corpus cannot offer, helping the model to infer more accurately. Taking online courses as an example, available domain data includes, but is not limited to, video subtitles, courseware, and training materials. Here, the original domain data is collectively referred to as set G = {c i}, where c is called i This is the main text of the domain data. To reduce machine resource consumption, the main text c is limited. i The text length must not exceed 1024 characters. Eliminating the need for manual annotation, this example automatically constructs pseudo-tags by analyzing the goals and characteristics of the course tagging task and selecting the following method. i Firstly, select the main text c i First, obtain the top m words with the highest TF-IDF values ​​in the text; second, first obtain the main text c. i pseudo-summary s i This refers to the n sentences with the highest overlap with other sentences (see Jingqing et al., 2020). Specifically, the initial set of pseudo-summaries, s... i If empty, find the difference d between the text and the tags. i =c i -s i Choose a sentence from the set d such that the difference set d i And the pseudo-summary at this momenti The one with the highest Rouge-F1 score was included in the candidate pseudo-summaries. i In the process, repeat the above steps n times to obtain a pseudo-summary s containing n sentences. i Finally, the pseudo-abstract s was selected. i The top m words with the highest TF-IDF values ​​in the dataset. This yields the pseudo-label t. i Afterwards, the updated domain data is collectively referred to as set G = {c i} is a set G = {g i}={c i ,t i}, where g i This is a pseudo-labeled sample.

[0099] ② Generative pre-training: Based on the pseudo-labeled samples g constructed above i =(c i ,t i This example employs two pre-training tasks simultaneously: supervised pre-training and unsupervised pre-training. The supervised pre-training is a generative pseudo-labeling task, which involves generating pseudo-label samples from the text c. i The data is fed into an encoder-decoder model (e.g., the T5 model) to generate the label t of the pseudo-labeled sample. i The learning objective is to perform unsupervised pre-training as a generative MLM (Masked Language Model) task, which involves randomly replacing text c with [MASK] labels at a certain ratio. i Multiple words in the text are used to obtain a new body text. ～ i The words are then sequentially fed into the encoder-decoder model, with the learning objective being to reconstruct and generate the replaced words. After the model iterations are complete, the pre-trained model parameters are stored.

[0100] (2) Fine-tuning of generative domain models consists of the following three steps:

[0101] ① Addressing Tag Quality Issues: The course tag data used in this example consists of transcribed text / subtitle files of instructional video audio from online course scenarios, as well as multiple tags manually configured by users when uploading courses (the number of tags is not fixed). This example refers to the course tag data collectively as H = {h...} i}={x i ,y i Each course tag sample h i All are subtitle content x i and the tag set y i Composed of, where the tag set y i ={y ijIt consists of multiple tags. Due to the arbitrariness of users manually configuring tags, in order to make the tag set y i More reliable and comprehensive, this example provides a sample of each course tag h. i Match and add related tags. Simply put, first match the subtitle content x i The data is fed into the kNN model to obtain the k closest samples and k sets of labels. Each label in the k sets of labels is then compared with the subtitle content x. i The tag is sent to the course - tag matching model. If a match is found, the tag is added to the original tag set y. i In China. Specifically, such as... Figure 2 As shown, it consists of the following two steps:

[0102] 1) Training the Course-Label Matching Model: This example uses the BERT model for course-label matching. Positive and negative samples are constructed according to a preset a:b ratio, and for each subtitle content x... i Its tag set y i Each label in the set y is a positive sample label, and the set of labels y that are similar to it but do not belong to it is also a positive sample label. i The label y in j(j≠i) These are all negative sample labels. Limited by the maximum encoding length of 512, the subtitle content is first... i Divide into p subtitle segments {x ip Each subtitle segment is fed into the BERT model as a batch to obtain its encoded representation. To fuse the encoded representations of a set of subtitle segments and obtain the complete subtitle content x, the following steps are performed. i The encoding representation of A is given in this example using two methods: first, average pooling; second, padding the subtitle fragments to a preset maximum number of subtitle fragments to generate the encoding representation for each subtitle fragment, and then passing it through a fully connected layer of size 768. Then, it determines whether the current tag matches the subtitle content x. i The optimization objective is to match the values ​​of samples, meaning the output [CLS] value for positive samples should be close to 1, and the output [CLS] value for negative samples should be close to 0. After the model has completed its iterative training, the BERT model parameters are stored.

[0103] 2) Correct course tags: For each course sample {x i ,y i}, with subtitle content x i As input, a kNN model is used to select the k most similar course samples. From the k sets of labels of the k course samples, a label set y is selected. i Tags not included in the list are referred to as the tag set y in this example. j Then, the previously trained course-label matching model is loaded, and the caption content of the current course sample is sequentially xi and the tag set y j Each tag in the text is fed into the course-tag matching model to determine whether the current tag matches the subtitle content x. i Match. If a match is found, add the label to the label set y of the current course sample. i In the end, we obtained updated, higher-quality course label training data H. ～ ={h ～ i}={x i ,y ～ i}

[0104] ② Encoder-decoder generative training: Since the number of course labels is not fixed, this example uses a generative model to solve the course labeling task. The updated course label training data H from the previous text is used. ～ ={h ～ i}={x i ,y ～ i The training data is fed into the pre-trained pseudo-labeled domain model mentioned earlier. The goal of the course label generation task is to generate labels based on the given subtitle content x. i Automatically generate tag set text, where the tag set text is the tag set y ～ i This example concatenates the strings of various tags; no string concatenation symbols are specified. To enable the model to handle subtitle text exceeding 1024 characters, this example uses a sliding window w with a sliding step of w / 2, averaging multiple generation probabilities. After the model has completed iterative training, the encoder-decoder model parameters are stored.

[0105] ③ Course Tag Post-processing: In this example, tags in the manually constructed tag system tree are called official tags, and tags manually added by the user that do not belong to the tag system tree are called temporary tags. The tag library in this example includes both official and temporary tags. There is a certain hierarchical structure between official tags, while there is no explicit hierarchical structure between temporary tags, and due to the difficulty of manual maintenance, the number of temporary tags is often greater than that of official tags. For generated tags, if they do not belong to the tag library, a tag similarity matching algorithm is used to search for the tag closest to the generated tag in the tag library to replace the generated tag, thereby completing the tag normalization. If the generated tag is a tag in the tag library, then if the generated tag is an official tag, the missing tags can be filled in by tracing the tag tree path; in addition, character-level normalization can also be performed, such as capitalization and typo correction.

[0106] (3) Generative domain model prediction inference consists of the following two steps.

[0107] ① Encoder-decoder generative inference: Load the parameters of the pre-trained encoder-decoder model, and input the subtitle content of the course sample to be inferred. i The data is fed into the model to obtain the generated tag set text, which is then segmented using concatenation symbols to obtain the tag set y^. i .

[0108] ② Course tag post-processing: Similar to the course tag post-processing mentioned above in the training phase, it will not be repeated here.

[0109] As can be seen from the above examples, this example uses three innovative methods—pseudo-label generation pre-training, sample label quality optimization, and course label generation and post-processing—to solve the problems caused by the small sample size, high cost of manual annotation (long viewing time), poor quality of course labels, and non-fixed number of labels in online course label recognition.

[0110] The various methods of the embodiments of this application have been described above. Apparatus for implementing the above methods will now be provided.

[0111] Please refer to Figure 3 This application also provides a training apparatus for a label recognition model, comprising:

[0112] Generation module 31 is used to collect domain text and generate pseudo tags corresponding to the domain text;

[0113] The first training module 32 is used to construct pseudo-label samples based on the domain text and its corresponding pseudo-labels, and to pre-train the label recognition model using the pseudo-label samples to obtain the pre-trained label recognition model.

[0114] The construction module 33 is used to receive the text content input by the user and the original tag set corresponding to the text content, expand the original tag set corresponding to the text content to obtain an expanded tag set, and construct training samples based on the text content and its corresponding expanded tag set.

[0115] The second training module 34 is used to fine-tune the pre-trained label recognition model using the training samples to obtain the final label recognition model.

[0116] Through the above modules, the embodiments of this application can, to some extent, compensate for the problem of insufficient samples and reduce the reliance on manual annotation by introducing domain information.

[0117] Optionally, the generation module 31 is further configured to generate pseudo-tags corresponding to the domain text in at least one of the following ways:

[0118] Calculate the importance index of each word in the domain text, and extract the first number of words with the highest importance index as pseudo-tags corresponding to the domain text;

[0119] A pseudo-summary of the domain text is generated using a preset summarization algorithm; the importance index of each word in the pseudo-summary is calculated, and the second number of words with the highest importance index are extracted as pseudo-tags corresponding to the domain text.

[0120] Optionally, the first training module 32 is further configured to simultaneously train the label recognition model through a first training task and a second training task to obtain a pre-trained label recognition model; wherein,

[0121] The first training task is to input the domain text of the pseudo-label sample into the label recognition model, and train the model with the pseudo-labels generated by the label recognition model corresponding to the domain text as the optimization objective. The second training task is to randomly replace the words in the domain text of the pseudo-label sample with mask labels to generate new domain text; input the new domain text into the label recognition model, and train the model with the reconstruction of the words replaced by the mask labels as the optimization objective.

[0122] Optionally, the construction module 33 is further configured to, for the current text content, search for at least one similar text content that is closest to the current text content from all text content using a similar text search algorithm; identify tags that match the current text content from the original tag set corresponding to the similar text content, and add them to the original tag set corresponding to the current text content to obtain an extended tag set corresponding to the current text content.

[0123] Optionally, the construction module 33 is further configured to input each tag in the original tag set corresponding to the similar text content, together with the current text content, into a pre-trained matching model, wherein the matching model is used to identify whether the tag matches the text; and to obtain the tag that matches the current text content based on the identification result output by the matching model.

[0124] Optionally, the above-mentioned device further includes:

[0125] The third training module is used to train the matching model according to the following steps:

[0126] Construct positive and negative samples, wherein positive samples include text content and positive tags, and the positive tags are tags in the original tag set corresponding to the text content; negative samples include text content and negative tags, and the negative tags are tags that do not belong to the original tag set corresponding to the text content.

[0127] The text content from the positive and negative samples is input into the matching model to generate an encoded representation of the text content. The model is then iteratively trained with the optimization objective of whether the current label matches the current text content, thus obtaining the matching model.

[0128] Optionally, the third training module is further configured to generate an encoded representation of the text content in the following manner:

[0129] The text content is divided into multiple segments, an encoded representation of each segment is generated, and the encoded representations of the multiple segments are subjected to average pooling to obtain the encoded representation of the text content.

[0130] or,

[0131] The text content is divided into multiple segments. If the number of segments obtained is less than a preset number, padding is performed to obtain the preset number of segments. An encoded representation of each segment is generated and input into a fully connected layer to obtain the encoded representation of the text content.

[0132] Optionally, the second training module 34 is further configured to input the text content in the training samples into the pre-trained label recognition model, generate a set of labels text obtained by concatenating at least one label, and perform iterative training with the goal of optimizing the generated set of labels text to be close to the extended set of labels text corresponding to the text content, to obtain the final label recognition model, wherein the extended set of labels text is a string formed by concatenating each label in the extended set of labels.

[0133] Optionally, the second training module 34 is further configured to, when generating a generated tag set text obtained by concatenating at least one tag, determine whether the currently generated tag is a tag in the tag library, wherein the tag library includes formal tags in the tag system tree and temporary tags added by the user outside the tag system tree; if the currently generated tag is not a tag in the tag library, replace the currently generated tag with the closest tag in the tag library; if the currently generated tag is a tag in the tag library, perform character-level normalization processing on the currently generated tag according to the corresponding tag in the tag library.

[0134] Optionally, the second training module 34 is further configured to, when generating a generated tag set text obtained by concatenating at least one tag, supplement the parent tag of the currently generated tag by tracing the path in the tag system tree if the currently generated tag is an official tag in the tag library.

[0135] Optionally, the above-mentioned device further includes:

[0136] The reasoning module is used to reason about the text to be reasoned after obtaining the final label recognition model, and generate a set of reasoning labels corresponding to the text to be reasoned.

[0137] Optionally, the aforementioned reasoning module is further configured to, when generating the reasoning tag set corresponding to the text to be reasoned, determine whether the tags in the reasoning tag set are tags in the tag library, wherein the tag library includes formal tags in the tag system tree and temporary tags added by the user outside the tag system tree; if the tags in the reasoning tag set are not tags in the tag library, replace the tags in the reasoning tag set with the closest tags in the tag library; if the tags in the reasoning tag set are tags in the tag library, perform character-level normalization processing on the tags in the reasoning tag set according to the corresponding tags in the tag library.

[0138] Optionally, the above-mentioned reasoning module is further configured to, when generating the reasoning tag set corresponding to the text to be reasoned, supplement the parent tags of the tags in the reasoning tag set by performing path tracing in the tag system tree if the tags in the reasoning tag set are official tags in the tag library.

[0139] It should be noted that the device in this embodiment corresponds to the method described above, and the implementation methods in each of the above embodiments are applicable to the device embodiment and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail here.

[0140] Please refer to Figure 4 This application also provides a tag recognition model training device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the computer program is executed by the processor 401, it implements the various processes of the tag recognition model training method embodiment executed by the terminal described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0141] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described tag recognition model training method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0142] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0144] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A training method for a label recognition model, characterized in that, include: Collect domain text and generate pseudo-tags corresponding to the domain text, including at least one of the following: Calculate the importance index of each word in the domain text, and extract the first number of words with the highest importance index as pseudo-tags corresponding to the domain text; A pseudo-summary of the domain text is generated using a preset summarization algorithm; Calculate the importance index of each word in the pseudo-summary, and extract the second number of words with the highest importance index as pseudo-tags corresponding to the domain text; Based on the domain text and its corresponding pseudo-labels, pseudo-label samples are constructed, and the pseudo-label samples are used to pre-train the label recognition model to obtain the pre-trained label recognition model. The system receives text content input by the user and a set of original tags corresponding to the text content. It then expands the set of original tags corresponding to the text content to obtain an expanded tag set. Based on the text content and its corresponding expanded tag set, it constructs training samples, including: For the current text content, a similar text search algorithm is used to search for at least one similar text content that is closest to the current text content from all text content. Using the training samples, the pre-trained label recognition model is fine-tuned to obtain the final label recognition model; The original tag set corresponding to the text content is expanded to obtain an expanded tag set, which also includes: From the original tag set corresponding to the similar text content, identify the tags that match the current text content and add them to the original tag set corresponding to the current text content to obtain the extended tag set corresponding to the current text content; From the original tag set corresponding to the similar text content, identify tags that match the current text content, including: Each tag in the original tag set corresponding to the similar text content is input together with the current text content into a pre-trained matching model, wherein the matching model is used to identify whether the tag matches the text; Based on the recognition results output by the matching model, a tag matching the current text content is obtained.

2. The method according to claim 1, characterized in that, The label recognition model is pre-trained using the pseudo-label samples to obtain a pre-trained label recognition model, including: The label recognition model is trained simultaneously through a first training task and a second training task to obtain a pre-trained label recognition model; wherein... The first training task is to input the domain text of the pseudo-label sample into the label recognition model, and train the model with the pseudo-labels generated by the label recognition model corresponding to the domain text as the optimization objective. The second training task is to randomly replace the words in the domain text of the pseudo-label sample with mask labels to generate new domain text; input the new domain text into the label recognition model, and train the model with the reconstruction of the words replaced by the mask labels as the optimization objective.

3. The method according to claim 1, characterized in that, Also includes: The matching model is trained by following these steps: Construct positive and negative samples, wherein positive samples include text content and positive tags, and the positive tags are tags in the original tag set corresponding to the text content; negative samples include text content and negative tags, and the negative tags are tags that do not belong to the original tag set corresponding to the text content. The text content from the positive and negative samples is input into the matching model to generate an encoded representation of the text content. The model is then iteratively trained with the optimization objective of whether the current label matches the current text content, thus obtaining the matching model.

4. The method according to claim 3, characterized in that, The encoded representation of the generated text content includes: The text content is divided into multiple segments, an encoded representation of each segment is generated, and the encoded representations of the multiple segments are subjected to average pooling to obtain the encoded representation of the text content. or, The text content is divided into multiple segments. If the number of segments obtained is less than a preset number, padding is performed to obtain the preset number of segments. An encoded representation of each segment is generated and input into a fully connected layer to obtain the encoded representation of the text content.

5. The method according to claim 1, characterized in that, The step of fine-tuning the pre-trained label recognition model using the training samples includes: The text content in the training samples is input into the pre-trained label recognition model to generate a generated label set text obtained by concatenating at least one label. The model is iteratively trained with the goal of optimizing the generated label set text to be close to the extended label set text corresponding to the text content, so as to obtain the final label recognition model. The extended label set text is a string formed by concatenating each label in the extended label set.

6. The method according to claim 5, characterized in that, When generating a text set of generated tags by concatenating at least one tag, the method further includes: Determine whether the currently generated tag is a tag in the tag library, which includes official tags in the tag system tree and temporary tags added by the user that are outside the tag system tree; If the currently generated tag is not a tag in the tag library, replace the currently generated tag with the closest tag in the tag library; If the currently generated tag is a tag in the tag library, the currently generated tag is normalized at the character level according to the corresponding tag in the tag library.

7. The method according to claim 6, characterized in that, When generating a text set of generated tags by concatenating at least one tag, the method further includes: If the currently generated tag is an official tag in the tag library, the parent tag of the currently generated tag is completed by tracing the path in the tag system tree.

8. The method according to claim 1, characterized in that, After obtaining the final label recognition model, the method further includes: The label recognition model is used to infer the text to be inferred, and a set of inference labels corresponding to the text to be inferred is generated.

9. The method according to claim 8, characterized in that, When generating the set of inference tags corresponding to the text to be inferred, the method further includes: Determine whether the tags in the inference tag set are tags in the tag library, the tag library including official tags in the tag system tree and temporary tags added by the user outside the tag system tree; If a tag in the inference tag set is not a tag in the tag library, the tag in the inference tag set is replaced with the closest tag in the tag library; If the tags in the inference tag set are tags in the tag library, then the tags in the inference tag set are normalized at the character level according to the corresponding tags in the tag library.

10. The method according to claim 9, characterized in that, When generating the set of inference tags corresponding to the text to be inferred, the method further includes: If a tag in the inference tag set is an official tag in the tag library, the parent tags of the tags in the inference tag set are supplemented by tracing the path in the tag system tree.

11. A training device for a label recognition model, characterized in that, include: The generation module is used to collect domain text and generate pseudo-tags corresponding to the domain text, including at least one of the following: Calculate the importance index of each word in the domain text, and extract the first number of words with the highest importance index as pseudo-tags corresponding to the domain text; A pseudo-summary of the domain text is generated using a preset summarization algorithm; Calculate the importance index of each word in the pseudo-summary, and extract the second number of words with the highest importance index as pseudo-tags corresponding to the domain text; The first training module is used to construct pseudo-label samples based on the domain text and its corresponding pseudo-labels, and to pre-train the label recognition model using the pseudo-label samples to obtain the pre-trained label recognition model. A construction module is used to receive text content input by the user and the original tag set corresponding to the text content, expand the original tag set corresponding to the text content to obtain an expanded tag set, and construct training samples based on the text content and its corresponding expanded tag set, including: For the current text content, a similar text search algorithm is used to search for at least one similar text content that is closest to the current text content from all text content. The second training module is used to fine-tune the pre-trained label recognition model using the training samples to obtain the final label recognition model. The original tag set corresponding to the text content is expanded to obtain an expanded tag set, which also includes: From the original tag set corresponding to the similar text content, identify the tags that match the current text content and add them to the original tag set corresponding to the current text content to obtain the extended tag set corresponding to the current text content; From the original tag set corresponding to the similar text content, identify tags that match the current text content, including: Each tag in the original tag set corresponding to the similar text content is input together with the current text content into a pre-trained matching model, wherein the matching model is used to identify whether the tag matches the text; Based on the recognition results output by the matching model, a tag matching the current text content is obtained.

12. A training device for a label recognition model, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 10.

Images