Model training method, multi-label qualitative word recognition method and related device

Abstract

This application discloses a multi-model training method, a multi-label qualitative word recognition method, and related equipment. The training method includes: selecting N target characters corresponding to N multi-label qualitative words from sample text containing multi-label qualitative words; encoding the N target characters after masking processing using a pre-trained language model of a word recognition model based on the sample text and the N multi-label qualitative words to obtain a representation vector corresponding to each target character; classifying and recognizing the N multi-label qualitative words based on each representation vector using a classifier of the word recognition model to obtain the predicted word label attributes corresponding to each multi-label qualitative word; adjusting the model parameters of the word recognition model based on the predicted word label attributes corresponding to the N multi-label qualitative words and the N sample word label attributes of the sample text, wherein the sample word label attributes are used to represent the true semantics of the corresponding multi-label qualitative words in the sample text.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a multi-model training method, a multi-label qualitative word recognition method, and related equipment. Background Technology

[0002] In voice quality inspection systems, it is typically necessary to identify multi-label qualitative words in sentences to assess call quality. Multi-label qualitative words refer to words whose semantics differ in different contexts but whose category attributes are the same. For example, the texts “Sir, you owe money and haven't paid it back, and you're going to complain? That doesn't make sense.”, “Sir, you can complain, but you owe money and haven't paid it back, so it still needs to be resolved.”, “Sir, you're the one who owes money and hasn't paid it back, and you're going to complain about me? Will complaining about me make you not pay?”, and “Go ahead and complain, it's your business.” all contain the multi-label qualitative word “go and complain.” This word is classified as non-professional terminology, but while it doesn't imply responsibility in the first and third texts, it does imply responsibility in the second and fourth texts.

[0003] In related technologies, the recognition of multi-label qualitative words usually adopts the Named Entity Recognition (NER) method. The multi-label qualitative words and their semantics in the sample text are pre-annotated. Then, the sample text and the annotation information it carries are used to train the named entity recognition model. Then, the text to be recognized is input into the trained named entity recognition model, which can output the multi-label qualitative words in the text to be recognized and their semantics in the text to be recognized.

[0004] However, the NER task has the problem of recognition boundary, such as misidentifying "TV station" as "television". Also, because the named entity recognition model has a certain generalization ability, it will recognize unlabeled words, which will affect the recognition accuracy. Summary of the Invention

[0005] The purpose of this application is to provide a multi-model training method, a multi-label qualitative word recognition method, and related equipment to solve the problem of low recognition accuracy in multi-label qualitative word recognition methods in related technologies.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a method for training a word recognition model, comprising:

[0008] Select N target characters corresponding to the N multi-label qualitative words from the sample text containing multi-label qualitative words. Each multi-label qualitative word refers to a word with different semantics in different contexts but the same attribute category; one multi-label qualitative word corresponds to one target character; N is an integer greater than 1;

[0009] Based on the sample text and the N multi-label qualitative words, the pre-trained language model of the word recognition model encodes the N target characters after masking to obtain the representation vector corresponding to each target character.

[0010] The classifier of the word recognition model classifies and identifies the N multi-label qualitative words based on each representation vector, and obtains the predicted word label attributes corresponding to each of the N multi-label qualitative words. Each predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text.

[0011] Based on the predicted word label attributes corresponding to each of the N multi-label qualitative words and the N sample word label attributes of the sample text, the model parameters of the word recognition model are adjusted. Each sample word label attribute is used to represent the true semantics of the corresponding multi-label qualitative word in the sample text. The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0012] The word recognition model training method provided in this application adopts a technical approach of text classification task plus post-processing task. First, it utilizes the text understanding, prediction, and text representation capabilities of a pre-trained language model to select the target characters corresponding to N multi-label qualitative words from the sample text. Then, the pre-trained language model encodes the N target characters after masking based on the sample text and the N multi-label qualitative words. This ensures that each representation vector obtained by the pre-trained language model is obtained through character prediction and semantic representation based on a thorough learning and understanding of the semantics of the sample text and the true semantics of each multi-label qualitative word in the sample text. Therefore, each representation vector can be used as the word label attribute vector of the corresponding multi-label qualitative word to accurately represent that multi-label qualitative word. The model identifies the true semantics of the sample text. Based on this, a text classification task is performed using the classifier of the word recognition model. Based on each representation vector, the corresponding multi-label qualitative words are classified and identified to obtain the predicted word label attributes corresponding to each multi-label qualitative word. Based on the predicted word label attributes corresponding to N multi-label qualitative words and the label attributes of N sample words in the sample text, the model parameters of the word recognition model are adjusted. The word recognition model trained in this way can perform multi-label qualitative word recognition based on accurately understanding the semantics of the text and the semantics of the words in the text. Compared with the NER model that performs the NER task, it can effectively avoid the problems of boundary recognition errors or recognition of unlabeled words that affect the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0013] Secondly, embodiments of this application provide a multi-label qualitative word recognition method, including:

[0014] Obtain the text to be recognized;

[0015] The text to be identified is input into a word recognition model to obtain predicted multi-label qualitative words in the text to be identified and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attributes are used to represent the semantics of the predicted multi-label qualitative words in the text to be identified.

[0016] The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text to be recognized based on the representation vector of the text to be recognized to obtain predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The word recognition model is trained based on the training method of the word recognition model described in the first aspect.

[0017] The multi-label qualitative word recognition method provided in this application uses a pre-trained language model of a trained word recognition model to perform multi-label qualitative word recognition on the text to be recognized. Since the trained word recognition model can perform multi-label qualitative word recognition based on an accurate understanding of the semantics of the text and the semantics of the words in the text, compared with the NER model performing the NER task, it can effectively avoid the problem of boundary recognition errors or recognition of unlabeled words that affect the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0018] Thirdly, embodiments of this application provide a training apparatus for a word recognition model, comprising:

[0019] The acquisition unit is used to select the target character corresponding to the multi-label qualitative words from the sample text containing multi-label qualitative words. The multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category.

[0020] The encoding unit is used to encode the target character after masking based on the sample text and the multi-label qualitative words using a pre-trained language model of the word recognition model, so as to obtain the representation vector corresponding to the target character;

[0021] The recognition unit is used to classify and recognize the multi-label qualitative words based on the representation vector using the classifier of the word recognition model, and obtain the predicted word label attribute corresponding to the multi-label qualitative words, wherein the predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative words in the sample text;

[0022] The adjustment unit is used to adjust the model parameters of the word recognition model based on the predicted word label attributes and sample word label attributes corresponding to the multi-label qualitative words. The sample word label attributes are used to represent the true semantics of the corresponding multi-label qualitative words in the sample text. The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0023] Fourthly, embodiments of this application provide a multi-tag qualitative word recognition device, comprising:

[0024] The acquisition unit is used to acquire the text to be recognized;

[0025] The recognition unit is used to input the text to be recognized into the word recognition model to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attributes are used to represent the semantics of the predicted multi-label qualitative words in the text to be recognized.

[0026] The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text to be recognized based on the representation vector of the text to be recognized to obtain predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The word recognition model is trained based on the training method of the word recognition model described in the first aspect.

[0027] Fifthly, embodiments of this application provide an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the word recognition model training method as described in the first aspect; or, the processor is configured to execute the instructions to implement the multi-label qualitative word recognition method as described in the second aspect.

[0028] In a sixth aspect, embodiments of this application provide a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the word recognition model training method as described in the first aspect; or, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the multi-label qualitative word recognition method as described in the second aspect. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 A flowchart illustrating a training method for a word recognition model provided in one embodiment of this application;

[0031] Figure 2 A flowchart illustrating a training method for a word recognition model provided in another embodiment of this application;

[0032] Figure 3 A flowchart illustrating a multi-label qualitative word recognition method provided for one embodiment of this application;

[0033] Figure 4 A flowchart illustrating a multi-label qualitative word recognition method provided for another embodiment of this application;

[0034] Figure 5 A schematic diagram of the structure of a training device for a word recognition model provided in one embodiment of this application;

[0035] Figure 6 A schematic diagram of the structure of a multi-label qualitative word recognition device provided in one embodiment of this application;

[0036] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0039] Explanation of some concepts:

[0040] Named Entity Recognition (NER) refers to the identification of entities with specific meanings in text, mainly including names of people, places, organizations, and proper nouns.

[0041] LSTM (Long Short-Term Memory): A type of recurrent neural network designed to address the long-term dependency problem inherent in general recurrent neural networks, and capable of performing better on longer sequences.

[0042] BiLSTM (Bi-directional Long Short-Term Memory): A bidirectional LSTM, which is composed of a forward LSTM and a backward LSTM.

[0043] BERT: A pre-trained language model that uses a bidirectional encoder representation from a Transformer. Pre-trained BERT representations can be fine-tuned with an additional output layer, making it suitable for building state-of-the-art models across a wide range of tasks.

[0044] MLM (Masked Language Model): One of the learning tasks in BERT pre-training. It randomly selects a portion of the input text, replaces it with a mask, and then trains the model to correctly predict the masked characters.

[0045] NSP (Next Sentence Prediction): Predicting sentence pair relationships, one of the learning tasks in BERT pre-training. Given two sentences, determine whether sentence B is the next sentence after sentence A.

[0046] As described in the background section, in related technologies, the Named Entity Recognition (NER) method is typically used to identify multi-label qualitative words. This method utilizes BIO to pre-annotate multi-label qualitative words and their semantics in the sample text. For example, the following is a sample text and its annotated multi-label qualitative words, where O represents a character that is not a multi-label qualitative word, B represents the first character of a multi-label qualitative word, I represents a non-first character of a multi-label qualitative word, GW represents a category attribute, and WZ represents a semantic meaning.

[0047] You O

[0048] To O

[0049] Electricity B-GW-WZ

[0050] I-GW-WZ

[0051] Taiwan I-GW-WZ

[0052] Go to O

[0053] On O

[0054] Class O

[0055] It is O

[0056] No O

[0057] It is O

[0058] ? O

[0059] Then, the named entity recognition model is trained using sample text and its accompanying annotation information. The text to be recognized is then input into the trained model, which outputs the multi-labeled qualitative words in the text and their semantic meaning. However, the NER task suffers from recognition boundary issues, such as misidentifying "TV station" as "television." Furthermore, because the named entity recognition model has a certain generalization ability, it may recognize unlabeled words, affecting the accuracy of the recognition.

[0060] In view of this, the embodiments of this application aim to propose a training method for a word recognition model. The method selects the target characters corresponding to N multi-label qualitative words from the sample text and encodes the masked N target characters using a pre-trained language model based on the sample text and the N multi-label qualitative words. This ensures that each representation vector obtained by the pre-trained language model is obtained through character prediction and semantic representation based on a thorough learning and understanding of the semantics of the sample text and the true semantics of each multi-label qualitative word in the sample text. Therefore, each representation vector can be used as the word label attribute vector of the corresponding multi-label qualitative word to accurately represent the true semantics of the multi-label qualitative word in the sample text. Based on this, the method further... The word recognition model performs text classification tasks using a classifier. Based on each representation vector, it classifies and identifies the corresponding multi-label qualitative words, obtaining the predicted word label attributes corresponding to each multi-label qualitative word. Based on the predicted word label attributes corresponding to N multi-label qualitative words and the label attributes of N sample words in the sample text, the model parameters of the word recognition model are adjusted. The word recognition model trained in this way can perform multi-label qualitative word recognition based on an accurate understanding of the semantics of the text and the semantics of the words in the text. Compared with the NER model performing the NER task, it can effectively avoid boundary recognition errors or the problem of recognizing unlabeled words, which affects the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0061] Based on the trained word recognition model, this application also proposes a multi-label qualitative word recognition method. The method utilizes the pre-trained language model of the trained word recognition model to perform multi-label qualitative word recognition on the text to be recognized. Since the trained word recognition model can perform multi-label qualitative word recognition based on an accurate understanding of the semantics of the text and the semantics of the words in the text, compared with the NER model performing the NER task, it can effectively avoid the problems of boundary recognition errors or recognition of unlabeled words that affect the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0062] It should be understood that the training method for the word recognition model and the multi-label qualitative word recognition method proposed in the embodiments of this application can both be executed by electronic devices or software installed in electronic devices. The electronic devices referred to herein may include terminal devices, such as smartphones, tablets, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, smartwatches, vehicle terminals, aircraft, etc.; or, the electronic devices may also include servers, such as independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0063] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0064] Please see Figure 1 The following is a flowchart illustrating a training method for a word recognition model, provided as an embodiment of this application. The method may include the following steps:

[0065] S102, select N target characters corresponding to N multi-label qualitative words from the sample text containing multi-label qualitative words.

[0066] In this context, each multi-label qualitative word refers to a word whose semantics differs across contexts but whose attribute category remains the same. One multi-label qualitative word corresponds to one target character, and N is an integer greater than 1. For each multi-label qualitative word, it possesses different word label attributes in different contexts. The word label attribute corresponding to a multi-label qualitative word in a given context represents its true semantic meaning within that context. Multi-label qualitative words also share the same category attribute across different contexts. The sample word label attribute represents the true semantic meaning of the corresponding multi-label qualitative word in the sample text, while the category attribute represents the category to which the word belongs in a classification task. For example, taking a word classification task in a speech quality inspection scenario, word label attributes may include, but are not limited to, "responsible" and "not responsible," while category attributes may include, but are not limited to, "high-risk," "risky," "unprofessional terminology," and "serious attitude."

[0067] In this embodiment, each multi-tag qualitative word in the sample text has a corresponding sample word tag attribute and a category attribute. Here, the sample word tag attribute represents the true semantic meaning of the corresponding multi-tag qualitative word in its respective sample text. In practical applications, the sample text may include at least one multi-tag qualitative word, and each multi-tag qualitative word has a corresponding sample word tag attribute and a category attribute. For example, Table 1 shows some sample texts containing multi-tag qualitative words and the corresponding sample word tag attributes and category attributes of the multi-tag qualitative words.

[0068] Table 1

[0069]

[0070] In the above S102, various methods can be used to select the target characters corresponding to the multi-label qualitative words, and the specific method can be selected according to actual needs.

[0071] Optionally, in S102 above, a subset of characters can be randomly selected from the sample text as the target characters corresponding to the multi-label qualitative terms. Specifically, when the sample text contains multiple multi-label qualitative terms, for each multi-label qualitative term, a subset of characters can be randomly selected from the sample text as the target characters corresponding to that multi-label qualitative term.

[0072] For example, taking the first sample text in Table 1 above, we can randomly select some characters from the sample text, such as "going to work," as the target characters corresponding to the multi-label qualitative word "TV station." Similarly, taking the fourth sample text in Table 1 above, we can randomly select some characters from the sample text, such as "not having," as the target characters corresponding to the first multi-label qualitative word "quality."

[0073] Understandably, using random character selection can improve the text understanding ability of the pre-trained language model in the word recognition model during the training process, thereby enhancing the semantic representation ability of the pre-trained language model.

[0074] Optionally, in S102 above, the target character corresponding to the multi-label qualitative word can be randomly selected from the sentence containing the multi-label qualitative word in the sample text. Specifically, when the sample text contains multiple multi-label qualitative words, firstly, the sentence containing the target multi-label qualitative word in the sample text is determined, wherein the target multi-label qualitative word is any one of the multiple multi-label qualitative words; further, a portion of characters are randomly selected from the sentence containing the target multi-label qualitative word as the target character corresponding to the target multi-label qualitative word.

[0075] For example, taking the second sample text in Table 1 above, for the multi-tag qualitative word "TV station", some characters can be randomly selected from the sentence "I'll find the TV station to expose you", such as "expose", as the target character corresponding to the multi-tag qualitative word. Similarly, taking the third sample text in Table 1 above, for the first multi-tag qualitative word "quality", some characters can be randomly selected from the sentence "How can you speak so rudely", such as "speak", as the target character corresponding to the multi-tag qualitative word. And for the second multi-tag qualitative word "quality", some characters can be randomly selected from the sentence "You should still pay attention to your quality", such as "pay attention", as the target character corresponding to the multi-tag qualitative word.

[0076] It is understandable that, since sample text may contain multiple multi-label qualitative words with the same characters, but these multi-label qualitative words may have different true meanings in the sample text, if the target characters for these identical multi-label qualitative words are selected separately from the entire sample text, there is a possibility that the selected target characters may be the same, causing the pre-trained language model to be unable to distinguish the true meanings of these multi-label qualitative words, thereby affecting the training effect of the word recognition model. In the above implementation, since the sentence containing each multi-label qualitative word can reflect the context of each multi-label qualitative word, randomly selecting the target characters corresponding to each multi-label qualitative word from the sentence containing each multi-label qualitative word is beneficial for the pre-trained language model to distinguish the true meanings of each multi-label qualitative word, thereby improving the training effect of the word recognition model.

[0077] This application embodiment illustrates a specific implementation of S102 described above. It should be understood that S102 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0078] S104 uses a pre-trained language model of the word recognition model to encode the N target characters after masking based on the sample text and N multi-label qualitative words, thereby obtaining the representation vector corresponding to each target character.

[0079] In this embodiment, the pre-trained language model refers to a language model that has been pre-trained and can predict the masked character using the context of the masked character, and then perform semantic representation (i.e., encoding) on ​​the predicted character to obtain a representation vector that can be recognized and processed by a computer to represent the predicted character. In practical applications, the pre-trained language model can be any model with character prediction and semantic representation capabilities, such as BERT, ROBERT, etc., and can be selected according to actual needs. This embodiment does not limit this selection.

[0080] Based on this, the sample text and multi-label qualitative words can serve as the context of the target characters. In S104 above, a pre-trained language model can predict the N target characters after masking based on the sample text and N multi-label qualitative words, obtaining the predicted character corresponding to each target character. Furthermore, the pre-trained language model encodes the predicted character corresponding to each target character, obtaining the representation vector corresponding to each target character. Since this representation vector is obtained by encoding the predicted character based on the understanding of the semantics of the sample text and the true semantics of the corresponding multi-label qualitative words in the sample text by the pre-trained language model, this representation vector can reflect the semantics of the multi-label qualitative words to a certain extent. Therefore, the representation vector corresponding to each target character can be used as the word label attribute vector of the corresponding multi-label qualitative word to represent the word label attribute of the multi-label qualitative word.

[0081] In order for the pre-trained language model to fully learn and understand the semantics of the target characters corresponding to the multi-label qualitative words from the sample text and the multi-label qualitative words, and to perform character prediction and encoding, the above S104 may include the following steps:

[0082] S141, at least the sample text, the multi-label qualitative words in the sample text, and the target characters corresponding to the multi-label qualitative words are concatenated to obtain the concatenated text.

[0083] In the first optional implementation, the sample text, the multi-tag qualitative words in the sample text, and the target characters corresponding to the multi-tag qualitative words can be concatenated according to a preset format to obtain the concatenated text. For example, the preset format can be:

[0084] [CLS]text[SEP]keyword_1,string_1[SEP]keyword_2,string_2…[SEP]

[0085] In the above preset format, [CLS] is a start identifier used to indicate the beginning of the concatenated text; [SEP] is a split identifier used to indicate the split between sentences, and each sentence ends with [SEP]; text represents the sample text; keyword_1, keyword_2, ... represent the first multi-tag qualitative word, the second multi-tag qualitative word, etc. in the sample text, respectively; string_1, string_2 represent the target characters corresponding to the first multi-tag qualitative word, the second multi-tag qualitative word, etc., respectively.

[0086] For example, taking the first and fourth sample texts shown in Table 1 above as examples, assuming that the target character corresponding to the multi-tag qualitative term "TV station" in the first sample text is "going to work", and the target character corresponding to the first multi-tag qualitative term "quality" in the fourth sample text is "not", the concatenated text shown below can be obtained:

[0087] The concatenated text corresponding to the first sample text: [CLS]Are you going to work at the TV station? [SEP]TV station, work [SEP];

[0088] The concatenated text corresponding to sample text #4: [CLS] Don't keep saying those things all the time, it only proves that you have no manners at all. [SEP] No manners, no manners. [SEP]

[0089] It is understandable that the concatenated text obtained in the above manner can clearly define the boundaries between the sample text, each multi-label qualitative word, and the target character, which is beneficial for the pre-trained language model to accurately learn and understand the semantics of the target character corresponding to the multi-label qualitative word and perform character prediction and encoding.

[0090] In the second optional implementation, the positions of multi-tag qualitative words in the sample text can be obtained, and the sample text, the multi-tag qualitative words in the sample text, the positions of the multi-tag qualitative words in the sample text, and the target characters corresponding to the multi-tag qualitative words can be concatenated according to a preset format to obtain the concatenated text. For example, the preset format can be:

[0091] [CLS]text[SEP]keyword_1,position_1,string_1[SEP]keyword_2,,position_1,string_2…[SEP].

[0092] In the above preset format, [CLS] is a start identifier used to indicate the beginning of the concatenated text; [SEP] is a separator identifier used to indicate the separation between sentences, and each sentence ends with [SEP]; text represents the sample text; keyword_1, keyword_2… represent the first multi-tag qualitative word, the second multi-tag qualitative word, etc., in the sample text, respectively; position_1, position_2… represent the position of the first multi-tag qualitative word, the position of the second multi-tag qualitative word, etc., in the sample text, respectively; string_1, string_2… represent the target character corresponding to the first multi-tag qualitative word, the target character corresponding to the second multi-tag qualitative word, etc. The position of the multi-tag qualitative word in the sample text can include the sequence number of the first character and the sequence number of the last character of the multi-tag qualitative word in the sample text.

[0093] For example, continuing with the above example, the concatenated text corresponding to the first sample text and the concatenated text corresponding to the fourth sample text obtained by using the above concatenation method are shown below:

[0094] The concatenated text corresponding to the first sample text is: [CLS]Are you going to work at the TV station? [SEP]TV station,2,4,work [SEP];

[0095] The concatenated text corresponding to sample text #4: [CLS] Don't keep saying those things all the time, it only proves that you have no manners at all [SEP] Manners, 22, 23 No [SEP].

[0096] Among them, in the spliced text corresponding to the first sample text, 2 indicates that the serial number of the first character '电' of the multi-label qualitative word '电视台' in the first sample text is 2; 4 indicates that the serial number of the last character '台' of the multi-label qualitative word '电视台' in the first sample text is 4. In the spliced text corresponding to the fourth sample text, 22 indicates that the serial number of the first character '素' of the multi-label qualitative word '素质' in the fourth sample text is 22, and 23 indicates that the serial number of the last character '质' of the multi-label qualitative word '素质' in the fourth sample text is 23.

[0097] [[ID=३]]It can be understood that the spliced text spliced by the above splicing method can not only clarify the boundaries between the sample text, each multi-label qualitative word and the target character, thus facilitating the pre-trained language model to accurately learn and understand the semantics of the target character corresponding to the multi-label qualitative word for character prediction and encoding, but also enables the pre-trained language model to distinguish different multi-label qualitative words based on the position information of the multi-label qualitative word. Especially in the case where the sample text contains multiple multi-label qualitative words with the same characters but different real semantics, so as to accurately understand and represent the semantic information of the multi-label qualitative word in the sample text.

[0098] S142. Mask the target characters in the spliced text to obtain a masked text.

[0099] In the above S142, the target characters in the spliced text can be masked in various appropriate ways to obtain a masked text. Optionally, replacing the target characters in the spliced text with a preset masked character can obtain the masked text. Among them, the preset masked character can be set according to actual needs. For example, the preset masked character can be [MASK] to be clearly distinguished from the characters in the spliced text, etc. The form of the preset masked character is not limited in the embodiments of the present application.

[0100] Exemplarily, continuing to use the example in the above S141, for the spliced text corresponding to the first sample text and the spliced text corresponding to the fourth sample text obtained by the first implementation method, by masking the target characters in the spliced text, the following texts can be obtained:

[0101] Masked text corresponding to the first sample text: [CLS] Are you going to work at the TV station? [SEP] TV station, [MASK] [SEP];

[0102] Masked text corresponding to the fourth sample text: [CLS] Don't keep saying those words of yours all the time. It only proves that you have no quality at all [SEP] Quality, [MASK] [SEP].

[0103] For the concatenated text corresponding to the first sample text and the concatenated text corresponding to the fourth sample text obtained through the second implementation method, by masking the target characters in the concatenated text, the following text can be obtained:

[0104] The masked text corresponding to the first sample text is: [CLS]Are you going to work at the TV station? [SEP]TV station,2,4, [MASK] [SEP];

[0105] The masked text corresponding to sample text #4: [CLS] Don't keep those words on your lips all the time, it only proves that you have no manners at all [SEP] manners, 22, 23 [MASK] [SEP].

[0106] Understandably, replacing the target characters in the concatenated text with preset mask characters is simple to implement, highly efficient in masking, and beneficial for improving the training efficiency of word recognition models.

[0107] This application embodiment illustrates a partial implementation of masking target characters in concatenated text. Of course, it should be understood that masking of target characters can also be achieved using various techniques commonly used in the art, and this application embodiment does not limit this approach.

[0108] S143 uses a pre-trained language model to encode the masked target characters based on the masked text, thus obtaining the representation vector corresponding to the target characters.

[0109] Specifically, the masked text and the sample word labels corresponding to the multi-label qualitative words in the masked text can be used as input to the pre-trained language model. The sample word labels corresponding to the multi-label qualitative words provide supervision signals, enabling the pre-trained language model to learn and understand the contextual information of the target character from the masked text based on the supervision signals, understand the overall semantics of the masked text, and then predict the target character after masking to obtain the predicted character corresponding to the target character. The predicted character is then encoded to obtain the representation vector corresponding to the target character, which can be used as the word label attribute vector corresponding to the multi-label qualitative words of the target character.

[0110] This application embodiment illustrates a specific implementation of S104 described above. It should be understood that S104 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0111] S106, the classifier of the word recognition model classifies and identifies N multi-label qualitative words based on each representation vector, and obtains the predicted word label attributes corresponding to each of the N multi-label qualitative words.

[0112] In this context, the predicted word label attribute corresponding to the multi-label qualitative word is used to represent the predicted semantics of the multi-label qualitative word in the sample text. In practical applications, the classifier can be any model with classification and recognition capabilities commonly used in this field, and the specific model can be selected according to actual needs. This application embodiment does not limit this selection.

[0113] S108, based on the predicted word label attributes corresponding to N multi-label qualitative words and the N sample word label attributes of the sample text, adjust the model parameters of the word recognition model.

[0114] The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0115] The model parameters of the word recognition model can include the model parameters of the pre-trained language model and the model parameters of the classifier. The model parameters of the pre-trained language model can include the number of processing nodes (e.g., neurons) in the pre-trained language model, the connection relationships and edge weights between processing nodes in different network layers, and the biases corresponding to the processing nodes in each network layer. Similarly, the model parameters of the classifier can include the number of processing nodes (e.g., neurons) in the classifier, the connection relationships and edge weights between processing nodes in different network layers, and the biases corresponding to the processing nodes in each network layer.

[0116] Optionally, in order for the word recognition model to accurately identify multi-label qualitative words and their true semantics in the text, the above S108 may include: determining the recognition loss of the word recognition model based on the predicted word label attributes and sample word label attributes corresponding to each of the N multi-label qualitative words; and adjusting the model parameters of the word recognition model based on the recognition loss of the word recognition model.

[0117] In this context, the recognition loss of the word recognition model represents the difference between the true semantics and the predicted semantics of multi-label qualitative words in the sample text. The recognition loss of the word recognition model can be calculated using various loss functions, such as the sigmoid function or the softmax function; this embodiment does not limit the specific loss function used. After determining the recognition loss of the word recognition model, a backpropagation algorithm can be used to sequentially adjust the model parameters of the classifier and the word recognition model based on the recognition loss.

[0118] It is worth noting that the number of sample texts used to train the word recognition model can be multiple. For each sample text, the predicted word label attributes corresponding to the multi-label qualitative words in the sample text can be obtained through S102~S106 above. Furthermore, in S108 above, the model parameters of the word recognition model are adjusted based on the predicted word label attributes corresponding to N multi-label qualitative words in multiple sample texts and the sample word label attributes corresponding to each multi-label qualitative word.

[0119] Furthermore, the above process is only a single training iteration for the word recognition model. In practical applications, multiple iterations of training may be required for the word recognition model. Therefore, steps S102 to S108 can be repeated multiple times until a preset training stopping condition is met. The preset training stopping condition may include: the recognition loss of the word recognition model is less than a preset loss threshold; or, the recognition accuracy of the word recognition model is greater than or equal to a preset accuracy threshold; or, the number of iterations reaches a preset number, etc. These conditions can be set according to actual needs, and this embodiment does not limit them.

[0120] To enable those skilled in the art to better understand the training method of the word recognition model provided in the embodiments of this application, the following is combined with... Figure 2 The training method of the word recognition model provided in the embodiments of this application will be described.

[0121] like Figure 2 As shown, firstly, target characters (target character 1, target character 2, etc.) corresponding to each multi-tag qualitative word (keyword_1, keyword_2, etc.) are selected from the sample text containing multi-tag qualitative words. Then, the sample text, each multi-tag qualitative word, the position of each multi-tag qualitative word in the sample text (position_1, position_2, etc.), and the target characters corresponding to each multi-tag qualitative word are concatenated according to a preset format. Finally, the target characters are masked to obtain the masked text.

[0122] [CLS]text[SEP]keyword_1,position_1,string_1[SEP]keyword_2,,position_1,string_2…[SEP]. Next, the pre-trained language model of the word recognition model predicts the preset mask characters [MASK] in the masked text based on the masked text, obtaining the predicted characters corresponding to each preset mask character [MASK]. The predicted characters are then encoded to obtain the representation vectors vec1, vec2, etc., corresponding to each target character. The representation vector corresponding to each target character can be used as the word label attribute vector of the multi-label qualitative word corresponding to each target character, so as to represent the true semantics of the multi-label qualitative word in the sample text.

[0123] Furthermore, the classifier of the word recognition model classifies and identifies the multi-label qualitative words corresponding to each target character based on the representation vectors vec1 and vec2 corresponding to each target character, and obtains the predicted word label attributes label_1', label_2', etc. corresponding to the multi-label qualitative words corresponding to each target character.

[0124] Finally, based on the predicted word label attributes and sample word label attributes (label_1, label_2, etc.) corresponding to each multi-label qualitative word in the sample text, the recognition loss of the word recognition model is determined, and the model parameters of the word recognition model are adjusted based on the recognition loss.

[0125] The word recognition model training method provided in one or more embodiments of this application adopts a technical approach of text classification task plus post-processing task. First, it utilizes the text understanding, prediction, and text representation capabilities of a pre-trained language model to select the target characters corresponding to N multi-label qualitative words from the sample text. Then, based on the sample text and the N multi-label qualitative words, the pre-trained language model encodes the N target characters after masking. This ensures that each representation vector obtained by the pre-trained language model is obtained through character prediction and semantic representation based on a thorough learning and understanding of the semantics of the sample text and the true semantics of each multi-label qualitative word in the sample text. Therefore, each obtained representation vector can be used as the word label attribute vector of the corresponding multi-label qualitative word to accurately represent the multi-label. The true semantic meaning of qualitative words in the sample text is determined. Based on this, a text classification task is performed by the classifier of the word recognition model. Based on each representation vector, the corresponding multi-label qualitative words are classified and identified to obtain the predicted word label attributes corresponding to each multi-label qualitative word. Based on the predicted word label attributes corresponding to N multi-label qualitative words and the label attributes of N sample words in the sample text, the model parameters of the word recognition model are adjusted. The word recognition model trained in this way can perform multi-label qualitative word recognition based on accurately understanding the semantic meaning of the text and the semantic meaning of the words in the text. Compared with the NER model that performs the NER task, it can effectively avoid the problem of boundary recognition error or recognition of unlabeled words that affects the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0126] This application also provides a multi-label qualitative word recognition method, which can perform multi-label qualitative word recognition tasks on any text based on the word recognition model trained using the above-described word recognition model training method. Please refer to... Figure 3 The following is a flowchart illustrating a multi-label qualitative word recognition method according to an embodiment of this application. The method may include the following steps:

[0127] S302, Obtain the text to be recognized.

[0128] S304, Input the text to be recognized into the word recognition model to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words.

[0129] Among them, predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attribute is used to represent the semantics of the predicted multi-label qualitative words in the text to be identified.

[0130] Specifically, the word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text based on the representation vector of the text to be recognized to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words.

[0131] Optionally, such as Figure 4 As shown, after S304 above, the multi-tag qualitative word recognition method provided in this application embodiment may further include:

[0132] S306 matches the predicted multi-label qualitative words with the preset multi-label qualitative words in the preset dictionary.

[0133] The preset dictionary records multiple preset multi-tag qualitative terms and the category attributes corresponding to each preset multi-tag qualitative term. In practical applications, the preset dictionary can be pre-constructed based on the multiple preset multi-tag qualitative terms and the category attributes corresponding to each multi-tag qualitative term. For example, Table 2 shows some multi-tag qualitative terms and the category attributes corresponding to each multi-tag qualitative term.

[0134] Table 2

[0135]

[0136] Based on Table 2 above, a preset dictionary Keyword_dict ={'TV station': high risk, 'Mayor's hotline': risk, 'quality': unprofessional terminology, 'unreasonable': unprofessional terminology, 'debt defaulter': serious attitude, …} can be constructed.

[0137] S308: Obtain the category attribute corresponding to the preset multi-label qualitative word that matches the predicted multi-label qualitative word from the preset dictionary and determine it as the category attribute corresponding to the predicted multi-label qualitative word.

[0138] Specifically, the preset dictionary can be traversed, and the currently traversed preset multi-label qualitative words can be matched with the predicted multi-label qualitative words. If the match is successful, the category attribute corresponding to the preset multi-label qualitative word is used as the category attribute corresponding to the predicted multi-label qualitative word.

[0139] For example, taking the text to be identified as "Then I will complain about you, then I will go to that TV station." as an example, after determining that the text to be identified includes the multi-label qualitative word "TV station", the attribute category corresponding to the multi-label qualitative word "TV station" can be determined as "high risk" by querying the preset dictionary.

[0140] It is understandable that since multi-label qualitative terms have the same category attribute in different contexts, that is, the category attribute corresponding to a multi-label qualitative term is unique, it is simple and efficient to determine and predict the category attribute corresponding to a multi-label qualitative term by pre-constructing a dictionary and querying the dictionary.

[0141] The multi-label qualitative word recognition method provided in one or more embodiments of this application utilizes a pre-trained language model of a trained word recognition model to perform multi-label qualitative word recognition on the text to be recognized. Since the trained word recognition model can perform multi-label qualitative word recognition based on an accurate understanding of the semantics of the text and the semantics of the words in the text, compared with the NER model performing the NER task, it can effectively avoid the problem of boundary recognition errors or the recognition of unlabeled words that affect the recognition accuracy, thereby improving the recognition accuracy of multi-label qualitative words.

[0142] Furthermore, comparative experiments were conducted on the multi-label qualitative word recognition method provided in the embodiments of this application. The comparison parameters and experimental results are as follows:

[0143] Related technical solutions: NER, BERT model;

[0144] This application's solution: text classification + post-processing, using the BERT model;

[0145] Experimental data: Training set: 11,000 texts; Validation set: 1,300 texts; Test set: 1,300 texts. Experimental parameters: Training epochs, learning rate, maximum text length, and other parameters remained consistent.

[0146] Experimental results:

[0147]

[0148] Experimental conclusion: Compared with related technical solutions, the technical solution of this application improves the accuracy by 8 points and the F1 score by 4 points.

[0149] The technical solutions provided in this application can be applied to various scenarios requiring multi-label qualitative word recognition, such as speech quality inspection, and are not limited thereto. The following description uses offline speech quality inspection as an example to illustrate the training method of the word recognition model and the multi-label qualitative word recognition method provided in this application.

[0150] In offline voice quality inspection systems, it is typically necessary to identify statements containing multi-label qualitative words on the agent's side to assess the quality of service and facilitate timely detection of agent misconduct and service attitude. First, the business side provides sample texts as shown in Table 1 above, along with the sample word label attributes and category attributes of the multi-label qualitative words within them. If a sample text contains multiple multi-label qualitative words, the sample word label attributes of these words do not need to be consistent. For example, in the third sample text in Table 1, the two "quality" items have different sample word label attributes: one is "responsible," and the other is "not responsible." In the offline voice quality inspection system, the word recognition model training method provided in this application is used to train a word recognition model based on sample text provided by the business party and the sample word tag attributes of the multi-label qualitative words therein. A pre-set dictionary is also constructed based on the sample text provided by the business party and the category attributes corresponding to the multi-label qualitative words therein. On this basis, the voice data of the agent to be inspected is converted into text, and the word recognition model provided in this application is used to perform multi-label qualitative word recognition on the text to obtain the predicted multi-label qualitative words in the text and the predicted word tag attributes corresponding to the predicted multi-label qualitative words. For the predicted multi-label qualitative word with the predicted word tag attribute of "responsible," the category attribute corresponding to the predicted multi-label qualitative word is obtained by querying the pre-constructed dictionary. Furthermore, the service quality of the agent is evaluated by combining multiple texts of the agent and the multi-label qualitative word recognition results corresponding to each text.

[0151] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0152] With the above Figure 1 Corresponding to the training method of the word recognition model shown, this application also proposes a training device for the word recognition model. Please refer to... Figure 5 The diagram below illustrates the structure of a word recognition model training device 500 according to an embodiment of this application. The device 500 includes:

[0153] The acquisition unit 510 is used to select N target characters corresponding to the N multi-label qualitative words from the sample text containing multi-label qualitative words. Each multi-label qualitative word refers to a word with different semantics in different contexts but the same attribute category; one multi-label qualitative word corresponds to one target character; N is an integer greater than 1.

[0154] The encoding unit 520 is used to encode the N target characters after masking based on the N sample texts and the multi-label qualitative words using the pre-trained language model of the word recognition model, so as to obtain the representation vector corresponding to each target character.

[0155] The recognition unit 530 is used to classify and recognize the N multi-label qualitative words based on each representation vector through the classifier of the word recognition model, and obtain the predicted word label attributes corresponding to each of the N multi-label qualitative words, wherein each predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text.

[0156] The adjustment unit 540 is used to adjust the model parameters of the word recognition model based on the predicted word label attributes corresponding to each of the N multi-label qualitative words and the N sample word label attributes of the sample text. Each sample word label attribute is used to represent the true semantics of the corresponding multi-label qualitative word in the sample text. The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0157] Optionally, when the encoding unit 520 encodes the N target characters after masking based on the sample text and the N multi-label qualitative words using the pre-trained language model of the word recognition model to obtain the representation vector corresponding to each target character, it performs the following steps:

[0158] The pre-trained language model predicts the N target characters after masking based on the sample text and the N multi-label qualitative words, and obtains the predicted character corresponding to each target character.

[0159] The predicted character corresponding to each target character is encoded by the pre-trained language model to obtain the representation vector corresponding to each target character.

[0160] Optionally, when the encoding unit 520 encodes the N target characters after masking based on the sample text and the N multi-label qualitative words using the pre-trained language model of the word recognition model to obtain the representation vector corresponding to each target character, it performs the following steps:

[0161] At least the sample text, the N multi-label qualitative words, and the N target characters are concatenated to obtain the concatenated text;

[0162] The N target characters in the concatenated text are masked to obtain masked text.

[0163] The pre-trained language model encodes the N target characters after masking based on the masked text to obtain a representation vector corresponding to each target character.

[0164] Optionally, when the encoding unit 520 concatenates at least the sample text, the N multi-label qualitative words, and the N target characters to obtain the concatenated text, it performs the following steps:

[0165] Obtain the position of each of the N multi-label qualitative terms in the sample text;

[0166] The sample text, the N multi-label qualitative words, the positions of the N multi-label qualitative words in the sample text, and the N target characters are concatenated according to a preset format to obtain the concatenated text.

[0167] Optionally, when the encoding unit 520 selects N target characters corresponding to N multi-label qualitative words from sample text containing multi-label qualitative words, it performs the following steps:

[0168] Identify the sentence containing each multi-label qualitative word in the sample text;

[0169] Randomly select a portion of characters from the sentence containing each multi-label qualitative word, and use them as the target characters corresponding to the multi-label qualitative word.

[0170] Optionally, when adjusting the model parameters of the word recognition model based on the predicted word label attributes corresponding to each of the N multi-label qualitative words and the N sample word label attributes of the sample text, the adjustment unit 540 performs the following steps:

[0171] Based on the predicted word label attributes and sample word label attributes corresponding to each of the N multi-label qualitative words, the recognition loss of the word recognition model is determined;

[0172] Based on the recognition loss of the word recognition model, the model parameters of the word recognition model are adjusted.

[0173] Obviously, the training device for the word recognition model provided in this application embodiment can be used as... Figure 1 The entity executing the training method of the word recognition model shown, for example Figure 1 In the training method of the word recognition model shown, step S102 can be performed by... Figure 5 The acquisition unit 510 in the training device of the word recognition model shown executes step S104, which can be performed by... Figure 5 The word recognition model training device shown executes the encoding unit 520, and step S106 can be performed by... Figure 5The recognition unit 530 in the training device of the word recognition model shown executes step S108, which can be performed by... Figure 5 The adjustment unit 540 in the training device of the word recognition model shown is executed.

[0174] According to another embodiment of this application, Figure 5 The units in the training device of the word recognition model shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the training device of the word recognition model may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0175] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 1 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 5 The diagram illustrates a training apparatus for a word recognition model, and a training method for implementing the word recognition model according to embodiments of this application. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0176] With the above Figure 3 Corresponding to the multi-label qualitative word recognition method shown, this application also proposes a multi-label qualitative word recognition device. Please refer to... Figure 6 The image below is a schematic diagram of the structure of a multi-tag qualitative word recognition device 600 provided in one embodiment of this application. The device 600 includes:

[0177] Acquisition unit 610 is used to acquire the text to be recognized;

[0178] The recognition unit 620 is used to input the text to be recognized into the word recognition model to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attributes are used to represent the semantics of the predicted multi-label qualitative words in the text to be recognized.

[0179] The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text to be recognized based on the representation vector of the text to be recognized to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The word recognition model is trained based on the training method of the word recognition model provided in the embodiments of this application.

[0180] Optionally, the identification unit 620 is further configured to:

[0181] After inputting the text to be identified into the word recognition model to obtain the predicted multi-label qualitative words in the text to be identified and the predicted word label attributes corresponding to the predicted multi-label qualitative words, the predicted multi-label qualitative words are matched with the preset multi-label qualitative words in the preset dictionary. The preset dictionary records multiple preset multi-label qualitative words and the category attributes corresponding to each preset multi-label qualitative word.

[0182] The category attribute corresponding to the preset multi-label qualitative word that matches the predicted multi-label qualitative word is obtained from the preset dictionary and determined as the category attribute corresponding to the predicted multi-label qualitative word.

[0183] Obviously, the multi-tag qualitative word recognition device provided in this application embodiment can be used as... Figure 3 The execution entity of the multi-label qualitative word recognition method shown is, for example... Figure 3 In the multi-label qualitative word recognition method shown, step S302 can be performed by... Figure 6 The acquisition unit 610 in the multi-label qualitative word recognition device shown executes step S304, which can be performed by... Figure 6 The recognition unit 620 in the multi-label qualitative word recognition device shown is executed.

[0184] According to another embodiment of this application, Figure 6The units in the multi-tag qualitative word recognition device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the multi-tag qualitative word recognition device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0185] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as CPU, RAM, and ROM, can be used to run an application capable of performing tasks such as... Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 6 The multi-tag qualitative word recognition device shown herein, and the multi-tag qualitative word recognition method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0186] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 7 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0187] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0188] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0189] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a training device for the word recognition model at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0190] Select N target characters corresponding to the N multi-label qualitative words from the sample text containing multi-label qualitative words. Each multi-label qualitative word refers to a word with different semantics in different contexts but the same attribute category; one multi-label qualitative word corresponds to one target character; N is an integer greater than 1;

[0191] Based on the sample text and the N multi-label qualitative words, the pre-trained language model of the word recognition model encodes the N target characters after masking to obtain the representation vector corresponding to each target character.

[0192] The classifier of the word recognition model classifies and identifies the N multi-label qualitative words based on each representation vector, and obtains the predicted word label attributes corresponding to each of the N multi-label qualitative words. Each predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text.

[0193] Based on the predicted word label attributes corresponding to each of the N multi-label qualitative words and the N sample word label attributes of the sample text, the model parameters of the word recognition model are adjusted. Each sample word label attribute is used to represent the true semantics of the corresponding multi-label qualitative word in the sample text. The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0194] Alternatively, the processor reads the corresponding computer program from non-volatile memory into memory and runs it, forming a multi-tag qualitative word recognition device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0195] Obtain the text to be recognized;

[0196] The text to be identified is input into a word recognition model to obtain predicted multi-label qualitative words in the text to be identified and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attributes are used to represent the semantics of the predicted multi-label qualitative words in the text to be identified.

[0197] The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text to be recognized based on the representation vector of the text to be recognized to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The word recognition model is trained based on the training method of the word recognition model provided in the embodiments of this application.

[0198] The above is as stated in this application. Figure 1 The illustrated embodiment discloses a method for training a word recognition model, or the method described above in this application. Figure 3 The method executed by the multi-tag qualitative word recognition device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0199] The electronic device can also perform Figure 1 The method, and the implementation of the word recognition model training device in Figure 1 , Figure 2 The illustrated embodiment may also perform the functions of the electronic device, or the electronic device may also perform the functions of the embodiment shown. Figure 3The method, and the implementation of a multi-label qualitative word recognition device in Figure 3 , Figure 4 The functions of the embodiments shown are not described in detail here.

[0200] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0201] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations:

[0202] Select N target characters corresponding to the N multi-label qualitative words from the sample text containing multi-label qualitative words. Each multi-label qualitative word refers to a word with different semantics in different contexts but the same attribute category; one multi-label qualitative word corresponds to one target character; N is an integer greater than 1;

[0203] Based on the sample text and the N multi-label qualitative words, the pre-trained language model of the word recognition model encodes the N target characters after masking to obtain the representation vector corresponding to each target character.

[0204] The classifier of the word recognition model classifies and identifies the N multi-label qualitative words based on each representation vector, and obtains the predicted word label attributes corresponding to each of the N multi-label qualitative words. Each predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text.

[0205] Based on the predicted word label attributes corresponding to each of the N multi-label qualitative words and the N sample word label attributes of the sample text, the model parameters of the word recognition model are adjusted. Each sample word label attribute is used to represent the true semantics of the corresponding multi-label qualitative word in the sample text. The adjusted word recognition model is used to perform multi-label qualitative word recognition on any text.

[0206] Alternatively, the one or more programs include instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 3 The method of the illustrated embodiment is specifically used to perform the following operations:

[0207] Obtain the text to be recognized;

[0208] The text to be identified is input into a word recognition model to obtain predicted multi-label qualitative words in the text to be identified and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The predicted multi-label qualitative words refer to words with different semantics in different contexts but the same attribute category. The predicted word label attributes are used to represent the semantics of the predicted multi-label qualitative words in the text to be identified.

[0209] The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the text to be recognized to obtain a representation vector of the text to be recognized. The classifier is used to classify and recognize the text to be recognized based on the representation vector of the text to be recognized to obtain the predicted multi-label qualitative words in the text to be recognized and the predicted word label attributes corresponding to the predicted multi-label qualitative words. The word recognition model is trained based on the training method of the word recognition model provided in the embodiments of this application.

[0210] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0211] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0212] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0213] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0214] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for training a word recognition model, the method comprising: The method comprises the steps of: selecting N target characters corresponding to N multi-label qualitative words from sample text containing the multi-label qualitative words, each multi-label qualitative word referring to a word with different semantics but the same attribute category in different contexts, one multi-label qualitative word corresponding to one target character, and N being an integer greater than 1; encoding the N target characters after mask processing based on the sample text and the N multi-label qualitative words through a pre-training language model of a word recognition model to obtain a representation vector corresponding to each target character; classifying and identifying the N multi-label qualitative words based on each representation vector through a classifier of the word recognition model to obtain a prediction word label attribute corresponding to each of the N multi-label qualitative words, wherein each prediction word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text; adjusting model parameters of the word recognition model based on the prediction word label attribute corresponding to each of the N multi-label qualitative words and N sample word label attributes of the sample text, wherein each sample word label attribute is used to represent the real semantics of the corresponding multi-label qualitative word in the sample text, and the adjusted word recognition model is used for multi-label qualitative word recognition on any text.

2. The method of claim 1, wherein, The method comprises the steps of: encoding the N target characters after mask processing based on the sample text and the N multi-label qualitative words through a pre-training language model of a word recognition model to obtain a representation vector corresponding to each target character; encoding the prediction character corresponding to each target character through the pre-training language model to obtain a representation vector corresponding to each target character.

3. The method of claim 1, wherein, The method comprises the steps of: splicing at least the sample text, the N multi-label qualitative words, and the N target characters to obtain a spliced text; mask processing the N target characters in the spliced text to obtain a mask text; encoding the N target characters after mask processing based on the mask text through the pre-training language model to obtain a representation vector corresponding to each target character.

4. The method of claim 3, wherein, The method comprises the steps of: obtaining the positions of the N multi-label qualitative words in the sample text; splicing the sample text, the N multi-label qualitative words, the positions of the N multi-label qualitative words in the sample text, and the N target characters in a preset format to obtain the spliced text.

5. The method of claim 1, wherein, The method comprises the steps of: determine a sentence in which each multi-label qualitative word in the sample text is located; randomly select part of characters from the sentence in which each multi-label qualitative word is located as target characters corresponding to the multi-label qualitative word.

6. The method of claim 1, wherein, The model parameters of the word recognition model are adjusted based on the predicted word label attributes corresponding to the N multi-label qualitative words and the N sample word label attributes of the sample text, including: Based on the predicted word label attributes corresponding to the N multi-label qualitative words and the sample word label attributes, the recognition loss of the word recognition model is determined. Based on the recognition loss of the word recognition model, the model parameters of the word recognition model are adjusted.

7. A multi-label categorical word recognition method characterized by, It includes: obtaining a to-be-recognized text; inputting the to-be-recognized text into a word recognition model to obtain a predicted multi-label qualitative word in the to-be-recognized text and a predicted word label attribute corresponding to the predicted multi-label qualitative word, wherein the predicted multi-label qualitative word refers to a word with different semantics but the same attribute category in different contexts, and the predicted word label attribute is used to represent the semantics of the predicted multi-label qualitative word in the to-be-recognized text; The word recognition model includes a pre-trained language model and a classifier. The pre-trained language model is used to encode the to-be-recognized text to obtain a representation vector of the to-be-recognized text. The classifier is used to classify and recognize the to-be-recognized text based on the representation vector of the to-be-recognized text to obtain the predicted multi-label qualitative word in the to-be-recognized text and the predicted word label attribute corresponding to the predicted multi-label qualitative word. The word recognition model is trained based on the training method of the word recognition model in any one of claims 1 to 6.

8. The method of claim 7, wherein, After inputting the to-be-recognized text into the word recognition model to obtain the predicted multi-label qualitative word in the to-be-recognized text and the predicted word label attribute corresponding to the predicted multi-label qualitative word, the method further includes: matching the predicted multi-label qualitative word with a preset multi-label qualitative word in a preset dictionary, the preset dictionary recording a plurality of preset multi-label qualitative words and a category attribute corresponding to each preset multi-label qualitative word; obtaining the category attribute corresponding to the preset multi-label qualitative word matched with the predicted multi-label qualitative word from the preset dictionary and determining it as the category attribute corresponding to the predicted multi-label qualitative word. 9.A device for training a word recognition model, comprising: It includes: An acquisition unit is configured to select N target characters corresponding to N multi-label qualitative words from a sample text containing multi-label qualitative words, each multi-label qualitative word being a word with different semantics but the same attribute category in different contexts, one multi-label qualitative word corresponding to one target character, and N being an integer greater than 1. An encoding unit is configured to encode the N target characters processed by a mask based on a sample text and the N multi-label qualitative words by a pre-trained language model of a word recognition model to obtain a representation vector corresponding to each target character. The identification unit is configured to identify the N multi-label qualitative words based on the classifier of the word identification model, to obtain N multi-label qualitative words each corresponding to a predicted word label attribute, wherein each predicted word label attribute is used to represent the predicted semantics of the corresponding multi-label qualitative word in the sample text. The adjustment unit is configured to adjust the model parameters of the word identification model based on the N multi-label qualitative words each corresponding to a predicted word label attribute and the N sample word label attributes of the sample text, wherein each sample word label attribute is used to represent the true semantics of the corresponding multi-label qualitative word in the sample text, and the adjusted word identification model is used for multi-label qualitative word identification of any text.

10. A multi-label categorical word recognition apparatus, characterized by comprising: The method comprises: The acquisition unit is configured to acquire a to-be-identified text; The identification unit is configured to input the to-be-identified text into a word identification model to obtain a predicted multi-label qualitative word in the to-be-identified text and a predicted word label attribute corresponding to the predicted multi-label qualitative word, wherein the predicted multi-label qualitative word refers to a word with different semantics but the same attribute category in different contexts, and the predicted word label attribute is used to represent the semantics of the predicted multi-label qualitative word in the to-be-identified text; The word identification model comprises a pre-trained language model and a classifier, the pre-trained language model is used to encode the to-be-identified text to obtain a representation vector of the to-be-identified text, and the classifier is used to classify and identify the to-be-identified text based on the representation vector of the to-be-identified text to obtain the predicted multi-label qualitative word in the to-be-identified text and the predicted word label attribute corresponding to the predicted multi-label qualitative word; and the word identification model is trained based on the training method of the word identification model in any one of claims 1 to 6.

11. An electronic device, comprising: The processor; The memory for storing the processor-executable instructions; The processor is configured to execute the instructions to implement the training method of the word identification model in any one of claims 1 to 6; or the processor is configured to execute the instructions to implement the multi-label qualitative word identification method in claim 7 or 8. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the training method of the word identification model in any one of claims 1 to 6; or when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the multi-label qualitative word identification method in claim 7 or 8.

12. A computer-readable storage medium, characterized in that, ​

Images