Multi-label qualitative word recognition method and device, electronic equipment and storage medium

Abstract

The application discloses a multi-label qualitative word recognition method and device, electronic equipment and a storage medium, wherein the method comprises: obtaining a text to be recognized; classifying the text to be recognized through a text classification model to obtain a text label attribute corresponding to the text to be recognized, the text label attribute being used to represent a sentiment tendency reflected by the text to be recognized in a target application scenario; matching a plurality of preset multi-label qualitative words with words in the text to be recognized to obtain multi-label qualitative words in the text to be recognized, wherein the multi-label qualitative words refer to words with different semantics but the same category attribute in different contexts; and if it is determined through matching that the text to be recognized includes a target multi-label qualitative word, determining a word label attribute corresponding to the multi-label qualitative word in the text to be recognized based on the text label attribute corresponding to the text to be recognized. The application can improve the multi-label qualitative word recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a multi-label qualitative word recognition method, device, electronic device and storage medium. Background Technology

[0002] In voice quality inspection systems, it is typically necessary to identify multi-label qualitative words in sentences to assess call quality. Multi-label qualitative words are words whose semantics differ in different contexts but share the same category attribute. For example, the texts "Sir, it's your own problem, why are you complaining? That's unreasonable.", "Sir, you can complain, but the problems you caused us still need to be resolved.", "Sir, you're the one at fault, why are you complaining about me? Do you think complaining about me will make this matter closed?", and "Go ahead and complain, it's your business." all contain the multi-label qualitative word "complain." This word is classified as unprofessional terminology. While it doesn't imply responsibility in the first and third texts, it does imply responsibility in the second and fourth texts. "Responsible" implies that the speaker is acting out of negative emotions and bears responsibility for the subsequent problems. This expression of "responsible" or "not responsible" is commonly used in customer service and user calls.

[0003] In related technologies, the recognition of multi-label qualitative words usually adopts the Named Entity Recognition (NER) method. The multi-label qualitative words and their semantics in the sample text are pre-annotated. Then, the sample text and the annotation information it carries are used to train the named entity recognition model. Then, the text to be recognized is input into the trained named entity recognition model, which can output the multi-label qualitative words in the text to be recognized and their semantics in the text to be recognized.

[0004] However, this method has the problem of identifying boundaries. For example, it may mistakenly identify "TV station" as "television". Also, because the named entity recognition model has a certain generalization ability, it may identify words that have not been labeled, which will affect the recognition accuracy. Summary of the Invention

[0005] The purpose of this application is to provide a multi-label qualitative word recognition method, apparatus, and electronic device to solve the problem of low recognition accuracy in related technologies.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: In a first aspect, embodiments of this application provide a multi-label qualitative word recognition method, including: Obtain the text to be recognized; The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified. The text label attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute labels corresponding to the sample text. The sample text attribute labels are determined based on the word label attributes of multi-label qualitative words in the sample text. Multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word label attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. Match multiple preset multi-label qualitative terms with the words in the text to be identified; If the matching determines that the text to be identified contains target multi-label qualitative words, the word label attributes corresponding to the multi-label qualitative words in the text to be identified are determined based on the text label attributes corresponding to the text to be identified.

[0007] Secondly, embodiments of this application provide a multi-tag qualitative word recognition device, comprising: The acquisition unit is used to acquire the text to be recognized; A classification unit is used to classify the text to be identified using a text classification model to obtain text label attributes corresponding to the text to be identified. The text label attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute labels corresponding to the sample text. The sample text attribute labels are determined based on the word label attributes of multi-label qualitative words in the sample text. Multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word label attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. The recognition unit is used to match multiple preset multi-label qualitative words with words in the text to be recognized; The recognition unit is further configured to, if the matching determines that the text to be recognized includes target multi-label qualitative words, determine the word label attribute corresponding to the multi-label qualitative words in the text to be recognized based on the text label attribute corresponding to the text to be recognized.

[0008] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in the first aspect.

[0010] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: It employs a text classification task plus a post-processing task approach. First, text classification technology is used to classify and identify the text to be identified as a whole, obtaining text tag attributes that represent the sentiment tendency reflected by the text in the target application scenario. Based on this, multiple preset multi-label qualitative words are matched with words in the text to be identified, thus identifying the target multi-label qualitative words in the text. Since the preset multiple multi-label qualitative words clearly define word boundaries, multi-label qualitative word identification based on word matching can avoid misidentification caused by identification boundary issues and the identification of unlabeled words due to model generalization ability, which is beneficial to improving the identification accuracy. Furthermore... Since the word label attributes of multi-label qualitative words can represent the semantics of multi-label qualitative words in their respective sample texts, when training a text classification model in advance, the sample text attribute labels used to provide supervision signals for the model training process are determined by referring to the word label attributes of multi-label qualitative words in the sample texts. Then, the text recognition model trained based on the sample texts and their corresponding sample text attribute labels will also consider the semantics of each word in the text to be recognized when classifying the text to be recognized, so as to accurately identify the sentiment tendency reflected by the text and obtain the text label attributes corresponding to the text to be recognized. Thus, based on the text label attributes corresponding to the text to be recognized, the semantics of the target multi-label qualitative words can be accurately determined, and the word label attributes corresponding to the target multi-label qualitative words can be obtained. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a multi-label qualitative word recognition method provided for one embodiment of this application; Figure 2 A flowchart illustrating a multi-label qualitative word recognition method provided for another embodiment of this application; Figure 3 A flowchart illustrating a multi-label qualitative word recognition method provided in another embodiment of this application; Figure 4 A schematic diagram of the structure of a multi-label qualitative word recognition device provided in one embodiment of this application; Figure 5This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in this specification and claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] Explanation of some concepts: Named Entity Recognition (NER) refers to the identification of entities with specific meanings in text, mainly including names of people, places, organizations, and proper nouns.

[0015] LSTM (Long Short-Term Memory): A type of recurrent neural network designed to address the long-term dependency problem inherent in general recurrent neural networks, and capable of performing better on longer sequences.

[0016] BiLSTM (Bi-directional Long Short-Term Memory): A bidirectional LSTM, which is composed of a forward LSTM and a backward LSTM.

[0017] BERT: A pre-trained language model that uses a bidirectional encoder representation from a Transformer. Pre-trained BERT representations can be fine-tuned with an additional output layer, making them suitable for building state-of-the-art models across a wide range of tasks.

[0018] As described in the background section, related technologies typically employ Named Entity Recognition (NER) to identify multi-labeled qualitative words. This involves pre-annotating sample text with multi-labeled qualitative words and their semantic meaning within the text. Then, the sample text and its annotation information are used to train a named entity recognition model. Finally, the text to be recognized is input into the trained model, which outputs the multi-labeled qualitative words and their semantic meaning within the text. However, this approach suffers from recognition boundary issues, such as misidentifying "TV station" as "television." Furthermore, because named entity recognition models possess a certain generalization ability, they may identify unlabeled words, affecting recognition accuracy.

[0019] In view of this, the embodiments of this application aim to propose a multi-label qualitative word recognition method. It adopts a technical approach of text classification plus post-processing. First, text classification technology is used to classify and recognize the text to be recognized as a whole, obtaining text label attributes that represent the sentiment tendency reflected by the text in the target application scenario. Based on this, multiple preset multi-label qualitative words are matched with words in the text to be recognized, thereby identifying the target multi-label qualitative words in the text. Since the preset multiple multi-label qualitative words clearly define word boundaries, multi-label qualitative word recognition based on word matching can avoid misidentification caused by recognition boundary issues and the recognition of unlabeled words due to model generalization ability, thus improving recognition accuracy. Furthermore, Since the word label attributes of multi-label qualitative words can represent the semantics of multi-label qualitative words in their respective sample texts, when training a text classification model in advance, the sample text attribute labels used to provide supervision signals for the model training process are determined by referring to the word label attributes of multi-label qualitative words in the sample texts. Then, the text recognition model trained based on the sample texts and their corresponding sample text attribute labels will also consider the semantics of each word in the text to be recognized when classifying the text to be recognized, so as to accurately identify the sentiment tendency reflected by the text and obtain the text label attributes corresponding to the text to be recognized. Thus, based on the text label attributes corresponding to the text to be recognized, the semantics of the target multi-label qualitative words can be accurately determined, and the word label attributes corresponding to the target multi-label qualitative words can be obtained.

[0020] It should be understood that the multi-label qualitative word recognition method proposed in this application embodiment can be executed by an electronic device or software installed in an electronic device. The electronic device referred to herein may include terminal devices, such as smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances, smartwatches, vehicle terminals, aircraft, etc.; or, the electronic device may also include a server, such as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0021] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] Please see Figure 1 The following is a flowchart illustrating a multi-label qualitative word recognition method according to an embodiment of this application. The method may include the following steps: S102, Obtain the text to be recognized.

[0023] The text to be identified refers to the text that requires multi-label qualitative word recognition. The text to be identified can be text generated in any scenario or text converted from speech. For example, the text to be identified can be the speech converted from a call between a customer service representative and a user, or it can be any section of text in a contract.

[0024] S104. The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified.

[0025] The text label attribute is used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. That is, the text label attribute varies depending on the target application scenario. For example, if the target application scenario is a voice quality inspection scenario, the text label attribute can be divided into two categories: responsible and irresponsible; responsible reflects a negative sentiment tendency, while irresponsible reflects a positive or neutral sentiment tendency. Similarly, if the target application scenario is a sentiment classification scenario, the text label attribute can be divided into two categories: positive and negative, and so on.

[0026] Here, a text classification model refers to a model with text classification capabilities. This model can be used to classify any text to obtain the corresponding text label attributes. In other words, in S104 above, inputting the text to be identified into the text classification model will yield the corresponding text label attributes. In this embodiment, the text classification model is trained based on sample text and its corresponding text label attributes. It is worth noting that the training of the text classification model is pre-performed based on sample text and its corresponding text label attributes. Subsequently, in the process of recognizing multi-label qualitative words, it is not necessary to train the text classification model every time, or the model can be periodically updated based on new sample text and its corresponding text label attributes.

[0027] Specifically, the training method for the text classification model includes: obtaining sample texts and their corresponding sample text label attributes, whereby the sample text label attributes represent the semantics of the corresponding sample text; and training the text classification model based on the sample texts and their corresponding sample text label attributes. Here, the sample text label attributes represent the semantics of the sample text. For example, taking the target application scenario of voice quality inspection as an example, the sample text label attributes can be divided into two categories: responsible and unresponsible. Table 1 below shows some sample texts and their corresponding sample text label attributes.

[0028] Table 1 In this embodiment of the application, the sample text and its corresponding sample text tag attributes can be obtained through various appropriate methods. You can choose the specific method according to your actual needs. This embodiment of the application does not limit this method.

[0029] In one optional implementation, the sample text tag attribute corresponding to the sample text is determined based on the word tag attribute of the multi-tag qualitative words in the sample text. Specifically, obtaining the sample text and its corresponding sample text tag attribute includes: obtaining the sample text containing preset multi-tag qualitative words and the sample word tag attribute corresponding to the preset multi-tag qualitative words, wherein the sample word tag attribute is used to represent the semantics of the corresponding multi-tag qualitative words in the sample text; further, the sample word tag attribute corresponding to the multi-tag qualitative words in the sample text is used as the sample text tag attribute corresponding to the sample text. Here, multi-tag qualitative words refer to words with different semantics in different contexts but the same category attribute. The category attribute of a word is used to represent the category to which the word belongs in the target application scenario. For example, taking a voice quality inspection scenario, the category attribute may include, but is not limited to: high-risk, risky, unprofessional terminology, serious attitude, etc.

[0030] For example, Table 2 shows some sample texts and the multi-tag word information in the sample texts. For each sample text, the sample word tag attribute corresponding to the multi-tag qualitative words it contains is used as the sample text tag attribute corresponding to that sample text, thus obtaining the sample texts and their corresponding sample text tag attributes shown in Table 1 above.

[0031] Table 2 Understandably, by using the sample word label attributes corresponding to the multi-label qualitative words in the sample text as the sample text label attributes, the sample text label attributes can accurately reflect the semantics of the multi-label qualitative words in the sample text. This facilitates the accurate identification of multi-label qualitative words and their corresponding word label attributes in subsequent text classification and post-processing tasks. Moreover, compared to related technologies that use BIO to annotate multi-label qualitative words and their semantics in the sample text, this method is simpler, faster, and has higher annotation accuracy.

[0032] In other alternative implementations, sample text can be manually labeled with corresponding sample text tag attributes. This approach only requires considering the semantics of the sample text, without needing to consider multi-label qualitative words within it, resulting in simple, quick, and highly accurate labeling.

[0033] In this embodiment of the application, after obtaining the sample text and its corresponding sample text label attributes, the format can be converted to obtain training data in a preset format; further, a machine learning algorithm is used to train the training data in the preset format to obtain a text classification model.

[0034] For example, taking the first and second sample texts in Table 1 above as examples, the following training data can be obtained after format conversion: Training Data 1: Do you work in the after-sales department? (No responsibility) Training data 2: I need to contact your after-sales service. (Responsible) In the training data above, \t represents a space.

[0035] In addition, the aforementioned machine learning algorithms may include, for example, at least one of the following algorithms: Bert, Robert, LSTM, BiLSTM, etc.

[0036] S106, Match multiple preset multi-label qualitative terms with words in the text to be identified.

[0037] Optionally, in order to improve the recognition efficiency and accuracy of the text to be recognized, the above S106 may include the following steps: S161, perform word segmentation on the text to be recognized to obtain the words in the text to be recognized.

[0038] For example, various word segmentation tools commonly used in this field, such as the jieba word segmentation tool, can be used to segment the text to be recognized and obtain the words in the text.

[0039] S162, match multiple multi-label qualitative words in the preset vocabulary with words in the text to be identified.

[0040] The preset vocabulary contains multiple preset multi-tag qualitative terms, and the target multi-tag qualitative term is any one of the preset multiple multi-tag qualitative terms.

[0041] S163, the multi-label qualitative words that match the words in the text to be identified in the preset vocabulary are identified as the target multi-label qualitative words contained in the text to be identified.

[0042] Specifically, the preset vocabulary can be iterated until all multi-tag qualitative words in the preset vocabulary have been matched. During the traversal, the currently traversed multi-tag qualitative word is matched with the words in the text to be recognized. If the words in the text to be recognized match the currently traversed multi-tag qualitative word, it is determined that the text to be recognized contains the currently matched multi-tag qualitative word.

[0043] For example, suppose the preset vocabulary Keyword = ['after-sales', 'suitable', 'unreasonable', 'you're right',...] includes multiple preset multi-tag qualitative words such as "after-sales", "suitable", "unreasonable", and "you're right". Suppose the text to be identified is "I'll contact your after-sales service to resolve this." The corresponding sample text tag attribute is "responsible". By matching the multi-tag qualitative words in the preset vocabulary with the words in the text to be identified, it can be determined that the target multi-tag qualitative word included in the text to be identified is "after-sales".

[0044] It is understandable that in the above implementation, since the preset multiple multi-label qualitative words clearly define the word boundaries, multi-label qualitative word recognition based on word matching can avoid problems such as misidentification caused by recognition boundary issues and the recognition of unlabeled words due to model generalization ability, which is conducive to improving recognition accuracy. In addition, the method of multi-label qualitative word recognition through word matching does not require complex processing such as multi-label qualitative word annotation and entity recognition model training, which is convenient and efficient, and helps to improve the recognition efficiency of multi-label qualitative words.

[0045] This application embodiment illustrates one specific implementation of S106 described above. It should be understood that S106 can also be implemented in other ways, and this application embodiment does not limit this implementation.

[0046] S108, if the matching determines that the text to be identified contains the target multi-label qualitative word, then based on the text label attribute corresponding to the text to be identified, determine the word label attribute corresponding to the target multi-label qualitative word in the text to be identified.

[0047] Since the word label attributes of multi-label qualitative words can represent the semantics of the multi-label qualitative words in their respective sample texts, the sample text attribute labels used to provide supervision signals during the pre-training of the text classification model are determined by referring to the word label attributes of multi-label qualitative words in the sample texts. Furthermore, the text recognition model trained based on the sample texts and their corresponding sample text attribute labels, when classifying the text to be recognized, also considers the semantics of each word in the text to accurately identify the sentiment tendency reflected by the text, thus obtaining the text label attributes corresponding to the text to be recognized. Therefore, based on the text label attributes corresponding to the text to be recognized, the semantics of the target multi-label qualitative words in the text to be recognized can be accurately determined, and the word label attributes corresponding to the target multi-label qualitative words can be obtained. Specifically, the text label attributes corresponding to the text to be recognized can be determined as the word label attributes corresponding to the target multi-label qualitative words in the text to be recognized.

[0048] For example, taking the text to be identified, “I will contact your after-sales service to resolve this.” as an example, the sample text tag attribute corresponding to this text is “responsible”. After identifying that the text to be identified contains the multi-tag qualitative word “after-sales service”, it can be determined that the word tag attribute corresponding to the multi-tag qualitative word “after-sales service” is “responsible”.

[0049] Optionally, the multi-label qualitative word recognition result corresponding to the text to be recognized may further include the attribute category corresponding to the target multi-label qualitative word in the text to be recognized. In this case, after S106 above, the multi-label qualitative word recognition method provided in this application embodiment may further include: S110, if the matching determines that the text to be recognized contains the target multi-label qualitative word, then query the category attribute corresponding to the target multi-label qualitative word from the preset dictionary.

[0050] The preset dictionary records the category attributes corresponding to each of the aforementioned multi-tag qualitative terms. In practical applications, the preset dictionary can be pre-constructed based on multiple multi-tag qualitative terms in a preset vocabulary and the category attributes corresponding to each multi-tag qualitative term. For example, Table 3 shows some of the multi-tag qualitative terms in the preset vocabulary and the category attributes corresponding to each multi-tag qualitative term.

[0051] Table 3 Based on Table 3 above, a preset dictionary Keyword_dict = {'After-sales': High risk, 'Suitable': Risk, 'Unreasonable': Unprofessional terminology, 'You're right': Serious attitude, …} can be constructed. For example, still using the text to be identified as "I'll contact your after-sales service for a solution.", after determining that the text includes the multi-tag qualitative term "after-sales," the preset dictionary can be used to determine that the attribute category corresponding to the multi-tag term "after-sales" is "High risk."

[0052] Optionally, the multi-label qualitative word recognition result corresponding to the text to be recognized may further include the position of the target multi-label qualitative word in the text to be recognized. After S106 above, the multi-label qualitative word recognition method provided in this application embodiment may further include: if the matching determines that the text to be recognized includes the target multi-label qualitative word, then based on the arrangement order of each character in the target multi-label qualitative word in the text to be recognized, the position of the target multi-label qualitative word in the text to be recognized is determined.

[0053] The position of the multi-label qualitative word can include the order of the first character and the last character of the multi-label qualitative word in the text to be recognized. For example, still using the text to be recognized as "I'll contact your after-sales service for a solution.", the multi-label qualitative word recognition method described above can yield the following multi-label qualitative word recognition result: [['After-sales', 'High-risk', 5, 6]], where 'TV station' represents the multi-label qualitative word in the text to be recognized, 'High-risk' represents the category attribute corresponding to the multi-label qualitative word, 5 indicates that the first character of the multi-label qualitative word is the 5th character in the text to be recognized, and 6 indicates that the last character of the multi-label qualitative word is the 6th character in the text to be recognized.

[0054] The multi-label qualitative word recognition method provided in the above embodiments of this application adopts a technical approach of text classification task plus post-processing task. First, text classification technology is used to classify and recognize the text to be recognized as a whole, obtaining text label attributes that represent the sentiment tendency reflected by the text in the target application scenario. Based on this, multiple preset multi-label qualitative words are matched with words in the text to be recognized, thereby identifying the target multi-label qualitative words in the text. Since the preset multiple multi-label qualitative words clearly define word boundaries, multi-label qualitative word recognition based on word matching can avoid misidentification caused by recognition boundary issues and the recognition of unlabeled words due to model generalization ability, which is beneficial to improving recognition accuracy. Furthermore, since multiple... The word label attributes of a multi-label qualitative word can represent the semantics of the multi-label qualitative word in the sample text to which it belongs. When training a text classification model in advance, the sample text attribute labels used to provide supervision signals for the model training process are determined by referring to the word label attributes of the multi-label qualitative words in the sample text. Then, the text recognition model trained based on the sample text and its corresponding sample text attribute labels will also consider the semantics of each word in the text to be recognized when classifying the text to be recognized, so as to accurately identify the sentiment tendency reflected by the text and obtain the text label attributes corresponding to the text to be recognized. Thus, based on the text label attributes corresponding to the text to be recognized, the semantics of the target multi-label qualitative word can be accurately determined, and the word label attributes corresponding to the target multi-label qualitative word can be obtained.

[0055] Furthermore, comparative experiments were conducted on the multi-label qualitative word recognition method provided in the embodiments of this application. The comparison parameters and experimental results are as follows: Related technical solutions: NER, BERT model; This application's solution: text classification + post-processing, using the BERT model; Experimental data: 1539 training data entries; 170 validation data entries; 170 test data entries; Experimental parameters: The number of training epochs, learning rate, maximum text length, and other parameters were kept constant. Experimental results: Experimental conclusion: Compared with related technical solutions, the technical solution of this application improves the accuracy by 6 points and the F1 score by 3 points.

[0056] The multi-label qualitative word recognition method provided in this application can be applied to various scenarios that require multi-label qualitative word recognition, such as speech quality inspection and sentiment classification. This application does not limit the application to these scenarios. The following uses the speech quality inspection scenario as an example to illustrate the multi-label qualitative word recognition method provided in this application.

[0057] In the target application scenario of voice quality inspection, the text tag attributes can be divided into two categories: responsible and not responsible. The word tag attributes corresponding to multi-tag qualitative words can also be divided into two categories: responsible and not responsible. The category attributes corresponding to multi-tag qualitative words can be divided into multiple categories such as high-risk, risky, unprofessional terminology, and serious attitude. The text to be identified is any user dialogue text in the service dialogue to be inspected, which is any historical dialogue between customer service and the user.

[0058] Please see Figure 2 The following is a flowchart illustrating a multi-label qualitative word recognition method according to another embodiment of this application. The method may include the following steps: S202, Obtain the text to be recognized.

[0059] For example, you can iterate through the dialogues of the service to be inspected and use the currently iterated user dialogue text as the text to be identified.

[0060] S204. The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified.

[0061] If the text tag attribute corresponding to the text to be recognized indicates that the text to be recognized does not reflect the preset emotional tendency in the speech quality inspection scenario, then an empty value [] is output, and the next text to be recognized is obtained; if the text tag attribute corresponding to the text to be recognized indicates that the text to be recognized reflects the preset emotional tendency in the speech quality inspection scenario, then the following step S206 is executed.

[0062] For example, the preset sentiment tendency may include "responsible". That is, if the text tag attribute corresponding to the text to be identified is "unresponsible", then an empty value [] is output, and the next text to be identified is obtained; if the text tag attribute corresponding to the text to be identified is "responsible", then step S206 is executed. The specific implementation process of S204 above is similar to the specific implementation method of S104 above. Please refer to the detailed description of S104 above, which will not be repeated here.

[0063] S206, Match multiple preset multi-label qualitative terms with words in the text to be identified.

[0064] If the text to be identified does not contain multi-label qualitative words, then an empty value [] is output, and the next text to be identified is obtained; if the text to be identified contains multi-label qualitative words, then the following steps S208~S210 are executed. The specific implementation process of S206 above is similar to the specific implementation method of S106 above. Please refer to the detailed description of S106 above, which will not be repeated here.

[0065] S208, Based on the text tag attributes corresponding to the text to be identified, determine the word tag attributes corresponding to the target multi-tag qualitative words in the text to be identified.

[0066] The specific implementation process of S208 is similar to that of S108 mentioned above. Please refer to the detailed explanation of S108 mentioned above. It will not be repeated here.

[0067] S210, query the category attribute corresponding to the multi-label qualitative words in the text to be recognized from the preset dictionary.

[0068] The specific implementation process of S210 is similar to that of S110 mentioned above. Please refer to the detailed explanation of S110 mentioned above. It will not be repeated here.

[0069] Understandably, in voice quality inspection scenarios, user dialogue text that reflects a preset emotional tendency can reflect the service quality of customer service to a certain extent. Therefore, in this application scenario, we usually only focus on user dialogue text that reflects a preset emotional tendency, such as user dialogue text with the semantic meaning of "responsible". Based on this, post-processing when user dialogue text reflects a preset emotional tendency is beneficial to reduce the consumption of processing resources and further improve the efficiency of multi-label qualitative word recognition of the text to be recognized.

[0070] Please see Figure 3 The following is a flowchart of a multi-label qualitative word recognition method provided in another embodiment of this application. The method may include the following steps: S302, Obtain the text to be recognized.

[0071] For example, you can iterate through the dialogues of the service to be inspected and use the currently iterated user dialogue text as the text to be identified.

[0072] S304. The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified.

[0073] If the text tag attribute corresponding to the text to be recognized does not reflect the preset sentiment tendency in the speech quality inspection scenario, then an empty value [] is output, and the next text to be recognized is obtained; if the text tag attribute corresponding to the text to be recognized indicates that the text to be recognized reflects the preset sentiment tendency in the speech quality inspection scenario, then the following step S306 is executed.

[0074] For example, the preset sentiment tendency may include "responsible". That is, if the text label attribute corresponding to the text to be identified is "unresponsible", then an empty value [] is output, and the next text to be identified is obtained; if the text label attribute corresponding to the text to be identified is "responsible", then step S306 is executed.

[0075] S306, Match multiple preset multi-label qualitative terms with words in the text to be identified.

[0076] If the text to be identified does not contain multi-label qualitative words, then an empty value [] is output, and the next text to be identified is obtained; if the text to be identified contains multi-label qualitative words, then the following steps S308~S314 are executed. S308, based on the text label attributes corresponding to the text to be identified, the word label attributes corresponding to the target multi-label qualitative words in the text to be identified are determined.

[0077] The specific implementation process of S308 is similar to that of S108 mentioned above. Please refer to the detailed explanation of S108 mentioned above. It will not be repeated here.

[0078] S310: Query the category attribute corresponding to the multi-label qualitative words in the text to be recognized from the preset dictionary.

[0079] The specific implementation process of S310 is similar to that of S110 mentioned above. Please refer to the detailed explanation of S110 mentioned above. It will not be repeated here.

[0080] S312, If the text to be recognized reflects a preset sentiment tendency in the speech quality inspection scenario, then the text to be recognized is added to the abnormal text set.

[0081] The abnormal text set is used to store multiple user dialogue texts with preset emotional biases in voice quality inspection scenarios.

[0082] For example, taking the text to be identified, "I'll contact your after-sales service to resolve this." as an example, if the text to be identified reflects the preset sentiment "responsible" as determined by the above steps S304~S310, then the text to be identified is added to the abnormal text set. Thus, all user dialogue texts in the abnormal text set have a preset semantic (such as "responsible").

[0083] Optionally, after S312 above, the multi-tag qualitative word recognition method provided in this application embodiment may further include: S314, In response to a quality inspection assessment event, perform quality inspection processing on the dialogue to be inspected based on the abnormal text set.

[0084] The quality inspection assessment event indicator is used to trigger quality inspection processing. In practical applications, the quality inspection assessment event can be set according to actual needs, such as receiving a quality inspection assessment request from a user, etc. This application embodiment does not limit this.

[0085] Specifically, if the number of user dialogue texts in the abnormal text set is less than a preset threshold, it indicates that the user is relatively satisfied with the service quality of customer service, and thus it can be determined that the service dialogue to be inspected has passed the quality inspection.

[0086] If the number of user dialogue texts in the abnormal text set is greater than or equal to a preset threshold, the service quality of the service dialogue to be inspected is evaluated based on each user dialogue text in the abnormal text set and the customer service dialogue text associated with each user dialogue text in the service dialogue to be inspected.

[0087] For each user dialogue text, the customer service dialogue text associated with that user dialogue text in the service dialogue to be inspected can include the preceding and following customer dialogue texts adjacent to that user dialogue text. Since the user dialogue texts in the abnormal text set can reflect user dissatisfaction with customer service quality to a certain extent, and the customer service dialogue texts associated with these user dialogue texts can reflect the customer service attitude during the service process, by combining the user dialogue texts in the abnormal text set and the customer service dialogue texts associated with each user dialogue text in the service dialogue to be inspected, the service quality of the service dialogue to be inspected can be accurately assessed.

[0088] More specifically, the service quality of the service dialogue to be inspected can be evaluated as follows: Based on the category attributes corresponding to the multi-tag qualitative words in the target user dialogue text, the anomaly level of the target user dialogue text is determined, where the target user dialogue text is any user dialogue text in the abnormal text set; then, the target customer service dialogue text is scored based on a preset quality inspection strategy to obtain a quality score for the target customer service dialogue text, where the target customer service dialogue text is the customer service dialogue text associated with the target user dialogue text in the service dialogue to be inspected; based on the anomaly level corresponding to each user dialogue text in the abnormal text set, the quality scores of the customer service dialogue texts associated with each user dialogue text in the abnormal text set are weighted and summed to determine the service quality of the service dialogue to be inspected. The preset quality inspection strategy can be set according to actual needs, such as including preset mapping relationships between the indicator values ​​and quality scores of multiple evaluation indicators, etc., which is not limited in this embodiment.

[0089] For example, taking the multi-tag qualitative words and their corresponding category attributes shown in Table 3 above as an example, assuming that the anomaly levels corresponding to each category attribute are: high risk > risk > serious attitude > unprofessional terminology, then corresponding weights can be set for each category attribute according to the anomaly level, where the category attribute with a higher anomaly level has a higher weight. Based on this, after determining the category attributes corresponding to the multi-tag qualitative words in the target user's dialogue text, the weight corresponding to the category attribute can be determined as the weight of the target customer service dialogue text. Further, based on the weights corresponding to the customer service dialogue texts associated with each user dialogue text in the abnormal text set, the quality scores of the customer service dialogue texts associated with each user dialogue text in the abnormal text set are weighted and summed to obtain the quality score of the service dialogue to be inspected. If the quality score exceeds the preset score, the service to be inspected is determined to have passed the quality inspection; if the quality score does not exceed the preset score, the service to be inspected is determined to have failed the quality inspection.

[0090] The multi-label qualitative word recognition method provided in the above embodiments of this application adopts a technical approach of text classification task plus post-processing task. First, text classification technology is used to classify and recognize the text to be recognized as a whole, obtaining text label attributes that represent the sentiment tendency reflected by the text to be recognized in the target application scenario. On this basis, multiple preset multi-label qualitative words are matched with words in the text to be recognized, thereby recognizing the target multi-label qualitative words in the text to be recognized. Since the multiple preset multi-label qualitative words clearly define the word boundaries, multi-label qualitative word recognition based on word matching can avoid misrecognition caused by recognition boundary issues and the recognition of unlabeled words due to model generalization ability, which is conducive to improving recognition accuracy. Furthermore, since the word label attributes of multi-label qualitative words can represent the multi-label qualitative words in their respective sample texts... In the semantics of text classification, during the pre-training of the text classification model, the sample text attribute labels used to provide supervision signals for the model training process are determined by referring to the word label attributes of multi-label qualitative words in the sample text. Furthermore, the text recognition model trained based on the sample text and its corresponding sample text attribute labels, when classifying the text to be recognized, also considers the semantics of each word in the text to accurately identify the sentiment reflected by the text, obtaining the text label attributes corresponding to the text to be recognized. Thus, based on the text label attributes corresponding to the text to be recognized, the semantics of the target multi-label qualitative words can be accurately determined, obtaining the word label attributes corresponding to the target multi-label qualitative words. On this basis, user dialogue texts with preset semantics and multi-label qualitative words selected from the service dialogues to be inspected can accurately evaluate whether the service dialogues to be inspected pass the quality inspection.

[0091] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0092] With the above Figure 1 Corresponding to the multi-label qualitative word recognition method shown, this application also proposes a multi-label qualitative word recognition device. Please refer to... Figure 4 The diagram below illustrates the structure of a multi-tag qualitative word recognition device 400 according to an embodiment of this application. The device 400 includes: Acquisition unit 410 is used to acquire the text to be recognized; The classification unit 420 is used to classify the text to be identified using a text classification model to obtain the text label attributes corresponding to the text to be identified. The text label attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute labels corresponding to the sample text. The sample text attribute labels are determined based on the word label attributes of multi-label qualitative words in the sample text. The multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word label attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. The recognition unit 430 is used to match multiple preset multi-label qualitative words with words in the text to be recognized; The recognition unit 430 is further configured to, if the matching determines that the text to be recognized includes target multi-label qualitative words, determine the word label attribute corresponding to the multi-label qualitative words in the text to be recognized based on the text label attribute corresponding to the text to be recognized.

[0093] Optionally, the recognition unit 430 is further configured to, after matching a plurality of preset multi-label qualitative words with words in the text to be recognized, if the matching determines that the text to be recognized includes the target multi-label qualitative word, determine the position of the target multi-label qualitative word in the text to be recognized based on the arrangement order of each character in the target multi-label qualitative word in the text to be recognized.

[0094] Optionally, the identification unit 430 is further configured to: after matching a plurality of preset multi-label qualitative words with words in the text to be identified, if the matching determines that the text to be identified contains a target multi-label qualitative word, then query the category attribute corresponding to the target multi-label qualitative word from a preset dictionary, wherein the preset dictionary records the category attribute corresponding to each of the plurality of multi-label qualitative words.

[0095] Optionally, the target application scenario is a voice quality inspection scenario, the text to be recognized is any user dialogue text in the service dialogue to be inspected, and the service dialogue to be inspected is any historical dialogue between customer service and user. The recognition unit 430 is further configured to: before matching the preset multiple multi-label qualitative words with the words in the text to be recognized, determine whether the text to be recognized reflects a preset emotional tendency in the speech quality inspection scenario based on the text label attributes corresponding to the text to be recognized; When the recognition unit 430 matches the preset multiple multi-label qualitative words with the words in the text to be recognized, it performs the following steps: if the text to be recognized reflects the preset emotional tendency in the speech quality inspection scenario, then the preset multiple multi-label qualitative words are matched with the words in the text to be recognized.

[0096] The device 400 further includes: An adding unit is used to add the text to be identified to an abnormal text set if the text to be identified reflects the preset emotional tendency in the voice quality inspection scenario, as described in the recognition unit 430; the abnormal text set is used to store multiple user dialogue texts that have a preset emotional tendency in the voice quality inspection scenario.

[0097] Optionally, the device 400 further includes: The quality inspection unit is used to perform quality inspection processing on the service dialogue to be inspected based on the abnormal text set in response to a quality inspection evaluation event after the adding unit adds the text to be identified to the abnormal text set.

[0098] Optionally, when the quality inspection unit performs quality inspection processing on the service dialogue to be inspected based on the abnormal text set, it performs the following steps: if the number of user dialogue texts in the abnormal text set is less than a preset number threshold, then it determines that the service dialogue to be inspected has passed the quality inspection.

[0099] Optionally, when the quality inspection unit performs quality inspection processing on the service dialogue to be inspected based on the abnormal text set, it performs the following steps: if the number of user dialogue texts in the abnormal text set is greater than or equal to a preset number threshold, then the service quality of the service dialogue to be inspected is evaluated based on each user dialogue text in the abnormal text set and the customer service dialogue text associated with each user dialogue text in the service dialogue to be inspected.

[0100] Optionally, when the quality inspection unit evaluates the service quality of the service dialogue to be inspected based on each user dialogue text in the abnormal text set and the customer service dialogue text associated with each user dialogue text in the service dialogue to be inspected, the following steps are performed: Based on the category attributes corresponding to the multi-tag qualitative words in the target user dialogue text, determine the abnormality level corresponding to the target user dialogue text, wherein the target user dialogue text is any user dialogue text in the abnormal text set; score the target customer service dialogue text based on a preset quality inspection strategy to obtain the quality score of the target customer service dialogue text, wherein the target customer service dialogue text is the customer service dialogue text associated with the target user dialogue text in the service dialogue to be inspected; based on the abnormality level corresponding to each user dialogue text in the abnormal text set, perform a weighted sum of the quality scores of the customer service dialogue texts associated with each user dialogue text in the abnormal text set to determine the service quality of the service dialogue to be inspected.

[0101] Obviously, the multi-tag qualitative word recognition device provided in this application embodiment can be used as... Figure 1 The execution entity of the multi-label qualitative word recognition method shown is, for example... Figure 1 In the multi-label qualitative word recognition method shown, step S102 can be performed by... Figure 4 The acquisition unit 410 in the multi-label qualitative word recognition device shown executes step S104, which can be performed by... Figure 4 The classification unit 420 in the multi-label qualitative word recognition device shown executes steps S106 and S108, which can be performed by... Figure 4 The recognition unit 430 in the multi-label qualitative word recognition device shown is executed.

[0102] According to another embodiment of this application, Figure 4 The units in the multi-tag qualitative word recognition device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the multi-tag qualitative word recognition device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0103] According to another embodiment of this application, a general-purpose computing device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can run an application capable of performing tasks such as... Figure 1 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 4 The multi-tag qualitative word recognition device shown herein, and the multi-tag qualitative word recognition method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, and may be transferred to and run in an electronic device via such a medium.

[0104] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0105] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0106] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0107] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a multi-tag qualitative word recognition device at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Obtain the text to be recognized; The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified. The text label attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute labels corresponding to the sample text. The sample text attribute labels are determined based on the word label attributes of multi-label qualitative words in the sample text. Multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word label attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. Match multiple preset multi-label qualitative terms with the words in the text to be identified; If the matching determines that the text to be identified contains target multi-label qualitative words, the word label attributes corresponding to the multi-label qualitative words in the text to be identified are determined based on the text label attributes corresponding to the text to be identified.

[0108] The above is as stated in this application. Figure 1The method executed by the multi-tag qualitative word recognition device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0109] The electronic device can also perform Figure 1 The method, and the implementation of a multi-label qualitative word recognition device in Figure 1 , Figure 2 , Figure 3 The functions of the embodiments shown are not described in detail here.

[0110] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0111] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment is specifically used to perform the following operations: Obtain the text to be recognized; The text to be identified is classified using a text classification model to obtain the text label attributes corresponding to the text to be identified. The text label attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute labels corresponding to the sample text. The sample text attribute labels are determined based on the word label attributes of multi-label qualitative words in the sample text. Multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word label attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. Match multiple preset multi-label qualitative terms with the words in the text to be identified; If the matching determines that the text to be identified contains target multi-label qualitative words, the word label attributes corresponding to the multi-label qualitative words in the text to be identified are determined based on the text label attributes corresponding to the text to be identified.

[0112] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0113] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0114] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A multi-label qualitative word recognition method, characterized in that, include: Obtain the text to be recognized; The text to be identified is classified using a text classification model to obtain text tag attributes corresponding to the text to be identified. The text tag attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute tags corresponding to the sample text. The sample text attribute tags are determined based on the word tag attributes of multi-label qualitative words in the sample text. Multi-label qualitative words refer to words with different semantics in different contexts but the same category attribute. The word tag attributes are used to represent the semantics of the corresponding multi-label qualitative words in the sample text to which they belong. The text to be identified is any user dialogue text in the quality inspection service dialogue. Match multiple preset multi-label qualitative terms with the words in the text to be identified; If the matching determines that the text to be identified contains the target multi-label qualitative word, then based on the text label attribute corresponding to the text to be identified, the word label attribute corresponding to the target multi-label qualitative word in the text to be identified is determined; If the text to be identified reflects a preset sentiment in the target application scenario, then the text to be identified is added to the abnormal text set; Based on the category attributes corresponding to the multi-label qualitative words in the target user's dialogue text, the anomaly level corresponding to the target user's dialogue text is determined, wherein the target user's dialogue text is any user dialogue text in the set of abnormal texts; The target customer service dialogue text is scored based on a preset quality inspection strategy to obtain a quality score for the target customer service dialogue text. The target customer service dialogue text is the customer service dialogue text associated with the target user dialogue text in the service dialogue to be inspected. Based on the anomaly level corresponding to each user dialogue text in the abnormal text set, the quality scores of the customer service dialogue texts associated with each user dialogue text in the abnormal text set are weighted and summed to determine the service quality of the service dialogue to be inspected.

2. The method according to claim 1, characterized in that, After matching multiple preset multi-label qualitative terms with the words in the text to be identified, the method further includes: If the matching determines that the text to be identified includes a target multi-label qualitative word, then the position of the target multi-label qualitative word in the text to be identified is determined based on the arrangement order of each character in the target multi-label qualitative word in the text to be identified.

3. The method according to claim 1, characterized in that, After matching multiple preset multi-label qualitative terms with the words in the text to be identified, the method further includes: If the matching determines that the text to be identified contains the target multi-label qualitative word, then the category attribute corresponding to the target multi-label qualitative word is queried from the preset dictionary, wherein the preset dictionary records the category attribute corresponding to each of the multiple multi-label qualitative words.

4. The method according to any one of claims 1 to 3, characterized in that, The target application scenario is a voice quality inspection scenario, and the text to be recognized is any user dialogue text in the service dialogue to be inspected, and the service dialogue to be inspected is any historical dialogue between customer service and user. Before matching the preset multiple multi-label qualitative terms with the words in the text to be identified, the method further includes: Based on the text tag attributes corresponding to the text to be identified, determine whether the text to be identified reflects the preset emotional tendency in the voice quality inspection scenario; The step of matching multiple preset multi-label qualitative terms with words in the text to be identified includes: If the text to be identified reflects the preset emotional tendency in the speech quality inspection scenario, then multiple preset multi-label qualitative words are matched with the words in the text to be identified.

5. The method according to claim 4, characterized in that, The abnormal text set is used to store multiple user dialogue texts with the preset emotional tendency in the voice quality inspection scenario.

6. The method according to claim 1, characterized in that, After adding the text to be identified to the abnormal text set, the method further includes: In response to a quality inspection assessment event, the dialogue of the service to be inspected is processed based on the abnormal text set.

7. The method according to claim 6, characterized in that, The quality inspection process for the service dialogue to be inspected based on the abnormal text set includes: If the number of user dialogue texts in the abnormal text set is less than a preset threshold, then the service dialogue to be inspected is determined to have passed the quality inspection.

8. The method according to claim 6, characterized in that, The quality inspection process for the service dialogue to be inspected based on the abnormal text set includes: If the number of user dialogue texts in the abnormal text set is greater than or equal to a preset threshold, then the service quality of the service dialogue to be inspected is evaluated based on each user dialogue text in the abnormal text set and the customer service dialogue text associated with each user dialogue text in the service dialogue to be inspected.

9. A multi-label qualitative word recognition device, characterized in that, include: The acquisition unit is used to acquire the text to be recognized; A classification unit is used to classify the text to be identified using a text classification model to obtain text tag attributes corresponding to the text to be identified. The text tag attributes are used to represent the sentiment tendency reflected by the text to be identified in the target application scenario. The text classification model is trained based on sample text and the sample text attribute tags corresponding to the sample text. The sample text water-level tags are determined based on the word tag attributes of multi-tag qualitative words in the sample text, and the sample text attribute tags are determined based on the word tag attributes of multi-tag qualitative words in the sample text. Multi-tag qualitative words refer to words with different semantics in different contexts but the same category attribute. The word tag attributes are used to represent the semantics of the corresponding multi-tag qualitative words in their respective sample texts. The text to be identified is any user dialogue text in the quality inspection service dialogue. The recognition unit is used to match multiple preset multi-label qualitative words with words in the text to be recognized; The recognition unit is further configured to, if the matching determines that the text to be recognized includes a target multi-label qualitative word, then, based on the text label attribute corresponding to the text to be recognized, determine the word label attribute corresponding to the target multi-label qualitative word in the text to be recognized; An adding unit is used to add the text to be identified to the abnormal text set if the text to be identified reflects a preset sentiment tendency in the target application scenario. The quality inspection unit is used to determine the anomaly level of the target user dialogue text based on the category attributes corresponding to the multi-tag qualitative words in the target user dialogue text, wherein the target user dialogue text is any user dialogue text in the abnormal text set; to score the target customer service dialogue text based on a preset quality inspection strategy to obtain the quality score of the target customer service dialogue text, wherein the target customer service dialogue text is the customer service dialogue text associated with the target user dialogue text in the service dialogue to be inspected; and to perform a weighted sum of the quality scores of the customer service dialogue texts associated with each user dialogue text in the abnormal text set based on the anomaly level of each user dialogue text in the abnormal text set to determine the service quality of the service dialogue to be inspected.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1 to 8.

Images