Text classification method and device, electronic equipment and storage medium

Abstract

Embodiments of the present specification provide a text classification method and device, electronic equipment and a storage medium, wherein the text classification method comprises: obtaining a target text to be classified; performing vector conversion processing on the target text to obtain a target word vector and a target sentence vector; inputting the target word vector into a first prediction model to predict a word score, and outputting a first prediction result; inputting the target sentence vector into a second prediction model to predict a sentence score, and outputting a second prediction result; the word score is used to represent the professional degree of the target word vector and the professional degree of the word collocation; the sentence score is used to represent the professional degree of the target sentence vector; and according to the first prediction result and the second prediction result, a classification result of the target text is obtained, and the classification result is used to represent the quality level of the target text. In this way, the evaluation accuracy of the text classification result is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent quality inspection technology, and in particular to a text classification method, apparatus, electronic device and storage medium. Background Technology

[0002] In daily life, the volume of service data is expanding rapidly. Faced with this massive amount of data, manual quality inspection is far too inefficient to meet the rapidly growing demand. Utilizing machine learning or deep learning for intelligent quality inspection of service data is gradually becoming a popular trend in the service industry. Intelligent quality inspection refers to using natural language algorithms and predefined rules to analyze the dialogues between customer service agents and customers in call center scenarios, thereby achieving quality checks and improving agent efficiency and customer satisfaction.

[0003] In real-world scenarios, when customer service agents provide services to customers, in addition to their service attitude, other factors can significantly influence the customer's impression of the service. If the quality of the text used by customer service agents is evaluated solely from the perspective of service attitude, the evaluation may be less accurate. Summary of the Invention

[0004] This application provides a service language quality assessment method and apparatus to address how to improve the accuracy of text quality assessment.

[0005] In a first aspect, embodiments of this application provide a text classification method, including:

[0006] Obtain the text of the service terms to be evaluated;

[0007] The service term text is subjected to vector transformation processing to obtain target word vectors and target sentence structure vectors;

[0008] The target word vector is input into the word expressiveness prediction model for a first prediction process, and a first prediction result is output; the target sentence structure vector is input into the sentence expressiveness prediction model for a second prediction process, and a second prediction result is output.

[0009] Based on the first prediction result and the second prediction result, the service language text is classified according to its quality level to obtain the quality level classification result of the service language text.

[0010] Secondly, embodiments of this application provide a method for training a prediction model, including:

[0011] Obtain multiple training text samples;

[0012] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0013] Based on the training text samples after the multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model.

[0014] And / or, train an initial second prediction model based on the training text samples after the multiple vector transformations to obtain a second prediction model.

[0015] Thirdly, embodiments of this application provide a method for training a text classification model, including:

[0016] Obtain multiple training text samples;

[0017] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0018] Based on the training text samples after the multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model; and based on the training text samples after the multiple vector transformations, an initial second prediction model is trained to obtain the second prediction model.

[0019] A text classification model is trained based on the first prediction result and the second prediction result; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0020] Fourthly, embodiments of this application provide a text processing method, including:

[0021] Get the text of the response to be replied to;

[0022] The text to be replied to is subjected to vector transformation to obtain target word vectors and target sentence structure vectors;

[0023] The target word vector is input into the first prediction model to predict word scores, and the first prediction result is output; the target sentence structure vector is input into the second prediction model to predict sentence structure scores, and the second prediction result is output.

[0024] Based on the first prediction result and the second prediction result, the classification result of the response text to be replied to is obtained, and the classification result is used to characterize the quality level of the response text to be replied to;

[0025] If a non-quality response text that matches the response text to be replied to exists in the pre-configured mapping library, then at least one quality response text that has a mapping relationship with the non-quality response text is called from the mapping library to replace the response text to be replied to.

[0026] Fifthly, embodiments of this application provide a text classification device, including:

[0027] The first acquisition module is used to acquire the target text to be classified.

[0028] The first conversion module is used to perform vector conversion processing on the target text to obtain target word vectors and target sentence vectors;

[0029] The first prediction module is used to input the target word vector into a first prediction model to predict word scores and output a first prediction result; input the target sentence vector into a second prediction model to predict sentence scores and output a second prediction result; the word scores are used to characterize the word professionalism and word collocation professionalism of the target word vector; the sentence scores are used to characterize the sentence professionalism of the target sentence vector.

[0030] The first classification module is used to obtain the classification result of the target text based on the first prediction result and the second prediction result, and the classification result is used to characterize the quality level of the target text.

[0031] Sixthly, embodiments of this application provide a training apparatus for a prediction model, comprising:

[0032] The second acquisition module is used to acquire multiple training text samples;

[0033] The second conversion module is used to perform vector conversion processing on each training text sample to obtain multiple training text samples after vector conversion.

[0034] The first training module is used to train an initial first prediction model based on the training text samples after the multiple vector transformations to obtain a first prediction model; and / or to train an initial second prediction model based on the training text samples after the multiple vector transformations to obtain a second prediction model.

[0035] Seventhly, embodiments of this application provide a training apparatus for a text classification model, comprising:

[0036] The third acquisition module is used to acquire multiple training text samples;

[0037] The third conversion module is used to perform vector transformation processing on each training text sample to obtain multiple training text samples after vector transformation.

[0038] The second training module is used to train an initial first prediction model based on the training text samples after the multiple vectors have been transformed and processed, to obtain the first prediction model, and to train an initial second prediction model based on the training text samples after the multiple vectors have been transformed and processed, to obtain the second prediction model.

[0039] The third training module is used to train a text classification model based on the first prediction result and the second prediction result; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0040] Eighthly, embodiments of this application provide a text processing apparatus, including:

[0041] The fourth module is used to obtain the text of the response to be replied to;

[0042] The fourth conversion module is used to perform vector conversion processing on the response text to be replied to, to obtain target word vectors and target sentence structure vectors;

[0043] The second prediction module is used to input the target word vector into the first prediction model to predict word scores and output a first prediction result; and to input the target sentence structure vector into the second prediction model to predict sentence structure scores and output a second prediction result.

[0044] The second classification module is used to obtain the classification result of the response text to be replied to based on the first prediction result and the second prediction result. The classification result is used to characterize the quality level of the response text to be replied to.

[0045] The calling module is configured to, if a non-quality response text that matches the response text to be replied to exists in a pre-configured mapping library, call at least one quality response text that has a mapping relationship with the non-quality response text in the mapping library to replace the response text to be replied to.

[0046] Ninthly, embodiments of this application provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform a text classification method as described in the first aspect, or a prediction model training method as described in the second aspect, or a text classification model training method as described in the third aspect, or a text processing method as described in the fourth aspect.

[0047] In a tenth aspect, embodiments of this application provide a computer-readable storage medium for storing computer-executable instructions, which, when executed by a processor, implement the text classification method as described in the first aspect, or the prediction model training method as described in the second aspect, or the text classification model training method as described in the third aspect, or the text processing method as described in the fourth aspect.

[0048] As can be seen, in this embodiment, firstly, the target text to be classified is obtained; secondly, the target text is processed by vector transformation to obtain target word vectors and target sentence structure vectors; then, the target word vectors are input into a first prediction model to predict word scores, and a first prediction result is output; the target sentence structure vectors are input into a second prediction model to predict sentence structure scores, and a second prediction result is output; the word scores are used to characterize the word professionalism and word collocation professionalism of the target word vectors; the sentence structure scores are used to characterize the sentence structure professionalism of the target sentence structure vectors; finally, based on the first and second prediction results, the classification result of the target text is obtained, and the classification result is used to characterize the quality level of the target text. Thus, the first prediction result of the word scores of the target text is obtained through the first prediction model, and the second prediction result of the sentence structure scores of the target text is obtained through the second prediction model. Based on the first and second prediction results, the classification result of the target text is determined, and the quality level of the target text is comprehensively evaluated from multiple dimensions such as word transfer degree, word collocation transfer degree, and sentence structure professionalism, thereby improving the accuracy of the evaluation. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a text classification method provided in this application embodiment;

[0051] Figure 2 A flowchart illustrating another text classification method provided in this application embodiment;

[0052] Figure 3 A flowchart illustrating a training method for a prediction model provided in an embodiment of this application;

[0053] Figure 4 A flowchart illustrating a training method for a text classification model provided in this application embodiment;

[0054] Figure 5 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0055] Figure 6 A flowchart illustrating another text processing method provided in this application embodiment;

[0056] Figure 7 A schematic diagram of a text classification device provided in an embodiment of this application;

[0057] Figure 8 A schematic diagram of a training device for a prediction model provided in an embodiment of this application;

[0058] Figure 9 A schematic diagram of a training device for a text classification model provided in an embodiment of this application;

[0059] Figure 10 A schematic diagram of a text processing device provided in an embodiment of this application;

[0060] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0062] Intelligent quality inspection is often used to evaluate text quality from the perspective of service attitude. However, when customer service agents provide services, factors beyond service attitude can significantly influence a customer's impression of the service. Evaluating the quality of text used by customer service agents solely based on service attitude may lead to low accuracy. To address this issue, this application provides a text classification method.

[0063] Figure 1 This is a flowchart illustrating a text classification method provided in an embodiment of this application. (Refer to...) Figure 1 The service term quality assessment method provided in this embodiment specifically includes steps S102 to S108.

[0064] Step S102: Obtain the target text to be classified.

[0065] Acquiring target text can be done by collecting voice data and converting it into target text, receiving target text input from the user, or reading target text from a specified storage area, and so on.

[0066] The target text can be text that has a quality level classification requirement.

[0067] The quality rating reflects the quality of the target text.

[0068] Quality levels can include high quality and low quality; they can also include levels 1 through N, with each level corresponding to decreasing quality from best to worst; quality levels can also include other quality levels determined according to preset grading rules, and so on. For example, in a consultation scenario, the target text can be the response text from the consultant to the consultant's question. The higher the quality level of the response text, the better the consultant's response, the better the answer to the consultant's question, and the higher the consultant's satisfaction; the lower the quality level of the response text, the worse the consultant's response, the worse the answer to the consultant's question, and the lower the consultant's satisfaction. As another example, in a call center scenario, the target text can be the service language used by customer service agents to customers. The higher the quality level of the service language, the more appropriate the wording of the customer service agent, the better the service provided, and the higher the customer's satisfaction; the lower the quality level of the service language, the more inappropriate the wording of the customer service agent, the worse the service provided, and the lower the customer's satisfaction.

[0069] Service language can be a pending response text. For example, a customer may ask a question or make another statement to an agent. After receiving the customer's question or statement, the agent may submit a pending response text for that question or statement, which is a response text that the agent plans to send but has not yet sent.

[0070] Taking the target text as a service term as an example, the target text could be something like, "Do you need to handle service x?" or "Yes, we have a product y that is quite suitable for your needs, let me introduce it to you," and so on.

[0071] In another embodiment, the target text may further include text that has a quality level classification requirement, the preceding context text text_pre, and the following context text text_post.

[0072] For example, in a call center scenario, `text` can be a selected service phrase used for quality level classification. `text_pre` can be n service phrases preceding `text`, and `text_post` can be n service phrases following `text`, where `n` can be any natural number.

[0073] For example, text = "What do you think of Activity A?", text_pre = "What service do you need?", text_post = "You will receive an SMS notification later, please check your inbox".

[0074] When the target text includes text, text_pre, and text_post, obtaining the target text can be done by obtaining multiple consecutive service terms, wherein text_pre and text are separated by one or more preset delimiters, and text and text_post are separated by delimiters.

[0075] For example: Hello sir, how can I help you? #Well, we have a product that seems to suit your needs, let me introduce it to you. #Okay, if you're interested, I'll order it for you now.

[0076] In this context, "#" is the default separator, text-pre = "Hello sir, how can I help you?", text = "Well, we have a product that suits your needs, let me introduce it to you", and text_post = "Okay, if you need it, I'll order it for you now".

[0077] Step S104: Perform vector transformation on the target text to obtain target word vectors and target sentence structure vectors.

[0078] Optionally, the target text is subjected to vector transformation processing to obtain target word vectors and target sentence vectors, including: performing word segmentation processing on the target text to obtain the corresponding word segmentation results; performing sentence matching processing on the target text based on the word segmentation results and a pre-built sentence library, and determining the sentence patterns in the sentence library that match the target text as sentence matching results; performing vector transformation processing on the word segmentation results to obtain the corresponding target word vectors; and performing vector transformation processing on the sentence matching results to obtain the corresponding target sentence vectors.

[0079] In practice, pre-set terminology databases such as scenario-related professional terminology databases and service terminology databases can be introduced. Based on each pre-set terminology database, the target text is segmented into multiple words, and each segmented word is determined as the segmentation result corresponding to the target text.

[0080] For example, in a call center scenario, if the service utterance text is "Well, we have a product that fits your needs quite well, let me introduce it to you.", the professional terminology library and service utterance library for the call center scenario can be introduced to perform word segmentation processing on the text, and the word segmentation result of the text can be represented in the following manner: Well|,|we|have|a|product|that|fits|your|needs|quite|well|,|let|me|introduce|it|to|you. Here, the spacer "|" is used to separate two adjacent words in the word segmentation result. That is, the word segmentation result of the aforementioned text includes "Well", ",", "we", "have", "a", "product", "that", "fits", "your", "needs", "quite", "well", ",", "let", "me", "introduce", "it", "to", and "you". In one implementation, after performing word segmentation processing on the target text, the word segmentation result may be input into a pre-trained text type detection model for prediction processing, so as to obtain a prediction result of whether the target text belongs to a preset type of text. The pre-trained text type detection model may be one of a roberta model, a bert model, and a mengzi model. The prediction result output by the text type detection model may include: belonging to the preset type of text, and not belonging to the preset type of text.

[0081] If the prediction result is: not belonging to the preset type of text, the text classification process for this target text is ended; if the prediction result is: belonging to the preset type of text, subsequent steps S106 to S108 are continued to be executed.

[0082] By setting the preset type of text that has a quality level classification requirement, and predicting whether the target text belongs to the preset type of text through the text type detection model, target texts that do not belong to the preset type of text can be filtered out, and the text classification process is ended in advance, thereby indirectly improving the text classification efficiency.

[0083] Performing sentence pattern matching processing on the target text according to the word segmentation result and a pre-constructed sentence pattern library, and determining a sentence pattern in the sentence pattern library that matches the target text as a sentence pattern matching result.

[0084] The sentence pattern library may be custom constructed in advance, and stores a plurality of pre-stored sentence patterns, as shown in Table 1. Table 1 illustrates a custom constructed sentence pattern library provided in the embodiments of the present application.

[0085]

[0086] Table 1

[0087] In a specific example, the sentence pattern library includes multiple preset sentence patterns, such as "Please... is that okay?", "Excuse me...", etc. The word segmentation results of the target text include "please", "tell", "I", "you", "of", "phone number", "can", and "ma". Based on the word segmentation results of the target text and the various preset sentence patterns in the sentence pattern library, sentence pattern matching is performed on the target text. This will match the target text with the preset sentence pattern "Please... is that okay?" and identify "Please... is that okay?" as the sentence pattern matching result.

[0088] Optionally, based on the word segmentation results and a pre-built sentence pattern library, sentence pattern matching processing is performed on the target text, and the sentence patterns in the sentence pattern library that match the target text are determined as sentence pattern matching results. This includes: calculating the sentence pattern vector of the target text based on the word segmentation results; obtaining the sentence pattern vector of each pre-stored sentence pattern in the sentence pattern library; calculating the similarity between the sentence pattern vector of the target text and the sentence pattern vector of each pre-stored sentence pattern; and determining the pre-stored sentence pattern with the highest similarity as the sentence pattern matching result.

[0089] In practice, based on the word segmentation results and a pre-built sentence pattern library, sentence pattern matching is performed on the target text. This can be achieved by using contrastive learning (SIMCSE) to calculate the sentence pattern vector of the target text based on the word segmentation results. Specifically, firstly, the sentence pattern vector of each pre-stored sentence pattern in the sentence pattern library can be obtained. Next, the Facebook AI Similarity Search (FAISS) tool is used to calculate the difference between the sentence pattern vector of the target text and the sentence pattern vector of each pre-stored sentence pattern, thus obtaining the sentence pattern vector distance between the sentence pattern vector of the target text and the sentence pattern vector of each pre-stored sentence pattern.

[0090] Faiss is a tool developed by Facebook's AI team for large-scale similarity retrieval problems. It is written in C++, has a Python interface, and can achieve millisecond-level retrieval performance for indexes of up to 1 billion.

[0091] Sentence vector distance can be used to characterize the similarity between two sentences; the shorter the sentence vector distance, the higher the similarity between the two sentences.

[0092] For any pre-stored sentence pattern in the sentence pattern library, the shorter the sentence vector distance between the sentence pattern vector of the target text and the sentence pattern vector of the pre-stored sentence pattern, the higher the similarity between the target text and the pre-stored sentence pattern; the longer the sentence vector distance between the target text and the pre-stored sentence pattern, the lower the similarity between the target text and the pre-stored sentence pattern. Based on the calculated sentence vector distance for each sentence pattern, the similarity between the target text and each pre-stored sentence pattern can be determined.

[0093] After determining the similarity between the target text and each pre-stored sentence pattern, the pre-stored sentence pattern with the highest similarity can be identified as the sentence pattern matching result.

[0094] Optionally, the word segmentation results are processed into vectors to obtain the corresponding target word vectors. This includes: inputting the word segmentation results into a pre-trained first language model for embedding and positional encoding, outputting word embedding vectors and positional encoding vectors; summing the word embedding vectors and positional encoding vectors to obtain the target word vectors. The sentence matching results are also processed into vectors to obtain the corresponding target sentence vectors. This includes: inputting the sentence matching results into a pre-trained second language model for embedding and positional encoding, outputting sentence embedding vectors and sentence positional encoding vectors; summing the sentence embedding vectors and sentence positional encoding vectors to obtain the target sentence vectors.

[0095] The pre-trained first language model can be a BERT model, a Roberta model, a Mengzi model, or other language models. The pre-trained second language model can also be a BERT model, a Roberta model, a Mengzi model, or other pre-trained language models. The first and second language models can be language models of the same type but with different configuration parameters, or they can be language models of different types with different configuration parameters. The use of "first" and "second" here is merely for distinguishing two models with different purposes and has no practical meaning.

[0096] The word segmentation results are input into a first language model for embedding and positional encoding, outputting a word embedding vector and a positional encoding vector. The word embedding vector and the positional encoding vector are added together to calculate the target word embedding vector.

[0097] In one embodiment, the first language model may include a first sub-model and a second sub-model. The first sub-model is used for embedding the word segmentation results, and the second sub-model is used for positional encoding of the word segmentation results. The "first" and "second" in "first sub-model" and "second sub-model" are merely for the purpose of distinguishing two sub-models with different functions and have no actual meaning.

[0098] The word segmentation results can be input into the first language model for embedding and positional encoding, and the output can be word embedding vector and positional encoding vector. Alternatively, the word segmentation results can be input into the first sub-model for embedding and the output can be word embedding vector. At the same time, the word segmentation results can be input into the second sub-model for positional encoding and the output can be positional encoding vector.

[0099] In another embodiment, the first language model can be used to simultaneously perform embedding and positional encoding processing on the word segmentation results. The word segmentation results are input into the first language model for embedding, outputting word embedding vectors. This can be achieved by converting each word in the segmentation results into a vector form using the first language model, obtaining the word embedding vector for each word. The word segmentation results are then input into the first language model for positional encoding, outputting positional encoding vectors. This can be achieved by generating corresponding positional encoding vectors based on the positional information of each word in the sentence using the first language model. The positional encoding vectors corresponding to each word in the word segmentation results can reflect the position of each word in the sentence, thereby indirectly reflecting the order of the words in the target text.

[0100] After obtaining the word embedding vector and the position encoding vector, summing the word embedding vector and the position encoding vector yields the target word vector.

[0101] The sentence structure matching results are input into a pre-trained second language model for embedding and positional encoding, outputting a syntactical structure embedding and a structurepositional encoding vector. The syntactical structure embedding and the structurepositional encoding vector are added together to calculate the target sentence structure vector structureembedding.

[0102] In one embodiment, the second language model may include a third sub-model and a fourth sub-model. The third sub-model is used for embedding the sentence matching results, and the fourth sub-model is used for positional encoding of the sentence matching results. The terms "third" and "fourth" in "third sub-model" are merely for distinguishing between two sub-models with different functions and have no actual meaning.

[0103] The sentence matching results can be input into the second language model for embedding and position encoding, and the output sentence embedding vector and sentence position encoding vector can be obtained. Alternatively, the sentence matching results can be input into the third sub-model for embedding and the output sentence embedding vector can be obtained. At the same time, the word segmentation results can be input into the fourth sub-model for position encoding and the output sentence position encoding vector can be obtained.

[0104] In another embodiment, the second language model can be used to simultaneously perform embedding and positional encoding on the sentence matching results.

[0105] The sentence pattern extraction results are input into a second language model for embedding processing, outputting sentence pattern embedding vectors. This can be achieved by converting the sentence pattern extraction results into vector form using the second language model, thus obtaining the corresponding sentence pattern embedding vectors. The sentence pattern extraction results are then input into a second language model for positional encoding processing, outputting sentence pattern positional encoding vectors. This can be achieved by generating corresponding sentence pattern positional encoding vectors based on the positional information of each word in the sentence from the sentence pattern extraction results using the second language model.

[0106] After obtaining the sentence embedding vector and the sentence position encoding vector, the sentence embedding vector and the sentence position encoding vector can be summed to obtain the target sentence vector.

[0107] Step S106: Input the target word vector into the first prediction model to predict the word score and output the first prediction result; input the target sentence structure vector into the second prediction model to predict the sentence structure score and output the second prediction result; the word score is used to characterize the word professionalism and word collocation professionalism of the target word vector; the sentence structure score is used to characterize the sentence structure professionalism of the target sentence structure vector.

[0108] The first prediction model can be a decision tree classification model used for predicting word scores based on target word vectors. Word scores characterize the degree of professionalism and collocation of each word in the target text.

[0109] The level of professionalism of words can reflect, on the one hand, the similarity of each word to preset professional vocabulary, and on the other hand, the feedback evaluation of whether each word is professional in the corresponding scenario.

[0110] The target text may include one or more specialized terms, where each specialized term may be a word from a pre-configured specialized thesaurus, and the preset professional vocabulary corresponding to each specialized term may be preset preferred terms. The preset preferred terms corresponding to each specialized term may be determined based on the context of that specialized term.

[0111] The word segmentation results of the target text can include both general terms and specialized terms. General terms include, for example, quantifiers, modal particles, personal pronouns, etc. Specialized terms can be specific nouns or verbs used in specific business scenarios; for example, in a call center scenario, "payment method" is a specialized term.

[0112] The predicted word score of the target text, i.e., the predicted word score value, may include a first sub-score, which is used to characterize the similarity between each specialized word in the target text and the preset preferred words. For example, the higher the value of the first sub-score of the target text, the higher the similarity between each specialized word in the target text and the preset preferred words; the lower the value of the first sub-score of the target text, the lower the similarity between each specialized word in the target text and the preset preferred words.

[0113] Taking the first sub-score a1 with a value range of 0 ≤ a1 ≤ 100 as an example, in a specific example, the target text may include specialized words x1, x2, x3, and x4. x1 corresponds to the preset preferred word y1, x2 corresponds to the preset preferred word y2, x3 corresponds to the preset preferred word y3, and x4 corresponds to the preset preferred word y4. In the target text, the first sub-score for x1 can be 75 points, the first sub-score for x2 can be 97 points, the first sub-score for x3 can be 60 points, and the first sub-score for x4 can be 56 points. The first sub-score of the target text can be the average of the aforementioned four first sub-scores, or the average calculated after removing the maximum and minimum values ​​from the aforementioned four first sub-scores, or a value calculated based on the aforementioned four first sub-scores and other preset rules. The values ​​of the aforementioned first sub-scores are merely illustrative.

[0114] The predicted word scores for the target text may include a second sub-score, which characterizes whether each word in the target text provides a professional evaluation in the corresponding scenario. For example, a higher second sub-score indicates a better evaluation of the professionalism of each specialized term in the target text within the corresponding scenario; conversely, a lower second sub-score indicates a worse evaluation of the professionalism of each specialized term in the target text within the corresponding scenario.

[0115] Taking the second sub-score a2 with a value range of 0 ≤ a2 ≤ 100 as an example, in a specific example, the target text may include specialized term x1, specialized term x2, specialized term x3, and specialized term x4. In the target text, the second sub-score for x1 can be 78 points, the second sub-score for x2 can be 25 points, the second sub-score for x3 can be 46 points, and the second sub-score for x4 can be 77 points. The second sub-score of the target text can be the average of the aforementioned four second sub-scores, or the average calculated after removing the maximum and minimum values ​​from the aforementioned four second sub-scores, or a value calculated based on the aforementioned four second sub-scores and other preset rules. The values ​​of the aforementioned second sub-scores are merely illustrative.

[0116] The professionalism of word collocation can reflect, on the one hand, the similarity between the word collocations in the target text and the optimal collocations, and on the other hand, the feedback and evaluation of whether the word collocations are professional in the corresponding context.

[0117] The target text may include one or more specialized terms, where each specialized term can be a word from a pre-configured specialized thesaurus, and the preferred collocations corresponding to each specialized term and its preceding and following collocations can be preset preferred collocations. The predicted word score for the target text may include a third sub-score, which characterizes the similarity between the word collocations in the target text and the preferred collocations. The preferred collocations can be determined based on the collocations used by other high-quality users for the same specialized term.

[0118] For example, the higher the value of the third sub-score of the target text, the higher the similarity between the word collocations in the target text and the preferred collocations; the lower the value of the third sub-score of the target text, the lower the similarity between the word collocations in the target text and the preferred collocations.

[0119] Taking the value range of the third sub-score a3 as 0≤a3≤100 as an example, in a specific example, the target text may include special word x1, special word x2, special word x3 and special word x4. The word combination consisting of x1 and the words before and after x1 corresponds to the preset preferred word combination z1, the word combination consisting of x2 and the words before and after x2 corresponds to the preset preferred word combination z2, the word combination consisting of x3 and the words before and after x3 corresponds to the preset preferred word combination z3, and the word combination consisting of x4 and the words before and after x4 corresponds to the preset preferred word combination z4. In the target text, the third sub-score for the word collocation corresponding to x1 can be 35 points, the third sub-score for the word collocation corresponding to x2 can be 26 points, the third sub-score for the word collocation corresponding to x3 can be 77 points, and the third sub-score for the word collocation corresponding to x4 can be 85 points. The third sub-score of the target text can be the average of the aforementioned four third sub-scores, or the average calculated after removing the maximum and minimum values ​​from the aforementioned four third sub-scores, or a value calculated based on the aforementioned four third sub-scores and other preset rules. The values ​​of the aforementioned third sub-scores are merely illustrative.

[0120] The predicted word score for the target text may include a fourth sub-score, which characterizes whether the word collocations in the target text provide professional feedback in the corresponding scenario. For example, a higher fourth sub-score indicates better professional feedback in the target text's word collocations within the corresponding scenario; conversely, a lower second sub-score indicates worse professional feedback in the target text's word collocations within the corresponding scenario.

[0121] Taking the fourth sub-score a4 with a value range of 0 ≤ a4 ≤ 100 as an example, in a specific example, the target text may include specialized term x1, specialized term x2, specialized term x3, and specialized term x4. In the target text, the fourth sub-score for x1 could be 53 points, the fourth sub-score for x2 could be 96 points, the fourth sub-score for x3 could be 27 points, and the fourth sub-score for x4 could be 48 points. Therefore, the fourth sub-score of the target text could be the average of the aforementioned four fourth sub-scores, or the average calculated after removing the maximum and minimum values ​​from the aforementioned four fourth sub-scores, or a value calculated based on the aforementioned four fourth sub-scores and other preset rules. The values ​​of the aforementioned fourth sub-scores are merely illustrative.

[0122] The first, second, third, and fourth sub-scores mentioned above are merely used to distinguish the evaluation scores of the target text in different dimensions and have no actual meaning. The same applies to the fifth, sixth, seventh, and eighth sub-scores mentioned below, which will not be elaborated upon further.

[0123] In one embodiment, the predicted word score of the target text may include one or more of the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score of the target text. Alternatively, it may be a value obtained by calculating the average of one or more of the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score of the target text, or a value calculated based on one or more of the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score of the target text and other preset word score calculation rules.

[0124] The input parameter of the first prediction model can be the target word vector, and the output parameter of the first prediction model can be the prediction result of the word score corresponding to the target word vector, that is, the first prediction result.

[0125] The second prediction model can be another decision tree classification model used for predicting sentence structure scores based on the target sentence structure vector. The sentence structure score is used to characterize the sentence structure sophistication of the target sentence structure vector.

[0126] The professionalism of sentence structure can reflect, on the one hand, the similarity between the sentence structure and the preset professional sentence structure, and on the other hand, the feedback evaluation of whether the sentence structure is professional in the corresponding scenario.

[0127] The preset professional sentence structure corresponding to the sentence structure of the target text can be a preset preferred sentence structure. The predicted result of the sentence structure score of the target text, i.e., the predicted sentence structure score value, can include a fifth sub-score, which is used to characterize the similarity between the sentence structure of the target text and the preset preferred sentence structure. For example, the higher the value of the fifth sub-score of the target text, the higher the similarity between the sentence structure of the target text and the preset preferred sentence structure; the lower the value of the fifth sub-score of the target text, the lower the similarity between the sentence structure of the target text and the preset preferred sentence structure.

[0128] Taking the fifth sub-score a5 with a value range of 0 ≤ a5 ≤ 100 as an example, in a specific example, the sentence structure of the target text can be represented by m, and the preset preferred sentence structure corresponding to m is m'. The similarity between m and m' can be 79%, so the fifth sub-score of the target text can be 79. The aforementioned values ​​of the fifth sub-score are merely illustrative.

[0129] The predicted sentence structure score for the target text may include a sixth sub-score, which characterizes whether the sentence structure of the target text provides a professional evaluation in the corresponding scenario. For example, a higher sixth sub-score indicates a better evaluation of the professionalism of the target text's sentence structure in the corresponding scenario; conversely, a lower sixth sub-score indicates a worse evaluation of the professionalism of the target text's sentence structure in the corresponding scenario.

[0130] Taking the sixth sub-score a6 as an example, where the value range is 0 ≤ a6 ≤ 100, in a specific example, the sentence structure of the target text can be represented by m. The sixth sub-score of the target text can be 70 points. The aforementioned values ​​for the sixth sub-score are merely illustrative.

[0131] In one embodiment, the predicted sentence structure score of the target text may include one or more of the fifth sub-score and the sixth sub-score of the target text, or it may be a value obtained by calculating the average of the fifth sub-score and the sixth sub-score of the target text, or it may be a value calculated based on one or more of the fifth sub-score and the sixth sub-score of the target text and other preset sentence structure score calculation rules.

[0132] The input parameter of the second prediction model can be the target sentence vector, and the output parameter of the second prediction model can be the prediction result of the sentence score corresponding to the target sentence vector, i.e., the second prediction result.

[0133] For example, given the target text "Sir, you can choose three processing methods, which I will send to you via SMS later," a target word vector a1 and a target sentence structure vector b1 are generated based on this target text. a1 is input into the first prediction model for word scoring prediction, outputting the first prediction result: a word score of "10 points." Simultaneously, b1 is input into the second prediction model for sentence structure scoring prediction, outputting the second prediction result: a sentence structure score of "9 points."

[0134] By performing vector transformation on the target text, a target sentence structure vector is obtained. The target sentence structure vector is then input into a second prediction model to predict the sentence structure score, and the second prediction result is output. This sentence structure score is used to characterize the sentence structure professionalism of the target sentence structure vector. It can make full use of historical sentence structures that are frequently used in historical data and have received good feedback to evaluate the quality level of the target text.

[0135] Step S108: Based on the first prediction result and the second prediction result, the classification result of the target text is obtained. The classification result is used to characterize the quality level of the target text.

[0136] Optionally, the classification result of the target text is obtained based on the first prediction result and the second prediction result, including: determining the target prediction result of the target text based on the first prediction result and the second prediction result; inputting the target prediction result into the first logistic regression model for classification processing, and outputting the classification result of the target text.

[0137] The first prediction result can be the prediction result of the word score corresponding to the target text, i.e., the word score prediction value. The second prediction result can be the prediction result of the sentence score corresponding to the target text, i.e., the sentence score prediction value.

[0138] The implementation methods for determining the target prediction result of the target text based on the first prediction result and the second prediction result include, but are not limited to:

[0139] (a1) The word score prediction value and the sentence structure score prediction value are summed to obtain the first comprehensive prediction value, and the first comprehensive prediction value is determined as the target prediction result of the target text.

[0140] (a2) Calculate the average of the word score prediction value and the sentence structure score prediction value to obtain the second comprehensive prediction value, and determine the second comprehensive prediction value as the target prediction result of the target text.

[0141] (a3) The word score prediction value and the sentence structure score prediction value are weighted according to the pre-set weights to obtain the third comprehensive prediction value, and the third comprehensive prediction value is determined as the target prediction result of the target text.

[0142] The three implementation methods listed above are merely exemplary. The embodiments of this application can also generate target prediction results for target text based on other pre-configured comprehensive prediction value generation rules, first prediction results, and second prediction results.

[0143] The first logistic regression model, also known as the softmax model, is a classification model that can output multiple classification results and the probability of each classification result.

[0144] The quality grade classification results can include excellent and non-excellent, or grade 1 to grade N, with the quality of each grade decreasing from excellent to poor, and so on.

[0145] By inputting the target prediction results into the first logistic regression model for classification, the classification results of the target text can be obtained.

[0146] Optionally, the classification result of the target text is obtained based on the first prediction result and the second prediction result, including: inputting the first prediction result and the second prediction result into a text classification model to perform text classification and obtain the classification result of the target text.

[0147] The text classification model can be obtained through pre-training. The parameters to be trained are the first weight of the first prediction result and the second weight of the second prediction result. Here, the first and second weights are merely for distinguishing the two weight parameters and have no practical meaning. In practice, multiple data samples carrying sample labels can be obtained, and these labeled data samples can be input into the initial text classification model for iterative training to obtain the text classification model.

[0148] The first and second prediction results are input into the text type model to perform text classification based on the trained first and second weights, thereby obtaining the classification result of the target text.

[0149] Optionally, the target text includes the statement to be evaluated and its context statements; vector transformation processing is performed on the target text to obtain target word vectors and target sentence structure vectors, including: performing vector transformation processing on the statement to be evaluated to obtain target word vectors and target sentence structure vectors; performing vector transformation processing on the context statements of the statement to be evaluated to obtain context sentence structure vectors; after performing vector transformation processing on the target text to obtain target word vectors and target sentence structure vectors, the process further includes: inputting the target sentence structure vectors and context sentence structure vectors into a third prediction model to predict logical coherence scoring, and outputting a third prediction result; based on the first prediction result and the second prediction result, the classification result of the target text is obtained, including: performing quality level classification processing on the target text based on the first prediction result, the second prediction result, and the third prediction result to determine the quality level classification result of the target text.

[0150] The target text can include the statement to be evaluated and its contextual statements. The statement to be evaluated requires a quality level classification, and its contextual statements can include the preceding and following texts. Specifically, refer to the aforementioned `text`, the preceding text `text_pre`, and the following text `text_post`.

[0151] The statement to be evaluated is subjected to vector transformation to obtain the target word vector and the target sentence structure vector; the context statement to be evaluated is subjected to vector transformation to obtain the context sentence structure vector. The vector transformation process for the statement to be evaluated and the vector transformation process for the context statement can be referred to the aforementioned step S104, and will not be repeated here.

[0152] The target sentence structure vector and its context sentence structure vectors are input into a third prediction model to predict the logical coherence score, and the third prediction result is output. The third prediction model can be a decision tree classification model used for predicting the logical coherence score based on the target sentence structure vector and its context sentence structure vectors. The logical coherence score is used to characterize the level of professionalism of the logical coherence of the sentence pair consisting of the sentence to be evaluated and its context sentences.

[0153] The professionalism of the logical coherence of sentence pairs can be reflected on the similarity between the sentence pairs and the pre-set coherent expressions, and on the other hand, it can reflect the feedback evaluation of whether the sentence pairs are professional in the corresponding scenarios.

[0154] The predicted result of the logical coherence score of the target text, i.e., the predicted logical coherence score value, may include a seventh sub-score, which is used to characterize the similarity between sentence pairs in the target text and the preset coherent expressions. For example, the higher the value of the seventh sub-score of the target text, the higher the similarity between the sentence pairs in the target text and the preset coherent expressions; the lower the value of the seventh sub-score of the target text, the lower the similarity between the sentence pairs in the target text and the preset coherent expressions.

[0155] Taking the seventh sub-score a7, with a value range of 0 ≤ a7 ≤ 100, as an example, in a specific instance, the sentence pairs in the target text can be represented by n, and the pre-defined coherent expression corresponding to n is n'. The similarity between n and n' can be 35%, so the seventh sub-score of the target text can be 35. The aforementioned values ​​for the seventh sub-score are merely illustrative.

[0156] The predicted logical coherence score of the target text may include an eighth sub-score, which characterizes whether the sentence pairs in the target text provide a professional feedback evaluation in the corresponding scenario. For example, a higher value for the eighth sub-score indicates a better professional feedback evaluation of the sentence pairs in the corresponding scenario; conversely, a lower value for the eighth sub-score indicates a worse professional feedback evaluation of the sentence pairs in the corresponding scenario.

[0157] Taking the eighth sub-score a8 as an example, where the value range is 0 ≤ a8 ≤ 100, in a specific example, the sentence pairs in the target text can be represented by n. The eighth sub-score of the target text can be 36 points. The aforementioned values ​​for the eighth sub-score are merely illustrative.

[0158] In one embodiment, the predicted logical coherence score of the target text may include one or more of the seventh sub-score and the eighth sub-score of the target text, or it may be a value obtained by calculating the average of the seventh sub-score and the eighth sub-score of the target text, or it may be a value calculated based on one or more of the seventh sub-score and the eighth sub-score of the target text and other preset logical coherence score calculation rules.

[0159] After step S102 is executed, the target sentence vector and the context sentence vector can be input into the third prediction model to predict the logical coherence score and output the third prediction result.

[0160] Furthermore, the target text can be classified according to its quality level based on the first, second, and third prediction results to determine the quality level classification result. Optionally, after classifying the target text according to its quality level based on the first, second, and third prediction results to determine the quality level classification result, the process further includes: if the quality level classification result belongs to non-high-quality terms, determining the dimensions to be optimized in the target text based on the first, second, and third prediction results.

[0161] In a call center scenario, the target text is classified into quality levels based on the first, second, and third prediction results. This classification allows for a comprehensive evaluation of the text quality across multiple dimensions, including word count, sentence structure, and logical coherence, rather than solely relying on interjections to assess the service attitude of agents using the target text. This improves the accuracy of the quality level classification, and the prediction results for different dimensions reflect the text quality in those dimensions. Based on these predictions, it is possible to determine which dimensions still have room for optimization and which do not require optimization.

[0162] In one implementation, the quality grade classification result may include N quality grades, where i quality grades belong to high-quality terms and (Ni) quality grades belong to low-quality terms. N is a natural number greater than 1, i is a natural number greater than or equal to 1, and N is greater than i.

[0163] After determining the quality level classification result of the target text, it can be determined whether the quality level classification result is one of the i quality levels of the aforementioned high-quality terms. If so, it means that the quality of the target text is good and no further optimization is needed; if not, the dimension to be optimized of the target text is determined based on the first prediction result, the second prediction result and the third prediction result.

[0164] Optionally, the evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result. Based on the first prediction result, the second prediction result, and the third prediction result, the dimensions to be optimized of the target text are determined, including: among the first prediction result, the second prediction result, and the third prediction result, the prediction result with the smallest value is determined as the lowest score; and the evaluation dimension corresponding to the lowest score is determined as the dimension to be optimized of the target text.

[0165] The evaluation dimensions of the target text include, but are not limited to, word dimension, sentence structure dimension, and logical coherence dimension; among them, word dimension corresponds to the first prediction result; sentence structure dimension corresponds to the second prediction result; and logical coherence dimension corresponds to the third prediction result.

[0166] The first prediction result is the word score prediction value of the target text, the second prediction result is the sentence structure score prediction value of the target text, and the third prediction result is the logical coherence score prediction value of the target text.

[0167] Among the predicted values ​​of word scores, sentence structure scores, and logical coherence scores, the lowest value can be determined as the minimum score, and the evaluation dimension corresponding to the minimum score can be determined as the dimension to be optimized in the target text.

[0168] This dimension to be optimized can reflect why the quality level classification result of the target text belongs to the non-high quality level.

[0169] For example, if the predicted word score is 5, the predicted sentence structure score is 9, and the predicted logical coherence score is 8, with 5 being the smallest value, the word dimension corresponding to the predicted word score is determined as the dimension to be optimized in the target text. Furthermore, optimization can be performed on the target text based on the word dimension.

[0170] Considering that in some text classification applications, the purpose of determining the quality level classification result of a target text is not limited to penalizing poor-quality target text, but can also be to identify the reasons why the quality level classification result belongs to non-high-quality language, and to clarify from which dimension the target text should be optimized subsequently. By determining the evaluation dimension corresponding to the lowest score in each prediction result as the dimension to be optimized for the target text, the quality level classification result of the target text can be improved by subsequently targeting the evaluation dimension that most needs optimization.

[0171] Optionally, the evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result. Based on the first prediction result, the second prediction result, and the third prediction result, the dimensions to be optimized of the target text are determined, including: comparing the first prediction result with a preset score threshold to obtain a first comparison result; comparing the second prediction result with a preset score threshold to obtain a second comparison result; comparing the third prediction result with a preset score threshold to obtain a third comparison result; determining the target comparison result that meets the optimization conditions among the first comparison result, the second comparison result, and the third comparison result; and determining the evaluation dimension corresponding to the target comparison result as the dimensions to be optimized of the target text.

[0172] The evaluation dimensions of the target text include, but are not limited to, word dimension, sentence structure dimension, and logical coherence dimension; among them, word dimension corresponds to the first prediction result; sentence structure dimension corresponds to the second prediction result; and logical coherence dimension corresponds to the third prediction result.

[0173] The first prediction result is the word score prediction value of the target text, the second prediction result is the sentence structure score prediction value of the target text, and the third prediction result is the logical coherence score prediction value of the target text.

[0174] The predicted scores for words are compared with preset score thresholds to obtain the first comparison result; the predicted scores for sentence structure are compared with preset score thresholds to obtain the second comparison result; and the predicted scores for logical coherence are compared with preset score thresholds to obtain the third comparison result.

[0175] In another embodiment, different evaluation dimensions can correspond to different score thresholds. A first comparison result is obtained by comparing the predicted value of a word score with a first score threshold; a second comparison result is obtained by comparing the predicted value of a sentence structure score with a second score threshold; and a third comparison result is obtained by comparing the predicted value of a logical coherence score with a third score threshold.

[0176] The first score threshold corresponds to the word dimension, the second score threshold corresponds to the sentence structure dimension, and the third score threshold corresponds to the logical coherence dimension. The values ​​of any two of the first, second, and third score thresholds can be the same or different.

[0177] Among the first, second, and third comparison results, the target comparison result that meets the optimization condition is determined. The optimization condition can be that the predicted result value is less than or equal to the corresponding preset score threshold.

[0178] For example, if the first comparison result indicates that the predicted value of the word score is greater than the preset score threshold, the second comparison result indicates that the predicted value of the sentence structure score is less than the preset score threshold, and the third comparison result indicates that the predicted value of the logical coherence score is greater than the third score threshold, then the second comparison result can be determined as the target comparison result that meets the optimization conditions.

[0179] After determining the target comparison results, the evaluation dimensions corresponding to the target comparison results can be identified as the dimensions to be optimized for the target text. These dimensions can reflect why the target text's quality classification result belongs to a non-high-quality level.

[0180] For example, if the word score is 3, the sentence structure score is 4, and the logical coherence score is 8, with a preset score threshold of 7, and both the word score and sentence structure score are less than 7, it indicates that both the word dimension and sentence structure dimension of the target text need optimization. The predicted word score is 3, the predicted sentence structure score is 4, and the predicted logical coherence score is 8, with a preset score threshold of 7. Since 3 is less than 7 and 4 is less than 7, the word dimension and sentence structure dimension of the target text are identified as dimensions to be optimized. Furthermore, optimization can be performed on the target text based on the word dimension and sentence structure dimension.

[0181] Considering that in some text classification applications, the purpose of determining the quality level classification result of a target text is not limited to penalizing poor-quality target text, but can also be to identify the reasons why the quality level classification result belongs to non-high-quality language, and to clarify from which dimension the target text should be optimized subsequently. By judging whether the comparison result of the target result with the preset score threshold meets the optimization adjustment, one or more dimensions of the target text to be optimized can be identified, so that the quality level classification result of the target text can be improved in a targeted manner based on the one or more dimensions to be optimized.

[0182] In such Figure 1In the illustrated text classification method embodiment, firstly, the target text to be classified is obtained; secondly, the target text undergoes vector transformation processing to obtain target word vectors and target sentence structure vectors; next, the target word vectors are input into a first prediction model to predict word scores, and a first prediction result is output; the target sentence structure vectors are input into a second prediction model to predict sentence structure scores, and a second prediction result is output; word scores are used to characterize the word professionalism and word collocation professionalism of the target word vectors; sentence structure scores are used to characterize the sentence structure professionalism of the target sentence structure vectors; finally, based on the first and second prediction results, the classification result of the target text is obtained, and the classification result is used to characterize the quality level of the target text. The service language text to be evaluated is obtained; the service language text undergoes vector transformation processing to obtain target word vectors and target sentence structure vectors; the target word vectors are input into a word expressiveness model. The prediction model performs a first prediction process and outputs a first prediction result. The target sentence structure vector is input into the sentence structure expressiveness prediction model for a second prediction process, outputting a second prediction result. Based on the first and second prediction results, the service language text is classified according to its quality level, resulting in a quality level classification result. Thus, the first prediction model obtains the first prediction result for the word score of the target text, and the second prediction model obtains the second prediction result for the sentence structure score. Based on the first and second prediction results, the classification result of the target text is determined. The quality level of the target text is comprehensively evaluated from multiple dimensions, including word transfer degree, word collocation transfer degree, and sentence structure professionalism, improving the accuracy of the evaluation. Combining word expressiveness and sentence structure expressiveness, the quality level of service language is comprehensively evaluated, improving the efficiency and accuracy of service language quality evaluation.

[0183] Based on the same technical concept as the aforementioned method embodiments, this application also provides another embodiment of a text classification method applied to a call center scenario. Figure 2 A flowchart illustrating another text classification method provided in this application embodiment. See also... Figure 2 The text classification method specifically includes steps S202 to S224.

[0184] Step S202: Obtain service terms.

[0185] In a call center setting, the target text can be service-related language.

[0186] Step S204: Introduce the professional thesaurus and service terminology library for this scenario, and perform word segmentation on the service terms.

[0187] Step S206: Perform word embedding processing on the word segmentation results to obtain word embedding vectors.

[0188] Step S208: Perform position encoding processing on the word segmentation results to obtain the position encoding vector.

[0189] Step S210: Summate the word vector and the position encoding vector to obtain the target word vector.

[0190] Step S212: Predict word scores.

[0191] Step S214: Perform sentence pattern matching on the service terms after word segmentation to obtain the sentence pattern matching result.

[0192] Step S216: Embed the sentence pattern matching results to obtain the sentence pattern embedding vector.

[0193] Step S218: Perform position encoding processing on the sentence matching results to obtain the sentence position encoding vector.

[0194] Step S220: Summate the sentence embedding vector and the sentence position encoding vector to obtain the target sentence vector.

[0195] Step S222: Predict sentence structure scoring.

[0196] Step S224: Calculate the overall score.

[0197] Since the technical concept is the same, the description in this embodiment is relatively simple. For the relevant parts, please refer to the corresponding descriptions of the text classification method embodiments provided above.

[0198] Based on the same technical concept as the aforementioned text classification method embodiments, this application also provides an embodiment of a prediction model training method. Figure 3 This is a flowchart illustrating a training method for a prediction model provided in an embodiment of this application.

[0199] like Figure 3 As shown, in step S302, multiple training text samples are obtained.

[0200] Each training text sample may include a training text and multiple corresponding evaluation labels.

[0201] Step S304: Perform vector transformation processing on each training text sample to obtain multiple vector-transformed training text samples.

[0202] By performing vector transformation on the training text in each training text sample, we can obtain the corresponding vector-transformed training text sample. Each vector-transformed training text sample can include the corresponding training word vector and the corresponding training sentence vector.

[0203] Step S306: Train an initial first prediction model based on the training text samples after multiple vector transformations to obtain a first prediction model; and / or train an initial second prediction model based on the training text samples after multiple vector transformations to obtain a second prediction model.

[0204] Optionally, an initial first prediction model is trained based on multiple vector-transformed training text samples to obtain a first prediction model, including: training an initial first prediction model based on multiple training sentence vectors to obtain a first prediction model.

[0205] Optionally, the training text sample includes training text, a first evaluation label, and a second evaluation label; the training text includes at least one specialized word; each specialized word is a word from a pre-configured specialized lexicon; the first evaluation label is used to characterize the similarity between each specialized word and a preset preferred word; the second evaluation label is used to characterize the feedback evaluation of each specialized word; training an initial first prediction model based on multiple training sentence vectors to obtain the first prediction model includes: inputting each training word vector, the first evaluation label, and the second evaluation label into the initial first prediction model for iterative training to obtain the first prediction model.

[0206] The first evaluation label is used to characterize the similarity between each specific term and the preset preferred terms.

[0207] For example, the first evaluation label can be represented by any real number between [-1, +1]. For instance, in the training text, the context of a specialized term might indicate that the specialized term to be used is 'a', but the training text actually uses another term, 'b'. Since 'a' and 'b' are two completely different concepts, -1 can be used to represent the similarity between 'a' and 'b'. Similarly, in the training text, the context of a specialized term might indicate that the specialized term to be used is 'a', but the training text actually uses another term, 'a'. Since 'a' and 'a' have very similar meanings, 0.9 can be used to represent the similarity between 'a' and 'a'.

[0208] The first evaluation label can be generated manually and associated with the training text, or it can be obtained by preprocessing the training text based on historical data and preset first evaluation label generation rules and then associated with the training text.

[0209] The second evaluation label is used to characterize the feedback evaluation of each special term. This feedback evaluation can be manually labeled or generated based on the feedback terms in the training text in historical data. For example, if we take a dialogue from historical data: Customer: xxxx, Agent: [Training Text], Customer: [Positive Feedback Terms], then we can generate the corresponding feedback evaluation based on the positive feedback terms.

[0210] Generally, if a sentence contains multiple specialized terms, the feedback evaluation may not clearly show the individual opinions on each term, and can be approximated as having the same feedback evaluation for each specialized term within the same sentence. However, when a client gives a negative evaluation of the training text, they may specifically raise objections to a particular specialized term, and a corresponding feedback evaluation can be generated based on this.

[0211] Each training word vector, the first evaluation label, and the second evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model. This first prediction model can be applied to the aforementioned text classification method embodiments.

[0212] Optionally, the training text sample includes training text, a third evaluation label, and a fourth evaluation label; the training text includes at least one specialized word; each specialized word is a word from a pre-configured specialized lexicon; the third evaluation label is used to characterize the similarity between the words preceding and following each specialized word and the preset preferred words; the fourth evaluation label is used to characterize the feedback evaluation of the words preceding and following each specialized word; the initial first prediction model is trained based on multiple training sentence vectors to obtain the first prediction model, including: inputting each training word vector, the third evaluation label, and the fourth evaluation label into the initial first prediction model for iterative training to obtain the first prediction model.

[0213] The third evaluation label is used to characterize the similarity between the words preceding and following each specialized term and the preset preferred words.

[0214] For example, the third evaluation label can be represented by any real number between [-1, +1]. In a call center scenario, the preferred collocation can be the collocation most frequently used by the Class A agents with the highest service quality rating. Agent types can include Class A, Class B, and Class C agents, where Class A agents have the highest service quality rating and the highest customer satisfaction; Class B agents have a medium service quality rating and lower customer satisfaction than Class A agents; and Class B agents have the lowest service quality rating and lower customer satisfaction than Class C agents.

[0215] For specialized terms, the most frequently used collocation word for the highest-rated A-class agents is 'a', but in reality, the agents use other collocation words 'b'. Since 'a' and 'b' are two completely different concepts, the similarity between 'a' and 'b' can be represented by -1. Similarly, for specialized terms, the most frequently used collocation word for the highest-rated A-class agents is 'a', but in reality, the agents use other collocation words 'a'. Since 'a' and 'a' have very similar meanings, the similarity between 'a' and 'a' can be represented by 0.8.

[0216] The third evaluation label can be generated manually and associated with the training text, or it can be obtained by preprocessing the training text based on historical data and preset third evaluation label generation rules and then associated with the training text.

[0217] The fourth evaluation label is used to characterize the feedback evaluation of the words preceding and following each specialized term. This feedback evaluation can be manually annotated or generated based on the feedback language of the training text in historical data. For example, if we take a dialogue from historical data: Customer: xxxx, Agent: [Training Text], Customer: [Positive Feedback Language], then we can generate the corresponding feedback evaluation based on that positive feedback language.

[0218] Generally, if a sentence contains multiple specialized terms, the feedback evaluation may not reveal individual opinions on the collocations before and after each specialized term. It can be approximated that the feedback evaluations for the collocations before and after each specialized term are the same. However, when a client gives a negative evaluation of the training text, they may specifically raise objections to the collocations before and after a particular specialized term. Based on this, a corresponding feedback evaluation can be generated for the collocations before and after that specialized term.

[0219] Each training word vector, the third evaluation label, and the fourth evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model, which can be applied to the aforementioned text classification method embodiment. Optionally, an initial second prediction model is trained based on multiple vector-transformed training text samples to obtain the second prediction model, including: training the initial second prediction model based on multiple training sentence vectors to obtain the second prediction model.

[0220] Optionally, the training text sample includes training text, a fifth evaluation label, and a sixth evaluation label; the fifth evaluation label is used to characterize the similarity between the sentence structure of the training text and the preset preferred sentence structure; the sixth evaluation label is used to characterize the feedback evaluation of the sentence structure of the training text; the initial second prediction model is trained based on multiple training sentence structure vectors to obtain the second prediction model, including: inputting each training sentence structure vector, the fifth evaluation label, and the sixth evaluation label into the initial second prediction model for iterative training to obtain the second prediction model.

[0221] The fifth evaluation label is used to characterize the similarity between the sentence structure of the training text and the preset preferred sentence structure.

[0222] For example, the fifth evaluation label can be represented by any real number between [-1, +1]. In a call center scenario, the preset preferred phrase can be the preferred phrase most frequently used by the Class A agent with the highest service quality score in response to the intent of the training text. The agent types can include Class A agents, Class B agents, and Class C agents, where Class A agents have the highest service quality score and the highest customer satisfaction; Class B agents have a medium service quality score and lower customer satisfaction than Class A agents; and Class B agents have the lowest service quality score and lower customer satisfaction than Class C agents.

[0223] For example, if the most frequently used preferred phrase for a type A agent with the same intention is 'a', but the agent actually uses another phrase 'b', and 'a' and 'b' are two completely different concepts, then -1 can be used to represent the similarity between 'a' and 'b'. Similarly, if the most frequently used preferred phrase for a type A agent with the same intention is 'a', but the agent actually uses another phrase 'a', and 'a' and 'a' have very similar meanings, then 0.8 can be used to represent the similarity between 'a' and 'a'.

[0224] The fifth evaluation label can be generated manually and associated with the training text, or it can be obtained by preprocessing the training text based on historical data and preset fifth evaluation label generation rules and then associated with the training text.

[0225] The sixth evaluation label is used to characterize the feedback evaluation of the sentence structure of the training text. This feedback evaluation can be manually labeled or generated based on the feedback language of the training text in historical data. For example, if we take a dialogue from historical data: Customer: xxxx, Agent: [Training Text], Customer: [Positive Feedback Language], then we can generate the corresponding feedback evaluation based on that positive feedback language.

[0226] By inputting each training word vector, the fifth evaluation label, and the sixth evaluation label into the initial second prediction model for iterative training, a second prediction model can be obtained. This second prediction model can be applied to the aforementioned text classification method implementation examples.

[0227] Optionally, the training text samples include training text, context statements of the training text, a seventh evaluation label, and an eighth evaluation label; the training text samples after multiple vector transformations include training sentence vectors corresponding to the training text samples, the training sentence vectors include a first sentence vector corresponding to the training text, and a second sentence vector corresponding to the context statements; the seventh evaluation label is used to characterize the similarity between the sample paragraph and the preset coherent expression; the sample paragraph consists of training text and context statements; the eighth evaluation label is used to characterize the feedback evaluation of the sample paragraph; the method further includes: inputting each first sentence vector, second sentence vector, seventh evaluation label, and eighth evaluation label into the initial third prediction model for iterative training to obtain the third prediction model.

[0228] The seventh evaluation label is used to characterize the similarity between the sample paragraph and the preset coherent expression. The sample paragraph consists of training text and contextual statements.

[0229] The seventh evaluation label can be represented by any real number between [-1, +1].

[0230] For example, in a sample paragraph, if the industry requires the explanation of ABC in sequence for the same intent, but the agent actually says ACB, which is different from the industry requirement, then -0.4 can be used to represent the similarity between the sample paragraph and the preset coherent expression. As another example, in the training text, if the industry requires the explanation of ABC in sequence for the same intent, but the agent actually says A-B'-C, which is very similar, then 0.8 can be used to represent the similarity between the sample paragraph and the preset coherent expression.

[0231] The eighth evaluation label is used to characterize the feedback evaluation of the sample paragraph.

[0232] The feedback evaluation can be manually annotated or generated based on feedback phrases from training segments in historical data. For example, if a dialogue is taken from historical data: Customer: xxxx, Agent: [Training Segment], Customer: [Positive Feedback Phrases], then the corresponding feedback evaluation can be generated based on those positive feedback phrases.

[0233] By inputting each training word vector, the seventh evaluation label, and the eighth evaluation label into the initial third prediction model for iterative training, a third prediction model can be obtained. This third prediction model can be applied to the aforementioned text classification method implementation examples.

[0234] In such Figure 3 In an embodiment of the training method for the prediction model shown, firstly, multiple training text samples are acquired; then, each training text sample undergoes vector transformation processing to obtain multiple vector-transformed training text samples; finally, an initial first prediction model is trained based on the multiple vector-transformed training text samples to obtain a first prediction model; and / or, an initial second prediction model is trained based on the multiple vector-transformed training text samples to obtain a second prediction model. Thus, by using the vector-transformed training text samples, the initial first prediction model and / or the initial second prediction model can be trained to obtain prediction models for one or more evaluation dimensions.

[0235] Since the technical concept is the same, the description in this embodiment is relatively simple. For the relevant parts, please refer to the corresponding descriptions of the text classification method embodiments provided above.

[0236] Based on the same technical concept as the aforementioned text classification method embodiments, this application embodiment also provides a method for training a text classification model. Figure 4 This is a flowchart illustrating a training method for a text classification model provided in an embodiment of this application.

[0237] like Figure 4 As shown, in step S402, multiple training text samples are obtained.

[0238] Step S404: Perform vector transformation processing on each training text sample to obtain multiple training text samples after vector transformation.

[0239] Step S406: Train an initial first prediction model based on the training text samples after multiple vector transformations to obtain the first prediction model; and train an initial second prediction model based on the training text samples after multiple vector transformations to obtain the second prediction model.

[0240] Step S408: Train a text classification model based on the first prediction result and the second prediction result; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0241] After training the first and second prediction models, the initial text classification model to be trained can be iteratively trained based on the outputs of the first and second prediction models to obtain a text classification model. This text classification model can be applied to the aforementioned text classification method embodiments.

[0242] In such Figure 4 In the embodiment of the training method for the text classification model shown, firstly, multiple training text samples are acquired; secondly, each training text sample undergoes vector transformation processing to obtain multiple vector-transformed training text samples; next, an initial first prediction model is trained based on the multiple vector-transformed training text samples to obtain the first prediction model, and an initial second prediction model is trained based on the multiple vector-transformed training text samples to obtain the second prediction model; finally, the text classification model is trained based on the first prediction result and the second prediction result; the first prediction result is obtained through the first prediction model; the second prediction result is obtained through the second prediction model. Thus, using the vector-transformed training text samples, an initial first prediction model and an initial second prediction model can be trained to obtain two prediction models with different evaluation dimensions, thereby combining the prediction results obtained from the prediction models with different dimensions to train the text classification model.

[0243] Since the technical concept is the same, the description in this embodiment is relatively simple. For the relevant parts, please refer to the corresponding descriptions in the above-mentioned text classification method embodiment and prediction model training method embodiment.

[0244] Based on the same technical concept as the aforementioned text classification method embodiments, this application also provides an embodiment of a text processing method. Figure 5 This is a flowchart illustrating a text processing method provided in an embodiment of this application.

[0245] like Figure 5 As shown, in step S502, obtain the text of the response to be replied to.

[0246] The response text to be replied to can be the text that needs to be replied to after receiving the question or other comments from the questioner.

[0247] For example, in a call center scenario, the questioner can be a customer, and the response text to be replied to can be a response text that an agent plans to send in response to the customer's question or other questions but has not yet sent. Step S504 involves performing vector transformation processing on the response text to be replied to, obtaining target word vectors and target sentence structure vectors.

[0248] Step S506: Input the target word vector into the first prediction model to predict the word score and output the first prediction result; input the target sentence structure vector into the second prediction model to predict the sentence structure score and output the second prediction result.

[0249] Step S508: Based on the first prediction result and the second prediction result, obtain the classification result of the response text to be replied to. The classification result is used to characterize the quality level of the response text to be replied to.

[0250] Step S510: If the classification result belongs to non-high-quality terms and there is a non-high-quality response text in the pre-configured mapping library that matches the response text to be replied to, then at least one high-quality response text that has a mapping relationship with the non-high-quality response text is called in the mapping library to replace the response text to be replied to.

[0251] In one embodiment, if the classification result belongs to non-high-quality terms, it can be determined whether there is a non-high-quality response text in the pre-configured mapping library that matches the response text to be replied to. If so, at least one high-quality response text that has a mapping relationship with the non-high-quality response text is called in the mapping library; if not, it means that there is no need to recommend high-quality response text, and the text processing process ends.

[0252] Specifically, the system can query the mapping library to see if there are any non-high-quality response texts with a similarity greater than a preset similarity threshold. If so, it can call the high-quality response text mapped from the non-high-quality response text; otherwise, it can end the text processing process.

[0253] After identifying the non-high-quality response text that matches the text to be replied to, at least one high-quality response text that has a mapping relationship with the non-high-quality response text can be queried in the mapping library. If there are more than one high-quality response text, these multiple high-quality response texts can be displayed for the user to select one of them, and then the text to be replied to can be replaced with the high-quality response text selected by the user.

[0254] If the classification result of the response text to be replied to is "non-high-quality language", it means that the response text to be replied to is not suitable to be sent to the questioner.

[0255] In a call center scenario, at least one high-quality response text that has a mapping relationship with a non-high-quality response text can be one of the N most frequently used high-quality service phrases by Class A agents.

[0256] Optionally, the mapping library is configured as follows: An intent library is constructed, comprising multiple intent categories and multiple response texts corresponding to each intent category; any two response texts in the intent library are identified as a pair to be detected; each pair to be detected is input into a pre-trained third language model for vector transformation, outputting two corresponding sentence vectors to be processed; the sentence vector similarity of each pair to be detected is determined based on the two sentence vectors to be processed; if the sentence vector similarity is within a preset threshold range, the pair to be detected is identified as the target pair to be detected; if there is no semantic feature difference between the two response texts in the target pair to be detected, the two response texts in the target pair to be detected are classified separately to obtain two corresponding classification results; the classification results are used to characterize the quality level of the response text; if the two classification results corresponding to the target pair to be detected satisfy the mapping pair establishment conditions, a mapping relationship is constructed based on the target pair to be detected to obtain the mapping pair corresponding to the target pair to be detected; the mapping pairs corresponding to each pair of target pairs to be detected are stored in the mapping library.

[0257] Build an intent library that includes multiple intent categories and multiple response texts for each intent category.

[0258] An intent library can be built in the following ways:

[0259] (b1) First, j intent categories can be pre-set; for example: advance payment, negotiated payment, inquiry about payment intent, closing remarks, etc.

[0260] (b2) Next, k typical response texts can be assigned to each intent to form an initial intent library of size j*k.

[0261] (b3) Then, a new response text is received, denoted as s1. s1 is input into the pre-trained intent classification model for intent classification processing to obtain the intent classification result.

[0262] (b4) Based on the intent classification result, the new response text is filled into the initial intent library so that the new response text is associated with the corresponding intent category and stored.

[0263] (b5) Repeat steps (b3) and (b4) until the number of response texts in the initial intent library after filling reaches the preset threshold, and the intent library is obtained.

[0264] Next, any two response texts in the intent library can be identified as a pair to be detected; each pair to be detected is input into the pre-trained third language model for vector transformation processing, and the corresponding two sentence vectors to be processed are output.

[0265] The pre-trained third language model can be a BERT model, a Roberta model, a Mengzi model, or other language models. The third language model can be a language model of the same type as the first or second language model but with different configuration parameters, or it can be a language model of a different type but with different configuration parameters.

[0266] The terms "first," "second," and "third" here are merely for distinguishing three models with different purposes and have no practical meaning.

[0267] Based on the two sentence vectors corresponding to each pair of test pairs, the sentence vector similarity of each pair of test pairs is determined. This can be achieved by using the Faiss tool to calculate the interpolation between the two sentence vectors and obtaining the sentence vector distance between them.

[0268] Sentence vector distance can be used to characterize the similarity between two sentences; the shorter the sentence vector distance, the higher the similarity between the two sentences.

[0269] The shorter the sentence vector distance between the two sentence vectors corresponding to each pair of test cases, the higher the similarity between the two sentence structures in that pair; conversely, the longer the sentence vector distance, the lower the similarity between the two sentence structures in that pair. Based on the calculated sentence vector distance, the similarity between the two sentence structures in each pair of test cases can be determined, i.e., the sentence vector similarity of each pair of test cases.

[0270] If the sentence vector similarity is within a preset threshold range, the pair to be detected is determined as the target pair to be detected. If there is no semantic feature difference between the two response texts in the target pair to be detected, the two response texts in the target pair to be detected are classified separately to obtain the corresponding two classification results. If the two classification results corresponding to the target pair to be detected satisfy the conditions for establishing a mapping pair, a mapping relationship is constructed based on the target pair to be detected to obtain the mapping pair corresponding to the target pair to be detected.

[0271] The semantic feature difference can be determined based on the semantic features of the two response texts. For example, the two response texts include statement A and statement B.

[0272] Intent: Negotiated payment

[0273] Statement A: Hello sir, here are some payment options for you. You can choose any one of them: Payment Method A or Payment Method B.

[0274] The quality level classification result of statement A belongs to high-quality response text.

[0275] Statement B: You can choose to use either payment method A or payment method B.

[0276] The quality level classification result of statement B is that of a non-high-quality response text.

[0277] The semantic feature of expression A is that the payer is the recipient themselves, and the semantic feature of expression B is that the payer is the recipient's friend.

[0278] Based on the semantic features of expression A and expression B, it can be determined that the difference in semantic features between expression A and expression B is that the payment subjects are different.

[0279] The mapping pair can be established when, in two classification results, one belongs to the high-quality response text and the other does not.

[0280] For example,

[0281] Intent: Negotiated payment

[0282] Statement A: Hello sir, here are some payment options for you. You can choose any one of them: Payment Method A or Payment Method B.

[0283] The quality level classification result of statement A belongs to high-quality response text.

[0284] Statement B: You can choose to use either payment method A or payment method B.

[0285] The quality level classification result of statement B is that of a non-high-quality response text.

[0286] Statement C: Hello sir, here are a few payment methods recommended for your friend. Your friend can choose any one of them: Payment Method A and Payment Method B.

[0287] The quality level classification result of statement C belongs to high-quality response text.

[0288] Statement D: You tell your friend to either use payment method A or payment method B.

[0289] The quality level classification result of this D statement belongs to non-high-quality response text.

[0290] Table 2 shows a mapping pair constructed from expressions A, B, C, and D provided in an embodiment of this application.

[0291]

[0292] Table 2

[0293] As shown in Table 2, expressions A and B have the same semantic features, and one belongs to the category of high-quality service terms while the other belongs to the category of non-high-quality service terms. Therefore, mapping pair 1 can be constructed based on expressions A and B. Expressions C and D have the same semantic features, and one belongs to the category of high-quality service terms while the other belongs to the category of non-high-quality service terms. Therefore, mapping pair 2 can be constructed based on expressions C and D.

[0294] After constructing the mapping pairs, the mapping pairs corresponding to each set of target pairs to be detected can be stored in the mapping library.

[0295] Optionally, after constructing the mapping relationship based on the target pair to be detected, the text processing method further includes: inputting the semantic feature values ​​of the target pair to be detected into the second logistic regression model and outputting the confidence of the target pair to be detected; obtaining the mapping pair confidence of the target pair to be detected based on the confidence of the target pair to be detected and the sentence vector similarity of the target pair to be detected; and storing the mapping pair confidence of the target pair to be detected in the mapping library in association with the mapping pair corresponding to the target pair to be detected.

[0296] The second logistic regression model can be the Softmax model, which is a classification model that can output multiple classification results and the probability of each classification result.

[0297] The first and second logistic regression models are merely for distinguishing between two models with different functions and have no practical meaning.

[0298] Input the semantic feature values ​​of the target pair to be detected into the second logistic regression model, and output the confidence score of the target pair to be detected.

[0299] In statistics, a confidence interval for a probability sample is an interval estimate of a population parameter within that sample. The confidence interval shows the degree to which the true value of this parameter has a certain probability of falling within the range of the measured result. It provides the range of confidence regarding the measured value of the measured parameter, i.e., the "certain probability" mentioned earlier. This probability is called the confidence level.

[0300] The confidence of the mapping pair of the target pair can be obtained by calculating the product of the confidence of the target pair and the sentence vector similarity of the target pair.

[0301] That is, the confidence level of the mapping pair = confidence level * sentence vector similarity.

[0302] In the mapping library, the confidence scores of the mapping pairs of the target detection pairs are associated and stored with the mapping pairs corresponding to the target detection pairs.

[0303] Table 3 shows the stored data in a mapping library provided in one embodiment of this application.

[0304]

[0305] Table 3

[0306] As shown in Table 3, the mapping pair constructed from statements A and B is associated with a mapping pair confidence of 0.95, and the mapping pair constructed from statements C and D is associated with a mapping pair confidence of 0.8.

[0307] In a call center scenario, after step S510, if the actual response text used by the agent is the replaced high-quality response text, the agent's service quality score can be automatically increased to improve their service quality evaluation score. Simultaneously, the frequency of service quality optimization adoption can be incremented by 1. The recommended adoption rate measures the performance of the file processing system running the file processing method. Recommended adoption rate = Adoption frequency / Total number of times the high-quality response text is called.

[0308] Based on such Figure 5The illustrated text processing method embodiment first obtains the response text to be replied to; second, it performs vector transformation on the response text to be replied to, obtaining target word vectors and target sentence structure vectors; then, it inputs the target word vectors into a first prediction model to predict word scores and outputs a first prediction result; it inputs the target sentence structure vectors into a second prediction model to predict sentence structure scores and outputs a second prediction result; then, based on the first and second prediction results, it obtains the classification result of the response text to be replied to, which is used to characterize the quality level of the response text to be replied to; finally, if the classification result belongs to non-high-quality terms and there is a non-high-quality response text in the pre-configured mapping library that matches the response text to be replied to, then at least one high-quality response text that has a mapping relationship with the non-high-quality response text is called from the mapping library to replace the response text to be replied to. In this way, the word score of the response text to be replied to is predicted by the first prediction model to obtain the first prediction result, and the sentence structure score of the response text to be replied to is predicted by the second prediction model to obtain the second prediction result. Based on the first prediction result and the second prediction result, the classification result of the quality level of the response text to be replied to is determined. In order to optimize the quality of the response text to be replied to, if the classification result belongs to non-high-quality words and there is a non-high-quality response text that matches the response text to be replied to in the pre-configured mapping library, at least one high-quality response text that can replace the response text to be replied to is called.

[0309] Since the technical concept is the same, the description in this embodiment is relatively simple. For the relevant parts, please refer to the corresponding descriptions of the method embodiments provided above.

[0310] Based on the same technical concept as the aforementioned text classification method embodiments, this application also provides another embodiment of a text processing method applied to a call center scenario. Figure 6 A flowchart illustrating another text processing method provided in this application embodiment. See also... Figure 6 The text processing method specifically includes steps S602 to S630.

[0311] Step S602: Construct the initial intent library.

[0312] Step S604: By incorporating new intent sentences, a large-scale intent library is gradually formed.

[0313] A large-scale intent library can be an intent library where the number of intent sentences exceeds a preset threshold.

[0314] Step S606: Combine all sentences in the same intent in the intent library into pairs to form a pair to be detected.

[0315] Step S608: Obtain sentence vectors between different sentences using a pre-trained third language model.

[0316] Step S610: Calculate the sentence vector similarity of the pair to be detected.

[0317] In step S612, sentences whose vector similarity is within a certain threshold are determined to be semantically closely related.

[0318] Step S614: Subtract the sentence vectors of semantically closely related sentences, and use the difference as the semantic feature value.

[0319] Step S616: The obtained sentences are processed by a text classification model to detect high-quality and low-quality words.

[0320] Step S618: Sentences that meet all the above conditions can be used as mapping pairs. The mapping pairs and their confidence scores are included in the mapping library to complete the construction of the mapping library.

[0321] Step S620: Receive a new service phrase.

[0322] Step S622: Perform intent recognition based on the pre-trained intent classification model.

[0323] After obtaining the intent recognition result of the new service term, the new service term can be added to the intent library as a new intent sentence based on the intent recognition result.

[0324] Step S624: Input the text into the text classification model to detect high-quality and low-quality words.

[0325] Step S626: Provide top-n high-quality alternative expressions for non-high-quality terms detected by agents of category B and below.

[0326] Category A seats are those with the highest service quality ratings, while Category B and below seats are those with service quality ratings that need improvement.

[0327] Step S628: Obtain the response text to be replied to; based on the response text, retrieve the top-n high-quality responses from the mapping library to encourage agents to use high-quality responses.

[0328] In step S630, if an agent uses any of the top-n scripts, then add points to the agent's service quality score. Simultaneously, increment the frequency of service quality optimization by 1.

[0329] Since the technical concept is the same, the description in this embodiment is relatively simple. For the relevant parts, please refer to the corresponding descriptions of the method embodiments provided above.

[0330] In the above embodiments, a text classification method is provided, and correspondingly, a text classification device is also provided, which will be described below with reference to the accompanying drawings.

[0331] Figure 7 This is a schematic diagram of a text classification device provided in an embodiment of this application.

[0332] This embodiment provides a text classification device, including:

[0333] The first acquisition module 701 is used to acquire the target text to be classified.

[0334] The first conversion module 702 is used to perform vector conversion processing on the target text to obtain target word vectors and target sentence vectors;

[0335] The first prediction module 703 is used to input the target word vector into the first prediction model to predict the word score and output the first prediction result; input the target sentence vector into the second prediction model to predict the sentence score and output the second prediction result; the word score is used to characterize the word professionalism and word collocation professionalism of the target word vector; the sentence score is used to characterize the sentence professionalism of the target sentence vector.

[0336] The first classification module 704 is used to obtain the classification result of the target text based on the first prediction result and the second prediction result. The classification result is used to characterize the quality level of the target text.

[0337] Optionally, the first conversion module 702 includes:

[0338] The word segmentation unit is used to segment the target text into words and obtain the corresponding word segmentation results;

[0339] The matching unit is used to perform sentence pattern matching on the target text based on the word segmentation results and the pre-built sentence pattern library, and to determine the sentence pattern in the sentence pattern library that matches the target text as the sentence pattern matching result;

[0340] The transformation unit is used to perform vector transformation processing on the word segmentation results to obtain the corresponding target word vectors; and to perform vector transformation processing on the sentence matching results to obtain the corresponding target sentence vectors.

[0341] Optionally, the conversion unit is specifically used for:

[0342] The word segmentation results are input into a pre-trained first language model for embedding and position encoding processing, and the output word embedding vectors and position encoding vectors are generated.

[0343] The target word vector is obtained by summing the word embedding vector and the position encoding vector.

[0344] The sentence pattern matching results are input into a pre-trained second language model for embedding and position encoding, and the output sentence pattern embedding vector and sentence position encoding vector are output.

[0345] The target sentence vector is obtained by summing the sentence embedding vector and the sentence position encoding vector.

[0346] Optionally, the first classification module 704 is specifically used for:

[0347] Based on the first and second prediction results, determine the target prediction result for the target text;

[0348] The target prediction result is input into the first logistic regression model for classification, and the classification result of the target text is output.

[0349] Optionally, the target text includes the statement to be evaluated and the context statements of the statement to be evaluated; the transformation unit is specifically used for:

[0350] The statement to be evaluated is processed by vector transformation to obtain the target word vector and the target sentence structure vector; the context statement to be evaluated is processed by vector transformation to obtain the context sentence structure vector.

[0351] The text classification device also includes:

[0352] The coherence prediction module is used to input the target sentence vector and the context sentence vector into the third prediction model to predict the logical coherence score and output the third prediction result.

[0353] The first classification module 704 is specifically used for:

[0354] Based on the first, second, and third prediction results, the target text is classified according to its quality level, and the quality level classification result of the target text is determined.

[0355] Optionally, the text classification device also includes:

[0356] The determination module is used to determine the dimensions to be optimized in the target text based on the first prediction result, the second prediction result, and the third prediction result when the quality level classification result belongs to non-high-quality terms.

[0357] Optionally, the evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result; the determination module is specifically used for:

[0358] Among the first, second, and third prediction results, the prediction result with the smallest value is determined as the lowest score;

[0359] The evaluation dimension corresponding to the lowest score is identified as the dimension to be optimized in the target text.

[0360] Optionally, the evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result; the determination module is specifically used for:

[0361] The first prediction result is compared with a preset score threshold to obtain a first comparison result; the second prediction result is compared with a preset score threshold to obtain a second comparison result; and the third prediction result is compared with a preset score threshold to obtain a third comparison result.

[0362] Among the first comparison result, the second comparison result, and the third comparison result, determine the target comparison result that meets the conditions to be optimized;

[0363] The evaluation dimension corresponding to the target comparison result is determined as the dimension to be optimized in the target text.

[0364] Optionally, the matching unit is specifically used for:

[0365] Based on the word segmentation results, calculate the sentence vector of the target text; obtain the sentence vector of each pre-stored sentence in the sentence library;

[0366] Calculate the similarity between the sentence vector of the target text and the sentence vector of each pre-stored sentence;

[0367] The pre-stored sentence pattern with the highest similarity is selected as the sentence pattern matching result.

[0368] Optionally, the text classification module 704 is specifically used for:

[0369] The first and second prediction results are input into the text classification model to classify the text and obtain the classification result of the target text.

[0370] In this embodiment, a first acquisition module is used to acquire the target text to be classified; a first conversion module is used to perform vector conversion processing on the target text to obtain target word vectors and target sentence structure vectors; a first prediction module is used to input the target word vectors into a first prediction model to predict word scores and output a first prediction result; and to input the target sentence structure vectors into a second prediction model to predict sentence structure scores and output a second prediction result; the word scores are used to characterize the word professionalism and word collocation professionalism of the target word vectors; the sentence structure scores are used to characterize the sentence structure professionalism of the target sentence structure vectors; and a first classification module is used to obtain the classification result of the target text based on the first and second prediction results, and the classification result is used to characterize the quality level of the target text. Thus, by obtaining the first prediction result of the word scores of the target text through the first prediction model, obtaining the second prediction result of the sentence structure scores of the target text through the second prediction model, and determining the classification result of the target text based on the first and second prediction results, the quality level of the target text is comprehensively evaluated from multiple dimensions such as word transfer degree, word collocation transfer degree, and sentence structure professionalism, thereby improving the accuracy of the evaluation.

[0371] In the above embodiments, a method for training a prediction model is provided, and correspondingly, a device for training a prediction model is also provided, which will be described below with reference to the accompanying drawings.

[0372] Figure 8 This is a schematic diagram of a training device for a prediction model provided in an embodiment of this application.

[0373] This embodiment provides a training device for a prediction model, including:

[0374] The second acquisition module 801 is used to acquire multiple training text samples;

[0375] The second conversion module 802 is used to perform vector conversion processing on each training text sample to obtain multiple training text samples after vector conversion.

[0376] The first training module 803 is used to train an initial first prediction model based on training text samples after multiple vector transformations to obtain a first prediction model; and / or to train an initial second prediction model based on training text samples after multiple vector transformations to obtain a second prediction model.

[0377] Optionally, the first training module 803 includes:

[0378] The first training unit is used to train an initial first prediction model based on multiple training statement vectors to obtain the first prediction model.

[0379] Optionally, the first training module 803 is specifically used for:

[0380] The second training unit is used to train an initial second prediction model based on multiple training sentence vectors to obtain the second prediction model.

[0381] Optionally, the training text sample includes training text, a first evaluation label, and a second evaluation label; the training text includes at least one specialized term; each specialized term is a word from a pre-configured specialized lexicon; the first evaluation label is used to characterize the similarity between each specialized term and a preset preferred term; the second evaluation label is used to characterize the feedback evaluation of each specialized term; the first training unit is specifically used for:

[0382] Each training word vector, the first evaluation label, and the second evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model.

[0383] Optionally, the training text sample includes training text, a third evaluation label, and a fourth evaluation label; the training text includes at least one specialized word; each specialized word is a word from a pre-configured specialized vocabulary; the third evaluation label is used to characterize the similarity between the words preceding and following each specialized word and the preset preferred words; the fourth evaluation label is used to characterize the feedback evaluation of the words preceding and following each specialized word; the first training unit is specifically used for:

[0384] Each training word vector, the third evaluation label, and the fourth evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model.

[0385] Optionally, the training text samples include training text, a fifth evaluation label, and a sixth evaluation label; the fifth evaluation label is used to characterize the similarity between the sentence structure of the training text and the preset preferred sentence structure; the sixth evaluation label is used to characterize the feedback evaluation of the sentence structure of the training text; the second training unit is specifically used for:

[0386] Each training sentence vector, the fifth evaluation label, and the sixth evaluation label are input into the initial second prediction model for iterative training to obtain the second prediction model.

[0387] Optionally, the training text samples include training text, context statements of the training text, a seventh evaluation label, and an eighth evaluation label; the training text samples after multiple vector transformations include training sentence vectors corresponding to the training text samples, the training sentence vectors include a first sentence vector corresponding to the training text, and a second sentence vector corresponding to the context statements; the seventh evaluation label is used to characterize the similarity between the sample paragraph and the preset coherent expression; the sample paragraph consists of training text and context statements; the eighth evaluation label is used to characterize the feedback evaluation of the sample paragraph; the training device for the prediction model also includes:

[0388] The iterative training module is used to input each first sentence vector, second sentence vector, seventh evaluation label, and eighth evaluation label into the initial third prediction model for iterative training to obtain the third prediction model.

[0389] This application provides a training apparatus for a prediction model, comprising: a second acquisition module, a second conversion module, and a first training module; wherein: the second acquisition module is used to acquire multiple training text samples; the second conversion module is used to perform vector conversion processing on each training text sample to obtain multiple vector-converted training text samples; the first training module is used to train an initial first prediction model based on the multiple vector-converted training text samples to obtain a first prediction model; and / or, to train an initial second prediction model based on the multiple vector-converted training text samples to obtain a second prediction model. Through the vector-converted training text samples, the initial first prediction model and / or the initial second prediction model can be trained to obtain prediction models for one or more evaluation dimensions.

[0390] In the above embodiments, a method for training a text classification model is provided, and correspondingly, a training device for a text classification model is also provided, which will be described below with reference to the accompanying drawings.

[0391] Figure 9 This is a schematic diagram of a training device for a text classification model provided in an embodiment of this application.

[0392] This embodiment provides a training device for a text classification model, including:

[0393] The third acquisition module 901 is used to acquire multiple training text samples;

[0394] The third conversion module 902 is used to perform vector conversion processing on each training text sample to obtain multiple training text samples after vector conversion.

[0395] The second training module 903 is used to train an initial first prediction model based on training text samples after multiple vector transformations to obtain the first prediction model, and to train an initial second prediction model based on training text samples after multiple vector transformations to obtain the second prediction model.

[0396] The third training module 904 is used to train a text classification model based on the first prediction result and the second prediction result; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0397] The text classification model training device provided in this application includes: a third acquisition module, a third conversion module, a second training module, and a third training module, wherein: the third acquisition module is used to acquire multiple training text samples; the third conversion module is used to perform vector conversion processing on each training text sample to obtain multiple vector-converted training text samples; the second training module is used to train an initial first prediction model based on the multiple vector-converted training text samples to obtain a first prediction model, and to train an initial second prediction model based on the multiple vector-converted training text samples to obtain a second prediction model; the third training module is used to train a text classification model based on the first prediction result and the second prediction result; the first prediction result is obtained by predicting through the first prediction model; the second prediction result is obtained by predicting through the second prediction model. Through the vector-converted training text samples, the initial first prediction model and the initial second prediction model can be trained to obtain two prediction models with different evaluation dimensions, thereby combining the prediction results obtained from the prediction models with different dimensions to train the text classification model. In the above embodiments, a text processing method is provided, and correspondingly, a text processing device is also provided, which will be described below with reference to the accompanying drawings.

[0398] Figure 10 This is a schematic diagram of a text processing device provided in an embodiment of this application.

[0399] This embodiment provides a text processing device, including:

[0400] The fourth acquisition module 1001 is used to acquire the text of the response to be replied to;

[0401] The fourth conversion module 1002 is used to perform vector conversion processing on the response text to be replied to, so as to obtain the target word vector and the target sentence structure vector;

[0402] The second prediction module 1003 is used to input the target word vector into the first prediction model to predict the word score and output the first prediction result; and to input the target sentence vector into the second prediction model to predict the sentence score and output the second prediction result.

[0403] The second classification module 1004 is used to obtain the classification result of the response text to be replied to based on the first prediction result and the second prediction result. The classification result is used to characterize the quality level of the response text to be replied to.

[0404] Module 1005 is used to call at least one high-quality response text that has a mapping relationship with the non-high-quality response text in the mapping library to replace the response text if the classification result belongs to non-high-quality terms and there is a non-high-quality response text in the pre-configured mapping library that matches the response text to be replied to.

[0405] Optionally, the mapping library can be configured as follows:

[0406] Construct an intent library that includes multiple intent categories and multiple response texts corresponding to each intent category;

[0407] Define any two response texts in the intent library as a pair to be detected;

[0408] Each pair of sentences to be detected is input into a pre-trained third language model and vector transformation is performed to output the corresponding two sentence vectors to be processed.

[0409] Based on the two sentence vectors corresponding to each pair of pairs to be detected, determine the sentence vector similarity of each pair of pairs to be detected;

[0410] If the sentence vector similarity is within the preset threshold range, then the pair to be detected is determined as the target pair to be detected;

[0411] If there is no semantic feature difference between the two response texts in the target detection pair, then the two response texts in the target detection pair are classified separately to obtain two corresponding classification results; the classification results are used to characterize the quality level of the response texts.

[0412] If the two classification results corresponding to the target pair to be detected satisfy the conditions for establishing a mapping pair, then a mapping relationship is constructed based on the target pair to be detected to obtain the mapping pair corresponding to the target pair to be detected.

[0413] Store the mapping pairs corresponding to each group of target pairs to be detected in the mapping library.

[0414] Optionally, the text processing device further includes:

[0415] The third classification module is used to input the semantic feature values ​​of the target detection pair into the second logistic regression model and output the confidence score of the target detection pair.

[0416] The calculation module is used to obtain the mapping pair confidence of the target detection pair based on the confidence of the target detection pair and the sentence vector similarity of the target detection pair;

[0417] The storage module is used to associate and store the confidence scores of the target detection pair with the corresponding mapping pair in the mapping library.

[0418] In this embodiment, the text processing device includes a fourth acquisition module, a fourth conversion module, a second prediction module, a second classification module, and a calling module. The fourth acquisition module acquires the response text to be replied to. The fourth conversion module performs vector conversion processing on the response text to be replied to, obtaining target word vectors and target sentence structure vectors. The second prediction module inputs the target word vectors into a first prediction model to predict word scores and outputs a first prediction result; it also inputs the target sentence structure vectors into a second prediction model to predict sentence structure scores and outputs a second prediction result. The second classification module obtains a classification result of the response text to be replied to based on the first and second prediction results, the classification result representing the quality level of the response text to be replied to. The calling module, if a non-high-quality response text matching the response text to be replied to exists in a pre-configured mapping library, calls at least one high-quality response text that has a mapping relationship with the non-high-quality response text from the mapping library to replace the response text to be replied to. In this way, the word score of the response text to be replied to is predicted by the first prediction model to obtain the first prediction result, and the sentence structure score of the response text to be replied to is predicted by the second prediction model to obtain the second prediction result. Based on the first prediction result and the second prediction result, the classification result of the quality level of the response text to be replied to is determined. In order to optimize the quality of the response text to be replied to, if the classification result belongs to non-high-quality words and there is a non-high-quality response text that matches the response text to be replied to in the pre-configured mapping library, at least one high-quality response text that can replace the response text to be replied to is called.

[0419] Corresponding to the text classification method, prediction model training method, text classification model training method, or text processing method described above, based on the same technical concept, embodiments of this application also provide an electronic device for executing the aforementioned text classification method, prediction model training method, text classification model training method, or text processing method. Figure 11 This is a schematic diagram of the structure of a service term quality assessment device provided in an embodiment of this application.

[0420] like Figure 11As shown, electronic devices can vary considerably due to differences in configuration or performance. They may include one or more processors 1101 and memory 1102, with memory 1102 storing one or more application programs or data. Memory 1102 can be temporary or persistent storage. The application programs stored in memory 1102 may include one or more modules (not shown), each module including a series of computer-executable instructions from a service quality assessment device. Furthermore, processor 1101 may be configured to communicate with memory 1102, executing the series of computer-executable instructions stored in memory 1102 on the electronic device. The electronic device may also include one or more power supplies 1103, one or more wired or wireless network interfaces 1104, one or more input / output interfaces 1105, one or more keyboards 1106, etc.

[0421] In one specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the electronic device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0422] Obtain the target text to be classified;

[0423] The target text is processed by vector transformation to obtain target word vectors and target sentence structure vectors;

[0424] The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output; the target sentence structure vector is input into the second prediction model to predict the sentence structure score, and the second prediction result is output; the word score is used to characterize the word professionalism and word collocation professionalism of the target word vector; the sentence structure score is used to characterize the sentence structure professionalism of the target sentence structure vector.

[0425] Based on the first and second prediction results, the classification result of the target text is obtained, and the classification result is used to characterize the quality level of the target text.

[0426] Alternatively, in another specific embodiment, the electronic device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the dialogue role recognition device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0427] Obtain multiple training text samples;

[0428] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0429] Based on the training text samples after multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model.

[0430] And / or, train an initial second prediction model based on training text samples after multiple vector transformations to obtain a second prediction model.

[0431] Alternatively, in another specific embodiment, the electronic device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the dialogue role recognition device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0432] Obtain multiple training text samples;

[0433] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0434] Based on the training text samples after multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model; and based on the training text samples after multiple vector transformations, an initial second prediction model is trained to obtain the second prediction model.

[0435] A text classification model is trained based on the first and second prediction results; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0436] Alternatively, in yet another specific embodiment, the electronic device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the dialogue role recognition device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0437] Get the text of the response to be replied to;

[0438] The response text is subjected to vector transformation to obtain target word vectors and target sentence structure vectors;

[0439] The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output; the target sentence structure vector is input into the second prediction model to predict the sentence structure score, and the second prediction result is output.

[0440] Based on the first and second prediction results, the classification result of the response text to be replied to is obtained. The classification result is used to characterize the quality level of the response text to be replied to.

[0441] If a non-quality response text that matches the response text to be replied to exists in the pre-configured mapping library, then at least one quality response text that has a mapping relationship with the non-quality response text is called from the mapping library to replace the response text to be replied to.

[0442] Corresponding to the text classification method described above, based on the same technical concept, this application also provides a computer-readable storage medium.

[0443] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0444] Obtain the target text to be classified;

[0445] The target text is processed by vector transformation to obtain target word vectors and target sentence structure vectors;

[0446] The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output; the target sentence structure vector is input into the second prediction model to predict the sentence structure score, and the second prediction result is output; the word score is used to characterize the word professionalism and word collocation professionalism of the target word vector; the sentence structure score is used to characterize the sentence structure professionalism of the target sentence structure vector.

[0447] Based on the first and second prediction results, the classification result of the target text is obtained, and the classification result is used to characterize the quality level of the target text.

[0448] It should be noted that the embodiments of computer-readable storage media in this specification and the embodiments of text classification methods in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0449] Corresponding to the training method of the prediction model described above, based on the same technical concept, this application also provides a computer-readable storage medium.

[0450] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0451] Obtain multiple training text samples;

[0452] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0453] Based on the training text samples after multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model.

[0454] And / or, train an initial second prediction model based on training text samples after multiple vector transformations to obtain a second prediction model.

[0455] It should be noted that the embodiments of computer-readable storage media in this specification and the embodiments of the training method of prediction model in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0456] Corresponding to the training method of the text classification model described above, based on the same technical concept, this application embodiment also provides a computer-readable storage medium.

[0457] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0458] Obtain multiple training text samples;

[0459] Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation.

[0460] Based on the training text samples after multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model; and based on the training text samples after multiple vector transformations, an initial second prediction model is trained to obtain the second prediction model.

[0461] A text classification model is trained based on the first and second prediction results; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

[0462] It should be noted that the embodiments of computer-readable storage media in this specification and the embodiments of the training method of text classification model in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0463] Corresponding to the text processing method described above, based on the same technical concept, this application also provides a computer-readable storage medium.

[0464] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0465] Get the text of the response to be replied to;

[0466] The response text is subjected to vector transformation to obtain target word vectors and target sentence structure vectors;

[0467] The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output; the target sentence structure vector is input into the second prediction model to predict the sentence structure score, and the second prediction result is output.

[0468] Based on the first and second prediction results, the classification result of the response text to be replied to is obtained. The classification result is used to characterize the quality level of the response text to be replied to.

[0469] If a non-quality response text that matches the response text to be replied to exists in the pre-configured mapping library, then at least one quality response text that has a mapping relationship with the non-quality response text is called from the mapping library to replace the response text to be replied to.

[0470] It should be noted that the embodiments of computer-readable storage media in this specification and the embodiments of text processing methods in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0471] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0472] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, this specification can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0473] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0474] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0475] These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0476] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0477] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0478] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0479] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0480] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0481] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0482] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A text classification method, characterized in that, include: Obtain the target text to be classified; The target text is subjected to vector transformation processing to obtain target word vectors and target sentence structure vectors; The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output. The target sentence structure vector is input into a second prediction model to predict the sentence structure score, and a second prediction result is output. The word score is used to characterize the word professionalism and word collocation professionalism of the target word vector. The sentence structure score is used to characterize the sentence structure professionalism of the target sentence structure vector. The prediction result of the word score includes at least one of the following: a first sub-score, used to characterize the similarity between each specialized word in the target text and a preset preferred word; a second sub-score, used to characterize the feedback evaluation of whether each specialized word in the target text is professional in the corresponding scenario. The third sub-score is used to characterize the similarity between word collocations in the target text and preferred collocations; the fourth sub-score is used to characterize whether the word collocations in the target text provide professional feedback in the corresponding scenario; the prediction result of the sentence structure score includes at least one of the following: the fifth sub-score is used to characterize the similarity between the sentence structure of the target text and the preset preferred sentence structure; the sixth sub-score is used to characterize whether the sentence structure of the target text provides professional feedback in the corresponding scenario; Based on the first prediction result and the second prediction result, the target prediction result of the target text is determined; The target prediction result is input into a first logistic regression model for classification processing, and the classification result of the target text is output, which is used to characterize the quality level of the target text.

2. The method according to claim 1, characterized in that, The vector transformation process performed on the target text to obtain target word vectors and target sentence structure vectors includes: The target text is segmented into words to obtain the corresponding segmentation results; Based on the word segmentation results and the pre-built sentence pattern library, the target text is subjected to sentence pattern matching processing, and the sentence patterns in the sentence pattern library that match the target text are determined as sentence pattern matching results; The word segmentation results are processed by vector transformation to obtain the corresponding target word vectors; the sentence matching results are processed by vector transformation to obtain the corresponding target sentence vectors.

3. The method according to claim 2, characterized in that, The step of performing vector transformation on the word segmentation results to obtain the corresponding target word vectors includes: The word segmentation results are input into a pre-trained first language model for embedding and position encoding processing, and the output word embedding vector and position encoding vector are generated. The target word vector is obtained by summing the word embedding vector and the position encoding vector. The process of performing vector transformation on the sentence pattern matching result to obtain the corresponding target sentence pattern vector includes: The sentence pattern matching results are input into a pre-trained second language model for embedding and position encoding processing, and the sentence pattern embedding vector and sentence position encoding vector are output. The target sentence vector is obtained by summing the sentence embedding vector and the sentence position encoding vector.

4. The method according to claim 2, characterized in that, The target text includes the statement to be evaluated and the context statements of the statement to be evaluated; the vector transformation processing of the target text to obtain target word vectors and target sentence structure vectors includes: The statement to be evaluated is subjected to vector transformation processing to obtain the target word vector and the target sentence structure vector; the context statements of the statement to be evaluated are subjected to vector transformation processing to obtain the context sentence structure vector. After performing vector transformation processing on the target text to obtain target word vectors and target sentence structure vectors, the process further includes: The target sentence vector and the context sentence vector are input into the third prediction model to predict the logical coherence score, and the third prediction result is output. Based on the first prediction result, the second prediction result, and the third prediction result, the target text is classified into quality levels to determine the quality level classification result of the target text.

5. The method according to claim 4, characterized in that, After classifying the target text according to the first prediction result, the second prediction result, and the third prediction result, and determining the quality level classification result of the target text, the method further includes: If the quality level classification result is a non-high-quality term, the dimension to be optimized for the target text is determined based on the first prediction result, the second prediction result, and the third prediction result.

6. The method according to claim 5, characterized in that, The evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result. The step of determining the dimension to be optimized of the target text based on the first prediction result, the second prediction result, and the third prediction result includes: Among the first prediction result, the second prediction result, and the third prediction result, the prediction result with the smallest value is determined as the lowest score; The evaluation dimension corresponding to the lowest score is determined as the dimension to be optimized for the target text.

7. The method according to claim 5, characterized in that, The evaluation dimensions of the target text include word dimension, sentence structure dimension, and logical coherence dimension; the word dimension corresponds to the first prediction result; the sentence structure dimension corresponds to the second prediction result; and the logical coherence dimension corresponds to the third prediction result. The step of determining the dimension to be optimized of the target text based on the first prediction result, the second prediction result, and the third prediction result includes: The first prediction result is compared with a preset score threshold to obtain a first comparison result; the second prediction result is compared with the preset score threshold to obtain a second comparison result; and the third prediction result is compared with the preset score threshold to obtain a third comparison result. Among the first comparison result, the second comparison result, and the third comparison result, a target comparison result that meets the conditions to be optimized is determined. The evaluation dimension corresponding to the target comparison result is determined as the dimension to be optimized for the target text.

8. A method for training a prediction model, characterized in that, include: Obtain multiple training text samples; Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation. Based on the training text samples after the multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model. And / or, an initial second prediction model is trained based on the training text samples after the multiple vector transformations to obtain a second prediction model; the first prediction model is used to predict word scores, and the second prediction model is used to predict sentence structure scores; the word scores are used to characterize the word professionalism and word collocation professionalism of the training word vectors; the sentence structure scores are used to characterize the sentence structure professionalism of the training sentence structure vectors; the prediction results of the word scores include at least one of the following: a first sub-score, used to characterize the similarity between each specialized word in the training text sample and a preset preferred word; a second sub-score, used to characterize the feedback evaluation of whether each specialized word in the training text sample is professional in the corresponding scenario; The third sub-score is used to characterize the similarity between word collocations and preferred collocations in the training text sample; the fourth sub-score is used to characterize whether the word collocations in the training text sample are professional feedback evaluations in the corresponding scenario; the prediction result of the sentence structure score includes at least one of the following: the fifth sub-score is used to characterize the similarity between the sentence structure of the training text sample and the preset preferred sentence structure; the sixth sub-score is used to characterize whether the sentence structure of the training text sample is professional feedback evaluations in the corresponding scenario.

9. The method according to claim 8, characterized in that, The step of training an initial first prediction model based on the training text samples after transformation and processing of the multiple vectors to obtain the first prediction model includes: The initial first prediction model is trained based on multiple training statement vectors to obtain the first prediction model.

10. The method according to claim 8, characterized in that, The step of training an initial second prediction model based on the training text samples after the multiple vector transformations to obtain the second prediction model includes: The initial second prediction model is trained based on multiple training sentence vectors to obtain the second prediction model.

11. The method according to claim 9, characterized in that, The training text sample includes training text, a first evaluation label, and a second evaluation label; the training text includes at least one specialized word; each specialized word is a word from a pre-configured specialized vocabulary; the first evaluation label is used to characterize the similarity between each specialized word and a preset preferred word; The second evaluation label is used to characterize the feedback evaluation for each of the specialized terms; The step of training an initial first prediction model based on multiple training statement vectors to obtain the first prediction model includes: Each training word vector, the first evaluation label, and the second evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model.

12. The method according to claim 9, characterized in that, The training text sample includes training text, a third evaluation label, and a fourth evaluation label; the training text includes at least one specialized word; each specialized word is a word from a pre-configured specialized vocabulary; the third evaluation label is used to characterize the similarity between the words preceding and following each specialized word and the preset preferred words; The fourth evaluation label is used to characterize the feedback evaluation of the words that precede and follow each of the special terms; The step of training an initial first prediction model based on multiple training statement vectors to obtain the first prediction model includes: Each training word vector, the third evaluation label, and the fourth evaluation label are input into the initial first prediction model for iterative training to obtain the first prediction model.

13. The method according to claim 10, characterized in that, The training text sample includes training text, a fifth evaluation label, and a sixth evaluation label; the fifth evaluation label is used to characterize the similarity between the sentence structure of the training text and the preset preferred sentence structure; The sixth evaluation label is used to characterize the feedback evaluation of the sentence structure of the training text; The step of training an initial second prediction model based on multiple training sentence vectors to obtain a second prediction model includes: Each training sentence vector, the fifth evaluation label, and the sixth evaluation label are input into the initial second prediction model for iterative training to obtain the second prediction model.

14. The method according to claim 8, characterized in that, The training text samples include training text, contextual statements of the training text, a seventh evaluation label, and an eighth evaluation label; The training text samples after multiple vector transformations include training sentence vectors corresponding to the training text samples. The training sentence vectors include a first sentence vector corresponding to the training text and a second sentence vector corresponding to the context statement. The seventh evaluation label is used to characterize the similarity between the sample paragraph and the preset coherent expression. The sample paragraph consists of the training text and the context statements; The eighth evaluation label is used to characterize the feedback evaluation of the sample paragraph; the method further includes: Each of the first sentence vector, the second sentence vector, the seventh evaluation label, and the eighth evaluation label is input into the initial third prediction model for iterative training to obtain the third prediction model.

15. A training method for a text classification model, characterized in that, include: Obtain multiple training text samples; Each training text sample is subjected to vector transformation processing to obtain multiple training text samples after vector transformation. Based on the training text samples after the multiple vector transformations, an initial first prediction model is trained to obtain the first prediction model, and an initial second prediction model is trained based on the training text samples after the multiple vector transformations to obtain the second prediction model. The first prediction model is used to predict word scores, and the second prediction model is used to predict sentence structure scores. The word scores are used to characterize the word professionalism and word collocation professionalism of the word vectors in the training text samples. The sentence structure scores are used to characterize the sentence structure professionalism of the sentence structure vectors in the training text samples. The prediction results of the word scores include at least one of the following: a first sub-score, used to characterize the similarity between each specialized word in the training text samples and preset preferred words; a second sub-score, used to characterize the feedback evaluation of whether each specialized word in the training text samples is professional in the corresponding scenario. The third sub-score is used to characterize the similarity between word collocations and preferred collocations in the training text sample; the fourth sub-score is used to characterize whether the word collocations in the training text sample provide professional feedback in the corresponding scenario; the prediction result of the sentence structure score includes at least one of the following: the fifth sub-score is used to characterize the similarity between the sentence structure of the training text sample and the preset preferred sentence structure; the sixth sub-score is used to characterize whether the sentence structure of the training text sample provides professional feedback in the corresponding scenario; A text classification model is trained based on the first prediction result and the second prediction result; the first prediction result is obtained by the first prediction model; the second prediction result is obtained by the second prediction model.

16. A text processing method, characterized in that, include: Get the text of the response to be replied to; The text to be replied to is subjected to vector transformation to obtain target word vectors and target sentence structure vectors; The target word vector is input into the first prediction model to predict the word score, and the first prediction result is output. The target sentence structure vector is input into a second prediction model to predict sentence structure scores, and a second prediction result is output. The word score is used to characterize the word professionalism and word collocation professionalism of the target word vector. The sentence structure score is used to characterize the sentence professionalism of the target sentence structure vector. The prediction result of the word score includes at least one of the following: a first sub-score, used to characterize the similarity between each specialized word in the response text to be replied to and the preset preferred words; a second sub-score, used to characterize whether each specialized word in the response text to be replied to is a professional feedback evaluation in the corresponding scenario. The third sub-score is used to characterize the similarity between the word collocations in the response text to be replied to and the preferred collocations; the fourth sub-score is used to characterize whether the word collocations in the response text to be replied to are professional feedback evaluations in the corresponding scenario; the prediction result of the sentence structure score includes at least one of the following: the fifth sub-score is used to characterize the similarity between the sentence structure of the response text to be replied to and the preset preferred sentence structure; The sixth sub-score is used to characterize whether the sentence structure of the response text to be replied to is a professional feedback evaluation in the corresponding scenario; Based on the first prediction result and the second prediction result, determine the target prediction result of the response text to be replied to; The target prediction result is input into the first logistic regression model for classification processing, and the classification result of the response text to be replied to is output. The classification result is used to characterize the quality level of the response text to be replied to. If the classification result belongs to non-high-quality terms and there is a non-high-quality response text in the pre-configured mapping library that matches the response text to be replied to, then at least one high-quality response text that has a mapping relationship with the non-high-quality response text is called from the mapping library to replace the response text to be replied to.

17. The method according to claim 16, characterized in that, The mapping library is configured as follows: Construct an intent library that includes multiple intent categories and multiple response texts corresponding to each intent category; Any two response texts in the intent library are identified as a pair to be detected; Each group of test samples is input into a pre-trained third language model and vector transformation is performed to output two corresponding sentence vectors. Based on the two sentence vectors to be processed corresponding to each pair of pairs to be detected, the sentence vector similarity of each pair of pairs to be detected is determined. If the sentence vector similarity is within a preset threshold range, then the pair to be detected is determined as the target pair to be detected; If there is no semantic feature difference between the two response texts in the target detection pair, then the two response texts in the target detection pair are classified respectively to obtain two corresponding classification results; The classification results are used to characterize the quality level of the response text; If the two classification results corresponding to the target pair to be detected satisfy the conditions for establishing a mapping pair, then a mapping relationship is constructed based on the target pair to be detected to obtain the mapping pair corresponding to the target pair to be detected; The mapping pairs corresponding to each group of target pairs to be detected are stored in the mapping library.

18. The method according to claim 17, characterized in that, After constructing the mapping relationship based on the target to be detected, the process further includes: The semantic feature values ​​of the target detection pair are input into the second logistic regression model for classification, and the confidence score of the target detection pair is output. The mapping pair confidence of the target detection pair is obtained based on the confidence of the target detection pair and the sentence vector similarity of the target detection pair. The confidence scores of the mapping pairs of the target detection pairs are associated and stored in the mapping library to the mapping pairs corresponding to the target detection pairs.

19. An electronic device, characterized in that, The device includes: A processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform a text classification method as claimed in any one of claims 1-7, or a method for training a prediction model as claimed in any one of claims 8-14, or a method for training a text classification model as claimed in claim 15, or a text processing method as claimed in any one of claims 16-18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions, which, when executed by a processor, implement the text classification method as described in any one of claims 1-7, or the training method for a prediction model as described in any one of claims 8-14, or the training method for a text classification model as described in claim 15, or the text processing method as described in any one of claims 16-18.

Images