Complaint label classification method, device and equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a complaint label classification method and device, equipment and a medium, and the method comprises the steps: obtaining complaint data of a user, converting the complaint data into a complaint text, and inputting the complaint text into a large language model; the big language model analyzes and processes the complaint text and outputs complaint key information; performing label classification on the complaint key information according to a preset label framework to obtain a label result; according to the historical complaint labels, whether the label results are classified accurately or not is judged, and when the label results are classified accurately, label classification is completed. According to the method, the content of the complaint of the user is analyzed, the content of the complaint of the user is subjected to label classification, whether label classification is correct or not is judged through the historical complaint labels, and the accuracy of label classification is improved.

Description

Technical Field

[0001] The present invention relates to the fields of data processing technology and financial technology, and in particular to a complaint label classification method, device, equipment and medium. Background Art

[0002] In the field of financial technology, telephone operators are the first point of contact when handling customer complaints and feedback. Their personal qualities, abilities, and proficiency in the business are crucial to the accuracy of problem description, the breakdown of demands, and the identification of responsible departments.

[0003] In the quality inspection and analysis of customer complaint tickets, due to the diversity and complexity of customer complaint content, it is difficult to structure the complaint content with high quality, which makes attribution analysis complicated. The traditional processing method relies on the subjective judgment of business personnel, combined with manual operation and the assistance of information systems. It takes about 5 minutes to process each ticket, and different personnel have subjective differences in their understanding of the same complaint, making it difficult to achieve standardized processing.

[0004] To improve efficiency and accuracy, some companies have adopted intelligent solutions based on high-quality sample construction and training models. These models perform reasoning tasks by learning from training data. However, this approach requires a lot of time and resources to prepare samples and build models, and when the attribution system changes, the model needs to be retrained. In addition, changes in specific vocabulary or context may also cause large fluctuations in model accuracy, thereby increasing development costs. Summary of the invention

[0005] The present invention provides a complaint label classification method, apparatus, computer equipment and medium to solve the problem that the traditional label classification model is sensitive to changes in specific vocabulary and context, resulting in large fluctuations in label classification accuracy and high development costs.

[0006] In the first aspect, a complaint label classification method is provided, including:

[0007] Acquire complaint data of the user, convert the complaint data into complaint text, and input the complaint text into the large language model;

[0008] The large language model analyzes and processes the complaint text and outputs key complaint information;

[0009] Classify the key complaint information according to a preset label framework to obtain label results;

[0010] Whether the label result is accurately classified is determined based on the historical complaint labels. If the label result is accurately classified, the label classification is completed.

[0011] In a second aspect, a complaint label classification device is provided, comprising:

[0012] A data acquisition module, used to acquire user complaint data, pre-process the complaint data, and convert the complaint data into complaint text;

[0013] A data analysis module, used for the large language model to analyze and process the complaint text and output key complaint information;

[0014] A label classification module is used to classify the key complaint information according to a preset label framework to obtain label results;

[0015] The label judgment module is used to judge whether the label result is accurate based on the historical complaint labels. When the label result is accurate, the label classification is completed.

[0016] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned complaint label classification method when executing the computer program.

[0017] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned complaint label classification method are implemented.

[0018] The solution implemented by the above complaint label classification method, device, computer equipment and storage medium involves a series of automated and optimized steps to improve the efficiency and accuracy of data collection. The specific process is as follows: First, the system collects user complaint data, which may come from different channels, such as customer service hotlines, social media, emails, etc. The collected complaint data is converted into text format for subsequent analysis. This step ensures that all types of complaint data can be processed uniformly.

[0019] The converted complaint text is input into the pre-trained large language model. This model can understand and analyze natural language and is a key tool for processing complaint text. The large language model conducts in-depth analysis of the complaint text and extracts key information. This key information may include the main issues of the complaint, the products or services involved, the user's emotions, etc. According to the preset label framework, the extracted complaint key information is classified. This label framework may include a variety of predefined categories, such as product quality, service delays, price issues, etc. Using historical complaint data and known labels, the system will determine whether the current label classification result is accurate. This step is an important link to ensure the quality of classification. If the historical complaint label verification shows that the classification result is accurate, then the label classification process of the complaint is completed. This enables enterprises to respond quickly to user complaints and take corresponding measures. The present invention not only improves the automation level of complaint processing, but also improves the accuracy of label classification through historical data verification. This helps enterprises better understand customer complaints, optimize customer service processes, and improve customer satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0021] Figure 1 is a schematic diagram of an application environment of a complaint label classification method in an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a complaint label classification method in one embodiment of the present invention;

[0023] Figure 3 is a structural schematic diagram of a complaint label classification device in one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0025] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present invention or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0027] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] The complaint label classification method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment of , the client communicates with the server through the network. The server can receive user complaint data through the client, which may come from different channels, such as customer service hotlines, social media, emails, etc. The collected complaint data is converted into text format for subsequent analysis. This step ensures that all types of complaint data can be processed uniformly. The converted complaint text is input into the pre-trained large language model. This model can understand and analyze natural language and is a key tool for processing complaint text. The large language model conducts in-depth analysis of the complaint text and extracts key information. This key information may include the main problem of the complaint, the product or service involved, the user's emotions, etc. The extracted complaint key information is classified according to the preset label framework. This label framework may include multiple predefined categories, such as product quality, service delay, price issues, etc. Using historical complaint data and known labels, the system will determine whether the current label classification result is accurate. This step is an important part of ensuring the quality of classification. If the historical complaint label verification shows that the classification result is accurate, the label classification process for the complaint is completed. This enables enterprises to respond to user complaints quickly and take corresponding measures. Match the solution to the problem based on the label result and feedback it back to the client. The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0029] See also Figure 2 As shown, Figure 2 A flowchart of a complaint label classification method provided by an embodiment of the present invention includes the following steps:

[0030] S10, obtaining complaint data of the user, converting the complaint data into complaint text, and inputting the complaint text into the large language model;

[0031] S20, the large language model analyzes and processes the complaint text and outputs key complaint information;

[0032] S30. Classify the key complaint information according to a preset label framework to obtain label results.

[0033] In this embodiment, when processing customer complaint information, it is first necessary to collect data from different channels, which may exist in various forms such as text, voice or video. For non-text complaint data, such as voice recordings, it is necessary to convert them into text format through voice recognition technology for analysis. This step usually involves transcribing voice data into analyzable text.

[0034] For data that is already in text form, the processing steps may only include cleaning and formatting the data to ensure consistency and readability. Subsequently, these pre-processed texts are input into the big language model. The big language model has been trained with a large amount of text data and has the ability to understand and analyze natural language. It can receive text data and perform in-depth analysis.

[0035] During the analysis phase, the large language model will deeply understand the semantic content of the complaint text, identify sentiment, extract topics and key words, etc. The purpose of this step is to accurately grasp the core issues of the complaint and the customer's demands.

[0036] After the analysis is completed, the large language model will output key information about the complaint, such as the subject of the complaint, the product or service involved, the customer's emotional state, etc. This information will serve as the basis for the subsequent label classification process.

[0037] Next, using the key information extracted from the large language model, the complaints are classified according to a preset label framework. The label framework is a set of pre-defined classification labels used to classify complaints into different categories, such as "service delay", "product quality issue", etc. This process may involve using machine learning algorithms, such as classification algorithms, to assign the complaint text to the most appropriate label based on the extracted key information. A probabilistic model (such as the softmax function) is used to determine the probability that the complaint text belongs to each label and select the most likely label.

[0038] Ultimately, this process generates one or more tags, representing the classification results of the complaint text. By combining the natural language processing capabilities of the large language model and the preset tag framework, the automatic classification of customer complaints is achieved. This method not only improves processing efficiency, but also improves processing accuracy, providing strong support for enterprises to more effectively handle and respond to customer complaints.

[0039] S40. Determine whether the label result is accurately classified based on the historical complaint labels. If the label result is accurately classified, the label classification is completed.

[0040] In this embodiment, generative models have a wide range of applications in the field of natural language processing, such as text generation, translation, and summarization. These models generate new data instances by learning the distribution of a large amount of data. However, they may also produce so-called "hallucination facts", that is, the information generated or inferred by the model does not exist in the original data, but the model mistakenly believes that they are true. This phenomenon is particularly common in generative models because they need to "create" content without explicit instructions.

[0041] Large language models are very powerful when handling complex language tasks, but they can also produce hallucinations. For example, the model may mistakenly associate certain words or phrases with specific complaint categories, even if these associations do not exist in reality. In order to reduce the impact of hallucinations on the results, it is recommended to use the model again to verify the conclusions after the large language model makes a preliminary judgment. This step can help identify and correct possible errors or omissions. By rechecking, the model can re-evaluate the accuracy of its initial classification and consider whether there is a more reasonable explanation or classification.

[0042] In practice, this may involve designing a feedback mechanism where the model’s output is used as input to be analyzed again. For example, the model may first classify the complaint text and then analyze the classification results again to see if there are better matching labels or if any information was overlooked. To further enhance this process, historical complaint data and known labels can be combined to verify and adjust the model’s output. This approach can use existing accurate data to guide and improve the model’s predictions.

[0043] Through this double-check mechanism, the large language model is not only able to use its powerful generative capabilities to handle complex language tasks, but also improve the accuracy and reliability of its output through self-verification and correction. This approach is particularly important in areas that require a high degree of precision, such as handling customer complaints, and helps improve the efficiency and quality of customer service, while also providing companies with valuable data for analyzing and improving their products and services. In this way, companies can ensure that their complaint handling systems are not only efficient, but also accurate and reliable, so as to better serve customers and maintain their brand image.

[0044] Using universal large language models can effectively simplify the steps of data set construction and model training in traditional machine learning. These models have been pre-trained on a large amount of data and can quickly adapt to new tasks, such as customer complaint analysis. The text context understanding ability of large language models enables them to more accurately analyze complex contexts and scenarios, thereby improving the accuracy of understanding and classification of customer complaint content.

[0045] In the solution, the integrated sentiment analysis function enables the model to identify and classify emotional tendencies in customer complaints, such as anger, disappointment, or satisfaction. This kind of sentiment recognition is crucial for a deep understanding of customer experience and needs, and helps provide more targeted responses. By analyzing historical complaint data, the model is able to predict potential customer complaint trends, allowing companies to take measures in advance to prevent possible problems and thus improve service quality.

[0046] For niche content that contains specific product names, professional terms, or specific contexts, the large language model can quickly understand and identify these contents by optimizing the prompt words. This method avoids the problem of underfitting in training caused by insufficient samples in traditional deep learning models.

[0047] Through the unified semantic understanding of artificial intelligence, the simple repetitive work of front-line staff in handling complaints is reduced, and the consistent understanding of the same case by different staff is ensured, which improves the consistency and efficiency of handling. In addition, adding the large model's own checkpoints helps reduce the possible generation of hallucination facts and ensures the accuracy of label recognition. The solution can quickly identify the key points of user complaints and summarize user opinions, help companies analyze the main pain points of customers in the current business, and quickly optimize the solution to reduce customer complaints, improve customer satisfaction and maintain the company's reputation.

[0048] Overall, this solution not only improves the efficiency and accuracy of customer complaint handling by leveraging the power of large language models, but also enhances the understanding and response capabilities of customer experience through functions such as sentiment analysis and trend prediction. These innovations work together to improve customer service quality, reduce complaints, and enhance customer satisfaction and loyalty.

[0049] In one embodiment, obtaining the user's complaint data, converting the complaint data into complaint text, and inputting the complaint text into a large language model includes:

[0050] Acquire user complaint data, pre-process the complaint data, and convert the complaint data into complaint text;

[0051] The complaint text is converted into a complaint text vector, and the complaint text vector is input into a large language model.

[0052] In this embodiment, in order to realize automated and intelligent complaint processing, it is first necessary to collect user complaint data from multiple channels (telephone, email, social media, etc.). Next, preprocessing is performed, which is an important step in data preparation, the purpose of which is to convert the original complaint data into a format suitable for analysis. Preprocessing may include operations such as data cleaning (removing irrelevant information, filling missing values), text standardization (unifying vocabulary usage, correcting spelling errors), and word segmentation (breaking text into separate words or phrases).

[0053] After preprocessing, the data is converted into a unified text format to ensure that all complaint information exists in text form to facilitate subsequent text analysis. For example, if the original complaint is a voice recording, it needs to be transcribed into text first. In order to analyze text using a large language model, the text needs to be converted into a vector in numerical form. This step is called text vectorization. Commonly used vectorization methods include Bag of Words, TF-IDF (Term Frequency-Inverse Document Frequency), Word Embeddings (such as Word2Vec, GloVe), etc. These methods can convert text into numerical vectors that can reflect its semantic content.

[0054] The vectorized text vector is input into the pre-trained large language model, which can understand the semantic content of the text and perform further analysis and processing based on it. The large language model is usually pre-trained on large-scale text data and has rich language knowledge and strong text understanding capabilities.

[0055] Through these steps, user complaint data can be converted into a form that can be processed by a large language model, laying the foundation for subsequent text analysis and label classification. This process is a key link in realizing automated and intelligent complaint processing, which helps improve processing efficiency, ensures a quick response to user complaints, and thus improves customer satisfaction.

[0056] In one embodiment, the label classification of the key complaint information according to the preset label framework to obtain the label result includes:

[0057] The softmax function is used to calculate the relevance probability of each label based on the key information of the complaint, and the probability result is output;

[0058] According to the probability result, a classification label of the complaint text is selected to obtain a label result.

[0059] In this example, after the large language model analyzes the complaint text and extracts key information, the next step is to determine the relevance of the text to each label in the predefined label framework. To do this, the model first calculates a raw score for each possible label. Then, the Softmax function is applied to these raw scores to convert them into probabilistic form. The formula of the Softmax function is defined as follows:

[0060]

[0061] In the context of the Softmax function, and represents the result of applying the exponential function to the original score. i and z j It is the raw score generated by the model for each possible label after processing the input data (in this case, the complaint text). Here is a detailed explanation:

[0062] Specifically, i is the raw score of the i-th label. In multi-classification problems, this score is usually given by the output layer of the model, indicating the strength of the correlation between the input data and the i-th label. In the case of complaint text classification, this may reflect the relevance of the text to a specific complaint category.

[0063] Yes i The exponent of e (the base of natural logarithms, approximately equal to 2.71828) i The purpose of calculating this exponent is to convert the original score into a non-negative number, the size of which can be used to represent the relative likelihood of the label. The exponential function can amplify the difference between scores, making high-scoring labels more prominent.

[0064] is the original score z of the jth label j In the Softmax function, all possible labels will have a corresponding exponential value.

[0065] The formula of the Softmax function ensures that the index values ​​of all labels are normalized, that is, the sum of the probabilities of all labels is 1. In this way, the probability of each label can be interpreted as the relative probability of selecting the label among all possible labels. Specifically: is the index score of the current tag i. The denominator ∑ j is the sum of all label index scores, ensuring that the result is a valid probability distribution.

[0066] In this way, the Softmax function converts the model's raw output scores into a probability form, so that each label has a probability value between 0 and 1, and the sum of these probability values ​​is 1. This allows the model to output a probability distribution indicating the probability that the input data belongs to each possible label, thus providing a basis for the final decision. In complaint text classification, this can help determine the complaint category to which the text is most likely to belong.

[0067] After applying the Softmax function, the model generates a probability value for each label, which represents the probability that the complaint text belongs to each label. These probability values ​​are the key output in the decision-making process because they provide a quantitative basis for the classification of the text. Subsequently, the label with the highest probability is selected as the classification result of the complaint text. This label is considered the best match because it has the highest relative likelihood, indicating that the model predicts that the complaint text is most relevant to this label.

[0068] The process of selecting the highest probability label is based on the assumption that among all possible classifications, the classification with the highest probability provides the most likely explanation or classification. Ultimately, this highest probability label is determined as the classification result of the complaint text and is used for subsequent processing, such as customer service response, complaint analysis, trend prediction, etc.

[0069] In this way, the Softmax function not only helps the model make choices among multiple possible classifications, but also provides a quantitative way to evaluate the relevance of each classification. This probability-based decision-making process increases the transparency and interpretability of the classification, helping to improve the accuracy and efficiency of the entire complaint handling system.

[0070] In one embodiment, judging whether the label result is accurately classified according to the historical complaint label includes:

[0071] Acquire historical complaint data, and convert the historical complaint data into a historical complaint vector;

[0072] Calculate the similarity between the historical complaint vector and the complaint text vector to obtain a similarity value;

[0073] Whether the label result is classified accurately is determined according to the similarity value.

[0074] In this embodiment, the historical complaint data includes customer complaint records that have been processed and classified in the past. These data are usually stored in the enterprise database and cover the complaint text content and its corresponding classification labels. In order to compare with the current complaint text, the historical complaint text also needs to be vectorized, that is, the text is converted into a numerical vector. This step can use the same method as the current complaint text, such as TF-IDF, Word Embeddings, etc., to ensure consistency.

[0075] By calculating the similarity between the historical complaint vector and the current complaint text vector, we can evaluate how close they are. The similarity calculation can be done using different metrics, such as cosine similarity or Euclidean distance. The similarity value provides a quantitative indicator for measuring the similarity between the current complaint text and the historical complaint data.

[0076] Based on this similarity value, we can evaluate whether the current classification result is accurate. If the similarity value is high, it means that the current complaint text is very similar to a historical complaint case, so it can be considered that the current classification result is consistent with the classification label of the historical case, thereby verifying the accuracy of the classification. On the contrary, if the similarity value is low, it may mean that the current classification result does not match the historical data. At this time, we may need to reconsider the classification result, or further analyze the complaint text to determine a more accurate classification.

[0077] In this way, historical complaint data not only provides valuable contextual information for current complaints, but also helps improve the accuracy and reliability of classification results. This validation process based on historical data is an important part of machine learning model evaluation and optimization, which helps improve the predictive performance of the entire system. Through continuous optimization and adjustment, it can ensure that the complaint handling system continues to improve over time and better serve customers and enterprises.

[0078] In one embodiment, judging whether the label result is accurately classified according to the similarity value includes:

[0079] Selecting historical complaint cases according to the similarity value and obtaining label results of the historical complaint cases;

[0080] Determine whether the label result is the same as the label result of the historical complaint case;

[0081] When the label result is the same as the label result of the historical complaint case, the label result classification is accurate;

[0082] When the label result is different from the label result of the historical complaint case, the label result is misclassified.

[0083] In this embodiment, by calculating the similarity between the current complaint text vector and the historical complaint text vector, we can find the case most similar to the current complaint in the historical complaint database. The purpose of this step is to find the past case most relevant to the current complaint so as to refer to its label. After finding the most similar historical complaint case, we compare its label with the predicted label of the current complaint text, which is the core of the verification process.

[0084] If the predicted labels of the current complaint text match the labels of historical complaint cases, this generally means that the classification is accurate. This consistency indicates that the model is able to correctly understand and classify the complaint content and is consistent with the labeling system of historical data. Conversely, if the predicted labels are inconsistent with the historical complaint labels, this may indicate that the classification is wrong. This mismatch can be caused by a variety of factors, such as the model's inaccurate understanding of the complaint text, or the complaint text itself is ambiguous.

[0085] When it is found that the labeling results are inconsistent with the historical complaint labels, further measures may be required. This may include re-examining the complaint text, adjusting the model's parameters, or adding more similar cases to the model training to improve its future classification accuracy. Through this historical data-based verification method, the classification performance of the model can be effectively evaluated and improved.

[0086] This approach not only helps improve the accuracy of individual complaint handling, but also improves the quality and efficiency of the entire complaint handling system through continuous feedback and optimization. In addition, this approach helps ensure that the model's decision-making process is explainable and consistent, enhancing users' trust in the system. In this way, companies can ensure that their complaint handling systems are not only efficient, but also accurate and reliable, thereby better serving customers and maintaining their brand image.

[0087] In one embodiment, when the label result is the same as the label result of the historical complaint case, after the label result classification is accurate, the method further includes:

[0088] Determine the level of the label framework according to the label result;

[0089] When the label result belongs to the first-level label frame or the second-level label frame, label classification of the next-level label frame is performed;

[0090] When the label result belongs to the third-level label framework, the label classification is completed.

[0091] In this embodiment, in the multi-level labeling framework, the classification of complaint text may involve multiple levels, each level representing a different level of abstraction of the complaint content. For example, the first-level label is usually a broad classification, such as "service problem" or "product problem". As the level goes deeper, the label becomes more specific, such as "slow response time" or "poor service attitude" under "service problem".

[0092] After obtaining the labeling results of the complaint text, it is necessary to determine which level of the labeling framework this result belongs to in order to determine whether further classification processing is required. If the labeling results belong to the first-level or second-level labeling framework, this indicates that the classification of the complaint text has not yet reached the most specific level and needs to be further subdivided. For example, if the complaint text is initially classified under the first-level label of "service problem", it may then need to be subdivided into more specific second-level labels such as "slow response time" or "poor service attitude".

[0093] When the label result belongs to the third or deeper label framework, it usually means that the most specific level in the preset label framework has been reached. In this case, it can be considered that the label classification process has been completed because the complaint text has been classified as detailed as possible.

[0094] After label classification, this information can be used for a variety of purposes, including customer service improvement, complaint trend analysis, resource optimization, etc. The hierarchical label classification method ensures that the complaint text is systematically classified and each complaint can be accurately identified and handled. This method not only improves the efficiency and quality of customer service, but also provides companies with valuable data for analyzing and improving their products and services. Through this detailed classification, companies can have a deeper understanding of customer needs and problems, so as to take more effective measures to improve customer satisfaction and business performance.

[0095] In one embodiment, the labeling of the key complaint information according to a preset labeling framework to obtain labeling results further includes:

[0096] Classify the key complaint information according to the first-level labels to obtain first-level label results;

[0097] Classify the key complaint information according to the secondary label and the primary label results to obtain the secondary label results;

[0098] The key complaint information is labeled and classified according to the third-level label and the second-level label results to obtain the third-level label results.

[0099] In this embodiment, in the process of handling customer complaints, it is an effective method to classify the complaint text using a multi-level labeling system. This process usually starts with the preliminary classification of the complaint text using the first-level label. The first-level label provides the broadest classification, covering the main aspects of the complaint content, such as "service problems", "product defects", "billing disputes", etc. At this stage, the large language model analyzes the key information of the complaint text and determines the most appropriate first-level label based on the first-level label framework, providing a general classification for the complaint and laying the foundation for subsequent more detailed classification.

[0100] After obtaining the first-level label results, the second-level labels are used to further classify the complaint text. Second-level labels are more specific than first-level labels. For example, under "service issues", there may be second-level labels such as "slow response time" and "poor attitude of service staff". At this stage, the large language model combines the first-level label results to conduct a more detailed analysis of the complaint text and determine the most appropriate second-level labels, which helps to further refine the classification of complaints and make them more specific and accurate.

[0101] Finally, the complaint text is finally classified using the third-level labels. The third-level labels are usually the most specific classifications, which can describe the specific content of the complaint in detail. For example, under "product defect", there may be third-level labels such as "hardware failure" and "software failure". At this stage, the large language model combines the second-level label results to conduct an in-depth analysis of the complaint text and determine the most appropriate third-level labels to ensure that the complaint is classified as detailed as possible so that appropriate measures can be taken for specific complaint issues.

[0102] This hierarchical label classification approach ensures that complaint texts are systematically classified and that each complaint can be accurately identified and handled. This approach not only improves the efficiency and quality of customer service, but also provides companies with valuable data for analyzing and improving their products and services. In addition, this approach helps ensure that the model's decision-making process is explainable and consistent, enhancing users' trust in the system. In this way, companies can respond to and resolve customer complaints more effectively, thereby improving customer satisfaction and loyalty.

[0103] It can be seen that the above scheme is suitable for scenarios that require a large amount of data collection and analysis, especially in financial fields such as digital marketing, user behavior analysis, and product optimization. First, the system collects user complaint data, which may come from different channels, such as customer service hotlines, social media, emails, etc. The collected complaint data is converted into text format for subsequent analysis. This step ensures that all types of complaint data can be processed uniformly. The converted complaint text is input into the pre-trained large language model. This model can understand and analyze natural language and is a key tool for processing complaint text. The large language model conducts in-depth analysis of the complaint text and extracts key information. This key information may include the main problem of the complaint, the products or services involved, the user's emotions, etc. The extracted complaint key information is classified according to the preset label framework. This label framework may include multiple predefined categories, such as product quality, service delays, price issues, etc. Using historical complaint data and known labels, the system will determine whether the current label classification result is accurate. This step is an important part of ensuring the quality of classification. If the historical complaint label verification shows that the classification result is accurate, then the label classification process for the complaint is completed. This enables companies to quickly respond to user complaints and take appropriate measures. The present invention not only improves the automation level of complaint handling, but also improves the accuracy of label classification through historical data verification. This helps companies better understand customer complaints, optimize customer service processes, and improve customer satisfaction.

[0104] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0105] In one embodiment, a complaint label classification device is provided, which corresponds one-to-one to the complaint label classification method in the above embodiment. Figure 3 As shown, the complaint label classification device includes a data acquisition module 101, a data analysis module 102, a label classification module 103 and a label judgment module 104. The detailed description of each functional module is as follows:

[0106] The data acquisition module 101 is used to acquire the user's complaint data, convert the complaint data into complaint text, and input the complaint text into the large language model;

[0107] Data analysis module 102, used for the large language model to analyze and process the complaint text and output key complaint information;

[0108] The label classification module 103 is used to classify the key complaint information according to a preset label framework to obtain a label result;

[0109] The label judgment module 104 is used to judge whether the label result is accurate based on the historical complaint labels. When the label result is accurate, the label classification is completed.

[0110] In one embodiment, the data acquisition module 101 includes:

[0111] A complaint text unit, used to obtain complaint data of the user, pre-process the complaint data, and convert the complaint data into complaint text;

[0112] The text vector unit is used to convert the complaint text into a complaint text vector and input the complaint text vector into a large language model.

[0113] In one embodiment, the tag classification module 103 includes:

[0114] The probability result unit is used to calculate the relevance probability of each label according to the key information of the complaint by using the softmax function and output the probability result;

[0115] The label result unit is used to select the classification label of the complaint text according to the probability result to obtain the label result.

[0116] In one embodiment, the tag determination module 104 includes:

[0117] A historical complaint unit, used for acquiring historical complaint data and converting the historical complaint data into a historical complaint vector;

[0118] A similarity calculation unit, used to calculate the similarity between the historical complaint vector and the complaint text vector to obtain a similarity value;

[0119] A similarity judgment unit is used to judge whether the label result is classified accurately according to the similarity value.

[0120] In one embodiment, the tag determination module 104 includes:

[0121] A similarity selection unit, used to select historical complaint cases according to the similarity value and obtain label results of the historical complaint cases;

[0122] A label comparison unit, used to determine whether the label result is the same as the label result of the historical complaint case;

[0123] A classification accuracy unit, used for determining that when the label result is the same as the label result of the historical complaint case, the label result classification is accurate;

[0124] The classification error unit is used to classify the label result as incorrect when the label result is different from the label result of the historical complaint case.

[0125] In one embodiment, the tag determination module 104 further includes:

[0126] A level judgment unit, used to judge the level of the label framework according to the label result;

[0127] The one-two frame unit is used to classify the labels of the next-level label frame when the label result belongs to the first-level label frame or the second-level label frame;

[0128] The three-level framework unit is used when the label result belongs to the three-level label framework, and the label classification is completed.

[0129] In one embodiment, the tag classification module 103 further includes:

[0130] A primary label unit, used to classify the key complaint information according to the primary label to obtain a primary label result;

[0131] A secondary label unit, used to classify the key complaint information according to the secondary label and the primary label result to obtain a secondary label result;

[0132] The third-level label unit is used to classify the key complaint information according to the third-level label and the second-level label results to obtain the third-level label results.

[0133] For the specific definition of the complaint label classification device, please refer to the definition of the intelligent question and answer method above, which will not be repeated here. Each module in the above complaint label classification device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0134] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a complaint label classification method.

[0135] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a client side of a complaint label classification method

[0136] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0137] Acquire complaint data of the user, convert the complaint data into complaint text, and input the complaint text into the large language model;

[0138] The large language model analyzes and processes the complaint text and outputs key complaint information;

[0139] Classify the key complaint information according to a preset label framework to obtain label results;

[0140] Whether the label result is accurately classified is determined based on the historical complaint labels. If the label result is accurately classified, the label classification is completed.

[0141] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0142] Acquire complaint data of the user, convert the complaint data into complaint text, and input the complaint text into the large language model;

[0143] The large language model analyzes and processes the complaint text and outputs key complaint information;

[0144] Classify the key complaint information according to a preset label framework to obtain label results;

[0145] Whether the label result is accurately classified is determined based on the historical complaint labels. If the label result is accurately classified, the label classification is completed.

[0146] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0148] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0150] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A complaint label classification method, characterized in that: include: Acquire complaint data of the user, convert the complaint data into complaint text, and input the complaint text into the large language model; The large language model analyzes and processes the complaint text and outputs key complaint information; Classify the key complaint information according to a preset label framework to obtain label results; Whether the label result is accurately classified is determined based on the historical complaint labels. If the label result is accurately classified, the label classification is completed.

2. The complaint label classification method according to claim 1, characterized in that: The obtaining of user complaint data, converting the complaint data into complaint text, and inputting the complaint text into a large language model includes: Acquire user complaint data, pre-process the complaint data, and convert the complaint data into complaint text; The complaint text is converted into a complaint text vector, and the complaint text vector is input into a large language model.

3. The complaint label classification method according to claim 1, characterized in that: The label classification of the key complaint information according to the preset label framework to obtain the label result includes: The softmax function is used to calculate the relevance probability of each label based on the key information of the complaint, and the probability result is output; According to the probability result, a classification label of the complaint text is selected to obtain a label result.

4. The complaint label classification method according to claim 2, characterized in that: The step of judging whether the label result is accurately classified according to the historical complaint label includes: Acquire historical complaint data, and convert the historical complaint data into a historical complaint vector; Calculate the similarity between the historical complaint vector and the complaint text vector to obtain a similarity value; Whether the label result is classified accurately is determined according to the similarity value.

5. The complaint label classification method according to claim 4, characterized in that: The determining whether the label result is accurately classified according to the similarity value includes: Selecting historical complaint cases according to the similarity value and obtaining label results of the historical complaint cases; Determine whether the label result is the same as the label result of the historical complaint case; When the label result is the same as the label result of the historical complaint case, the label result classification is accurate; When the label result is different from the label result of the historical complaint case, the label result is misclassified.

6. The complaint label classification method according to claim 5, characterized in that: When the label result is the same as the label result of the historical complaint case, the label result classification is accurate, and further includes: Determine the level of the label framework according to the label result; When the label result belongs to the first-level label frame or the second-level label frame, label classification of the next-level label frame is performed; When the label result belongs to the third-level label framework, the label classification is completed.

7. The complaint label classification method according to claim 1, characterized in that: The labeling of the key complaint information according to the preset labeling framework to obtain labeling results also includes: Classify the key complaint information according to the first-level labels to obtain first-level label results; Classify the key complaint information according to the secondary label and the primary label results to obtain the secondary label results; The key complaint information is labeled and classified according to the third-level label and the second-level label results to obtain the third-level label results.

8. A complaint label classification device, characterized in that: include: A data acquisition module, used to acquire user complaint data, convert the complaint data into complaint text, and input the complaint text into a large language model; A data analysis module, used for the large language model to analyze and process the complaint text and output key complaint information; A label classification module is used to classify the key complaint information according to a preset label framework to obtain label results; The label judgment module is used to judge whether the label result is accurate based on the historical complaint labels. When the label result is accurate, the label classification is completed.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the complaint label classification method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the complaint label classification method as described in any one of claims 1 to 7 are implemented.

Images