Multi-level complaint classification method, device and equipment based on large model and storage medium

Abstract

The invention discloses a multi-level complaint classification method, device and equipment based on a large model and a storage medium, and relates to the technical field of data processing.The method comprises the steps that a multi-level classification system table and complaint texts subjected to data desensitization and normalization processing are obtained; semantic feature extraction is conducted on the complaint text according to a multi-potential head attention big model and the multi-level classification system table, an initial classification probability vector is obtained, and the multi-potential head attention big model is a pre-training language model adopting a multi-potential head attention structure to replace a grouping query attention structure; performing hierarchical constraint classification according to the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths containing confidence; and performing backtracking and re-judgment according to the candidate classification path and a preset confidence threshold to obtain a target complaint classification path.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a multi-level complaint classification method, apparatus, device and storage medium based on a large model. Background Technology

[0002] In service industries such as finance, telecommunications, and the internet, efficiently and accurately processing massive amounts of user complaint texts is key to improving operational efficiency and customer satisfaction. Currently, the industry widely adopts automated or semi-automated classification technologies based on natural language processing and machine learning.

[0003] However, existing solutions suffer from a series of significant drawbacks in practical applications. First, classification accuracy is limited, especially when faced with colloquial, abbreviated, and context-dependent complaint texts. The models lack sufficient semantic understanding depth, making it difficult to perform effective fine-grained differentiation, resulting in low accuracy. Second, the coarse classification system cannot cover complex business scenarios. Many complaints with subtle differences are categorized into broad categories, weakening the value of the classification results in supporting precise policy implementation and root cause analysis. Third, existing models are mostly one-time flat classifications, ignoring the inherent hierarchical relationships between categories. The decision-making process is isolated, lacking mechanisms for optimization and error correction using structural information. Simultaneously, the models struggle to adapt to dynamic changes in business rules and complaint hotspots, lacking effective online learning and adaptive capabilities, resulting in high maintenance costs and slow response times. Furthermore, during peak user consultation periods, the models exhibit poor robustness and performance degradation when facing non-standard inputs and changes in data distribution. Finally, when the number of classification categories expands rapidly to thousands or even tens of thousands, traditional model architectures and serial processes face severe computational efficiency and scalability bottlenecks, making it difficult to meet the demands of high concurrency and low latency real-time processing while maintaining accuracy.

[0004] Therefore, how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios is an urgent problem to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a multi-level complaint classification method, apparatus, device, and storage medium based on a large model, aiming to solve the technical problem of how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios.

[0006] To achieve the above objectives, this application proposes a multi-level complaint classification method based on a large model, the method comprising: Obtain the multi-level classification system table and the complaint text after data anonymization and normalization; Semantic features are extracted from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure. Based on the initial classification probability vector and the multi-level classification system table, hierarchical constraint classification is performed to obtain candidate classification paths including confidence scores; The target complaint classification path is obtained by backtracking and re-judging based on the candidate classification path and the preset reliability threshold.

[0007] In one embodiment, the step of extracting semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector includes: The complaint text is input into a multi-potential head attention model to obtain the hidden state sequence of the last layer. The semantic feature vector is obtained by performing mean pooling on the hidden state sequence; The semantic feature vector is mapped to the multi-level category space corresponding to the multi-level classification system table through a fully connected layer to obtain the initial classification probability vector.

[0008] In one embodiment, the step of performing hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores includes: Based on the initial classification probability vector and the preset confidence threshold, the confidence levels corresponding to the first-level classifications in the multi-level classification system table are selected, and the first-level classification with the highest confidence level is taken as the first-level classification node. Starting from the first-level classification node, based on the initial classification probability vector and the preset confidence threshold, the corresponding sub-classification nodes and the confidence levels corresponding to the sub-classification nodes in the multi-level classification system table are traversed. The candidate classification path is obtained by connecting the first-level classification node, the sub-classification node and the corresponding confidence score in hierarchical order.

[0009] In one embodiment, the step of performing backtracking and re-judging based on the candidate classification path and a preset confidence threshold to obtain the target complaint classification path includes: Traverse the candidate classification path. When the confidence level corresponding to the current level is lower than the preset backtracking threshold, backtrack the classification level to the level above the current level. Then, starting from the classification node of the previous level, re-determine the sub-classification node of the next level based on the initial classification probability vector. The target complaint classification path is obtained by updating the corresponding level classification nodes and their confidence scores in the candidate classification path based on the redefined sub-classification nodes.

[0010] In one embodiment, before the step of extracting semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector, the method further includes: Obtain a pre-trained language model, and replace the grouped query attention structure in the pre-trained language model with a multi-potential head attention structure to obtain the target training model; Determine the corresponding complaint category label based on the multi-level classification system table; Align the intermediate layer features of the target training model with those of the pre-trained language model, and then perform end-to-end supervised training on the output of the target training model based on the complaint classification label after alignment to obtain a multi-potential head attention large model.

[0011] In one embodiment, after the steps of aligning the intermediate layer features of the target training model with those of the pre-trained language model, and performing end-to-end supervised training on the output of the target training model based on the complaint classification label after alignment to obtain a multi-potential head attention large model, the method further includes: Obtain the number of complaints and the average confidence level for each complaint category label within the current time period; When the increase in the number of complaints within a preset unit time period is greater than or equal to a preset hotspot warning threshold, the complaint category tags that are greater than or equal to the preset hotspot warning threshold are marked as hotspot events. When the average confidence level decreases by more than or equal to a preset drift warning threshold, the complaint classification labels that are greater than or equal to the preset drift warning threshold are marked with a distribution drift. Based on the hotspot event markers and / or the distribution drift markers, samples of the corresponding categories are extracted from a preset incremental learning sample pool to fine-tune the parameters of the multi-potential head attention model, thereby updating the multi-potential head attention model.

[0012] In one embodiment, after the step of performing backtracking and re-judging based on the candidate classification path and a preset confidence threshold to obtain the target complaint classification path, the method further includes: When the final confidence level of the target complaint classification path is lower than the preset manual review threshold, the complaint text and the target complaint classification path are pushed to the manual review queue. Receive the correction category label returned by the manual review queue, and replace the corresponding last-level node in the target complaint classification path according to the correction category label to form a manual correction path; The target complaint classification path is updated based on the manually corrected path.

[0013] Furthermore, to achieve the above objectives, this application also proposes a multi-level complaint classification device based on a large model, the device comprising: The data acquisition module is used to obtain a multi-level classification system table and complaint texts that have undergone data anonymization and normalization. The feature extraction module is used to extract semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure. The classification decision module is used to perform hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores. The complaint classification module is used to perform backtracking and re-judgment based on the candidate classification paths and preset confidence thresholds to obtain the target complaint classification path.

[0014] Furthermore, to achieve the above objectives, this application also proposes a multi-level complaint classification device based on a large model, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-level complaint classification method based on a large model as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the multi-level complaint classification method based on a large model as described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multi-level complaint classification method based on a large model as described above.

[0017] This application provides a multi-level complaint classification method based on a large model. The method includes: obtaining a multi-level classification system table and complaint text after data anonymization and normalization; extracting semantic features from the complaint text based on a multi-potential head attention large model and the multi-level classification system table to obtain an initial classification probability vector, wherein the multi-potential head attention large model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure; performing hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence levels; and performing backtracking and re-judgment based on the candidate classification paths and a pre-set confidence threshold to obtain the target complaint classification path. In summary, this application, by constructing a large model integrating an attention optimization mechanism and multi-level optimization decision-making, achieves accurate and rapid classification of massive non-standardized complaint texts, solving the adaptability and efficiency bottlenecks of traditional methods in ultra-large-scale classification scenarios, and improving the system's classification accuracy and efficiency. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the first embodiment of the multi-level complaint classification method based on a large model in this application. Figure 2 This is a flowchart illustrating the multi-level classification process in one embodiment of the multi-level complaint classification method based on a large model in this application. Figure 3 This is a flowchart illustrating the second embodiment of the multi-level complaint classification method based on a large model in this application. Figure 4 This is a flowchart illustrating the third embodiment of the multi-level complaint classification method based on a large model in this application. Figure 5 This is a schematic diagram of the module structure of the multi-level complaint classification device based on a large model, as described in an embodiment of this application. Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the multi-level complaint classification method based on a large model in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is as follows: Obtain a multi-level classification system table and complaint text after data anonymization and normalization; extract semantic features from the complaint text based on a multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure; perform hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence levels; and perform backtracking and re-judgment based on the candidate classification paths and a preset confidence threshold to obtain the target complaint classification path.

[0025] In service industries such as finance, telecommunications, and the internet, efficiently and accurately processing massive amounts of user complaint texts is key to improving operational efficiency and customer satisfaction. Currently, the industry widely adopts automated or semi-automated classification technologies based on natural language processing and machine learning.

[0026] However, existing solutions suffer from a series of significant drawbacks in practical applications. First, classification accuracy is limited, especially when faced with colloquial, abbreviated, and context-dependent complaint texts. The models lack sufficient semantic understanding depth, making it difficult to perform effective fine-grained differentiation, resulting in low accuracy. Second, the coarse classification system cannot cover complex business scenarios. Many complaints with subtle differences are categorized into broad categories, weakening the value of the classification results in supporting precise policy implementation and root cause analysis. Third, existing models are mostly one-time flat classifications, ignoring the inherent hierarchical relationships between categories. The decision-making process is isolated, lacking mechanisms for optimization and error correction using structural information. Simultaneously, the models struggle to adapt to dynamic changes in business rules and complaint hotspots, lacking effective online learning and adaptive capabilities, resulting in high maintenance costs and slow response times. Furthermore, during peak user consultation periods, the models exhibit poor robustness and performance degradation when facing non-standard inputs and changes in data distribution. Finally, when the number of classification categories expands rapidly to thousands or even tens of thousands, traditional model architectures and serial processes face severe computational efficiency and scalability bottlenecks, making it difficult to meet the demands of high concurrency and low latency real-time processing while maintaining accuracy. Therefore, how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios is an urgent problem to be solved.

[0027] It should be noted that the executing entity in this embodiment can be a multi-level complaint classification system based on a large model, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of implementing the aforementioned multi-level complaint classification function based on a large model. This embodiment does not specifically limit it in this way. The following uses a multi-level complaint classification system based on a large model as an example to describe this embodiment and the following embodiments.

[0028] Based on this, embodiments of this application provide a multi-level complaint classification method based on a large model, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-level complaint classification method based on a large model in this application.

[0029] In this embodiment, the multi-level complaint classification method based on a large model includes steps S10 to S40: Step S10: Obtain the multi-level classification system table and the complaint text after data anonymization and normalization.

[0030] It should be noted that in this step, the system loads a predefined multi-level classification system table. This table defines the hierarchical relationship of complaint categories in a tree structure, including nodes for first-level, second-level, third-level, and fourth-level categories, encompassing thousands of fine-grained categories. Simultaneously, the system receives the original complaint text from customer service channels and performs automated data anonymization and normalization processing on it.

[0031] Additionally, it should be noted that a multi-level classification system table refers to a pre-constructed, hierarchical knowledge structure of complaint categories. For example, a four-level classification system has a root node as a first-level category (such as marketing activities), each first-level category containing multiple second-level subcategories (such as marketing activity implementation), and so on down to the finest-grained leaf nodes (fourth-level categories). The data anonymization and normalization processing refers to a series of preprocessing operations on the original text, specifically including: data anonymization: automatically identifying and hiding sensitive personal information in the text, such as mobile phone numbers and ID card numbers, based on regular expressions and preset rules; text cleaning and normalization: including typo correction based on confusion matrices and context verification, removal of meaningless characters and symbols, and conversion of colloquial, abbreviated, and other non-standard expressions into standard written language. It can be understood that the purpose of this step is to provide a structured knowledge framework and high-quality input data for subsequent classification, reducing the interference of non-standard expressions on the model's understanding.

[0032] Step S20: Extract semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure.

[0033] It should be noted that in this step, the system inputs the preprocessed complaint text into a multi-potential head attention model. This model is based on a language model pre-trained on a large-scale corpus (such as a model with 7 billion parameters) and its attention mechanism has been specifically optimized. The model performs deep semantic encoding and understanding of the input text, and outputs an initial classification probability vector based on its understanding of all leaf nodes (i.e., the final subcategories) in the multi-level classification system table. In essence, this step leverages the powerful semantic understanding capabilities of the large model to perform a preliminary, comprehensive assessment of the classification probability of the complaint text.

[0034] Additionally, it should be noted that the initial classification probability vector is a numerical vector, the dimension of which is equal to the total number of leaf nodes (final classification categories) in the multi-level classification system (e.g., 5000+). Each element in the vector represents the initial probability or confidence score of the input text being classified into the corresponding leaf node. The Multi-Latent Head Attention (MLA) large model refers to a model obtained by structurally modifying a base pre-trained language model. Its core feature is replacing the original Group Query Attention (GQA) mechanism with a Multi-Latent Head Attention (MLA) mechanism. This structural optimization aims to reduce the KV-Cache memory usage during inference and adapt to inference acceleration technologies such as FlashMLA, thereby improving the model's efficiency in handling long sequences and high-concurrency requests.

[0035] Step S30: Perform hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores.

[0036] It's important to note that in this step, the system doesn't directly select the leaf node with the highest probability from the initial classification probability vector as the final result. Instead, it introduces hierarchical decision-making logic. The system utilizes the tree structure of the classification system to constrain and propagate the initial probabilities, simulating a coarse-to-fine decision-making process. This generates a candidate classification path from the root node to a specific leaf node, with each node on the path assigned a classification confidence score. Essentially, this step reduces the search space for each decision, leverages hierarchical prior knowledge to decrease misclassifications between sibling categories, and improves the logical rationality and accuracy of the classification.

[0037] Additionally, it should be noted that hierarchical constraint classification is a decision algorithm that follows a hierarchical structure of the classification system. It does not perform a one-time flat classification, but rather decomposes the classification task into multiple sequential sub-decisions, each of which is performed only within a finite set of subcategories determined by the higher-level nodes.

[0038] In one feasible implementation, step S30 specifically includes: Step S301: Based on the initial classification probability vector and the preset confidence threshold, filter the confidence levels corresponding to the first-level classifications in the multi-level classification system table, and take the first-level classification with the highest confidence level as the first-level classification node.

[0039] It's important to note that in this step, the system aggregates and calculates all confidence scores for each primary category based on the initial classification probability vector. Then, it checks whether the highest confidence score exceeds a predetermined high threshold α (e.g., 0.8). If it does, the primary category with the highest confidence score is designated as the primary classification node. Furthermore, it's worth noting that the preset confidence threshold is a threshold value used to judge the reliability of the model's decisions, and different thresholds can be set at different levels.

[0040] Step S302: Starting from the first-level classification node, based on the initial classification probability vector and the preset confidence threshold, traverse the corresponding sub-classification nodes and the confidence levels corresponding to the sub-classification nodes in the multi-level classification system table.

[0041] It should be noted that, as Figure 2 As shown, in this step, the system enters a layer-by-layer refinement phase. Under the determined first-level classification node, only its direct child nodes (second-level categories) are considered. Based on the probability information of the leaf nodes corresponding to these second-level categories in the initial classification probability vector, the second-level category with the highest confidence is determined as the second-level classification node. Then, using this node as a new starting point, this process is repeated in its child nodes (third-level categories) until the leaf node (fourth-level category) is reached. At each level, the selected node and its confidence are recorded. It can be understood that the purpose of this step is to use the hierarchical structure to simplify the complex global classification problem into a series of local, simpler classification subproblems, thereby reducing decision complexity.

[0042] Step S303: Connect the first-level classification node, the sub-classification node and the corresponding confidence score in hierarchical order to obtain the candidate classification path.

[0043] It should be noted that in this step, the system combines the determined nodes at each level (level 1, level 2, level 3, and level 4) and their associated confidence scores in a hierarchical order from root to leaf, forming a complete candidate classification path. Understandably, the purpose of this step is to integrate the decision results from each level to form an interpretable and structured preliminary classification.

[0044] Step S40: Perform backtracking and re-judgment based on the candidate classification path and the preset confidence threshold to obtain the target complaint classification path.

[0045] It's important to note that in this step, the system performs a final verification and optimization of the candidate classification paths. It checks the classification confidence at each layer of the path, especially the lower layers (such as levels three to four). If the confidence at a certain layer is found to be lower than a predetermined threshold β (e.g., 0.6), a backtracking and re-judgment mechanism is triggered. This step essentially adds a quality control checkpoint, preventing incorrect fine-grained classifications from being forced when evidence is insufficient, thereby further improving the reliability of the final result.

[0046] Additionally, it should be noted that the backtracking re-judgment refers to an error correction mechanism. When the system detects insufficient confidence in the classification of a lower-level node, it will no longer continue using that path but will instead revert to the previous level node to re-evaluate other subcategories or make a decision based on more contextual information. Alternatively, the decision can be left at a higher level (such as level three), and a prompt indicating the need for manual intervention will be output. The target complaint classification path is the final classification result path output by the system after verification and adjustment through the backtracking re-judgment mechanism. It may be the original candidate path, a path ending at a higher level, or a new path after adjustment.

[0047] This embodiment provides a multi-level complaint classification method based on a large model. The method includes: obtaining a multi-level classification system table and complaint text after data anonymization and normalization; extracting semantic features from the complaint text based on a multi-potential head attention large model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention large model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure; performing hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence levels; and performing backtracking and re-judgment based on the candidate classification paths and a pre-set confidence threshold to obtain the target complaint classification path. In summary, this embodiment, by constructing a large model integrating an attention optimization mechanism and multi-level optimization decision-making, achieves accurate and rapid classification of massive non-standardized complaint texts, solving the adaptability and efficiency bottlenecks of traditional methods in ultra-large-scale classification scenarios, and improving the system's classification accuracy and efficiency.

[0048] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the multi-level complaint classification method based on a large model in this application. Step S20 specifically includes: Step S201: Input the complaint text into the multi-potential head attention model to obtain the hidden state sequence of the last layer.

[0049] It's important to note that in this step, the preprocessed complaint text is converted into a sequence of tokens acceptable to the model and input into a multi-potential head attention model. The model performs forward computation sequentially through embedding layers and multiple Transformer layers. Finally, the model outputs the hidden state sequence of its last Transformer layer. The hidden state sequence of the last layer refers to the vector sequence output by the model's last encoding layer. Assuming the input text is segmented into N tokens, the hidden state sequence is a tensor of shape [N, D], where D is the hidden layer dimension of the model (e.g., 4096). Each vector in the sequence corresponds to an input token and incorporates the semantic information of that token and its context.

[0050] Step S202: Perform mean pooling on the hidden state sequence to obtain the semantic feature vector.

[0051] It should be noted that in this step, the system calculates the average of all vector elements along the sequence length dimension (N) of the hidden state sequence [N, D]. This operation compresses the variable-length sequence into a fixed-length vector [1, D], i.e., the semantic feature vector. Understandably, the purpose of this step is to aggregate the global semantic information of the entire text, forming a comprehensive feature representation for classification.

[0052] Step S203: Map the semantic feature vector through a fully connected layer to the multi-level category space corresponding to the multi-level classification system table to obtain the initial classification probability vector.

[0053] It should be noted that in this step, the system inputs the semantic feature vector [1, D] obtained in the previous step into the fully connected layer (classification head). The number of output neurons of this fully connected layer is equal to the total number K (e.g., 5000+) of the leaf nodes (final subcategories) in the multi-level classification system table. Through the linear transformation and non-linear activation function (such as Softmax) of this layer, the high-dimensional semantic features are mapped to the probability distribution of K categories, i.e., the initial classification probability vector [1, K].

[0054] In one feasible implementation, prior to step S20, the method further includes: Step A10: Obtain the pre-trained language model, replace the grouped query attention structure in the pre-trained language model with a multi-potential head attention structure, and obtain the target training model.

[0055] It should be noted that in this step, the system acquires a large language model pre-trained on a massive historical complaint corpus as a base (e.g., Qwen2.5-7B). Then, its internal attention mechanism module is modified: the original Group Query Attention (GQA) module is completely replaced with a Multi-Latent Head Attention (MLA) module, thereby obtaining a target training model with an optimized structure.

[0056] Additionally, it should be noted that the target training model refers to a model whose structural modifications have been completed but has not yet been trained (or fine-tuned) on a complaint classification task. The MLA structure, through the sharing of latent key-value pairs, can significantly reduce the GPU memory usage of the KV-Cache during inference.

[0057] Step A20: Determine the corresponding complaint category label based on the multi-level classification system table.

[0058] It should be noted that in this step, the system will construct or import the aforementioned multi-level classification system table based on business needs. Each leaf node (fourth-level category) in this system table defines an independent complaint classification label. The system organizes the labels corresponding to all leaf nodes into a list, which serves as the target space for the model to learn and predict. Furthermore, it should be noted that the complaint classification label refers to the unique identifier or name of each finest-grained category (leaf node) in the multi-level classification system table. During model training, these labels are converted into numerical IDs.

[0059] Step A30: Align the intermediate layer features of the target training model with those of the pre-trained language model, and after alignment, perform end-to-end supervised training on the output of the target training model based on the complaint classification label to obtain a multi-potential head attention large model.

[0060] It's important to note that the system employs a phased strategy for model training in this step. First, feature-level training is performed: using the same complaint text data, both the target training model and the original pre-trained language model are input, and the feature vectors output from certain intermediate network layers (i.e., intermediate layer features) are constrained to be as similar (aligned) as possible. This stage aims to ensure the modified model inherits the original model's general semantic understanding capabilities. Next, model-level training is performed: based on feature alignment, sample data with complaint classification labels is used to perform standard supervised learning (end-to-end training) on ​​the final output (i.e., classification probability) of the target training model to optimize its classification performance. After completing these two stages, a large multi-potential head attention model suitable for inference is obtained. In essence, this step aims to efficiently and stably adapt the structurally modified model to ultra-fine-grained complaint classification tasks.

[0061] Additionally, it should be noted that the intermediate layer feature alignment is a model distillation or transfer learning technique that, by minimizing the distance between the output features of the intermediate layers of two models (such as cosine similarity loss), enables the student model (target training model) to learn the internal representation of the teacher model (pre-trained model). End-to-end supervised training refers to the standard classification model training process, where the input is text and the output is classification probability. All weight parameters of the model are updated by comparing the predicted probability with the true label using the cross-entropy loss function.

[0062] In one feasible implementation, after step A30, the method further includes: Step A40: Obtain the number of complaints and the average confidence level for each complaint category label within the current time period.

[0063] It should be noted that the system continuously monitors business data during its online operation. It counts the number of complaint tickets categorized under each complaint category label (leaf node) within a recent preset time period (e.g., the past 24 hours), i.e., the number of complaints. Simultaneously, when classifying these tickets, the model outputs the average confidence score for the corresponding label. The average confidence score refers to the arithmetic mean of the classification confidence scores given by the model for a batch of complaint texts classified into the same category, reflecting the model's confidence level in its current determination of that category.

[0064] Step A50: When the increase in the number of complaints within a preset unit time period is greater than or equal to a preset hotspot warning threshold, the complaint category tags that are greater than or equal to the preset hotspot warning threshold are marked as hotspot events.

[0065] It's important to note that "hotspot events" refer to a business phenomenon characterized by an unusually high surge in the number of specific types of complaints within a short period. These typically require close attention and priority handling from the business side. In this step, the system compares the current number of complaints with the number of complaints in the previous comparable period (e.g., the previous 24 hours) and calculates the growth rate. If the growth rate of a certain category of complaints exceeds a preset threshold (e.g., a 50% increase), the system automatically marks that category as a hotspot event. This step aims to automatically identify concentrated complaint issues caused by sudden events, product malfunctions, or policy changes.

[0066] Step A60: When the average confidence level decreases by more than or equal to a preset drift warning threshold, the complaint classification labels that are greater than or equal to the preset drift warning threshold are marked with a distribution drift.

[0067] It's important to note that distribution drift refers to a significant change in the characteristics of actual online complaint data (such as expression style and complaint focus) compared to the historical data used to train the model. This causes the knowledge learned by the model based on older data to become inapplicable, manifesting as a decrease in model confidence. In this step, the system compares the model's average confidence for the same category between the current time period and historical time periods (such as one week ago). If the confidence decreases by more than a preset threshold (e.g., a 15% decrease), the system determines that the data in that category may have experienced distribution drift and marks it accordingly. Understandably, this step aims to detect the potential risk of a decline in model performance due to changes in the distribution of input data.

[0068] Step A70: Based on the hotspot event markers and / or the distribution drift markers, samples of the corresponding categories are extracted from the preset incremental learning sample pool to fine-tune the parameters of the multi-potential head attention model, so as to update the multi-potential head attention model.

[0069] It's important to note that in this step, the system maintains an incremental learning sample pool, continuously collecting recently verified (manually) collected complaint samples. When step A50 or A60 triggers an alert, the system automatically extracts new samples from the sample pool that are relevant to the labeled category within a recent period. Then, using a parameter-efficient fine-tuning method (such as LoRA), only a small portion of the additional parameters of the large model are updated, quickly retraining the model. Understandably, the purpose of this step is to enable the model to adaptively adjust based on the latest business feedback and data distribution, maintaining its classification accuracy and robustness.

[0070] In this embodiment, through the entire process design from model structure modification and multi-stage feature alignment training to real-time monitoring and incremental learning, the classification model achieves high accuracy, high inference efficiency and dynamic adaptive capability in ultra-large-scale fine-grained systems. It solves the computational bottlenecks of traditional methods in complex semantic understanding, data distribution drift and high concurrency processing, and significantly improves the accuracy, robustness and overall system performance of complaint classification.

[0071] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the multi-level complaint classification method based on a large model in this application. Step S40 specifically includes: Step S401: Traverse the candidate classification path. When the confidence level corresponding to the current level is lower than the preset backtracking threshold, backtrack the classification level to the level above the current level. Then, starting from the classification node of the previous level, re-determine the sub-classification node of the next level based on the initial classification probability vector.

[0072] It's important to note that in this step, the system performs quality checks on the candidate classification paths. Starting from the last node of the path (e.g., level four), it examines the classification confidence of each node layer by layer. When the confidence of a certain level falls below a pre-set backtracking threshold β (e.g., 0.6), the system determines that the classification evidence at that level is insufficient and the decision is unreliable. At this point, the system triggers a backtracking mechanism: abandoning the entire branch downwards from that node and reverting the current decision point back to its parent node (the previous level). Then, under the parent node, except for the previously selected incorrect child nodes, the confidence of other sibling child nodes is re-evaluated based on the initial classification probability vector, and the one with the highest confidence is selected as the new sub-classification node. In essence, this step introduces a proactive error correction and optimization logic, preventing the model from making erroneous fine-grained judgments when confidence is insufficient, thereby significantly improving the reliability of the classification results.

[0073] Step S402: Update the corresponding level of the classification nodes and their confidence scores in the candidate classification path according to the re-determined sub-classification nodes to obtain the target complaint classification path.

[0074] It should be noted that after backtracking and reselection, the system will replace all nodes in the original candidate classification path starting from the backtracking point with the newly determined sub-classification nodes and their corresponding confidence scores. If the reselected path after backtracking successfully reaches the leaf node (level four), a new, complete path will be generated; if insufficient confidence is encountered again at the new node after backtracking, only a path ending at a higher level will be generated (e.g., only up to level three). This updated path is the final determined target complaint classification path.

[0075] Understandably, the target complaint classification path is the final classification result output by the system after undergoing dual verification through internal hierarchical decision-making and a backtracking re-judgment mechanism. It represents the system's most reliable and reasonable classification judgment for the current complaint text.

[0076] In one feasible implementation, after step S40, the method further includes: Step S50: When the final confidence level of the target complaint classification path is lower than the preset manual review threshold, the complaint text and the target complaint classification path are pushed to the manual review queue.

[0077] It's important to note that even after backtracking and re-evaluation, the system still performs a final check on the target complaint classification path. The system checks the confidence level of the final-level nodes (leaf nodes) of the path. If this confidence level is lower than a more stringent preset human review threshold γ (e.g., 0.7), it indicates that the system's understanding of the case's classification is still insufficient. In this case, the system will not force an potentially erroneous automated result. Instead, it will package the complaint text, the model-recommended target complaint classification path, and the confidence levels at each level into a single review task and push it to a dedicated human review queue. This step serves as a crucial quality control measure, entrusting complex, ambiguous cases that the system struggles to handle to professionals, ensuring the absolute reliability of the final output and mitigating business risks associated with automated misjudgments.

[0078] Step S60: Receive the correction category label returned by the manual review queue, and replace the corresponding last-level node in the target complaint classification path according to the correction category label to form a manual correction path.

[0079] It should be noted that when processing tasks in the manual review queue, the reviewer will refer to the recommended paths and criteria provided by the system, and make a final judgment based on their own business knowledge. Then, they submit their confirmed or modified final category (i.e., corrected category label) through the interactive interface. The system will receive this feedback and replace the last-level node in the original target complaint classification path with the manually determined leaf node category, thus forming a manually corrected path.

[0080] Understandably, a manually corrected path refers to a complete classification path that is based on the hierarchical path recommended by the system, but is manually corrected only at the very last level.

[0081] Step S70: Update the target complaint classification path according to the manually corrected path.

[0082] It should be noted that in this step, the system will use the manually corrected path as the final result for the complaint case, overriding the target complaint classification path previously automatically generated by the system. The updated target complaint classification path will be officially output to downstream business systems and stored in the historical database. At the same time, this manually corrected path and its corresponding original complaint text will be used as a correction sample and stored in the incremental learning sample pool maintained by the adaptive learning module.

[0083] In this embodiment, a confidence-driven dynamic backtracking mechanism and human-machine collaborative feedback correction are used to achieve prudent verification and continuous optimization of the classification decision process. This solves the problems of traditional models forcibly classifying in the absence of sufficient evidence and lack of human supervision and feedback, and significantly improves the reliability, interpretability and overall adaptability of the classification results.

[0084] This application also provides a multi-level complaint classification device based on a large model, please refer to... Figure 5 The multi-level complaint classification device based on a large model includes: Data acquisition module 10 is used to acquire a multi-level classification system table and complaint text after data anonymization and normalization. Feature extraction module 20 is used to extract semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure. The classification decision module 30 is used to perform hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores. The complaint classification module 40 is used to perform backtracking and re-judgment based on the candidate classification path and the preset confidence threshold to obtain the target complaint classification path.

[0085] The multi-level complaint classification device based on a large model provided in this application, employing the multi-level complaint classification method based on a large model in the above embodiments, can solve the technical problem of how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios. Compared with the prior art, the beneficial effects of the multi-level complaint classification device based on a large model provided in this application are the same as those of the multi-level complaint classification method based on a large model provided in the above embodiments, and other technical features in the multi-level complaint classification device based on a large model are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0086] In one embodiment, the feature extraction module 20 is further configured to input the complaint text into a multi-potential head attention model to obtain the hidden state sequence of the last layer; perform mean pooling on the hidden state sequence to obtain a semantic feature vector; and map the semantic feature vector through a fully connected layer to a multi-level category space corresponding to the multi-level classification system table to obtain an initial classification probability vector.

[0087] In one embodiment, the classification decision module 30 is further configured to: filter the confidence levels corresponding to the first-level classifications in the multi-level classification system table based on the initial classification probability vector and a preset confidence threshold, and take the first-level classification with the highest confidence level as the first-level classification node; starting from the first-level classification node, traverse the corresponding sub-classification nodes and the confidence levels corresponding to the sub-classification nodes in the multi-level classification system table based on the initial classification probability vector and the preset confidence threshold; and connect the first-level classification nodes, the sub-classification nodes, and the corresponding confidence levels in hierarchical order to obtain candidate classification paths.

[0088] In one embodiment, the complaint classification module 40 is further configured to traverse the candidate classification paths, and when the confidence level corresponding to the current level is lower than a preset backtracking threshold, backtrack the classification level to the level above the current level, and use the classification node of the previous level as the starting point to re-determine the sub-classification node of the next level according to the initial classification probability vector; update the classification node and its confidence level of the corresponding level in the candidate classification path according to the re-determined sub-classification node to obtain the target complaint classification path.

[0089] In one embodiment, the complaint classification module 40 is further configured to push the complaint text and the target complaint classification path to the manual review queue when the final confidence level of the target complaint classification path is lower than a preset manual review threshold; receive the correction category tag returned by the manual review queue, and replace the corresponding final node in the target complaint classification path according to the correction category tag to form a manual correction path; and update the target complaint classification path according to the manual correction path.

[0090] In one embodiment, the multi-level complaint classification device based on a large model further includes an adaptive learning module 50, used to acquire a pre-trained language model, replace the grouped query attention structure in the pre-trained language model with a multi-potential head attention structure to obtain a target training model; determine the corresponding complaint classification label according to the multi-level classification system table; align the intermediate layer features of the target training model with those of the pre-trained language model, and perform end-to-end supervised training on the output of the target training model according to the complaint classification label after alignment to obtain a multi-potential head attention large model.

[0091] In one embodiment, the adaptive learning module 50 is further configured to obtain the number of complaints and the average confidence level of each complaint category label within the current time period; when the increase in the number of complaints within a preset unit time period is greater than or equal to a preset hotspot warning threshold, the complaint category labels that are greater than or equal to the preset hotspot warning threshold are marked as hotspot events; when the decrease in the average confidence level is greater than or equal to a preset drift warning threshold, the complaint category labels that are greater than or equal to the preset drift warning threshold are marked as distribution drift; based on the hotspot event marking and / or the distribution drift marking, samples of the corresponding category are extracted from a preset incremental learning sample pool to fine-tune the parameters of the multi-potential head attention model, so as to update the multi-potential head attention model.

[0092] This application provides a multi-level complaint classification device based on a large model. The multi-level complaint classification device based on a large model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multi-level complaint classification method based on a large model in the above embodiment 1.

[0093] The following is for reference. Figure 6 This document illustrates a structural schematic diagram of a multi-level complaint classification device based on a large model, suitable for implementing embodiments of this application. The multi-level complaint classification device based on a large model in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The multi-level complaint classification device based on a large model shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0094] like Figure 6 As shown, the multi-level complaint classification device based on a large model may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the multi-level complaint classification device based on the large model. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the multi-level complaint classification device based on a large model to communicate wirelessly or wiredly with other devices to exchange data. Although a multi-level complaint classification device based on a large model with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0095] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0096] The multi-level complaint classification device based on a large model provided in this application, employing the multi-level complaint classification method based on a large model in the above embodiments, can solve the technical problem of how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios. Compared with the prior art, the beneficial effects of the multi-level complaint classification device based on a large model provided in this application are the same as those of the multi-level complaint classification method based on a large model provided in the above embodiments, and other technical features in this multi-level complaint classification device based on a large model are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0097] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0099] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the multi-level complaint classification method based on a large model in the above embodiments.

[0100] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0101] The aforementioned computer-readable storage medium may be included in a multi-level complaint classification device based on a large model; or it may exist independently and not assembled into a multi-level complaint classification device based on a large model.

[0102] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a multi-level complaint classification device based on a large model, the multi-level complaint classification device based on a large model performs the following: acquires a multi-level classification system table and complaint text after data anonymization and normalization; extracts semantic features from the complaint text based on a multi-potential head attention large model and the multi-level classification system table to obtain an initial classification probability vector, wherein the multi-potential head attention large model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure; performs hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence levels; and performs backtracking and re-judgment based on the candidate classification paths and a preset confidence threshold to obtain the target complaint classification path.

[0103] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0106] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described multi-level complaint classification method based on a large model. This addresses the technical problem of improving the accuracy and efficiency of complaint classification in ultra-large-scale scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multi-level complaint classification method based on a large model provided in the above embodiments, and will not be elaborated upon here.

[0107] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-level complaint classification method based on a large model as described above.

[0108] The computer program product provided in this application can solve the technical problem of how to improve the accuracy and efficiency of complaint classification in ultra-large-scale scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-level complaint classification method based on large models provided in the above embodiments, and will not be repeated here.

[0109] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A multi-level complaint classification method based on a large model, characterized in that, The method includes: Obtain the multi-level classification system table and the complaint text after data anonymization and normalization; Semantic features are extracted from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure. Based on the initial classification probability vector and the multi-level classification system table, hierarchical constraint classification is performed to obtain candidate classification paths including confidence scores; The target complaint classification path is obtained by backtracking and re-judging based on the candidate classification path and the preset reliability threshold.

2. The method as described in claim 1, characterized in that, The step of extracting semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain the initial classification probability vector includes: The complaint text is input into a multi-potential head attention model to obtain the hidden state sequence of the last layer. The semantic feature vector is obtained by performing mean pooling on the hidden state sequence; The semantic feature vector is mapped to the multi-level category space corresponding to the multi-level classification system table through a fully connected layer to obtain the initial classification probability vector.

3. The method as described in claim 1, characterized in that, The step of performing hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores includes: Based on the initial classification probability vector and the preset confidence threshold, the confidence levels corresponding to the first-level classifications in the multi-level classification system table are selected, and the first-level classification with the highest confidence level is taken as the first-level classification node. Starting from the first-level classification node, based on the initial classification probability vector and the preset confidence threshold, the corresponding sub-classification nodes and the confidence levels corresponding to the sub-classification nodes in the multi-level classification system table are traversed. The candidate classification path is obtained by connecting the first-level classification node, the sub-classification node and the corresponding confidence score in hierarchical order.

4. The method as described in claim 1, characterized in that, The step of performing backtracking and re-judging based on the candidate classification path and a preset confidence threshold to obtain the target complaint classification path includes: Traverse the candidate classification path. When the confidence level corresponding to the current level is lower than the preset backtracking threshold, backtrack the classification level to the level above the current level. Then, starting from the classification node of the previous level, re-determine the sub-classification node of the next level based on the initial classification probability vector. The target complaint classification path is obtained by updating the corresponding level classification nodes and their confidence scores in the candidate classification path based on the redefined sub-classification nodes.

5. The method as described in claim 1, characterized in that, Before the step of extracting semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain the initial classification probability vector, the method further includes: Obtain a pre-trained language model, and replace the grouped query attention structure in the pre-trained language model with a multi-potential head attention structure to obtain the target training model; Determine the corresponding complaint category label based on the multi-level classification system table; Align the intermediate layer features of the target training model with those of the pre-trained language model, and then perform end-to-end supervised training on the output of the target training model based on the complaint classification label after alignment to obtain a multi-potential head attention large model.

6. The method as described in claim 5, characterized in that, After the steps of aligning the intermediate layer features of the target training model with those of the pre-trained language model, and then performing end-to-end supervised training on the output of the target training model based on the complaint classification labels to obtain a multi-latent head attention large model, the method further includes: Obtain the number of complaints and the average confidence level for each complaint category label within the current time period; When the increase in the number of complaints within a preset unit time period is greater than or equal to a preset hotspot warning threshold, the complaint category tags that are greater than or equal to the preset hotspot warning threshold are marked as hotspot events. When the average confidence level decreases by more than or equal to a preset drift warning threshold, the complaint classification labels that are greater than or equal to the preset drift warning threshold are marked with a distribution drift. Based on the hotspot event markers and / or the distribution drift markers, samples of the corresponding categories are extracted from a preset incremental learning sample pool to fine-tune the parameters of the multi-potential head attention model, thereby updating the multi-potential head attention model.

7. The method as described in claim 1, characterized in that, After the step of performing backtracking and re-judging based on the candidate classification path and a preset confidence threshold to obtain the target complaint classification path, the method further includes: When the final confidence level of the target complaint classification path is lower than the preset manual review threshold, the complaint text and the target complaint classification path are pushed to the manual review queue. Receive the correction category label returned by the manual review queue, and replace the corresponding last-level node in the target complaint classification path according to the correction category label to form a manual correction path; The target complaint classification path is updated based on the manually corrected path.

8. A multi-level complaint classification device based on a large model, characterized in that, The device includes: The data acquisition module is used to obtain a multi-level classification system table and complaint texts that have undergone data anonymization and normalization. The feature extraction module is used to extract semantic features from the complaint text based on the multi-potential head attention model and the multi-level classification system table to obtain an initial classification probability vector. The multi-potential head attention model is a pre-trained language model that replaces the grouped query attention structure with a multi-potential head attention structure. The classification decision module is used to perform hierarchical constraint classification based on the initial classification probability vector and the multi-level classification system table to obtain candidate classification paths including confidence scores. The complaint classification module is used to perform backtracking and re-judgment based on the candidate classification paths and preset confidence thresholds to obtain the target complaint classification path.

9. A multi-level complaint classification device based on a large model, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-level complaint classification method based on a large model as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the multi-level complaint classification method based on a large model as described in any one of claims 1 to 7.

Images