A large model customer service intelligent dispatching method and system fusing semantic understanding

Abstract

The application relates to a large model customer service intelligent order dispatching method and system based on fusion semantic understanding and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring multi-round interaction data of a customer, performing time alignment, and obtaining a conversation context data package; inputting the conversation context data package into a large language model, performing semantic understanding and noise resistance calibration, and generating robust work order semantic representation; based on the robust work order semantic representation, a dynamic graph of a work order and a candidate seat is constructed, the node and edge attributes of the dynamic graph are updated through graph propagation, and a dynamic order dispatching candidate set is formed; within the action boundary defined by the dynamic order dispatching candidate set, a service effect index value is combined to generate and execute an order dispatching instruction; in response to an abnormal state in the execution process of the order dispatching instruction, the dynamic graph is updated, a closed-loop re-dispatching is triggered, and a secondary order dispatching instruction is generated; and based on the actual processing result of the secondary order dispatching instruction, a semantic calibration model, a dynamic graph and an order dispatching decision model are updated to complete customer service intelligent order dispatching.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a large-scale customer service intelligent order dispatching method and system that integrates semantic understanding. Background Technology

[0002] With the rapid development of internet technology, e-commerce platforms and online service businesses are expanding rapidly, leading to an explosive growth in customer service requests. To improve service response speed and customer satisfaction while reducing manual operating costs, intelligent customer service systems are widely used. Among these systems, intelligent order dispatch, as a crucial link connecting customer needs with agent resources, directly determines service efficiency and user experience. How to accurately identify intent from massive and complex customer interaction data and quickly and precisely assign work orders to the most suitable agents has become a critical problem that urgently needs to be solved in the current customer service field.

[0003] Currently, existing methods typically involve performing simple keyword extraction or shallow semantic analysis on the text input by customers and matching it with preset agent skill tags; or using static knowledge graphs built based on historical statistical data to find candidate agents through fixed weight calculations.

[0004] However, existing dispatch methods rely on static rules or static matching, failing to perceive dynamic states such as agent load and remaining SLA time. This results in low dispatch accuracy, and semantic understanding is susceptible to verbal noise, leading to misjudgments. Furthermore, they lack a closed-loop calibration and update mechanism based on actual service results, making it impossible to self-correct misjudgments. The above statements are for providing background information related to this application only and do not necessarily constitute prior art. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a large-scale customer service intelligent dispatching method and system that integrates semantic understanding. This method can overcome the shortcomings of existing methods, which are based on static rules or static matching and cannot perceive dynamic states such as real-time agent load and remaining SLA time, resulting in low dispatching accuracy. Furthermore, the semantic understanding is easily affected by spoken noise, leading to misjudgments. It also lacks a closed-loop calibration and update mechanism based on real service results and cannot self-correct after misjudgment.

[0006] In a first aspect, the present invention proposes a large-scale customer service intelligent order dispatching method that integrates semantic understanding, the method comprising: S1: Obtain multi-round interaction data from the customer, perform time alignment, and obtain the session context data packet; S2, input the conversation context data packet into the large language model, perform semantic understanding and noise resistance calibration, and generate a robust work order semantic representation; S3, based on robust work order semantic representation, constructs a dynamic graph of work orders and candidate agents, and updates the node and edge attributes of the dynamic graph through graph propagation to form a dynamic dispatch candidate set; S4. Within the action boundaries defined by the dynamic dispatch candidate set, a dispatch instruction is generated and executed by combining the service performance index value. S5 responds to abnormal states during the execution of a dispatch instruction, updates the dynamic graph, triggers closed-loop redispatch, and generates a second dispatch instruction. S6 updates the semantic calibration model, dynamic graph, and dispatch decision model based on the actual processing results of the secondary dispatch instructions to complete intelligent customer service dispatch.

[0007] Furthermore, S1 specifically includes: S101, align the current interaction data, historical round data, and customer business-related data according to the order of the current interaction data, historical round data, and customer business-related data over time; S102, clean up duplicate text, blank text and invalid system prompts in the time-aligned data, and retain negative words, degree adverbs, time words and business entities; S103, combined with the preset field structure, encapsulates the cleaned data to obtain the session context data packet.

[0008] Furthermore, S2 specifically includes: S201, Input the session context data packet into the large language model to obtain the candidate intent and the current work order semantic vector; S202, based on candidate intents and the current work order semantic vector, for each intent category, obtain supporting samples and corresponding semantic vectors from historical work orders, and calculate the intra-class average consistency among supporting samples within the same intent category; S203. Based on the intra-class average consistency, perform Softmax normalization with temperature parameter to obtain the reliability weight of each support sample. Based on the reliability weight, perform weighted aggregation on the semantic vectors corresponding to each support sample to form a noise-resistant semantic prototype of the intent category. S204, calculate the distance between the current work order semantic vector and the noise-resistant semantic prototype, and convert the distance into the intent category calibration probability; S205, calibrate the probability based on the intent category to determine the target intent and intent confidence; S206 encodes the target intent, intent confidence, and current work order semantic vector in a unified manner to generate a robust work order semantic representation.

[0009] Furthermore, S2 also includes: S207, when the confidence level of an intent is lower than a preset low confidence threshold, select a preset number of candidate intents whose intent categories are ranked first by calibration probability, and deduplicate and merge the business skills corresponding to the multiple candidate intents to form an extended skill set.

[0010] Furthermore, S3 specifically includes: S301, based on robust work order semantic representation, sets the current work order as the work order side node and the candidate agent as the agent side node, establishes candidate service relationship edges, and initializes the work order node vector, agent node vector and edge attribute vector. S302, project the work order node vector, agent node vector and edge attribute vector onto the common relationship representation space, and calculate the fused relationship vector; S303, based on the fusion relation vector, calculate the unnormalized dynamic attention score, and perform normalization processing on the unnormalized dynamic attention score to obtain the normalized dynamic graph attention weight; S304, based on the normalized dynamic graph attention weights, perform weighted updates on the work order node vector and the agent node vector; S305, Based on the updated work order node vector and agent node vector, the edge attribute vector is corrected; S306, Repeat steps S302 to S305 until the preset number of layers is reached, and input the node vector of the final layer into the relationship probability prediction layer to obtain the dynamic matching probability; S307, based on dynamic matching probability, forms a dynamic order dispatch candidate set.

[0011] Furthermore, S4 specifically includes: S401, Based on service performance indicators, construct reward and evaluation rules for the customer service field; S402, constructs a random noise matrix by factoring Gaussian noise; S403: Input the current order dispatch decision state into the online Q network with parameterized noise, and perturb the network parameters of the online Q network through the elements in the random noise matrix to obtain the network layer output; S404, based on the customer service domain reward evaluation rules and network layer output, evaluates the value of actions through the Dueling structure, selects the action with the highest long-term action value from the set of legal dispatch actions in the dynamic dispatch candidate set, generates a dispatch instruction and executes it.

[0012] Furthermore, S5 specifically includes: S501 continuously monitors the target agent's order acceptance status, online status, number of concurrent work orders in real time, and remaining SLA time to obtain real-time monitoring data; S502: When real-time monitoring data detects that the target agent has refused to accept the call, is offline, has exceeded the preset limit for load, has exceeded the order acceptance waiting time, or has initiated an unprocessable transfer, the current order dispatch action will be marked as a failed action. S503, update the edge attributes in the dynamic graph based on real-time monitoring data; S504, re-input the updated edge attributes into the dynamic graph for graph propagation, and generate the current dynamic dispatch candidate set; S505 makes a dispatch decision based on the current dynamic dispatch candidate set, and masks failed actions during the decision-making process to generate a secondary dispatch instruction.

[0013] Furthermore, S5 also includes: S506: When a candidate agent who meets the hard constraints cannot accept the task or the remaining SLA time of the current work order is lower than the manual backup threshold, the current work order will be transferred to the manual backup queue with the corresponding business permissions, and the reason for triggering the manual backup will be recorded.

[0014] Furthermore, S6 specifically includes: S601, based on the actual processing results, according to the preset multi-step length, the actual service rewards corresponding to multiple dispatch decisions are attenuated and accumulated to obtain the discount reward; S602, Determine the Double DQN target value based on the discount reward; S603 calculates the timing difference error based on the Double DQN target value; S604 determines the basic priority of historical order dispatch experience based on the time series differential error, and extracts training batches based on the basic priority. S605 updates the order dispatch decision model through training batches; S606, based on the actual processing results, update the node vector and edge attributes of the actual processing agents in the dynamic graph; S607 updates the semantic support samples based on the true intent category, and updates the semantic calibration model based on the updated semantic support samples when preset conditions are met, so as to complete the intelligent customer service order dispatch.

[0015] In a second aspect, the present invention proposes a large-scale customer service intelligent dispatch system that integrates semantic understanding, comprising: a memory and a processor; The memory stores the application program, which is suitable for execution by the processor to implement the large-scale customer service intelligent dispatching method with fusion semantic understanding in the first aspect.

[0016] The beneficial effects of this invention are as follows: In this embodiment of the invention, by constructing a dynamic graph of work orders and candidate agents and using graph propagation to update node and edge attributes in real time, dynamic adjustments based on the real-time status of agents and the dynamic context of work orders are achieved, effectively overcoming the limitations of static matching mechanisms. Simultaneously, by combining multi-dimensional service performance indicators to determine dispatch instructions and using actual processing results to perform closed-loop updates on the semantic calibration model, dynamic graph, and dispatch decision model, the system is not only endowed with the ability to conduct long-term value assessments based on multi-dimensional indicators and achieve optimal resource allocation in dynamic environments, but also establishes a self-iteration and calibration mechanism based on actual service feedback, thereby improving the accuracy and adaptability of intelligent dispatch. Attached Figure Description

[0017] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0018] Figure 1 This is a flowchart illustrating a large-scale customer service intelligent order dispatching method that integrates semantic understanding, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the dynamic graph propagation and edge attribute correction mechanism provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the multi-model joint update closed loop provided in the embodiments of the present invention.

[0020] Figure 4 This is a schematic diagram of the structure of a large-scale customer service intelligent dispatch system that integrates semantic understanding, provided by an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.

[0024] This invention proposes a large-scale intelligent customer service dispatching method and system that integrates semantic understanding to solve the problems of existing dispatching methods being unable to perceive the real-time dynamic status of agents, semantic understanding being susceptible to noise interference, and lacking a closed-loop calibration mechanism.

[0025] Method Implementation Examples Reference Figure 1 The diagram illustrates a flowchart of a large-scale customer service intelligent dispatching method that integrates semantic understanding, provided by an embodiment of the present invention.

[0026] This invention provides a large-scale customer service intelligent order dispatching method that integrates semantic understanding. The method includes: Specifically, the method includes steps S1 to S6.

[0027] S1: Obtain multi-round interaction data from the customer, perform time alignment, and obtain the session context data packet.

[0028] In one possible implementation, S1 specifically includes: S101, align the current interaction data, historical round data, and customer business-related data according to the order of current interaction data, historical round data, and customer business-related data over time.

[0029] Specifically, this particular arrangement order is not set arbitrarily, but is designed to adapt to the attention mechanism characteristics of large language models, enabling them to prioritize the latest user demands while using recent history and business status as background anchors to effectively resolve complex linguistic phenomena such as referential resolution and omission recovery.

[0030] S102, clean up duplicate text, blank text and invalid system prompts in the time-aligned data, and retain negative words, degree adverbs, time words and business entities.

[0031] Specifically, unlike traditional natural language processing cleaning strategies that tend to remove stop words, this embodiment explicitly retains negation words, degree adverbs, and time words. This is because in customer service scenarios, these words often carry key semantic information that determines the direction of intent. For example, "do not refund" and "refund" differ by only one word but have completely opposite business meanings. If they are removed, it will lead to incorrect intent recognition.

[0032] S103, combined with the preset field structure, encapsulates the cleaned data to obtain the session context data packet.

[0033] S2 inputs the session context data packet into the large language model for semantic understanding and noise resistance calibration, generating a robust work order semantic representation.

[0034] In one possible implementation, S2 specifically includes: S201, input the session context data packet into the large language model to obtain the candidate intent and the current work order semantic vector.

[0035] Specifically, the large language model includes an input embedding module, a multi-layer Transformer encoding module, an intent classification head, and a semantic vector output module. The input embedding module maps segmented conversation context data packets into word vectors and superimposes positional encodings. The Transformer encoding module contains multiple sequentially concatenated encoding layers. Each encoding layer includes a multi-head self-attention sublayer, a feedforward network sublayer, and residual connections and layer normalization structures; the input of each subsequent encoding layer is the output of the previous one. Both the intent classification head and the semantic vector output module are connected to the output of the last encoding layer. The intent classification head outputs the candidate intent probability distribution through a fully connected layer and a normalization layer, while the semantic vector output module outputs the current work order semantic vector through a pooling layer.

[0036] Furthermore, in one specific implementation, the multi-layer Transformer encoding module has 24 encoding layers, each multi-head self-attention sub-layer has 32 attention heads, the hidden layer dimension is 4096, and the intermediate dimension of the feedforward network sub-layer is four times the hidden layer dimension. The total number of parameters in the large language model is 7 billion. The training process of the large language model includes: firstly, pre-training on a massive general corpus to obtain a basic model; then, supervised fine-tuning using historical conversation and intent annotation data from the customer service domain, freezing the bottom-level parameters and updating only the top-level parameters and classification heads to enable the model to acquire domain adaptability.

[0037] S202: Based on the candidate intent and the current work order semantic vector, for each intent category, obtain supporting samples and corresponding semantic vectors from historical work orders, and calculate the intra-class average consistency among supporting samples within the same intent category.

[0038] Furthermore, work orders that have been confirmed to be correctly labeled and resolved in one go are selected from the historical work order database and grouped according to the labeling intent category. For each intent category, a preset number of work orders are extracted based on the principle of time proximity (the preset number can be 20 or 50), and their semantic vectors are used as support samples for that category.

[0039] In this embodiment of the invention, intra-class average consistency is a core indicator for measuring sample reliability. Its physical meaning is that in a correctly labeled intent category, the vast majority of samples should be tightly clustered in the semantic space, while incorrectly labeled or semantically anomalous samples usually deviate from the cluster center. Therefore, by calculating the average similarity between a sample and other samples within the same category, the reliability of the sample can be quantified. As an optional implementation, intra-class average consistency can be measured using various methods such as Euclidean distance, cosine similarity, or dot product, as long as it reflects the semantic proximity between samples; this application does not limit this. Specifically, when using similarity-type measures such as cosine similarity or dot product, a larger value indicates higher consistency. When using distance-type measures such as Euclidean distance, the distance is first negative (or its reciprocal) to convert it into a consistency score before subsequent Softmax normalization is performed to ensure that a larger consistency score indicates greater semantic consistency between the sample and other samples within the same category.

[0040] S203. Based on the intra-class average consistency, perform Softmax normalization with temperature parameter to obtain the reliability weight of each support sample. Based on the reliability weight, perform weighted aggregation on the semantic vectors corresponding to each support sample to form a noise-resistant semantic prototype of the intent category.

[0041] Specifically, the role of introducing the temperature parameter is to adjust the sharpness of the weight distribution: lower temperatures make the weights of highly consistent samples more prominent, further suppressing the influence of low-consistency noisy samples. Higher temperatures make the weight distribution smoother, increasing the tolerance for marginal samples. The temperature parameter can be set to 0.5 or 1.0.

[0042] S204, calculate the distance between the current work order semantic vector and the noise-resistant semantic prototype, and convert the distance into the intent category calibration probability.

[0043] Specifically, the cosine distance between the current work order semantic vector and the noise-resistant semantic prototype of each intent category is calculated. Each distance is then negative and divided by a preset scaling factor, which can be 0.05 or 0.1. The scaling result for each intent category is then normalized, and the resulting normalized value is the calibration probability for each intent category.

[0044] S205, based on the probability calibration of intent category, determines the target intent and intent confidence.

[0045] Specifically, the intent category corresponding to the intent category with the highest calibration probability is determined as the target intent, and the value of this highest calibration probability is determined as the intent confidence level, that is, the intent confidence level is equal to the calibration probability corresponding to the target intent. This step only determines the target intent based on the intent category with the highest calibration probability, without performing additional threshold judgment; the confidence level threshold judgment is only used to trigger the extended skill set in step S207.

[0046] S206 encodes the target intent, intent confidence, and current work order semantic vector in a unified manner to generate a robust work order semantic representation.

[0047] Specifically, the embedding vector of the target intent and the vector of intent confidence mapped by the fully connected layer are concatenated with the current work order semantic vector, and then processed by the fully connected layer and the activation function in sequence to obtain a robust work order semantic representation with fixed dimensions.

[0048] It should be noted that the semantic calibration model includes a large language model, a support sample storage module, a prototype calculation module, and a calibration comparison module. The large language model outputs candidate intents and the semantic vector of the current work order. The support sample storage module stores manually confirmed support samples and their semantic vectors according to intent categories. The prototype calculation module is connected to both the support sample storage module and the large language model, and is used to calculate intra-class average consistency, generate reliability weights, and aggregate them to form noise-resistant semantic prototypes. The calibration comparison module is connected to both the prototype calculation module and the large language model, and is used to calculate the distance between the semantic vector of the current work order and each noise-resistant semantic prototype, outputting the intent category calibration probability and intent confidence. The initial training and subsequent update processes of this model are as described in steps S201 to S205 and step S607, respectively.

[0049] In one possible implementation, S2 further includes: S207, when the confidence level of an intent is lower than a preset low confidence threshold, select a preset number of candidate intents whose intent categories are ranked first by calibration probability, and deduplicate and merge the business skills corresponding to the multiple candidate intents to form an extended skill set.

[0050] Specifically, the low confidence threshold can be 0.6 or 0.65, and the preset number can be 3 or 5.

[0051] It should be noted that if the range of agent skills is forcibly limited based on a single highest probability intent, misjudgment could lead to the incorrect exclusion of agents with the ability to handle such cases. By merging the skills corresponding to a preset number of candidate intents before sorting, the system proactively expands the candidate agent selection space, improving fault tolerance and service continuity in edge cases.

[0052] S3, based on robust work order semantic representation, constructs a dynamic graph of work orders and candidate agents, and updates the node and edge attributes of the dynamic graph through graph propagation to form a dynamic dispatch candidate set.

[0053] It should be noted that the dynamic graph network includes a node feature initialization module, a feature projection module, an attention calculation module, a node update module, and an edge attribute correction module. The node feature initialization module constructs the work order node vector, the agent node vector, and the edge attribute vector. The feature projection module is connected to both the node feature initialization module and the edge attribute correction module, projecting the three vectors onto a common relation representation space and calculating the fused relation vector. The attention calculation module is connected to the feature projection module and calculates the normalized dynamic graph attention weights. The node update module is connected to the attention calculation module and updates the work order node vector and the agent node vector based on the attention weights. The edge attribute correction module is connected to the node update module and corrects the edge attribute vector based on the updated node vectors. The feature projection, attention calculation, node update, and edge attribute correction are sequentially and cyclically connected, forming a multi-layer graph propagation structure. The node vectors of the final layer are output to the relation probability prediction layer.

[0054] The training process of the dynamic graph network includes: using historical dispatch records as training data, using the actual acceptance results of historical work orders as labels (actual acceptance by the agent without triggering an anomaly is recorded as a positive sample, and rejection, transfer, or timeout is recorded as a negative sample), using the cross-entropy between the dynamic matching probability and the acceptance label as the loss function, and optimizing each learnable transformation matrix, attention vector, edge attribute correction matrix, and prediction layer parameters through gradient descent until the loss function converges.

[0055] Reference Figure 2 The diagram illustrates the dynamic graph propagation and edge attribute correction mechanism provided in an embodiment of the present invention.

[0056] Specifically, Figure 2 The top layer inputs robust work order semantic representations and agent feature vectors, which are initialized with node features to form work order nodes, agent nodes, and candidate service relationship edges. The middle graph propagation loop sequentially executes four modules: feature projection, attention calculation, node update, and edge attribute correction, and repeats the propagation to a preset number of layers. Finally, the relationship probability prediction layer outputs dynamic matching probabilities, generating a dynamic dispatch candidate set.

[0057] In one possible implementation, S3 specifically includes: S301, based on robust work order semantic representation, sets the current work order as the work order-side node and the candidate agent as the agent-side node, establishes candidate service relationship edges, and initializes the work order node vector, agent node vector, and edge attribute vector.

[0058] Specifically, the work order node vector is obtained by linear mapping from the robust work order semantic representation. The agent node vector is obtained by concatenating the normalized values ​​of skill tag encoding, historical one-time resolution rate, average processing time, and real-time load, and then mapping it through a fully connected layer. The edge attribute vector is initialized by concatenating normalized values ​​such as load ratio, expected waiting time, and remaining service level time limit. Candidate agents are those determined after skill recall and hard constraint filtering: skill recall determines the recalled agents based on the business skills corresponding to the target intent, or the skill range limited by the expanded skill set in low-confidence scenarios. Hard constraint filtering removes agents from the recalled agents who are currently offline, do not have corresponding business permissions, or whose load has reached the maximum capacity.

[0059] It should be noted that the edge attribute vector can also include real-time dynamic indicators such as the historical one-time resolution rate of target intent, the degree of matching between customer sentiment and agent service style, and recent transfer rate, rather than fixed static labels.

[0060] S302 projects the work order node vector, agent node vector, and edge attribute vector onto the common relationship representation space and calculates the fused relationship vector.

[0061] Specifically, since work order requirements, agent capabilities, and service relationships are heterogeneous information, direct interaction presents a semantic gap. Therefore, it is necessary to map them to the same semantic space through linear transformation.

[0062] Furthermore, the calculation of the fusion relationship vector involves linearly transforming the corresponding work order node vector, candidate agent node vector, and edge attribute vector through the learnable transformation matrices of their respective work order side node, agent side node, and edge attribute, and then summing the three transformed vectors. It should be understood that the edge attribute vector participates in the feature fusion of each layer, meaning that the real-time state of the service relationship directly affects the calculation of the matching degree, rather than merely serving as a post-filtering condition.

[0063] S303, based on the fusion relation vector, calculates the unnormalized dynamic attention score, and performs normalization processing on the unnormalized dynamic attention score to obtain the normalized dynamic graph attention weight.

[0064] Specifically, to adaptively identify which candidate agents are more important to the current work order, an attention mechanism is introduced. Based on the fused relationship vector obtained by fusing work order nodes, candidate agent nodes, and edge attribute vectors, an unnormalized dynamic attention score is calculated.

[0065] in, Represents the work order node With candidate seat nodes The unnormalized dynamic attention score between them Represents the work order node and candidate agent nodes The fusion relationship vector corresponding to the candidate service relationship edges in the current layer. Represents a trainable attention vector, with superscript indicating the direction of the vector. Indicates transpose. This represents a trainable weight matrix. This represents the activation function. The unnormalized dynamic attention score is a scalar score obtained by mapping the fused relationship vector to a unified space via a trainable weight matrix, projecting it onto the trainable attention vector, and then processing it through an activation function. Since the fused relationship vector is obtained by summing the work order-side projection component, the agent-side projection component, and the edge attribute-side projection component, the edge attribute vector directly participates in the calculation of the unnormalized dynamic attention score, allowing the real-time state of the service relationship to directly affect the distribution of attention weights.

[0066] It should be noted that the formula for calculating the unnormalized dynamic attention score is based on the fact that the correlation strength between work orders and different candidate agents is not uniformly distributed, requiring adaptive allocation of attention resources. Its advantages include the ability to automatically learn the dynamic correlation strength between nodes without manual rules, and computational complexity is linearly related to the number of edges, making it suitable for high-concurrency real-time work order dispatch scenarios.

[0067] Furthermore, the scores of all candidate agents are normalized using the Softmax function to obtain normalized dynamic graph attention weights. These weights reflect the degree of attention the work order node pays to each candidate agent in the current network layer; a higher weight indicates that the agent is more suitable to accept the work order in the current state.

[0068] S304, based on the normalized dynamic graph attention weights, perform weighted updates on the work order node vector and the agent node vector.

[0069] Specifically, S304 enables bidirectional information flow and aggregation. Work order nodes absorb the status information of their neighboring agents through weighted summation, thereby gaining an understanding of "which agents are available" and "the current busy level of these agents." Conversely, agent nodes also aggregate the work order request information they are connected to, updating their own understanding of the current task queue.

[0070] S305, based on the updated work order node vector and agent node vector, correct the edge attribute vector.

[0071] Specifically, the difference between this embodiment and static graph matching technology lies in the fact that the values ​​of edge attributes are dynamically modified during the graph propagation process. In traditional graph neural networks, edge attributes are usually considered as fixed inputs that do not change with the number of network layers. However, in actual customer service scenarios, service relationships are highly dynamic.

[0072] For example, when a service node senses that its load is about to reach saturation through graph propagation, its "effective service capacity" with work orders to be assigned should be reduced accordingly. To this end, this embodiment designs an edge attribute adaptive correction mechanism.

[0073] Specifically, the calculation logic is as follows: the work order node vector and the agent node vector from the previous layer are concatenated, transformed using a trainable edge attribute correction matrix, and then processed by a linear rectified activation function to obtain the correction increment. Subsequently, this correction increment is added to the edge attribute vector from the previous layer to obtain the updated edge attribute vector for the current layer. The activation function ensures that the correction amount is non-negative or has a specific directionality, which helps reduce meaningless oscillations in relationship strength.

[0074] S306. Repeat steps S302 to S305 until the preset number of layers is reached, and input the node vector of the final layer into the relationship probability prediction layer to obtain the dynamic matching probability.

[0075] The preset number of layers can be either 2 or 3.

[0076] Specifically, the multi-layer stacking allows the model to capture higher-order neighborhood information, such as indirect connections like "Agent A is busy, but their colleague B has just released capacity and their skills complement each other." The specific prediction process is as follows: the work order node vector of the final layer is concatenated with the agent node vector, and a fully connected transformation consisting of a trainable weight matrix and biases is input to obtain a scalar score. This scalar score is then input into a sigmoid activation function, mapping it to a dynamic matching probability between 0 and 1. The closer the value is to 1, the higher the effectiveness of the agent in handling the work order in the current real-time environment.

[0077] S307, based on dynamic matching probability, forms a dynamic order dispatch candidate set.

[0078] It should be noted that the dynamic dispatch candidate set output in this embodiment is not the final dispatch decision result, but a high-quality action space filtered by a dynamic graph. The system typically retains agents with a matching probability higher than the matching probability threshold or those ranked in the top K within this candidate set, while removing agents who, despite possessing static skills, have insufficient actual service capacity at the current moment due to factors such as load, SLA, or emotional state. This candidate set defines the legal action boundaries for subsequent reinforcement learning decisions, enabling the reinforcement learning agent to explore long-term optimal strategies within a feasible and safe scope. This reduces ineffective exploration while ensuring the real-time performance and reliability of dispatch decisions.

[0079] The matching probability threshold can be set to 0.7 or 0.8.

[0080] Furthermore, the specific dimensions of the edge attribute vector can be flexibly configured according to the business scenario. In addition to the load, SLA, and sentiment adaptation mentioned above, it can also include the historical average processing time for this type of problem by agents, the satisfaction rating of the most recent N services, and the average urgency of the current queued work orders. Moreover, the nonlinear activation function in the edge attribute correction formula can be replaced with other forms such as GELU (Gaussian Error Linear Unit) or Swish, depending on the training effect, as long as it can achieve the function of adaptively adjusting edge attributes based on node state. This flexibility allows the technical solution of this invention to adapt to the differentiated operational rules and service quality requirements of different customer service centers.

[0081] S4 generates and executes a dispatch instruction within the action boundaries defined by the dynamic dispatch candidate set, combining service performance indicator values.

[0082] In one possible implementation, S4 specifically includes: S401, construct reward and evaluation rules for the customer service field based on service performance indicator values.

[0083] Specifically, in order to guide the model to learn strategies that align with the long-term interests of the business, this embodiment designs a customer service domain reward evaluation rule, namely a comprehensive multi-dimensional reward function, the specific formula of which is as follows:

[0084] in, Let represent the multidimensional reward function corresponding to the t-th order dispatch decision. This represents the result of the first resolution in the t-th instance, used to measure whether the work order is effectively resolved in the first service call, thus avoiding duplicate calls. This indicates customer satisfaction results, reflecting users' subjective evaluation of the service. This represents the achievement result completed within the service level time limit for the t-th time, ensuring service timeliness. This represents the benefit of the t-th order dispatch to agent load balancing, encouraging the system to allocate work orders to agents with lower loads to maintain overall stability. This represents the penalty for the t-th transfer, used to suppress work order flow caused by skill mismatch or improper handling. This represents the penalty for exceeding the service level limit for the tth time, to prevent service timeouts. This represents the penalty for the target agent refusing to accept the order in the t-th instance, reducing the interference of invalid order dispatches on agents. to These represent the non-negative weights of the seven service performance indicators mentioned above, which are used to flexibly adjust the contribution of each indicator according to the focus of the actual business stage (such as focusing on SLA during major promotions and focusing on FCR in daily operations).

[0085] in, Specifically, it can be 0.3 or 0.4. Specifically, it can be taken as 0.2 or 0.15. Specifically, it can be taken as 0.2 or 0.25. Specifically, it can be 0.1 or 0.15. Specifically, it can be 0.2 or 0.3. Specifically, it can be 0.3 or 0.4. Specifically, 0.2 or 0.3 can be used.

[0086] It's important to note that the multidimensional reward function is based on the premise that customer service quality is the result of the combined effect of multiple mutually constraining indicators, and a single indicator cannot accurately depict the true service benefits. The principle involves assigning non-negative weights to positive and negative penalty indicators, then linearly weighting and summing them, transforming multi-objective evaluation into a single scalar optimization signal. The advantage lies in directly aligning the optimization direction of reinforcement learning with business objectives, and the weights can be flexibly configured according to different business stages, avoiding short-sighted behavior caused by optimizing a single indicator.

[0087] Before being included in the calculation, each service performance indicator is normalized to the range of [0,1]. The results of a single resolution and customer satisfaction are taken as a ratio, the service level achievement results are taken as a ratio of the time limit achievement, and the penalties for transfer, timeout and rejection are normalized according to the corresponding business upper limit. Therefore, the output of the multidimensional reward function is a dimensionless scalar, which makes the dimensions of each weighted item consistent.

[0088] It should be noted that the aforementioned service performance metrics are post-event results that can only be determined during or after service execution. This step uses them as reward signals for reinforcement learning, rather than as direct decision inputs at the time of order dispatch. During the training phase, each metric value is taken from the service records of completed historical work orders. The reward value for the corresponding dispatch action is calculated according to the reward evaluation rules of this step, constructing a state-action-reward training sample to update the parameters of the Q-network. In the application phase, the aforementioned metric values ​​for the current work order to be dispatched have not yet been generated at the time of dispatch. The actual values ​​are backfilled after the service is completed, serving as incremental samples for continuous updates to the Q-network.

[0089] S402 constructs a random noise matrix by factoring Gaussian noise.

[0090] Specifically, in order to make the exploration behavior state-dependent and computationally efficient, this embodiment adopts a factorized Gaussian noise generation mechanism. The specific calculation logic is as follows: the factorized random noise corresponding to any position in the weight matrix is ​​obtained by multiplying two independently sampled standard Gaussian random variables corresponding to the output dimension and the input dimension respectively after nonlinear transformation.

[0091] Furthermore, the specific computational logic of the nonlinear transformation is as follows: the sign of the input variable is taken, the square root of its absolute value is calculated, and the two are multiplied together. This transformation function preserves the sign direction of the original noise while compressing the amplitude of extreme values ​​through square root operation, resulting in a more stable noise distribution. Compared to independently sampling noise for each network parameter, the factorization method only needs to generate basic random variables of the same input and output dimensions to reconstruct the entire noise matrix, reducing computational overhead and storage requirements. Simultaneously, it maintains the correlation structure of noise between parameters, which is beneficial for achieving consistent exploration under complex conditions.

[0092] S403 inputs the current order dispatch decision state into an online Q-network with parameterized noise, and perturbs the network parameters of the online Q-network through the elements in the random noise matrix to obtain the network layer output.

[0093] Specifically, the computation process of the noisy network layer is as follows: First, the scale matrix of the weights is multiplied by the corresponding elements of the random noise matrix, and the result is added to the mean matrix of the weights to obtain the noisy weight matrix. Next, the noisy weight matrix is ​​multiplied by the feature input of the current dispatch decision state. Simultaneously, the scale vector of the bias is multiplied by the corresponding elements of the random noise vector, and the result is added to the mean vector of the bias to obtain the noisy bias vector. Finally, the result of the aforementioned multiplication operation is added to the noisy bias vector to obtain the final network layer output.

[0094] It's important to note that both the mean and scale matrices of the weights are trainable parameters. The mean and scale vectors of the biases correspond to the bias parameters. In this way, noise is directly injected into the network parameters rather than the input data, and the actual impact of the noise is adaptively adjusted by the learnable scale parameters. This means that in the early stages of network training, the model can explore extensively using larger scale parameters. As training converges, the scale parameters automatically decrease to reduce interference with deterministic policies. More importantly, since the noise is calculated based on the current state, the perturbation pattern experienced by the network differs in different states, thus achieving "state-dependent exploration" and avoiding the pitfalls of overexploration in some states and underexploration in others.

[0095] It should be noted that the online Q-network and the target Q-network have the same structure, both including an input layer, a parameterized noise hidden layer, a Dueling branch layer, and an output layer. The input layer receives the feature input of the current dispatch decision state. The parameterized noise hidden layer is connected to the input layer, and performs a nonlinear transformation after perturbing the weights and biases with factorized Gaussian noise. The Dueling branch layer is connected to the parameterized noise hidden layer, splitting the hidden features into a state value branch and an action advantage branch. The output layer is connected to the Dueling branch layer, and synthesizes the action value of each legal dispatch action by adding a corrected advantage to the state value and outputs it. The training process of the dispatch decision model includes: initializing the online Q-network and the target Q-network and making their parameters the same. Training batches are extracted from the priority experience replay pool, with an n-step Double DQN target value as the regression target, and the gradient of the online Q-network is updated with the temporal difference error loss corrected by importance sampling weights. Every preset update period, the parameters of the online Q-network are copied to the target Q-network, and the above process is repeated until convergence. Specific training details are shown in steps S601 to S605.

[0096] Furthermore, the parameter update method of the target Q network can be periodic hard copy or soft update. When using soft update, the target Q network parameters are slid slightly towards the online Q network parameters according to the update coefficient. The update coefficient can be 0.005 or 0.01, and the preset update period can be 200 steps or 500 steps.

[0097] Specifically, the current dispatch decision state is constructed by concatenating the work order-side features (work order urgency, remaining service level time limit, and current waiting time, all normalized to [0,1] according to their respective preset upper limits) and the work order-agent-side features (dynamic matching probability, agent real-time load ratio, historical one-time resolution rate, average processing time, and skill matching label, with values ​​assigned to each candidate agent in the dynamic dispatch candidate set) in a fixed order. Each candidate agent corresponds to a state vector, which is input into the Q-network to obtain the corresponding action value. The feature composition, dimensions, arrangement order, and encoding method of the state vector are completely consistent during the training and online inference phases.

[0098] S404, based on the customer service domain reward evaluation rules and network layer output, evaluates the value of actions through the Dueling structure, selects the action with the highest long-term action value from the set of legal dispatch actions in the dynamic dispatch candidate set, generates a dispatch instruction and executes it.

[0099] Specifically, when there are two or more candidate actions with the highest combined action value in the dynamic dispatch candidate set, a unique dispatch instruction is generated according to the following fixed priority rules: First, the action corresponding to the candidate agent with the lowest real-time agent load ratio is selected. If the load ratios are still tied, the action corresponding to the first candidate agent is selected after sorting by agent unique identifier in ascending order. Since the agent unique identifiers are all different, the target action obtained after the above rules is guaranteed to be unique. Furthermore, if the dynamic dispatch candidate set obtained after filtering by the S307 matching probability threshold and business constraints is an empty set, that is, there are no legal dispatch actions, then this step skips action selection, does not generate this dispatch instruction, and directly triggers the manual fallback mechanism, transferring the current work order to the manual customer service queue for allocation and processing. This situation occurs before the first dispatch, and the current work order has not entered the post-dispatch monitoring process, so the S5 closed-loop redispatch mechanism is not applicable.

[0100] It should be noted that the scope of action selection is strictly limited to the dynamic order dispatch candidate set and the set of legal order dispatch actions filtered by hard business constraints, ensuring the safety and compliance of the decision-making process. The Q-value decomposition of the Dueling structure is achieved by combining the state value function with the action advantage function.

[0101] Specifically, the total action value equals the state value plus a adjusted action advantage value, where the adjusted action advantage value is calculated by subtracting the maximum advantage value among all possible actions in the current state from the advantage value of the action to be evaluated. In this calculation, the state value function is used to evaluate the baseline level of reward obtained under the current overall customer service system environment (e.g., peak periods, downtime). The action advantage function is used to evaluate the quality of performing a specific action relative to the average level. Subtracting the maximum advantage value ensures the numerical centering of the advantage function, thereby enhancing training stability.

[0102] For example, when the system is extremely busy, the expected absolute returns of all actions are low. State value can accurately capture this global low point, while action advantage can identify the relatively better actions under the current predicament (such as "waiting" being better than "forced dispatch"). Ultimately, the system selects the action with the highest total action value, parses it into the target agent identifier, processing priority, and dispatch method, generates a dispatch instruction, and issues it for execution.

[0103] It should be noted that the metrics in the reward function are not fixed. For example, in specific business scenarios, metrics such as "Average Processing Time (AHT)," "First Response Time," or "Cross-Selling Success Rate" can be introduced, and the weight configuration adjusted accordingly. Similarly, the underlying distribution in factorized noise can be replaced from a standard Gaussian distribution to a uniform or Laplace distribution, as long as the purpose of parameter perturbation is achieved. Furthermore, in addition to using the maximum value operation, the advantage aggregation term in the Dueling structure can also use the average value operation to adapt to different training dynamics. These variations all fall within the protection scope of the technical solution of this application.

[0104] S5 responds to abnormal states during the execution of a dispatch instruction, updates the dynamic graph, triggers closed-loop redispatch, and generates a second dispatch instruction.

[0105] It should be noted that S5 aims to solve the problem that static one-time work order dispatch cannot cope with sudden anomalies in actual service. Through a complete closed loop of real-time monitoring, failure marking, dynamic graph updates and secondary decision-making, it ensures that a work order can still quickly find a suitable service provider after the first dispatch fails. At the same time, it sets up a manual backup as a service safety boundary.

[0106] In one possible implementation, S5 specifically includes: The S501 continuously monitors the target agent's order acceptance status, online status, number of concurrent work orders in real time, and remaining SLA time to obtain real-time monitoring data.

[0107] Specifically, the continuous monitoring here is not a simple periodic polling, but a hybrid mechanism combining event-driven and heartbeat detection. For example, the system subscribes to real-time event streams such as check-in / check-out and answer / hang-up from the agent workstation via a message queue, while simultaneously collecting heartbeat packets from agent terminals at a second-level frequency to verify their actual online status. The number of concurrent work orders in real time is read directly from the agent task queue, while the remaining SLA time is dynamically calculated based on the work order creation time, committed service duration, and current system time.

[0108] S502: When real-time monitoring data detects that the target agent has refused to accept the call, is offline, has exceeded the preset limit for load, has exceeded the order acceptance waiting time, or has initiated an unprocessable transfer, the current order dispatch action will be marked as a failed action.

[0109] Specifically, the above abnormal states cover the main service interruption scenarios in customer service: Rejection includes the agent actively clicking the reject button or not responding within the preset ringing time. Offline refers to loss of heartbeat or a status change to busy / rest. Load exceeds preset limit means the current number of concurrent work orders exceeds the maximum capacity configured for the agent's skill group. Exceeding order acceptance waiting time means the time from the issuance of the order dispatch instruction to the agent's confirmation of acceptance exceeds the system's tolerance threshold. Initiating an unprocessable transfer occurs when the agent, after previewing the work order, discovers their own inadequacy and proactively returns it.

[0110] The maximum capacity can be set to 5 or 8 units, and the system tolerance threshold can be set to 30 or 60 seconds.

[0111] Furthermore, these situations are uniformly abstracted into failure actions and written into a blacklist. The core technical effect is to provide clear negative feedback constraints for the reinforcement learning decision-making module, preventing the agent from repeatedly choosing paths that have been proven unusable due to value estimation lag during reassignment, thereby avoiding falling into an infinite loop of invalid dispatch.

[0112] S503 updates the edge attributes in the dynamic graph based on real-time monitoring data.

[0113] Specifically, step S503 is crucial for the closed-loop reassignment to take effect. Unlike the initial graph construction in step S3, the update here is based on incremental corrections of real-time status. For example, if the target agent is detected to be offline, the system directly sets the availability component in the edge attribute vector between that agent node and all associated work order nodes to zero or a minimum value. If the load exceeds the limit, the service capacity weight in the edge attribute is proportionally reduced. If the remaining SLA time is tight, the urgency factor in the edge attribute is increased. This update method does not require reconstructing the entire graph structure; it only requires modifying the attribute vectors of the affected edges, resulting in extremely low computational overhead and meeting the stringent real-time requirements of reassignment.

[0114] S504: Re-input the updated edge attributes into the dynamic graph for graph propagation to generate the current dynamic dispatch candidate set.

[0115] It should be noted that since the edge attributes have been corrected in step S503, when these updated attributes participate in the calculation of the fusion relation vector and the attention weight normalization described in step S3, the attention weight corresponding to the abnormal agent will naturally decrease or even become zero. Therefore, the current dynamic dispatch candidate set output after a new round of graph propagation has actually automatically eliminated those agents who, although possessing static skills, are unable to provide effective service at the current moment due to abnormalities. This process reflects the adaptive advantage of the dynamic graph architecture, that is, driving the evolution of the upper-layer matching results through changes in the underlying relation attributes, rather than relying on external hard-coded filtering rules.

[0116] S505 makes a dispatch decision based on the current dynamic dispatch candidate set, and masks failed actions during the decision-making process to generate a secondary dispatch instruction.

[0117] Specifically, during reinforcement learning decision-making, the system sets the action mask corresponding to the failed actions marked in step S502 to a prohibited state, for example, setting the corresponding logical value to negative infinity before Softmax. This means that even if a failed action still has a high evaluation value in the Q-network, it will not be selected. The final generated secondary dispatch instruction is the optimal solution determined based on the principle of maximizing long-term service revenue within the remaining legal action space after excluding all known risk options. This dual guarantee mechanism—soft screening at the dynamic graph level and hard masking at the decision level—improves the success rate and robustness of re-dispatch.

[0118] It should be noted that, in order to deal with the failure of automatic order dispatch in extreme cases, this embodiment also introduces a manual backup mechanism.

[0119] In one possible implementation, S5 further includes: S506: When a candidate agent who meets the hard constraints cannot accept the task or the remaining SLA time of the current work order is lower than the manual backup threshold, the current work order will be transferred to the manual backup queue with the corresponding business permissions, and the reason for triggering the manual backup will be recorded.

[0120] Specifically, a candidate agent who cannot accept a call due to hard constraints means that the current dynamic dispatch candidate set generated after step S504 is empty, or all agents in the candidate set are unable to be assigned a call due to lack of skills, insufficient permissions, or being completely inactive. The manual backup threshold is usually set to a stricter time limit than the standard SLA, such as less than 30 seconds remaining, to ensure there is still a buffer after manual intervention. Recording the triggering reasons (such as full staff availability, skill gaps, continuous call rejections, etc.) is not only used for operational monitoring and alarms, but also provides valuable anomaly samples for subsequent model joint updates, helping the system identify its blind spots and continuously improve.

[0121] For example, to illustrate the actual operation of the closed-loop reassignment mechanism, suppose the system dispatches an urgent refund work order to agent A during a peak sales period. Within 3 seconds of the dispatch instruction being issued, the monitoring system detects through the event stream that agent A initiated an unprocessable transfer. The system then executes step S502, marking the action as invalid. Step S503 is executed, minimizing the service compatibility of the edge attribute between agent A and the work order. Step S504 is executed, re-running the graph propagation. Because agent A's edge weight is extremely low, agent A is removed from the new dynamic dispatch candidate set, and agent B, which has just been released from load, is included. Finally, step S505 is executed. With agent A disabled, the decision module evaluates that agent B has the highest long-term benefit and generates a secondary dispatch instruction. The entire process, from anomaly detection to new instruction generation, takes less than 200 milliseconds. The customer completes the service resource rematch almost imperceptibly, effectively avoiding SLA default risk.

[0122] Reference Figure 3 The diagram illustrates a closed-loop multi-model joint update provided by an embodiment of the present invention.

[0123] Specifically, Figure 3 The top inputs the actual processing results and service performance metrics, which are then jointly updated and distributed to three parallel paths: the dispatch decision path undergoes n-step discount accumulation, Double DQN target value calculation, temporal differential error prediction, priority sampling, and weight processing to update the online Q-network parameters. The dynamic graph path updates node and edge attributes synchronously, integrates actual matching results, and reflects the actual service performance in reverse. The semantic calibration path updates semantic support samples; when the number of samples reaches a threshold, the prototype is recalculated and the comparison is calibrated; otherwise, no update is performed. After the three models are jointly updated, the feedback is sent to the next round of dispatch, realizing the periodic update and closed-loop feedback of the online Q-network.

[0124] S6 updates the semantic calibration model, dynamic graph, and dispatch decision model based on the actual processing results of the secondary dispatch instructions to complete intelligent customer service dispatch.

[0125] It should be noted that step S6 is the closed-loop endpoint of the entire intelligent dispatch technology chain and the starting point for the system's continuous self-evolution. Its core lies in using the same batch of real service feedback data to synchronously drive the joint optimization of the dispatch decision model, dynamic graph matching model, and semantic calibration model, overcoming the shortcomings of isolated updates and delayed feedback of modules in traditional systems.

[0126] In one possible implementation, S6 specifically includes: S601, based on the actual processing results, according to the preset multi-step length, attenuates and accumulates the actual service rewards corresponding to multiple dispatch decisions to obtain a discount reward.

[0127] Specifically, in customer service scenarios, the consequences of order dispatch often have significant delays. For example, a transfer or timeout may occur minutes or even hours after the order is dispatched. If only instant feedback is relied upon, the model will struggle to establish accurate causal relationships. Therefore, this embodiment employs an n-step reward mechanism to accumulate continuous rewards over a future period. The formula for calculating the discount reward is as follows:

[0128] in, This represents the discount reward calculated starting from step d. Indicates a preset multi-step length. Indicates the first Actual service rewards for each step This represents the future reward discount factor, used to weigh the proportion of immediate and long-term benefits. By introducing n-step accumulation, the system can effectively backpropagate the final evaluation (such as satisfaction level, whether the issue is resolved) at the end of the work order to earlier dispatch decision points, solving the learning challenges of sparse rewards and delayed feedback. Specifically, the discount factor can be 0.9 or 0.99, the multi-step length can be 3 or 5, and the actual service reward is calculated by the multi-dimensional reward function in step S401.

[0129] S602, determine the Double DQN target value based on the discount reward.

[0130] Specifically, to further improve the stability of value estimation and alleviate the overestimation problem, this embodiment adopts the Double DQN architecture. The specific formula for calculating the Double DQN target value is as follows:

[0131] in, This represents the Double DQN target value calculated at step d, which is the regression target signal used to train the Q-network. Indicates the first The state at step n, that is, the state of the environment after n steps. Indicates candidate actions, and ( ) represent the online Q-network and the target Q-network, respectively. This represents the network parameters of the online Q-network at step d. This represents the network parameters of the target Q-network at step d. This means iterating through all possible candidate actions to find the one that maximizes the value of the online Q-network. .

[0132] It should be noted that this formula is based on the fact that a single network simultaneously undertaking both action selection and value evaluation can lead to systematic overestimation bias. The principle is that the initial reward is accumulated with a discount decay, and the subsequent value is determined by the online Q-network selecting the optimal action after n steps, followed by the target Q-network evaluating the value of that action; the two are decoupled. The advantage is that it significantly alleviates the problem of inflated Q-values, making the order dispatch strategy learning more robust and achieving higher convergence quality.

[0133] S603 calculates the timing difference error based on the Double DQN target value.

[0134] Specifically, temporal difference error reflects the gap between the current estimate and the target value, and is a direct indicator of the learning value of experience samples. The specific calculation method is as follows: subtract the current estimate of the actual actions in historical states by the online Q-network based on the current parameters from the n-step Double DQN target value corresponding to the experience sample. The result is the temporal difference error of that experience sample. The larger the absolute value of the error, the more information the sample contains that is unexpected by the model, and the higher its value in correcting cognitive biases in the model.

[0135] S604 determines the basic priority of historical order dispatch experience based on the time-series differential error, and extracts training batches based on the basic priority.

[0136] Specifically, in massive customer service logs, the vast majority of samples are routine successful order dispatches, while the true determining factor for the system's upper limit is often the rare failure cases or extreme boundary situations. If uniform sampling is used, these high-value samples are easily overwhelmed. Therefore, this embodiment introduces a priority experience replay mechanism.

[0137] Furthermore, the calculation logic for the basic priority is as follows: take the absolute value of the temporal difference error corresponding to the empirical sample, and add a positive stabilizing term to obtain the basic priority of the empirical sample in the priority empirical replay pool. The positive stabilizing term is used to prevent samples with zero error from completely losing the opportunity to be sampled. The positive stabilizing term can be 0.01 or 0.001.

[0138] Furthermore, the extraction probability is determined based on the basic priority. The calculation process is as follows: the basic priority of the experience sample is raised to the power of the priority sampling degree control coefficient to obtain the numerator. At the same time, the basic priorities of all valid experience samples in the experience replay pool are raised to the same power and summed to obtain the denominator. The value obtained by dividing the numerator by the denominator is the probability that the experience sample is extracted from the priority experience replay pool.

[0139] The priority sampling control coefficient typically ranges from 0 to 1, and can be set to 0.4 or 0.6. This is used to smooth the priority distribution and avoid excessive bias towards high-error samples. It should be understood that because non-uniform sampling alters the data distribution, importance sampling weights must be introduced to correct the gradient in subsequent network updates to eliminate estimation bias and ensure convergence to the correct optimal strategy.

[0140] Furthermore, the correction process is as follows: The inverse of the sampling probability of each empirical sample in the training batch is multiplied by the number of valid empirical samples to obtain the original importance weights. Each original importance weight within a batch is divided by its maximum value for normalization, resulting in the corrected importance sampling weights. During gradient updates, the loss term corresponding to the temporal difference error of each sample is multiplied by the corrected weight of that sample, appropriately compressing the contribution of high-frequency samples while retaining the contribution of low-frequency, high-value samples, thereby restoring an unbiased data distribution.

[0141] S605 updates the order dispatch decision model by training batches.

[0142] Specifically, using the training batches that have undergone priority filtering and weight correction, the parameters of the online Q-network are updated using gradient descent, gradually approximating the true long-term service value function. This process enables the order dispatching decision strategy to continuously iterate based on real business feedback.

[0143] S606, based on the actual processing results, update the node vectors and edge attributes of the actual processing agents in the dynamic graph.

[0144] It should be noted that, in addition to updating the reinforcement learning strategy, this embodiment also emphasizes the synchronous optimization of the underlying matching and semantic modules, which is the key to the joint update. Unlike the temporary edge attribute correction performed in step S5 to deal with anomalies, the update here is a persistent capability profile evolution.

[0145] Specifically, after a work order is processed, the system incrementally updates or updates the corresponding capability dimensions in the agent's node vector based on the agent's performance (such as processing time, satisfaction rating, and whether the call was transferred). For example, if an agent successfully processes a new type of complex work order for the first time, the feature components related to that type of work order in their node vector will be enhanced. At the same time, the edge attribute statistics between the agent and various types of work orders (such as historical average FCR and mean sentiment fit) will also be recalculated and persistently stored.

[0146] S607 updates the semantic support samples based on the true intent category, and updates the semantic calibration model based on the updated semantic support samples when preset conditions are met, so as to complete the intelligent customer service order dispatch.

[0147] Specifically, this is the final link in the closed-loop chain, designed to cleanse and enhance semantic understanding capabilities by using manually verified or corrected true intent labels.

[0148] Specifically, when an agent confirms the target intent is correct, the system adds the semantic vector of the current work order as a new positive sample to the support sample set of the corresponding intent category. If the agent corrects the intent, the semantic vector is assigned to the corrected true category, and the original recognition result is marked as a misjudged sample. When the number of newly confirmed samples for a certain intent category reaches a preset threshold (which can be 50 or 100), or when a recent downward trend in the recognition accuracy of that category is detected, the system automatically triggers the recalculation process of the noise-resistant semantic prototype: recalculating the intra-class average consistency of all support samples (including newly added samples) within the category, updating the reliability weights, and generating a new noise-resistant semantic prototype. This mechanism ensures that the semantic calibration model can adaptively adjust with business development and changes in user expression habits, always maintaining its ability to suppress noise and generalize to new intents.

[0149] It should be noted that the triggering time for the above joint update can be executed in real time at the end of each work order, or it can be executed in batches at fixed time periods (such as every hour or every day). The specific timing can be flexibly configured according to the system's computing resources and business real-time requirements.

[0150] System Implementation Examples Reference Figure 4 The diagram shows a structural schematic of a large-scale customer service intelligent dispatch system that integrates semantic understanding, provided by an embodiment of the present invention.

[0151] This invention proposes a large-scale customer service intelligent dispatch system 30 that integrates semantic understanding, comprising: a memory 303 and a processor 301. The processor 301 and the memory 303 are connected, for example, via a bus 302.

[0152] The memory 303 stores an application program adapted to be executed by the processor 301 to implement the large-model customer service intelligent dispatching method of the method embodiment, which incorporates semantic understanding.

[0153] The structure of the large-scale customer service intelligent dispatch system 30 that integrates semantic understanding does not constitute a limitation on the embodiments of the present invention.

[0154] Processor 301 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0155] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI bus or an EISA bus, etc. Bus 302 may be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not mean that there is only one bus or one type of bus.

[0156] The memory 303 may be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A large-scale customer service intelligent order dispatching method integrating semantic understanding, characterized in that, The large-scale customer service intelligent dispatching method that integrates semantic understanding includes: S1: Obtain multi-round interaction data from the customer, perform time alignment, and obtain the session context data packet; S2, input the session context data packet into the large language model, perform semantic understanding and noise reduction calibration, and generate a robust work order semantic representation; S3. Based on the robust work order semantic representation, construct a dynamic graph of work orders and candidate agents, and update the node and edge attributes of the dynamic graph through graph propagation to form a dynamic dispatch candidate set. S4. Within the action boundary defined by the dynamic dispatch candidate set, a dispatch instruction is generated and executed by combining the service effect index value. S5, in response to an abnormal state during the execution of the first dispatch instruction, update the dynamic graph and trigger closed-loop redispatch to generate a second dispatch instruction. S6. Based on the actual processing result of the secondary dispatch instruction, update the semantic calibration model, the dynamic graph, and the dispatch decision model to complete intelligent customer service dispatch.

2. The large-scale customer service intelligent order dispatch method integrating semantic understanding as described in claim 1, characterized in that, S1 specifically includes: S101, align the current interaction data, the historical round data, and the customer business-related data according to the order of the current interaction data, the historical round data, and the customer business-related data by time. S102, clean up duplicate text, blank text and invalid system prompts in the time-aligned data, and retain negative words, degree adverbs, time words and business entities; S103, Combine the preset field structure to encapsulate the cleaned data to obtain the session context data packet.

3. The large-scale customer service intelligent order dispatch method integrating semantic understanding as described in claim 1, characterized in that, S2 specifically includes: S201, Input the session context data packet into the large language model to obtain the candidate intent and the current work order semantic vector; S202, based on the candidate intent and the current work order semantic vector, for each intent category, obtain supporting samples and corresponding semantic vectors from historical work orders, and calculate the intra-class average consistency among supporting samples within the same intent category. S203, based on the intra-class average consistency, perform Softmax normalization with temperature parameter to obtain the reliability weight of each support sample, and based on the reliability weight, perform weighted aggregation on the semantic vector corresponding to each support sample to form the noise-resistant semantic prototype of the intent category. S204, calculate the distance between the current work order semantic vector and the noise-resistant semantic prototype, and convert the distance into an intent category calibration probability; S205, Based on the calibration probability of the intent category, determine the target intent and intent confidence; S206, the target intent, the intent confidence, and the current work order semantic vector are uniformly encoded to generate the robust work order semantic representation.

4. The large-scale customer service intelligent order dispatching method integrating semantic understanding as described in claim 3, characterized in that, S2 further includes: S207, when the confidence level of the intent is lower than a preset low confidence threshold, select a preset number of candidate intents whose intent categories are ranked first by calibration probability, and deduplicate and merge the business skills corresponding to the multiple candidate intents to form an extended skill set.

5. The intelligent customer service order dispatching method based on a large-scale model integrating semantic understanding as described in claim 1, characterized in that, S3 specifically includes: S301, based on the robust work order semantic representation, set the current work order as a work order-side node and the candidate agent as an agent-side node, establish candidate service relationship edges, and initialize the work order node vector, agent node vector, and edge attribute vector. S302, Project the work order node vector, the agent node vector, and the edge attribute vector onto the common relationship representation space, and calculate the fused relationship vector; S303, Based on the fusion relationship vector, calculate the unnormalized dynamic attention score, and perform normalization processing on the unnormalized dynamic attention score to obtain the normalized dynamic graph attention weight; S304, based on the attention weight of the normalized dynamic graph, the work order node vector and the agent node vector are updated with weighted average; S305, The edge attribute vector is corrected based on the updated work order node vector and agent node vector; S306, Repeat steps S302 to S305 until the preset number of layers is reached, and input the node vector of the final layer into the relationship probability prediction layer to obtain the dynamic matching probability; S307, Based on the dynamic matching probability, form the dynamic dispatch candidate set.

6. The intelligent customer service order dispatching method based on a large-scale model integrating semantic understanding as described in claim 1, characterized in that, S4 specifically includes: S401, Based on service performance indicators, construct reward and evaluation rules for the customer service field; S402, constructs a random noise matrix by factoring Gaussian noise; S403, input the current order dispatch decision state into the online Q network with parameterized noise, and perturb the network parameters of the online Q network through the elements in the random noise matrix to obtain the network layer output; S404, based on the customer service domain reward evaluation rules and the network layer output, evaluate the value of the action through the Dueling structure, select the action with the highest long-term action value from the set of legal dispatch actions in the dynamic dispatch candidate set, generate the dispatch instruction and execute it.

7. The large-scale customer service intelligent order dispatch method integrating semantic understanding as described in claim 1, characterized in that, S5 specifically includes: S501 continuously monitors the target agent's order acceptance status, online status, number of concurrent work orders in real time, and remaining SLA time to obtain real-time monitoring data; S502, when the target agent is detected to have refused to accept the call, gone offline, exceeded the preset limit due to overload, exceeded the order waiting time, or initiated an unprocessable transfer based on the real-time monitoring data, the current order dispatch action is marked as a failed action. S503, Update the edge attributes in the dynamic graph based on the real-time monitoring data; S504, The updated edge attributes are re-input into the dynamic graph for graph propagation to generate the current dynamic dispatch candidate set; S505, based on the current dynamic order dispatch candidate set, make an order dispatch decision, and during the decision-making process, block the failed action and generate the secondary order dispatch instruction.

8. The large-scale customer service intelligent order dispatch method integrating semantic understanding according to claim 7, characterized in that, The S5 also includes: S506, when a candidate agent who meets the hard constraint cannot accept the task or the remaining SLA time of the current work order is lower than the manual backup threshold, the current work order is transferred to the manual backup queue with the corresponding business authority, and the reason for triggering the manual backup is recorded.

9. The large-scale customer service intelligent order dispatch method integrating semantic understanding according to claim 1, characterized in that, S6 specifically includes: S601, Based on the actual processing results, according to the preset multi-step length, the actual service rewards corresponding to multiple dispatch decisions are attenuated and accumulated to obtain a discount reward; S602, Determine the Double DQN target value based on the aforementioned discount reward; S603, Calculate the timing difference error based on the Double DQN target value; S604, Based on the time-series differential error, determine the basic priority of historical order dispatch experience, and extract training batches based on the basic priority; S605, Update the dispatch decision model using the training batch; S606, Based on the actual processing result, update the node vector and edge attributes of the actual processing agent in the dynamic graph; S607, Combine the true intent category, update the semantic support sample, and when the preset conditions are met, update the semantic calibration model based on the updated semantic support sample to complete the intelligent customer service order dispatch.

10. A large-scale customer service intelligent order dispatch system integrating semantic understanding, characterized in that, include: Memory and processor; The memory stores an application program adapted to be executed by the processor to implement the large-model customer service intelligent dispatching method according to any one of claims 1 to 9.

Images