Government customer service method, device, equipment, system, medium and product
Patent Information
- Application Number
- CN202510701065.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-28
- Publication Date
- 2026-02-10
Smart Images
Figure CN121504474A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment, system, medium and product for providing government customer service. Background Technology
[0002] Given the public's urgent need for efficient and convenient services, traditional manual windows and single question-and-answer models are insufficient to handle complex government consultation scenarios involving multiple regions and matters. Therefore, intelligent government customer service systems have emerged.
[0003] Current government customer service systems plan tasks based on user questions and execute them using fixed plans, or employ simple voting mechanisms to decide whether to adjust the plan and execute it, ultimately obtaining a response to the user's question. Specifically, a large language model agent votes on the planned solutions; if the number of votes in favor meets a certain condition, the plan is adjusted and executed. Both of these methods are prone to accumulating task deviations, failing to guarantee the overall task completion effect and affecting the accuracy of the responses. Summary of the Invention
[0004] This application provides a government customer service method, device, equipment, system, medium, and product to solve the problem in the prior art that task deviations are prone to accumulate, making it impossible to guarantee the overall task completion effect and affecting the accuracy of responses.
[0005] To achieve the above objectives, this application provides a government customer service method, including:
[0006] Get the user's current input text;
[0007] The input text is subjected to intent recognition. If the result of intent recognition is related to government affairs issues, task planning is performed based on the input text to obtain a planning scheme. The planning scheme includes a first agent corresponding to each subtask, ordered according to the execution steps.
[0008] According to the planning scheme, the subtasks in the planning scheme are assigned to the corresponding first slave agents, so that the first slave agents execute the subtasks to be executed in this execution step to generate answers and return the answers;
[0009] The received answer is scored using a pre-trained multi-index scoring model to obtain the current scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set;
[0010] The planning scheme is updated based on the current scoring result, and the process returns to the step of assigning subtasks in the planning scheme to the corresponding first slave agents, until the updated planning scheme is completed. The received answers are then summarized to obtain the response to the input text.
[0011] As an improvement to the above scheme, the multi-index scoring model is trained through the following steps:
[0012] Obtain the first sample set; the first sample set consists of the first question text for each first subtask and each first answer;
[0013] Calculate the score results of each indicator for each first answer and construct an indicator score matrix;
[0014] The proportion of the scoring results of each indicator in the index scoring matrix is used as the weight of the scoring results of the corresponding indicator, and the information entropy of each indicator in the index scoring matrix is calculated.
[0015] Based on the information entropy, construct the first loss function of the multi-index scoring model;
[0016] The multi-index scoring model is obtained by taking the index scoring matrix as input and training it by minimizing the first loss function.
[0017] As an improvement to the above scheme, the step of updating the planning scheme based on the current scoring result and returning to execute the step of assigning subtasks in the planning scheme to the corresponding first slave agents according to the planning scheme continues until the updated planning scheme is completed. The received answers are then summarized to obtain the response to the input text, including:
[0018] If there is no situation where the current score result is less than the first preset threshold, the execution of the first agent is not interrupted, and the process returns to the step of scoring the current answer using a pre-trained multi-index scoring model to obtain the current score result.
[0019] If the current score is less than the first preset threshold, the execution of the first slave agent is interrupted, the planning scheme is updated according to the current score that is less than the first preset threshold, and the execution of the step of allocating the sub-tasks in the planning scheme to the corresponding first slave agents is returned.
[0020] If the planning scheme or the updated planning scheme is completed, the received answers are summarized to obtain the reply of the input text.
[0021] As an improvement to the above scheme, the planning scheme also includes the maximum execution time for each subtask:
[0022] The government customer service methods also include:
[0023] The answer length range of each subtask in the planning scheme is obtained by using a pre-trained answer length prediction model; the answer length prediction model is trained based on a second sample set, which consists of the second question text of each second subtask generated according to the registration information of the first agent and the first agent corresponding to each second subtask;
[0024] Based on the preset generation rate of the first agent and the answer length range, the longest execution time of each subtask in the planning scheme is obtained.
[0025] As an improvement to the above scheme, the answer length prediction model is trained through the following steps:
[0026] Obtain the registration information;
[0027] Based on the registration information, the second question text for each second subtask is generated using the large model of the main agent;
[0028] For each second question text, the corresponding first agent is invoked to answer multiple times, resulting in multiple second answers for each second question text;
[0029] Based on the lengths of multiple second answers, the answer length range of the second question text is obtained;
[0030] A second loss function for the answer length prediction model is constructed based on the difference between the minimum and maximum values; the difference between the minimum values is the difference between the minimum value of the answer length range and the minimum value predicted by the model, and the difference between the maximum values is the difference between the minimum value of the answer length range and the maximum value predicted by the model.
[0031] The second sample set is used as input, and the model is trained using the second loss function to obtain the answer length prediction model.
[0032] As an improvement to the above solution, the government customer service method further includes:
[0033] During the subtask allocation and execution process, an agent exception message is received from the first target agent; the first target agent is the agent in the latest planning scheme and whose waiting time for receiving the third answer is greater than the longest execution time of the preceding subtask, and the third answer is the answer generated by executing the preceding subtask.
[0034] As an improvement to the above scheme, the answer generated by the first agent performing the subtask is obtained by the first agent through the following steps:
[0035] Receive the assigned subtask and extract several first question elements from the question text of the assigned subtask;
[0036] Each retrieval question is generated based on each of the first question elements;
[0037] A database search is performed for each of the search questions to obtain search results for each of the search questions.
[0038] Based on each of the search results, the answer generated by executing the assigned subtask is obtained.
[0039] As an improvement to the above scheme, obtaining the answer generated by executing the assigned subtask based on each of the search results includes:
[0040] The second question element to which each search result belongs is obtained through a pre-trained retrieval evaluation model; the retrieval evaluation model is trained using the search results of the subtask sample set for a multi-classification task.
[0041] Determine whether all elements of the second question include all elements of the first question;
[0042] If so, then all the search results are summarized to obtain the answer generated by the assigned subtask;
[0043] If not, then filter out the first question elements that are not included, regenerate a new search question, and continue the database search.
[0044] As an improvement to the above solution, after receiving the response to the input text, the government customer service method further includes:
[0045] The response is scored using the multi-index scoring model to obtain the score result of the response.
[0046] As an improvement to the above solution, after obtaining the current scoring result, the government customer service method further includes:
[0047] Based on the scoring results, calculate the average score of the first agent in this execution;
[0048] When the average score is less than the second preset threshold, a reminder is issued that the capabilities of the first agent in this execution need to be optimized.
[0049] As an improvement to the above solution, after performing intent recognition on the input text, the government customer service method further includes:
[0050] If the result of the intent recognition is unrelated to the government affairs issue, then the second intelligent agent is invoked to respond to the input text.
[0051] To achieve the above objectives, this application also provides a government customer service device, including:
[0052] The input module is used to obtain the user's current input text;
[0053] An intent recognition module is used to recognize the intent of the input text. If the result of intent recognition is related to government affairs issues, task planning is performed based on the input text to obtain a planning scheme. The planning scheme includes a first agent corresponding to each subtask, ordered according to the execution steps.
[0054] The allocation module is used to allocate the subtasks in the planning scheme to the corresponding first slave agents according to the planning scheme, so that the first slave agents can execute the subtasks to be executed in this execution according to the execution steps to generate answers and return the answers;
[0055] The scoring module is used to score the received answer using a pre-trained multi-index scoring model to obtain the current scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set;
[0056] The update module is used to update the planning scheme according to the current scoring result, and return to execute the step of assigning the sub-tasks in the planning scheme to the corresponding first slave agents according to the planning scheme, until the updated planning scheme is executed, and summarize the received answers to obtain the reply of the input text.
[0057] To achieve the above objectives, this application also provides a government customer service system, including:
[0058] The main intelligent agent is used to execute the aforementioned government customer service methods;
[0059] One or more first agents are used to perform task execution on assigned subtasks.
[0060] To achieve the above objectives, this application also provides a government customer service device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the government customer service method as described above.
[0061] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the government customer service method as described above.
[0062] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the government customer service method as described above.
[0063] Compared with the prior art, the government customer service method, apparatus, equipment, system, medium and product provided in this application embodiment score the received answer by using a pre-trained multi-index scoring model to obtain the current scoring result. Based on the current scoring result, the planning scheme of the user's current input text is updated, avoiding the problem of task deviation accumulation, thereby ensuring the overall task completion effect and improving the accuracy of the response. Attached Figure Description
[0064] Figure 1 This is a flowchart of a government customer service method provided in an embodiment of this application;
[0065] Figure 2 This is a structural block diagram of a government customer service device provided in an embodiment of this application;
[0066] Figure 3 This is a structural block diagram of a government customer service device provided in an embodiment of this application. Detailed Implementation
[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0068] See Figure 1 , Figure 1 This is a flowchart of a government customer service method provided in an embodiment of this application. The government customer service method includes:
[0069] S1. Obtain the user's current input text;
[0070] Specifically, the input text can be information entered by the user in the form of voice and / or text information.
[0071] S2. Perform intent recognition on the input text. If the result of intent recognition is related to government affairs issues, perform task planning based on the input text to obtain a planning scheme. The planning scheme includes sub-tasks ordered according to execution steps and a first slave agent corresponding to each sub-task.
[0072] Understandably, by performing semantic understanding on the input text, the intent of the input text can be identified, thereby determining whether the intent identification result is relevant to the government affairs issue; if relevant, task planning is performed based on the input text to obtain the planning scheme T = {(t i ,a i )}, i = 1, 2, ..., n, thus providing government customer service to users. Where t i Indicates a subtask i This indicates that subtask t needs to be executed. i The first step is from the intelligent agent. Generally speaking, a subtask t i This includes a question text, which contains multiple question elements, utilizing a first agent a. i Execute subtask t i Thus, the subtask t is obtained. i The answer to the question text, i.e., the answer to the subtask t i The answer is yes. First, intelligent agents can include any intelligent agent with basic or professional capabilities in the field of government affairs, such as intelligent agents for question answering, policy analysis, general knowledge, document generation, and network search, etc.
[0073] For example, the planning scheme T = {<(t1,a1), (t2,a2)>, (t3,a3), (t4,a4), ..., (t n ,a n )}, <(t1,a1),(t2,a2)> indicates that subtasks t1 and t2 can be executed synchronously and in parallel, which is the first step of the planning scheme T. Subtasks t1 and t2 are the subtasks to be executed in the first step, and the first slave agents a1 and a2 are the first slave agents that need to be executed in the first step; {(t i ,a i ), (t i+1 ,a i+1 )} represents subtask t i With subtask t i+1 It is executed sequentially, meaning that only one subtask is executed at a time, and subtask t i+1 It depends on the preceding subtask t iThe execution result, i.e., (t3, a3), is the second step of the planning scheme T. Subtask t3 is the subtask to be executed in the second step, and the first slave agent a3 is the first slave agent to be executed in the second step. (t4, a4) is the third step of the planning scheme T. Subtask t4 is the subtask to be executed in the third step, and the first slave agent a4 is the first slave agent to be executed in the third step. n ,a n This is the final step in the planning scheme T, subtask t. n This is the final subtask to be executed, starting with agent a. n This refers to the first agent that needs to perform the final step; where the first step, the second step, ..., the last step represent the execution steps.
[0074] Furthermore, the step of performing task planning based on the input text to obtain a planning scheme includes: rewriting the user's current input text by combining the user's historical input text to obtain a rewritten input text, and using the rewritten input text to perform task planning to obtain a planning scheme. Optionally, the user's historical input text includes: administrative division information, historical questions, etc.
[0075] This not only avoids problems caused by incomplete or ambiguous user input text, making the rewritten input text contain more comprehensive and accurate information, thus providing a better foundation for subsequent task planning, but also helps the system better understand user habits, preferences and context, enabling it to more accurately capture the user's true intentions and improve the accuracy and effectiveness of task planning.
[0076] Furthermore, the government customer service method also includes: if the result of the intent recognition is unrelated to the government issue, then invoking a second intelligent agent to reply to the input text.
[0077] In this embodiment, the second agent is an agent that does not possess basic or professional capabilities in the field of government affairs, such as a chat agent. The intent recognition result is obtained by performing intent recognition on the input text. If the intent recognition result is unrelated to the government affairs issue, the second agent is directly invoked to provide a response.
[0078] S3. According to the planning scheme, the sub-tasks in the planning scheme are assigned to the corresponding first slave agents, so that the first slave agents execute the sub-tasks to be executed in this execution step to generate answers and return the answers;
[0079] It is worth noting that this application provides a government customer service system based on multi-agent collaboration, employing a distributed architecture composed of a master agent and multiple slave agents (first slave agent, second slave agent, etc.). The master agent is responsible for scheduling, coordinating, and managing the slave agents, ensuring the efficient and stable operation of the entire system. Each slave agent is responsible for a specific functional module, and the agents communicate and collaborate through a message passing mechanism, forming an organic whole. Before joining the system, slave agents need to register with the master agent. Registration information includes the slave agent's capability description, preset generation rate, and task question-and-answer examples (i.e., example question text and corresponding answer examples).
[0080] For example, the subtask allocation process is as follows: Based on the planning scheme T, the first slave agent involved in the subtask execution is determined, forming a task group G, and a unique task number is assigned. All messages H of the planning scheme T and task group G are saved. Subtasks are allocated to the first slave agents in task group G, that is, according to the correspondence in planning scheme T, the subtasks in planning scheme T are assigned to the corresponding first slave agents. The allocated subtask information I includes: {subtask number, subtask question text, whether the subtask input depends on the output of some slave agents, routing information, whether the original basis for the question answer needs to be output}. Here, routing information refers to the routing information for the next step where the subtask output needs to be forwarded, including the master agent and the slave agent executing the next subtask. The slave agent input in the first step of planning scheme T is a preset task start identifier, and the routing message of the slave agent in the last step of planning scheme T is for the master agent, meaning the answers of each subtask need to be returned to the master agent for aggregation. Furthermore, the allocated subtask information I also includes: the maximum execution time of the subtask.
[0081] Subtask execution process: After receiving the assigned subtask, each first slave agent in task group G enters the task execution state. The first slave agent in the first step recognizes the task start marker and begins executing the first step's subtask. After this execution, the generated answer is forwarded to the next slave agent and the master agent according to the routing information. If no interrupt signal is received by the first slave agent in task group G, the assigned subtasks will be executed sequentially according to the execution steps.
[0082] S4. The received answer is scored using a pre-trained multi-index scoring model to obtain the scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set.
[0083] S5. Update the planning scheme based on the current scoring result, and return to execute the step of assigning the sub-tasks in the planning scheme to the corresponding first slave agents according to the planning scheme, until the updated planning scheme is executed. Summarize the received answers to obtain the reply of the input text.
[0084] The main agent uses a multi-index scoring model to score the received answer and decides whether to update the planning scheme based on the score result, thus realizing dynamic updating of the planning scheme and avoiding the problem of task deviation accumulation. This ensures the overall task completion effect and improves the accuracy of the response.
[0085] In an optional embodiment, the multi-index scoring model is trained through the following steps:
[0086] Obtain the first sample set; the first sample set consists of the first question text for each first subtask and each first answer;
[0087] Calculate the score results of each indicator for each first answer and construct an indicator score matrix;
[0088] The proportion of the scoring results of each indicator in the index scoring matrix is used as the weight of the scoring results of the corresponding indicator, and the information entropy of each indicator in the index scoring matrix is calculated.
[0089] Based on the information entropy, construct the first loss function of the multi-index scoring model;
[0090] The multi-index scoring model is obtained by taking the index scoring matrix as input and training it by minimizing the first loss function.
[0091] Optionally, each indicator includes at least one of the following:
[0092] (1) Perplexity: Used to evaluate the fluency of generated text (e.g., answers to subtasks, responses to input text / rewritten input text). The smaller the value, the closer the answer is to natural language. It can be implemented using a Python library.
[0093] (2) Answer Accuracy: Used to evaluate the effectiveness of the generated text. For each subtask, based on the question text and corresponding answers, it is determined whether each question element in the question text has been answered. Assuming the question text contains B question elements, and the answer contains b correctly answered question elements, then... B≥1, b≤B. Given the user's input text, based on the input text / rewritten input text and the corresponding response, determine whether each subtask in the input text / rewritten input text has been correctly executed. Assume the input text / rewritten input text contains C subtasks, and the response to the input text contains c correctly answered subtasks. C≥1, c≤C. Large model capabilities can be used for evaluation.
[0094] (3) Distinct: Used to evaluate the diversity of answers, it calculates the ratio of the number of unique n-grams in the generated text to the total number of n-grams.
[0095] (4) Task execution time: The total actual execution time of the planning scheme, or the actual execution time of a certain sub-task.
[0096] A multi-indicator scoring model is a method that combines multiple dimensions of indicators to comprehensively evaluate the effectiveness of generative tasks or the capabilities of models. The indicators mentioned above are just examples of commonly used evaluation indicators, and other indicators can be arbitrarily extended or adopted.
[0097] In this embodiment, a multi-index scoring model is constructed for multiple indicators of the answer. The training data consists of the scoring results of the first answer of each first subtask in the first sample set. Assuming the training batch size is m and the number of index items is n, the index scoring matrix R is first constructed. s :
[0098]
[0099] Among them, s ij This represents the score result of the j-th indicator of the first answer to the i-th first subtask;
[0100] Calculate the weight w of the j-th metric in the first answer of the i-th subtask. ij :
[0101]
[0102] Calculate the information entropy e of the j-th indicator. j :
[0103]
[0104] According to the definition of information entropy, the greater the dispersion of a certain indicator, the greater its impact on the scoring result, and the smaller its entropy value.
[0105] The first loss function L(λ) for constructing the multi-index scoring model is shown in the following equation. Of course, other expressions of the first loss function can also be constructed by combining information entropy, and it is not limited to the following equation:
[0106]
[0107] The indicator scoring matrix R s As input to the model, the model is trained by minimizing the first loss function to obtain a trained multi-index scoring model.
[0108] In an optional embodiment, the step of updating the planning scheme based on the current scoring result and returning to execute the step of assigning subtasks in the planning scheme to the corresponding first slave agents based on the planning scheme, until the updated planning scheme is executed, and summarizing the received answers to obtain the response to the input text, includes:
[0109] If there is no situation where the current score result is less than the first preset threshold, the execution of the first agent is not interrupted, and the process returns to the step of scoring the current answer using a pre-trained multi-index scoring model to obtain the current score result.
[0110] If the current score is less than the first preset threshold, the execution of the first slave agent is interrupted, the planning scheme is updated according to the current score that is less than the first preset threshold, and the execution of the step of allocating the sub-tasks in the planning scheme to the corresponding first slave agents is returned.
[0111] If the planning scheme or the updated planning scheme is completed, the received answers are summarized to obtain the reply of the input text.
[0112] It is understandable that in a certain step of the planning scheme execution, only one subtask can be executed, or multiple subtasks can be executed simultaneously. Correspondingly, the number of answers received this time can be one or more. If there is no case where the current score result is less than the first preset threshold, that is, the score result of each answer received this time is greater than or equal to the first preset threshold, it means that the planning scheme does not need to be updated at this time, and it continues to be executed normally according to the current planning scheme. Specifically, the execution of the first agent is not interrupted, that is, the execution of subsequent subtasks in the current planning scheme is not interrupted, and the step of scoring the current answer by using the pre-trained multi-index scoring model to obtain the current score result is returned.
[0113] If the score result is less than the first preset threshold, meaning the score result of a certain answer received is less than the first preset threshold, it indicates that there is an error in the execution result of the current planning scheme. To avoid further errors, the current planning scheme needs to be updated, and execution should continue according to the updated planning scheme. Specifically, the execution of the first slave agent is interrupted, that is, the execution of subsequent subtasks in the current planning scheme is interrupted, the task planning is re-performed based on the score result that is less than the first preset threshold, the current planning scheme is updated, and the execution returns to the step of assigning the subtasks in the planning scheme to the corresponding first slave agents according to the planning scheme.
[0114] This process iterates until the planned solution is completed, indicating that the task for the input text is finished. The answers generated by the agents executing subtasks are then summarized to obtain the response for the input text. In other words, each time the execution results of the preceding subtask are used to plan the content of the next subtask and the first agent executing the subtask, until the master agent determines that the task for the input text is finished. At this point, the master agent can also disband the task group corresponding to this task and clear the task group information and task number.
[0115] In an optional embodiment, the planning scheme further includes the maximum execution time for each subtask:
[0116] The government customer service methods also include:
[0117] The answer length range of each subtask in the planning scheme is obtained by using a pre-trained answer length prediction model; the answer length prediction model is trained based on a second sample set, which consists of the second question text of each second subtask generated according to the registration information of the first agent and the first agent corresponding to each second subtask;
[0118] Based on the preset generation rate of the first agent and the answer length range, the longest execution time of each subtask in the planning scheme is obtained.
[0119] In this embodiment of the application, the planning scheme T = {(t i ,a i ,t max_i )},t max_i Subtask t represents subtask t i The longest execution time, i.e., the time to complete the subtask. This is achieved by recording the subtask's execution time t. i The longest execution time is used to promptly identify abnormal situations in the first agent service, thereby improving the system's fault tolerance and robustness.
[0120] Specifically, by using an answer length prediction model to obtain the answer length range for any subtask, the maximum execution time of that subtask can be determined. The answer length range can be the token length range of the answer.
[0121] In an optional embodiment, the answer length prediction model is trained through the following steps:
[0122] Obtain the registration information;
[0123] Based on the registration information, the second question text for each second subtask is generated using the large model of the main agent;
[0124] For each second question text, the corresponding first agent is invoked to answer multiple times, resulting in multiple second answers for each second question text;
[0125] Based on the lengths of multiple second answers, the answer length range of the second question text is obtained;
[0126] A second loss function for the answer length prediction model is constructed based on the difference between the minimum and maximum values; the difference between the minimum values is the difference between the minimum value of the answer length range and the minimum value predicted by the model, and the difference between the maximum values is the difference between the minimum value of the answer length range and the maximum value predicted by the model.
[0127] The second sample set is used as input, and the model is trained using the second loss function to obtain the answer length prediction model.
[0128] In this embodiment of the application, the main agent uses an answer length prediction model to predict the execution of each subtask t. i The model generates answers to questions from a given text. The training dataset is constructed as follows: Based on the capability descriptions and task question-and-answer examples in the registration information of the first agent, a large model is used to generate second question texts for the second sub-task of that first agent, forming a question text set. Each second question text has a corresponding first agent, and these first agents form an agent set. During training, the corresponding first agent is invoked to generate multiple second answers for each second question text. For example, for a given second question text, the corresponding first agent is invoked multiple times to generate second answers; that is, each time a second answer for the second question text is generated, multiple second answers for that second question text are obtained. Then, the length of each second answer is calculated to obtain the answer length range for the second answer of the second question text.
[0129] In a specific example, let the set of question texts be X = {x f}, x fThe second problem text, f = 1, 2, ..., F, is the text of the second subtask, and the calls are made from the set of agents A = {a}. f}, a f Let f = 1, 2, ..., F represent the first agent, and each second question text has a corresponding first agent identifier. The input during model training is I = [X, A]. The answer length prediction model is specifically a linear regression model, and the second loss function L(θ) is constructed as shown in the following formula. Of course, other expressions of the second loss function can also be constructed by combining the difference between the minimum and maximum values, and are not limited to the following formula:
[0130]
[0131] In the formula, This represents the maximum value of the range of answer lengths for the second answer in the i-th second subtask. This represents the maximum predicted value of the answer length range for the second answer in the i-th second subtask, i.e., the maximum predicted value by the model. This represents the minimum value of the answer length range for the second answer of the i-th second subtask. This represents the minimum predicted value of the answer length range for the second answer of the i-th second subtask, i.e., the minimum predicted value of the model.
[0132] The input to the answer length prediction model is [subtask t] i The problem text, the subtask t i The corresponding first agent, using the answer length prediction model, predicts the subtask t. i The answer length range Y i =[y 1_i ,y 2_i ], y 1_i Subtask t represents subtask t i The predicted minimum value of the answer length range, y 2_i Subtask t represents subtask t i The predicted maximum value of the answer length range; the predicted maximum value y of the answer length range. 2_i As the maximum number of answers generated, then based on the execution of subtask t i The first preset generation rate v from the agent i The subtask t is calculated. i Longest execution time t max_i :
[0133]
[0134] In an optional embodiment, the government customer service method further includes:
[0135] During the subtask allocation and execution process, an agent exception message is received from the first target agent; the first target agent is the agent in the latest planning scheme and whose waiting time for receiving the third answer is greater than the longest execution time of the preceding subtask, and the third answer is the answer generated by executing the preceding subtask.
[0136] In one example, suppose a first agent a n The system is in a waiting state, awaiting the answer generated by the preceding subtask, i.e., the third answer. If the waiting time for the third answer exceeds the maximum execution time of the preceding subtask, it indicates that the first slave agent executing the preceding subtask has encountered an anomaly. Therefore, the first slave agent a... n An agent exception message is sent to the master agent to indicate that the first slave agent in the preceding subtask has encountered an exception. Upon receiving this exception message, the master agent can identify the malfunctioning first slave agent, thus improving the system's fault tolerance and robustness.
[0137] Furthermore, after receiving the agent exception message sent by the first agent, a task execution command is sent to the first agent indicated by the agent exception message. If the number of times the same agent exception message is received exceeds a preset threshold, the latest planning scheme is marked as having failed, the first agent indicated by the agent exception message is removed, and task planning is restarted. This improves the system's fault tolerance and robustness.
[0138] Optionally, the preset threshold number of attempts is two. The master agent sends the task execution command again to the first slave agent that encountered the anomaly. If the first slave agent still cannot return an answer, the planning scheme marked with the number of attempts fails, the first slave agent indicated by the agent's anomaly message is removed, and task planning is restarted. Furthermore, an alarm can be triggered for the anomaly of the first slave agent, requesting a restart or system investigation, until the alarm for the first slave agent is cleared and it is re-added to the task planning scope.
[0139] In one alternative embodiment, the answer generated by the first agent performing the subtask is obtained by the first agent through the following steps:
[0140] Receive the assigned subtask and extract several first question elements from the question text of the assigned subtask;
[0141] Each retrieval question is generated based on each of the first question elements;
[0142] A database search is performed for each of the search questions to obtain search results for each of the search questions.
[0143] Based on each of the search results, the answer generated by executing the assigned subtask is obtained.
[0144] It is worth noting that most government data varies by region and is frequently updated, such as regional policies and information on matters in different regions. This application's embodiment improves the accuracy of answers to government-related questions by calling a real-time updated database for retrieval. Specifically, the agent first extracts several first question elements from the question text of the assigned sub-task, such as the location for handling the matter and required application materials. Based on each first question element, a large model is used to generate corresponding questions to be retrieved, such as: Question 1: Where can I reissue my ID card? Question 2: What materials do I need to bring to reissue my ID card? A database search is performed for each question to obtain retrieval results. Based on these results, the answers generated by executing the assigned sub-task are obtained.
[0145] In one optional embodiment, obtaining the answer generated by executing the assigned subtask based on each of the search results includes:
[0146] The second question element to which each search result belongs is obtained through a pre-trained retrieval evaluation model; the retrieval evaluation model is trained using the search results of the subtask sample set for a multi-classification task.
[0147] Determine whether all elements of the second question include all elements of the first question;
[0148] If so, then all the search results are summarized to obtain the answer generated by the assigned subtask;
[0149] If not, then filter out the first question elements that are not included, regenerate a new search question, and continue the database search.
[0150] This application provides a retrieval enhancement generation (rag) iterative retrieval strategy that addresses the issues of domain-specific differences and frequent updates in government data. Specifically, a pre-trained retrieval evaluation model is used to evaluate each retrieval result, obtaining the second question element to which each result belongs. It is then determined whether all second question elements include all first question elements. If so, it means all question elements in the subtask have been answered, and all retrieval results are aggregated to obtain the answer generated by the assigned subtask. If not, it means only some question elements in the subtask have been answered, and the unanswered first question elements (i.e., those not included) are filtered out. A new retrieval question is regenerated and iteratively continued until all second question elements include all first question elements, obtaining the answer generated by the assigned subtask. This improves the accuracy of answering government questions.
[0151] Specifically, the input to the retrieval evaluation model is the retrieval results, and the model's category is defined as all the elements of the database records attached to the second agent.
[0152] In an optional embodiment, after receiving a response to the input text, the government customer service method further includes:
[0153] The response is scored using the multi-index scoring model to obtain the score result of the response.
[0154] In this embodiment of the application, the multi-index scoring model is also used to score the response to the input text, providing a comprehensive, objective and reasonable scoring standard for government customer service.
[0155] Furthermore, the system determines whether the user is satisfied based on the rating of the response to the input text. If satisfied, the response is output to the user; otherwise, the system rejects the response and forwards it to a human for further processing.
[0156] Specifically, the multi-index scoring model can be used for judgment. A softmax layer is added to the last layer of the multi-index scoring model for binary classification. Based on the rating results of the responses to the input text, the user's satisfaction level (satisfied or dissatisfied) is predicted. The training data for binary classification comes from webpage user interaction data. After each question, users can click the "satisfied" or "dissatisfied" buttons to complete the evaluation. This type of interaction data can be used to iteratively optimize the model's binary classification function, ensuring user satisfaction and improving the user experience.
[0157] In an optional embodiment, after obtaining the current scoring result, the government customer service method further includes:
[0158] Based on the scoring results, calculate the average score of the first agent in this execution;
[0159] When the average score is less than the second preset threshold, a reminder is issued that the capabilities of the first agent in this execution need to be optimized.
[0160] In this embodiment of the application, based on the current scoring result, the average score result of the first slave agent is calculated. The average score result is used to measure whether the capability of the first slave agent meets the requirements. When the average score result is less than the second preset threshold, it indicates that the capability of the first slave agent does not meet the requirements, and it is reminded that the capability of the first slave agent needs to be optimized, so as to realize the update of the first slave agent and improve the accuracy of the answer.
[0161] The government customer service method provided in this application uses a pre-trained multi-index scoring model to score the received answer, obtain the current scoring result, and then update the planning scheme of the user's current input text based on the current scoring result. This avoids the problem of task deviation accumulation, thereby ensuring the overall task completion effect and improving the accuracy of the answer.
[0162] See Figure 2 , Figure 2 This is a structural block diagram of a government customer service device 10 provided in an embodiment of this application. The government customer service device 10 includes:
[0163] Input module 11 is used to obtain the user's current input text;
[0164] The intent recognition module 12 is used to perform intent recognition on the input text. If the result of intent recognition is related to government affairs issues, task planning is performed based on the input text to obtain a planning scheme. The planning scheme includes a first agent corresponding to each subtask, ordered according to the execution steps.
[0165] The allocation module 13 is used to allocate the sub-tasks in the planning scheme to the corresponding first slave agents according to the planning scheme, so that the first slave agents can execute the sub-tasks to be executed in this execution according to the execution steps to generate answers and return the answers;
[0166] The scoring module 14 is used to score the received answer using a pre-trained multi-index scoring model to obtain the current scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set.
[0167] The update module 15 is used to update the planning scheme according to the current scoring result, and return to execute the step of assigning the sub-tasks in the planning scheme to the corresponding first slave agents according to the planning scheme, until the updated planning scheme is executed, and summarize the received answers to obtain the reply of the input text.
[0168] Optionally, the government customer service device 10 further includes:
[0169] The first training module is used to obtain the first sample set; the first sample set consists of the first question text of each first subtask and each first answer; calculate the score results of each indicator for each first answer and construct an indicator score matrix; use the proportion of the score results of each indicator in the indicator score matrix as the weight of the score results of the corresponding indicator, and calculate the information entropy of each indicator in the indicator score matrix; construct the first loss function of the multi-indicator scoring model based on the information entropy; use the indicator score matrix as input and train by minimizing the first loss function to obtain the multi-indicator scoring model.
[0170] Optionally, the update module 15 is specifically used for:
[0171] If there is no situation where the current score result is less than the first preset threshold, the execution of the first agent is not interrupted, and the process returns to the step of scoring the current answer using a pre-trained multi-index scoring model to obtain the current score result.
[0172] If the current score is less than the first preset threshold, the execution of the first slave agent is interrupted, the planning scheme is updated according to the current score that is less than the first preset threshold, and the execution of the step of allocating the sub-tasks in the planning scheme to the corresponding first slave agents is returned.
[0173] If the planning scheme or the updated planning scheme is completed, the received answers are summarized to obtain the reply of the input text.
[0174] Optionally, the planning scheme also includes the maximum execution time for each subtask:
[0175] The government customer service device 10 also includes:
[0176] The calculation module is used to obtain the answer length range of each subtask in the planning scheme through a pre-trained answer length prediction model; the answer length prediction model is trained based on a second sample set, which consists of the second question text of each second subtask generated according to the registration information of the first agent and the first agent corresponding to each second subtask; the maximum execution time of each subtask in the planning scheme is obtained according to the preset generation rate of the first agent and the answer length range.
[0177] Optionally, the government customer service device 10 further includes:
[0178] The second training module is used to acquire the registration information; generate a second question text for each second subtask using the large model of the main agent based on the registration information; for each second question text, call the corresponding first slave agent to answer multiple times to obtain multiple second answers for each second question text; obtain the answer length range of the second question text based on the length of the multiple second answers; construct a second loss function for the answer length prediction model based on the difference between the minimum and maximum values; the difference between the minimum values is the difference between the minimum value of the answer length range and the minimum value predicted by the model, and the difference between the maximum values is the difference between the minimum value of the answer length range and the maximum value predicted by the model; and train the answer length prediction model by using the second sample set as input and the second loss function.
[0179] Optionally, the government customer service device 10 further includes:
[0180] The receiving module is used to receive an agent exception message sent by the target first agent during the subtask allocation and subtask execution process; the target first agent is the first agent in the latest planning scheme and whose waiting time for receiving the third answer is greater than the longest execution time of the preceding subtask, and the third answer is the answer generated by executing the preceding subtask.
[0181] Optionally, the answer generated by the first agent performing the subtask is obtained by the first agent through the following steps:
[0182] Receive the assigned subtask and extract several first question elements from the question text of the assigned subtask;
[0183] Each retrieval question is generated based on each of the first question elements;
[0184] A database search is performed for each of the search questions to obtain search results for each of the search questions.
[0185] Based on each of the search results, the answer generated by executing the assigned subtask is obtained.
[0186] Optionally, obtaining the answer generated by executing the assigned subtask based on each of the search results includes:
[0187] The second question element to which each search result belongs is obtained through a pre-trained retrieval evaluation model; the retrieval evaluation model is trained using the search results of the subtask sample set for a multi-classification task.
[0188] Determine whether all elements of the second question include all elements of the first question;
[0189] If so, then all the search results are summarized to obtain the answer generated by the assigned subtask;
[0190] If not, then filter out the first question elements that are not included, regenerate a new search question, and continue the database search.
[0191] Optionally, the government customer service device 10 further includes:
[0192] The scoring module is used to score the response using the multi-index scoring model to obtain the score result of the response.
[0193] Optionally, the government customer service device 10 further includes:
[0194] The optimization module is used to calculate the average score of the first slave agent in this execution based on the current scoring result; when the average score is less than the second preset threshold, it prompts that the capabilities of the first slave agent in this execution need to be optimized.
[0195] Optionally, the intent recognition module 12 is further configured to:
[0196] If the result of the intent recognition is unrelated to the government affairs issue, then the second intelligent agent is invoked to respond to the input text.
[0197] It is worth noting that the working process of each module in the government customer service device 10 described in this application embodiment can refer to the working process of the government customer service method described in the above embodiment, and will not be repeated here.
[0198] The government customer service device 10 provided in this application embodiment scores the received answer using a pre-trained multi-index scoring model to obtain the current scoring result. Based on the current scoring result, the planning scheme for the user's current input text is updated, avoiding the problem of easy accumulation of task deviations, thereby ensuring the overall task completion effect and improving the accuracy of the answer.
[0199] Furthermore, this application embodiment also provides a government customer service system, including:
[0200] The main intelligent agent is used to execute the government customer service method described in any of the above embodiments;
[0201] One or more first slave agents are used to receive and execute subtasks assigned by the master agent.
[0202] It should be noted that, in this embodiment, the main intelligent agent can implement any step in the government customer service method embodiment applied to the main intelligent agent in this application, and the first slave intelligent agent can implement any step in the government customer service method embodiment applied to the first slave intelligent agent in this application, and achieve the same beneficial effect, which will not be elaborated here.
[0203] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the government customer service method as described in any of the above embodiments.
[0204] Furthermore, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the government customer service method as described in any of the above embodiments.
[0205] See Figure 3 , Figure 3 This is a structural block diagram of a government customer service device 20 provided in an embodiment of this application. The government customer service device 20 includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described government customer service method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.
[0206] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the government customer service device 20.
[0207] The government customer service device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the government customer service device 20 and does not constitute a limitation on the device. It may include more or fewer components than shown in the diagram, or combine certain components, or use different components. For example, the government customer service device 20 may also include input / output devices, network access devices, buses, etc.
[0208] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the government customer service equipment 20, connecting all parts of the equipment 20 via various interfaces and lines.
[0209] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the government customer service device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0210] If the modules / units integrated into the government customer service equipment 20 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.
[0211] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.
[0212] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.
Claims
1. A government customer service method, characterized in that, Applied to the main intelligent agent, including: Get the user's current input text; The input text is subjected to intent recognition. If the result of intent recognition is related to government affairs issues, task planning is performed based on the input text to obtain a planning scheme. The planning scheme includes sub-tasks ordered according to execution steps and a first slave agent corresponding to each sub-task. According to the planning scheme, the subtasks in the planning scheme are assigned to the corresponding first slave agents, so that the first slave agents execute the subtasks to be executed in this execution step to generate answers and return the answers; The received answer is scored using a pre-trained multi-index scoring model to obtain the current scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set; The planning scheme is updated based on the current scoring result, and the process returns to the step of assigning subtasks in the planning scheme to the corresponding first slave agents, until the updated planning scheme is completed. The received answers are then summarized to obtain the response to the input text.
2. The government customer service method as described in claim 1, characterized in that, The multi-index scoring model is trained through the following steps: Obtain the first sample set; the first sample set consists of the first question text for each first subtask and each first answer; Calculate the score results of each indicator for each first answer and construct an indicator score matrix; The proportion of the scoring results of each indicator in the index scoring matrix is used as the weight of the scoring results of the corresponding indicator, and the information entropy of each indicator in the index scoring matrix is calculated. Based on the information entropy, construct the first loss function of the multi-index scoring model; The multi-index scoring model is obtained by taking the index scoring matrix as input and training it by minimizing the first loss function.
3. The government customer service method as described in claim 1, characterized in that, The process involves updating the planning scheme based on the current scoring result, returning to the step of assigning subtasks in the planning scheme to the corresponding first slave agents, until the updated planning scheme is completed. Then, the received answers are summarized to obtain a response to the input text, including: If there is no situation where the current score result is less than the first preset threshold, the execution of the first agent is not interrupted, and the process returns to the step of scoring the current answer using a pre-trained multi-index scoring model to obtain the current score result. If the current score is less than the first preset threshold, the execution of the first slave agent is interrupted, the planning scheme is updated according to the current score that is less than the first preset threshold, and the execution of the step of allocating the sub-tasks in the planning scheme to the corresponding first slave agents is returned. If the planning scheme or the updated planning scheme is completed, the received answers are summarized to obtain the reply of the input text.
4. The government customer service method as described in claim 1, characterized in that, The planning scheme also includes the maximum execution time for each sub-task: The government customer service methods also include: The answer length range of each subtask in the planning scheme is obtained by using a pre-trained answer length prediction model; the answer length prediction model is trained based on a second sample set, which consists of the second question text of each second subtask generated according to the registration information of the first agent and the first agent corresponding to each second subtask; Based on the preset generation rate of the first agent and the answer length range, the longest execution time of each subtask in the planning scheme is obtained.
5. The government customer service method as described in claim 4, characterized in that, The answer length prediction model is trained through the following steps: Obtain the registration information; Based on the registration information, the second question text for each second subtask is generated using the large model of the main agent; For each second question text, the corresponding first agent is invoked to answer multiple times, resulting in multiple second answers for each second question text; Based on the lengths of multiple second answers, the answer length range of the second question text is obtained; A second loss function for the answer length prediction model is constructed based on the difference between the minimum and maximum values; the difference between the minimum values is the difference between the minimum value of the answer length range and the minimum value predicted by the model, and the difference between the maximum values is the difference between the minimum value of the answer length range and the maximum value predicted by the model. The second sample set is used as input, and the model is trained using the second loss function to obtain the answer length prediction model.
6. The government customer service method as described in claim 4, characterized in that, The government customer service methods also include: During the subtask allocation and execution process, an agent exception message is received from the first target agent; the first target agent is the agent in the latest planning scheme and whose waiting time for receiving the third answer is greater than the longest execution time of the preceding subtask, and the third answer is the answer generated by executing the preceding subtask.
7. The government customer service method as described in claim 1, characterized in that, The answer generated by the first agent performing the subtask is obtained by the first agent through the following steps: Receive the assigned subtask and extract several first question elements from the question text of the assigned subtask; Each retrieval question is generated based on each of the first question elements; A database search is performed for each of the search questions to obtain search results for each of the search questions. Based on each of the search results, the answer generated by executing the assigned subtask is obtained.
8. The government customer service method as described in claim 7, characterized in that, The step of obtaining the answer generated by executing the assigned subtask based on each of the search results includes: The second question element to which each search result belongs is obtained through a pre-trained retrieval evaluation model; the retrieval evaluation model is trained using the search results of the subtask sample set for a multi-classification task. Determine whether all elements of the second question include all elements of the first question; If so, then all the search results are summarized to obtain the answer generated by the assigned subtask; If not, then filter out the first question elements that are not included, regenerate a new search question, and continue the database search.
9. The government customer service method as described in any one of claims 1-8, characterized in that, After receiving a response to the input text, the government customer service method further includes: The response is scored using the multi-index scoring model to obtain the score result of the response.
10. The government customer service method as described in any one of claims 1-8, characterized in that, After obtaining the current scoring result, the government customer service method further includes: Based on the scoring results, calculate the average score of the first agent in this execution; When the average score is less than the second preset threshold, a reminder is issued that the capabilities of the first agent in this execution need to be optimized.
11. The government customer service method as described in any one of claims 1-8, characterized in that, After performing intent recognition on the input text, the government customer service method further includes: If the result of the intent recognition is unrelated to the government affairs issue, then the second intelligent agent is invoked to respond to the input text.
12. A government customer service device, characterized in that, include: The input module is used to obtain the user's current input text; An intent recognition module is used to recognize the intent of the input text. If the result of intent recognition is related to government affairs issues, task planning is performed based on the input text to obtain a planning scheme. The planning scheme includes a first agent corresponding to each subtask, ordered according to the execution steps. The allocation module is used to allocate the subtasks in the planning scheme to the corresponding first slave agents according to the planning scheme, so that the first slave agents can execute the subtasks to be executed in this execution according to the execution steps to generate answers and return the answers; The scoring module is used to score the received answer using a pre-trained multi-index scoring model to obtain the current scoring result; wherein the multi-index scoring model is trained based on the scoring results of the first answer of each first subtask in the first sample set; The update module is used to update the planning scheme according to the current scoring result, and return to execute the step of assigning the sub-tasks in the planning scheme to the corresponding first slave agents according to the planning scheme, until the updated planning scheme is executed, and summarize the received answers to obtain the reply of the input text.
13. A government customer service system, characterized in that, include: A main intelligent agent is used to execute the government customer service method according to any one of claims 1 to 11; One or more first slave agents are used to receive and execute subtasks assigned by the master agent.
14. A government customer service device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the government customer service method as described in any one of claims 1 to 11.
15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the government customer service method as described in any one of claims 1 to 11.
16. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the government customer service method as described in any one of claims 1 to 11.