Knowledge base expansion and correction method and system based on multi-agent driving

Abstract

The invention discloses a knowledge base expansion and correction method and system based on multi-agent driving, and belongs to the field of natural language processing, and the method comprises the steps: calling a first agent to carry out problem and priority classification on a work order, and generating a service label of the work order; calling a second agent to retrieve a knowledge base according to the service request of the work order to obtain question answers of the work order and retrieval frequency of similar questions; judging whether to start a knowledge base updating process or not based on a preset expansion condition; after a knowledge base updating process is started, calling a third agent to extract to-be-stored information of the work order and storing the to-be-stored information in a knowledge base; and according to the question score of the question answer, calling a fourth agent to perform redundant information processing on a knowledge block corresponding to the question answer in a knowledge base. The problems that in the prior art, manual expansion of the knowledge base is high in cost and insufficient in efficiency, and the retrieval efficiency is affected by more redundant information in the knowledge base can be solved.

Description

Technical Field

[0001] This application belongs to the field of natural language processing, and specifically relates to a knowledge base expansion and correction method and system based on multi-agent drive. Background Art

[0002] In today's business environment, customer service is crucial. In enterprise customer service centers, intelligent agents can quickly answer customer questions using a knowledge base, making knowledge base maintenance and expansion a key challenge. Traditional knowledge base management methods, which rely on manual intervention, are inefficient, especially in an era of progressive multi-model application. This is especially true when processing complex service requests, such as software system maintenance and usage issues. It's difficult to update the knowledge base in a timely and accurate manner, and similar new questions can't be promptly stored. This leads to inefficient customer service and the inability to answer customer questions in a timely manner. Furthermore, redundant or abnormal data in the knowledge base can also affect the output of answers.

[0003] Existing knowledge base expansion methods rely on manual review of information to be imported. However, manual expansion of the knowledge base requires significant manpower and is inefficient, failing to reflect the latest information in a timely manner and potentially compromising customer service quality. Furthermore, when redundant or invalid information is present in the knowledge base and not promptly cleared, it reduces answer retrieval efficiency. Summary of the Invention

[0004] This application proposes a knowledge base expansion and correction method and system based on multi-agent drive, which uses multiple agents to automatically and continuously update the knowledge base without affecting the service process, and promptly clear redundant information in the knowledge base, thereby reducing the cost of knowledge base expansion and improving the retrieval efficiency of the knowledge base.

[0005] The first aspect of the present application provides a method for expanding and revising a knowledge base based on multi-agent driving, the method comprising:

[0006] Invoking the first agent to classify and prioritize the received work order, and generating a business tag for the work order;

[0007] Invoke a second agent to search the knowledge base according to the service request of the work order, and obtain the answer to the question of the work order and the search frequency of similar questions;

[0008] According to the business tags and the search frequency of similar questions, based on the preset expansion conditions, it is determined whether to start the knowledge base update process;

[0009] After starting the knowledge base update process, calling the third agent to extract the to-be-entered information of the work order and storing the to-be-entered information in the knowledge base;

[0010] Based on the question score of the answer to the question, a fourth agent is called to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base.

[0011] The above solution uses multiple agents to manage the knowledge base in different ways, significantly reducing human intervention to achieve automatic knowledge extraction and dynamic updates, and improve the accuracy and timeliness of the knowledge base. First, the first agent is used to classify and prioritize work orders containing questions, providing data support for whether to store the question content in the work order in the knowledge base or delete it from the knowledge base. The second agent is used to retrieve the corresponding answer from the knowledge base based on the work order information and output it. After the knowledge base update process is initiated, the third agent is used to store the work order information to be stored in the knowledge base, completing the knowledge base update. The fourth agent determines whether the corresponding knowledge block is redundant data based on the score of the output question answer, determining whether the answer to the question satisfies the customer. If it is redundant data, it is deleted from the knowledge base or the corresponding search is stopped to improve retrieval efficiency and reduce the interference of redundant information on retrieving the correct answer.

[0012] In a possible implementation method of the first aspect, a first agent is called to classify and prioritize a received work order, and generate a service tag for the work order, specifically:

[0013] Analyzing the work order through the first agent to determine the problem type and business scenario of the work order;

[0014] Determine the priority label of the work order based on the problem type and business scenario through preset priority classification;

[0015] The business tag is generated according to the problem type, business scenario and the priority tag.

[0016] The above solution first uses the first agent to classify and prioritize work orders, determining the corresponding problem type and business scenario, providing data support for subsequent retrieval of answers. This prioritization also provides data support for the third and fourth agents to implement different processing strategies. For example, certain low-priority work orders do not need to participate in knowledge base expansion.

[0017] In a possible implementation method of the first aspect, a second agent is called to search a knowledge base according to the service request of the work order to obtain an answer to the question of the work order and a search frequency of similar questions, specifically:

[0018] Searching a knowledge base according to the service request by a second agent to determine a knowledge block corresponding to the service request;

[0019] Calling the knowledge block to output answers to questions in the work order, and recording the calling frequency of the knowledge block;

[0020] According to the question type of the business tag, the search frequency of similar questions corresponding to the question type is updated.

[0021] This approach uses a second agent to search the knowledge base for answers to work orders. It also records the frequency of searches for similar questions to determine whether the knowledge blocks corresponding to these questions are frequently accessed. If these blocks are frequently accessed, the agent will focus on them and perform expansion or redundant information processing to ensure timely updates to the knowledge base.

[0022] In a possible implementation method of the first aspect, whether to start the knowledge base update process is determined based on the business tag and the search frequency of similar questions based on preset expansion conditions, specifically:

[0023] Scoring the answer to the question in combination with the priority tag of the business tag to obtain a question score;

[0024] When the question type and business scenario of the business tag meet the expansion condition, and the search frequency of similar questions is greater than a first threshold and the question score is less than a second threshold, the knowledge base update process is started.

[0025] The above solution determines whether to initiate the knowledge base update process based on whether the answer to the ticket question receives a high score. If multiple searches for this type of question within a certain period of time fail to produce satisfactory results for the customer, it indicates that the answer to this type of question is of low quality. The answer to this type of question should be updated so that the next output is more in line with the customer's needs.

[0026] In a possible implementation method of the first aspect, based on the question score of the answer to the question, a fourth agent is called to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base, specifically:

[0027] When the question score is less than a third threshold, the knowledge base correction process is called by the fourth agent;

[0028] After calling the knowledge base correction process, extract the call information of the knowledge block corresponding to the answer to the question and mark the knowledge block as a redundant knowledge block; wherein the call information includes the call frequency, the repeated call frequency, the frequency of being marked as low-scoring, the marked low-scoring score and the gray list storage duration;

[0029] According to the call information, determining the processing method of the redundant knowledge block through the preset redundant information judgment conditions; wherein the processing method includes reducing the search weight, placing it in the gray list and placing it in the black list;

[0030] The redundant knowledge block is processed according to the processing manner.

[0031] In the above solution, if a customer is dissatisfied with the output answer to a question, the score will fall below the preset threshold. At this point, the knowledge base correction process needs to be initiated to address redundant information in the knowledge base and reduce the proportion of invalid answers. The first step is to identify the knowledge blocks corresponding to the low-scoring answers and mark them as redundant. Then, several options are offered: reducing the search weight, placing them on a graylist, and placing them on a blacklist. The corresponding treatment method is determined based on the degree of redundancy of the redundant knowledge blocks. In severe cases, the redundant information can be deleted, reducing the probability of redundant information being retrieved from the knowledge base and improving knowledge base retrieval efficiency.

[0032] In a possible implementation method of the first aspect, a processing method of the redundant knowledge block is determined according to the call information and a preset redundant information determination condition, specifically:

[0033] If the weekly decrease in the call frequency of the redundant knowledge block is higher than a fourth threshold, the retrieval weight of the redundant knowledge block is reduced; wherein the retrieval weight is related to the probability of the knowledge block being retrieved in the knowledge base;

[0034] Calculating a redundancy score of the redundant knowledge block based on the call information, and if the redundancy score is greater than a fifth threshold and less than or equal to a sixth threshold, or the frequency of being marked as low-scoring is lower than a seventh threshold, placing the redundant knowledge block in a gray list;

[0035] If the redundancy score is greater than a sixth threshold, the redundant knowledge block is put into a blacklist.

[0036] The above scheme proposes three ways to handle redundant information, namely, reducing the retrieval weight to reduce the possibility of redundant information being retrieved; placing the redundant knowledge block in the gray list but retaining the metadata, indicating that the redundant knowledge block is temporarily observed and can be retained when the redundant knowledge block subsequently receives a high score; placing the redundant knowledge block in the blacklist and deleting it, indicating that the redundant knowledge block itself is invalid information, and it is recommended to delete it to reduce the proportion of invalid information in the knowledge base.

[0037] In a possible implementation method of the first aspect, the redundancy score of the redundant knowledge block is calculated using the call information, specifically as follows:

[0038] The calculation formula of the redundancy score is:

[0039]

[0040] Where n is the number of dimensions, W i is the dimension weight, V iis a dimension value, including the call frequency, problem score and existence time, and N is the number of times the same problem occurs within a week.

[0041] In a possible implementation method of the first aspect, adding to the gray list and adding to the black list are specifically as follows:

[0042] After the redundant knowledge block is put into the grey list, its search in the knowledge base is stopped but the corresponding metadata is retained;

[0043] After the redundant knowledge block is put into the blacklist, the redundant knowledge block will be deleted from the knowledge base.

[0044] The second aspect of the present application provides a knowledge base expansion and correction system based on multi-agent drive, the system comprising: a business tag generation module, an answer retrieval module, a process startup module, a knowledge base update module and a redundant information processing module;

[0045] The service tag generation module is used to call the first agent to classify and prioritize the received work orders and generate service tags for the work orders;

[0046] The answer retrieval module is used to call the second agent to search the knowledge base according to the service request of the work order, and obtain the answer to the question of the work order and the search frequency of similar questions;

[0047] The process starting module is used to determine whether to start the knowledge base update process based on the business tag and the search frequency of similar questions and the preset expansion conditions;

[0048] The knowledge base update module is used to call the third agent to extract the to-be-entered information of the work order after starting the knowledge base update process, and store the to-be-entered information in the knowledge base;

[0049] The redundant information processing module is used to call the fourth agent to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base according to the question score of the answer to the question.

[0050] The third aspect of the present application provides a terminal device, which includes: a terminal device including a processor and a memory, the memory storing a computer program, and the processor implementing the steps of a multi-agent driven knowledge base expansion and correction method as described in any one of the embodiments of the present application when executing the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the implementation. Obviously, the drawings described below are only some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 This is a schematic diagram of a specific process of a multi-agent driven knowledge base expansion and correction method provided in one embodiment of the present application;

[0053] Figure 2 This is a traditional knowledge base management flow chart of a multi-agent driven knowledge base expansion and correction method provided in one embodiment of the present application;

[0054] Figure 3 This is a multi-model knowledge base management flow chart based on a multi-agent driven knowledge base expansion and correction method provided by a certain embodiment of the present application;

[0055] Figure 4 This is a specific structural diagram of a multi-agent driven knowledge base expansion and correction system provided by a certain embodiment of the present application;

[0056] Figure 5 A structural diagram of a terminal device is provided in accordance with a certain embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0058] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are to be executed.

[0059] First embodiment

[0060] Knowledge bases serve as the foundation for customer service centers to answer customer questions and require continuous maintenance and expansion to provide timely and accurate responses. However, traditional knowledge base management methods, such as manual updates, are still used. When processing complex service requests, similar new questions cannot be promptly entered into the database, resulting in inefficient customer service. Furthermore, the continuous addition of new knowledge leads to a continuous accumulation of redundant or invalid information in the knowledge base, hindering the delivery of accurate answers.

[0061] like Figure 1 As shown, Figure 1 A specific flow chart of a multi-agent driven knowledge base expansion and correction method is provided for a certain embodiment of the present application. The multi-agent driven knowledge base expansion and correction method of this embodiment includes steps S1 to S5, which are described in detail as follows:

[0062] Step S1: Call the first agent to classify the problem and determine the priority of the received work order, and generate a business tag for the work order.

[0063] In the embodiments of this application, multiple intelligent agents are used. Through the division of labor and cooperation among different intelligent agents, tasks such as question answering, knowledge base updating, and redundant information removal are completed, significantly reducing human intervention. The result is automatic knowledge extraction and dynamic updating, improving the accuracy and timeliness of the knowledge base, and reducing human intervention. The intelligent agent (also called an agent) is an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware, or a system, and has autonomy, adaptability, and interactive capabilities.

[0064] The traditional process of intelligent agent management knowledge base is shown in Figure 2 The entire process includes service request input (service), Agent processing, output results (Answer) and manual verification (manual check). Among them, service request input is to submit a work order containing customer questions to the intelligent agent, and manual verification is to manually confirm whether the output answer meets the questions in the work order. There are two important components inside the intelligent agent, which have a significant impact on the accuracy of the output results. One is the tool and the other is the knowledge base (RAG vector library is used here). However, the knowledge base within the enterprise is dynamic, and in the era of large-scale model applications, it is difficult to ensure that the knowledge base is comprehensive, complete, and can be dynamically updated. To achieve this goal, general human resources need to be invested in continuous operation and maintenance, so the upper limit of efficiency improvement is limited.

[0065] Based on the above defects, the embodiment of the present application provides a knowledge base expansion and correction method driven by multiple agents. The specific process is as follows: Figure 3 As shown. Figure 3 In addition to Agent B for outputting answers, the system also provides Agent A for question classification, business scenario identification and priority determination, Agent C for expanding the knowledge base, and Agent D for processing redundant information in the knowledge base.

[0066] Multiple intelligent agents are used to expand, modify and output the search answers to the knowledge base, triggering the update of the knowledge base while outputting the answers, and quickly responding to new questions in subsequent applications.

[0067] Before a work order containing an input question is submitted to the second agent, the first agent is called upon to perform multi-dimensional work order classification and priority determination based on the work order content, generating a corresponding business tag for the work order. This business tag can help quickly locate the knowledge block containing the answer during subsequent knowledge base searches, and output the answer to the question in a timely manner.

[0068] Specifically, the first intelligent agent first analyzes the work order to determine the problem type (including hardware / software) and business scenario (including consultation / analysis / log, etc.) of the work order, and then determines the priority label of the work order based on the preset priority classification according to the problem type and business scenario. Finally, the business label is generated based on the problem type, business scenario and the priority label.

[0069] For example, for a first work order, it is determined that this is a software problem, the business scenario to which it belongs is related functional consultation, and the priority is 1, then the corresponding business tag is generated: {"Classification Problem": "Software", "Business Scenario": "Functional Consultation"; "Priority": "1"}.

[0070] Furthermore, in the present embodiment, priority tags of 0-9 are set. For tags defined as 0-3, the knowledge base needs to be updated, and this can be directly updated and stored using a third agent. For tags defined as 4-7, different tools may be needed to obtain external data, which will be processed manually or by a third agent before being stored. For tags defined as 8-9, no processing is required for the time being. In other embodiments, priority tags of 10 or higher can be set to implement other functions.

[0071] Step S2: Call the second agent to search the knowledge base according to the service request of the work order to obtain the answer to the question of the work order and the search frequency of similar questions.

[0072] After generating a corresponding business tag for the work order, a second agent is called upon to retrieve the corresponding answer from the knowledge base based on the question type and business scenario provided by the business tag, as well as the service request question. During the retrieval process, the corresponding knowledge block is first located, then called upon to output the answer to the work order question. The frequency of calling the knowledge block is recorded, and the search frequency for similar questions corresponding to the question type is updated based on the question type of the business tag. For example, if the question type is hardware, the search frequency for similar questions corresponding to hardware problems is increased by one.

[0073] Step S3: judging whether to start the knowledge base update process based on the business tag and the search frequency of similar questions and the preset expansion conditions.

[0074] After the second agent outputs the answer to the question corresponding to the work order, the embodiment of the present application also adds a determination process to determine whether to trigger the knowledge base update process. This determination process is mainly based on the given question score, the question type and business scenario provided by the business tag, and the search frequency of similar questions to comprehensively determine whether to start the process and call the third agent. The purpose of setting this knowledge base expansion condition is to store frequently asked questions or more important types of questions in the knowledge base, so that timely and qualified answers can be provided in the next Q&A session.

[0075] The question score is calculated by weighting the priority tag assigned by the service tag and the user's rating of the output question answer. After receiving the output question answer, the user will rate it based on whether the answer solves the question posed. The rating ranges from 1-2: Very Unsatisfied, 3-4: Unsatisfied, 5-6: Average, 7-8: Good, and 9-10: Very Satisfied. Furthermore, during the weighted calculation process, the weights assigned to the priority tags and user ratings can be modified based on actual circumstances.

[0076] For example, the knowledge base expansion conditions provided in this embodiment are: when question type = software, business scenario = functional consultation, search frequency of similar questions > 5, and question score < 6, the knowledge base update process is initiated. In other embodiments, the judgment threshold of the above knowledge base expansion conditions can be modified according to actual requirements. For example, if the requirements for question scoring are more stringent, the question score can be set to < 5.

[0077] Step S4, after starting the knowledge base update process, calling the third agent to extract the to-be-entered information of the work order, and storing the to-be-entered information in the knowledge base.

[0078] After starting the knowledge base update process, the third agent is called to perform structured knowledge extraction on the work order for which the answer to the question has been output, and obtain the information to be stored in the work order.

[0079] For example, the embodiment of the present application provides information to be stored in JSON format: {"question", "step", "associated work order ID"}. The associated work order ID is a unique code specific to each work order, and "step" is the answer to the question output by the work order.

[0080] The third agent then stores the acquired information in the knowledge block corresponding to the question, completing the expansion of the knowledge base. Because the first and second agents can continue to perform the search and answer output processes while the third agent is expanding the knowledge base, the expansion of the knowledge base does not interrupt customer Q&A service. The knowledge base can be automatically and continuously updated without affecting the service process, greatly improving the user experience.

[0081] Step S5: Based on the question score of the answer to the question, call the fourth agent to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base.

[0082] In an embodiment of the present application, when the weighted calculated question score is lower than the set threshold, it means that the answer to the question cannot meet the user's expectations and is very likely redundant information, and the fourth agent needs to be called to start the knowledge base correction process.

[0083] After invoking the knowledge base correction process, the corresponding knowledge block is first determined based on the answer to the question with a low question score, and the knowledge block is marked as a redundant knowledge block. At the same time, the call information of the redundant knowledge block is extracted. The call information includes the call frequency, effective call frequency, invalid call frequency, frequency of being marked as low, the score of being marked as low, and the gray list storage duration. The low score is the question score of the answer to the question output from the knowledge block that is lower than the preset threshold. The low frequency is the number of times the answer to the question output from the knowledge block is given a low question score. The gray list storage duration is the length of time the redundant knowledge block is placed on the gray list. The call frequency is the number of times content related to the work order question can be found in the knowledge block for the first time. The effective call frequency is the number of times, after finding content related to the work order question in the knowledge block, the content is re-sorted and filtered to select the content with the greatest relevance to the work order question as the answer to the question.

[0084] Then, based on the call information, the processing method of the redundant knowledge block is determined by the preset redundant information judgment conditions. In the embodiment of this application, three processing methods are provided, including reducing the search weight, placing it in the gray list, and placing it in the black list. See the table below for details:

[0085]

[0086] The judgment conditions and processing thresholds in the above table will be adjusted according to actual conditions.

[0087] Among them, reducing the search weight is to reduce the probability of redundant knowledge blocks being retrieved in the knowledge base, thereby improving the quality of knowledge base retrieval. The gray list is used to store redundant knowledge blocks that are temporarily suspended and cannot be retrieved in the knowledge base, but the metadata of the redundant knowledge base is retained. When the problem score of the redundant knowledge block increases to a certain level, the redundant knowledge block can be released from the gray list and re-entered into the search. The blacklist is used to store redundant knowledge blocks with invalid information to be deleted. When a redundant knowledge block in the gray list has been stored for more than 30 days, it will be transferred to the blacklist for deletion. After the redundant knowledge block is placed on the blacklist, it will be deleted from the knowledge base.

[0088] The above redundancy score is calculated based on the call information, and the specific calculation formula is:

[0089]

[0090] Where n is the number of dimensions, W i is the dimension weight, V i It is a dimension value, including the call frequency, problem score, related work order resolution rate and existence time. The existence time is the length of time the knowledge block is stored in the knowledge base, calculated from the time of entry into the database; N is the number of times the same type of problem occurs within a week.

[0091] For example, for the call information of a redundant knowledge block, including: call frequency 120 times, invalid call frequency 98 times, average problem score 4.3 (out of 10 points), related ticket resolution rate 62%, knowledge base storage days 200 days, and similar problems appearing 10 times within a week, substitute into the above formula to obtain:

[0092] Redundancy score = 0.6*(1-98) / 120+0.3*(1-4.3 / 10)+0.1*(1-0.62)+log2(10+1);

[0093] The final result is 0.4+0.17+0.04+0.3.

[0094] In addition, the number of storage days is not calculated in the formula and is only used for marking. When the weights of knowledge blocks are consistent, knowledge blocks with shorter storage days are given priority.

[0095] The implementation of the embodiments of the present application has the following beneficial effects:

[0096] The embodiment of the present application uses multiple agents to manage the knowledge base differently, greatly reducing human access, so as to achieve automatic knowledge extraction and dynamic update, and improve the accuracy and timeliness of the knowledge base. First, the first agent is used to classify and prioritize the work orders containing questions, and provide data support for whether to store the problem content in the work order in the knowledge base or delete it from the knowledge base; the second agent is used to retrieve the corresponding answer in the knowledge base based on the work order information and output it; after the knowledge base update process is started, the third agent is used to store the information to be stored in the knowledge base of the work order to complete the update of the knowledge base; and the fourth agent is based on the score of the output question answer, and determines whether the answer to the question satisfies the customer to determine whether the corresponding knowledge block is redundant data. If it is redundant data, it will be deleted from the knowledge base or the corresponding search will be stopped to improve the retrieval efficiency and reduce the interference of redundant information on the retrieval of the correct answer.

[0097] Second embodiment

[0098] Furthermore, in order to implement the multi-agent driven knowledge base expansion and correction system corresponding to the above method embodiment to achieve the corresponding functions and technical effects, Figure 4 A structural diagram of a multi-agent driven knowledge base expansion and correction system is provided. For ease of explanation, only the parts related to this embodiment are shown. The multi-agent driven knowledge base expansion and correction system provided in this embodiment of the application includes:

[0099] The business tag generation module 201 is used to call the first agent to classify the problem and determine the priority of the received work order, and generate a business tag for the work order.

[0100] In the embodiments of this application, multiple intelligent agents are used. Through the division of labor and cooperation among different intelligent agents, tasks such as question answering, knowledge base updating, and redundant information removal are completed, significantly reducing human intervention. The result is automatic knowledge extraction and dynamic updating, improving the accuracy and timeliness of the knowledge base, and reducing human intervention. The intelligent agent (also called an agent) is an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware, or a system, and has autonomy, adaptability, and interactive capabilities.

[0101] Multiple intelligent agents are used to expand, modify and output the search answers to the knowledge base, triggering the update of the knowledge base while outputting the answers, and quickly responding to new questions in subsequent applications.

[0102] Before a work order containing an input question is submitted to the second agent, the first agent is called upon to perform multi-dimensional work order classification and priority determination based on the work order content, generating a corresponding business tag for the work order. This business tag can help quickly locate the knowledge block containing the answer during subsequent knowledge base searches, and output the answer to the question in a timely manner.

[0103] Specifically, the first intelligent agent first analyzes the work order to determine the problem type (including hardware / software) and business scenario (including consultation / analysis / log, etc.) of the work order, and then determines the priority label of the work order based on the preset priority classification according to the problem type and business scenario. Finally, the business label is generated based on the problem type, business scenario and the priority label.

[0104] For example, for a first work order, it is determined that this is a software problem, the business scenario to which it belongs is related functional consultation, and the priority is 1, then the corresponding business tag is generated: {"Classification Problem": "Software", "Business Scenario": "Functional Consultation"; "Priority": "1"}.

[0105] Furthermore, in the present embodiment, priority tags of 0-9 are set. For tags defined as 0-3, the knowledge base needs to be updated, and this can be directly updated and stored using a third agent. For tags defined as 4-7, different tools may be needed to obtain external data, which will be processed manually or by a third agent before being stored. For tags defined as 8-9, no processing is required for the time being. In other embodiments, priority tags of 10 or higher can be set to implement other functions.

[0106] The answer retrieval module 202 is used to call the second agent to search the knowledge base according to the service request of the work order, and obtain the answer to the question of the work order and the search frequency of similar questions.

[0107] After generating a corresponding business tag for the work order, a second agent is called upon to retrieve the corresponding answer from the knowledge base based on the question type and business scenario provided by the business tag, as well as the service request question. During the retrieval process, the corresponding knowledge block is first located, then called upon to output the answer to the work order question. The frequency of calling the knowledge block is recorded, and the search frequency for similar questions corresponding to the question type is updated based on the question type of the business tag. For example, if the question type is hardware, the search frequency for similar questions corresponding to hardware problems is increased by one.

[0108] The process starting module 203 is used to determine whether to start the knowledge base update process based on the business tag and the search frequency of similar questions and the preset expansion conditions.

[0109] In an embodiment of the present application, after the second agent outputs the answer to the question corresponding to the work order, the embodiment of the present application also adds a determination process for determining whether to trigger the knowledge base update process. The determination process is mainly based on the given question score, the question type and business scenario provided by the business tag, and the retrieval frequency of similar questions to comprehensively determine whether to start the process and call the third agent.

[0110] The question score is calculated by weighting the priority tag assigned by the service tag and the user's rating of the output question answer. After receiving the output question answer, the user will rate it based on whether the answer solves the question posed. The rating ranges from 1-2: Very Unsatisfied, 3-4: Unsatisfied, 5-6: Average, 7-8: Good, and 9-10: Very Satisfied. Furthermore, during the weighted calculation process, the weights assigned to the priority tags and user ratings can be modified based on actual circumstances.

[0111] For example, the knowledge base expansion conditions provided in this embodiment are: when question type = software, business scenario = functional consultation, search frequency of similar questions > 5, and question score < 6, the knowledge base update process is initiated. In other embodiments, the judgment threshold of the above knowledge base expansion conditions can be modified according to actual requirements. For example, if the requirements for question scoring are more stringent, the question score can be set to < 5.

[0112] The knowledge base updating module 204 is used to call the third agent to extract the to-be-entered information of the work order after starting the knowledge base updating process, and store the to-be-entered information in the knowledge base.

[0113] After starting the knowledge base update process, the third agent is called to perform structured knowledge extraction on the work order for which the answer to the question has been output, and obtain the information to be stored in the work order.

[0114] For example, the embodiment of the present application provides information to be stored in JSON format: {"question", "step", "associated work order ID"}. The associated work order ID is a unique code specific to each work order.

[0115] The third agent then stores the acquired information in the knowledge block corresponding to the question, completing the expansion of the knowledge base. Because the first and second agents can continue to perform the search and answer output processes while the third agent is expanding the knowledge base, the expansion of the knowledge base does not interrupt customer Q&A service. The knowledge base can be automatically and continuously updated without affecting the service process, greatly improving the user experience.

[0116] The redundant information processing module 205 is used to call the fourth agent to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base according to the question score of the answer to the question.

[0117] In an embodiment of the present application, when the weighted calculated question score is lower than the set threshold, it means that the answer to the question cannot meet the user's expectations and is very likely redundant information, and the fourth agent needs to be called to start the knowledge base correction process.

[0118] After invoking the knowledge base correction process, the corresponding knowledge block is first determined based on the answer to the question with a low question score, and the knowledge block is marked as a redundant knowledge block. At the same time, the call information of the redundant knowledge block is extracted. The call information includes the call frequency, effective call frequency, invalid call frequency, frequency of being marked as low, the score of being marked as low, and the gray list storage duration. The low score is the question score of the answer to the question output from the knowledge block that is lower than the preset threshold. The low frequency is the number of times the answer to the question output from the knowledge block is given a low question score. The gray list storage duration is the length of time the redundant knowledge block is placed on the gray list. The call frequency is the number of times content related to the work order question can be found in the knowledge block for the first time. The effective call frequency is the number of times, after finding content related to the work order question in the knowledge block, the content is re-sorted and filtered to select the content with the greatest relevance to the work order question as the answer to the question.

[0119] Then, based on the call information, the processing method of the redundant knowledge block is determined by the preset redundant information judgment conditions. In the embodiment of this application, three processing methods are provided, including reducing the search weight, placing it in the gray list, and placing it in the black list. See the table below for details:

[0120]

[0121] Among them, reducing the search weight is to reduce the probability of redundant knowledge blocks being retrieved in the knowledge base, thereby improving the quality of knowledge base retrieval. The gray list is used to store redundant knowledge blocks that are temporarily suspended and cannot be retrieved in the knowledge base, but the metadata of the redundant knowledge base is retained. When the problem score of the redundant knowledge block increases to a certain level, the redundant knowledge block can be released from the gray list and re-entered into the search. The blacklist is used to store redundant knowledge blocks with invalid information to be deleted. When a redundant knowledge block in the gray list has been stored for more than 30 days, it will be transferred to the blacklist for deletion. After the redundant knowledge block is placed on the blacklist, it will be deleted from the knowledge base.

[0122] The above redundancy score is calculated based on the call information, and the specific calculation formula is:

[0123]

[0124] Where n is the number of dimensions, W i is the dimension weight, V i is a dimension value, including the call frequency, problem score and existence time, and N is the number of times the same problem occurs within a week.

[0125] For example, for the call information of a redundant knowledge block, including: call frequency 120 times, repeated call frequency 98 times, average question score 4.3 (out of 10 points), related ticket resolution rate 62%, knowledge base storage days 200 days, and similar questions appearing 10 times within a week, substitute into the above formula to obtain:

[0126]

[0127] The final result is 0.699+1.04.

[0128] The implementation of the embodiments of the present application has the following beneficial effects:

[0129] The embodiment of the present application uses multiple agents to manage the knowledge base differently, greatly reducing human access, so as to achieve automatic knowledge extraction and dynamic update, and improve the accuracy and timeliness of the knowledge base. First, the first agent is used to classify and prioritize the work orders containing questions, and provide data support for whether to store the problem content in the work order in the knowledge base or delete it from the knowledge base; the second agent is used to retrieve the corresponding answer in the knowledge base based on the work order information and output it; after the knowledge base update process is started, the third agent is used to store the information to be stored in the knowledge base of the work order to complete the update of the knowledge base; and the fourth agent is based on the score of the output question answer, and determines whether the answer to the question satisfies the customer to determine whether the corresponding knowledge block is redundant data. If it is redundant data, it will be deleted from the knowledge base or the corresponding search will be stopped to improve the retrieval efficiency and reduce the interference of redundant information on the retrieval of the correct answer.

[0130] Further, Figure 5 This is a structural diagram of a terminal device provided in one embodiment of the present application. Figure 5 As shown, the terminal device 3 of this embodiment includes: at least one processor 30 (in Figure 5 Only one is shown) and a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor. When the processor 30 executes the computer program 32, the steps of a multi-agent driven knowledge base expansion and correction method described in any one of the embodiments of the present application can be implemented.

[0131] The terminal device 3 may be a computing device such as a desktop computer, a cloud server, or a laptop computer. The computing device may include but is not limited to a processor 30 and a memory 31 . Figure 5 This is merely an example of the terminal device 3 and does not constitute a limitation on the terminal device 3 , which may include more or fewer components than those shown in the figure.

[0132] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above description is merely a specific embodiment of this application and is not intended to limit the scope of protection of this application. In particular, it should be noted that for those skilled in the art, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A knowledge base expansion and correction method based on multi-agent driving, characterized in that: include: Invoking the first agent to classify and prioritize the received work order, and generating a business tag for the work order; Invoke a second agent to search the knowledge base according to the service request of the work order, and obtain the answer to the question of the work order and the search frequency of similar questions; According to the business tags and the search frequency of similar questions, based on the preset expansion conditions, it is determined whether to start the knowledge base update process; After starting the knowledge base update process, calling the third agent to extract the to-be-entered information of the work order and storing the to-be-entered information in the knowledge base; Based on the question score of the answer to the question, a fourth agent is called to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base.

2. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 1, characterized in that: The calling of the first agent performs problem classification and priority determination on the received work order, and generates a business tag for the work order, specifically: Analyzing the work order through the first agent to determine the problem type and business scenario of the work order; Determine the priority label of the work order based on the problem type and business scenario through preset priority classification; The business tag is generated according to the problem type, business scenario and the priority tag.

3. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 1, characterized in that: The calling of the second agent searches the knowledge base according to the service request of the work order to obtain the answer to the question of the work order and the search frequency of similar questions, specifically: Searching a knowledge base according to the service request by a second agent to determine a knowledge block corresponding to the service request; Calling the knowledge block to output answers to questions in the work order, and recording the calling frequency of the knowledge block; According to the question type of the business tag, the search frequency of similar questions corresponding to the question type is updated.

4. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 1, characterized in that: The process of determining whether to start the knowledge base update process based on the business tag and the search frequency of similar questions and the preset expansion conditions is as follows: Scoring the answer to the question in combination with the priority tag of the business tag to obtain a question score; When the question type and business scenario of the business tag meet the expansion condition, and the search frequency of similar questions is greater than a first threshold and the question score is less than a second threshold, the knowledge base update process is started.

5. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 1, characterized in that: According to the question score of the answer to the question, the fourth agent is called to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base, specifically: When the question score is less than a third threshold, the knowledge base correction process is called by the fourth agent; After calling the knowledge base correction process, extract the call information of the knowledge block corresponding to the answer to the question and mark the knowledge block as a redundant knowledge block; wherein the call information includes the call frequency, the repeated call frequency, the frequency of being marked as low-scoring, the marked low-scoring score and the gray list storage duration; According to the call information, determining the processing method of the redundant knowledge block through the preset redundant information judgment conditions; wherein the processing method includes reducing the search weight, placing it in the gray list and placing it in the black list; The redundant knowledge block is processed according to the processing manner.

6. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 5, characterized in that: The method of processing the redundant knowledge block is determined according to the call information and a preset redundant information determination condition, specifically: If the weekly decrease in the call frequency of the redundant knowledge block is higher than a fourth threshold, the retrieval weight of the redundant knowledge block is reduced; wherein the retrieval weight is related to the probability of the knowledge block being retrieved in the knowledge base; Calculating a redundancy score of the redundant knowledge block based on the call information, and if the redundancy score is greater than a fifth threshold and less than or equal to a sixth threshold, or the frequency of being marked as low-scoring is lower than a seventh threshold, placing the redundant knowledge block in a gray list; If the redundancy score is greater than a sixth threshold, the redundant knowledge block is put into a blacklist.

7. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 6, characterized in that: The redundancy score of the redundant knowledge block is calculated by using the call information, specifically: The calculation formula of the redundancy score is: Where n is the number of dimensions, W i is the dimension weight, V i is a dimension value, including the call frequency, problem score and existence time, and N is the number of times the same problem occurs within a week.

8. The method for expanding and revising a knowledge base based on multi-agent driving according to claim 5, characterized in that: The graylist and blacklist are specifically: After the redundant knowledge block is put into the grey list, its search in the knowledge base is stopped but the corresponding metadata is retained; After the redundant knowledge block is put into the blacklist, the redundant knowledge block will be deleted from the knowledge base.

9. A knowledge base expansion and correction system based on multi-agent drive, characterized by: include: Business tag generation module, answer retrieval module, process startup module, knowledge base update module and redundant information processing module; The service tag generation module is used to call the first agent to classify and prioritize the received work orders and generate service tags for the work orders; The answer retrieval module is used to call the second agent to search the knowledge base according to the service request of the work order, and obtain the answer to the question of the work order and the search frequency of similar questions; The process starting module is used to determine whether to start the knowledge base update process based on the business tag and the search frequency of similar questions and the preset expansion conditions; The knowledge base update module is used to call the third agent to extract the to-be-entered information of the work order after starting the knowledge base update process, and store the to-be-entered information in the knowledge base; The redundant information processing module is used to call the fourth agent to perform redundant information processing on the knowledge block corresponding to the answer to the question in the knowledge base according to the question score of the answer to the question.

10. A terminal device, characterized in that: It includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of a multi-agent driven knowledge base expansion and correction method according to any one of claims 1 to 8.

Images