Knowledge base update methods, devices, equipment and storage media

Abstract

This invention discloses a knowledge base updating method, apparatus, device, and storage medium, relating to the field of artificial intelligence technology. The method filters service queries in the knowledge base by using a mixed similarity between the question records of user queries and the question records of service queries. This ensures the identification of target service queries highly similar to existing service queries in the knowledge base, effectively improving the understanding and matching accuracy of user questions rejected by the service assistant. Then, a second filtering of the target service queries is performed using a large language model, accurately recalling user queries with different expressions but the same intent as those in the knowledge base. This allows for updating the knowledge base based on user queries, improving the efficiency of updating data in the knowledge base while optimizing the service assistant's knowledge base so that the service assistant does not reject user queries in the future, improving the service assistant's question-and-answer accuracy and enhancing its service capabilities.

Claims

1. A knowledge base update method, characterized in that it includes: The knowledge base updating device acquires query information, including user query information rejected by the service assistant and service query information in the knowledge base. Based on the mixed similarity between the question records of the user query information and the question records of the service query information, the knowledge base updating device determines target service query information matching the user query information from the service query information. The question records are used to record the query text, semantic vector, and keywords of the query information. The knowledge base updating device uses a large language model to verify the matching between the user query information and the target service query information, obtaining a verification result. If the verification result indicates that the matching between the user query information and the target service query information passes the verification, the knowledge base updating device updates the knowledge base based on the user query information. Before the knowledge base updating device acquires the query information, the method further includes: when the service query information includes service standard query information and service similar query information corresponding to the service standard query information, the knowledge base updating device divides all candidate service query information in the knowledge base using N service standard query information as the dividing dimension to obtain a first independent data layer corresponding to each service standard query information. The first independent data layer includes each service standard query information and the candidate service similar query information corresponding to each service standard query information. The knowledge base updating device adjusts the candidate service-similar query information in the first independent data layer based on the sampling benchmark number and the first number of candidate service-similar query information in the first independent data layer, to obtain a second independent data layer corresponding to each service standard query information; wherein, when the sampling benchmark number is less than the first number, the knowledge base updating device removes a second number of candidate service-similar query information from the first independent data layer to obtain the second independent data layer, the second number being the difference between the first number and the sampling benchmark number; when the sampling benchmark number is greater than the first number, the knowledge base updating device expands the candidate service-similar query information in the first independent data layer to obtain expanded service-similar query information; the expanded service-similar query information is added to the first independent data layer to obtain the second independent data layer, wherein the number of expanded service-similar query information is a third number, the third number being the difference between the sampling benchmark number and the first number; the knowledge base updating device determines the query information in the second independent data layer corresponding to each service standard query information as the service query information.

2. The method as described in request item 1, wherein, Before the knowledge base updating device acquires the query information, the method further includes: the knowledge base updating device acquiring candidate user query information rejected by the service assistant; the knowledge base updating device inputting the candidate user query information into a pre-trained language model to obtain a recognition result output by the pre-trained language model, wherein the pre-trained language model is used to determine whether the candidate user query information is invalid user query information; and the knowledge base updating device, if the recognition result indicates that the candidate user query information is not invalid user query information, confirming the candidate user query information as the user query information.

3. The method as described in request item 1, wherein, One service standard query corresponds to one first independent data layer, and the N service standard query information corresponds to N first independent data layers; the method further includes: the knowledge base updating device summarizing the candidate service similar query information in the N first independent data layers to obtain the summarized number of candidate service similar query information in the N first independent data layers; the knowledge base updating device determining the median or symmetric value of the summarized number as the sampling reference number.

4. The method as described in request item 1, wherein, The method further includes: the knowledge base updating device inputs the user query information and the first prompt word into the large language model to obtain the first keyword of the user query information output by the large language model, the large language model being used to determine the keyword in the query information according to the first prompt word, the first prompt word being a prompt word that guides the large language model to identify at least one of the following in the query information: service entity, service terminology, operation process, service intent; the knowledge base updating device inputs the first query text of the user query information into a text embedding model to obtain the first semantic vector of the user query information output by the text embedding model, the text embedding model being used to convert the query text into a high-dimensional semantic vector; the knowledge base updating device determines the first query text, the first semantic vector, and the first keyword of the user query information as the question record of the user query information.

5. The method as described in request item 1, wherein, The method further includes: the knowledge base updating device inputting the service query information and the first prompt word into the large language model to obtain the second keyword of the service query information output by the large language model, wherein the large language model is used to determine the keyword in the query information according to the first prompt word, wherein the first prompt word includes prompt words that guide the large language model to identify at least one of the following in the query information: service entity, service terminology, operation process words; the knowledge base updating device inputting the second query text of the service query information into a text embedding model to obtain the second semantic vector of each query information output by the text embedding model, wherein the text embedding model is used to convert the query text into a high-dimensional semantic vector; and the knowledge base updating device determining the second query text, the second semantic vector, and the second keyword of the service query information as the question record of the service query information.

6. The method as described in request item 1, wherein, The knowledge base updating device determines target service inquiry information that matches the user inquiry information based on the mixed similarity between the question records of the user inquiry information and the question records of the service inquiry information. This includes: the knowledge base updating device determining service inquiry information in the service inquiry information whose mixed similarity with the question records of the user inquiry information is greater than a preset threshold as the target service inquiry information.

7. The method as described in claim 1 or 6, wherein, Before the knowledge base updating device determines the target service query information matching the user query information from the service query information based on the mixed similarity of the question records of the user query information and the question records of the service query information, the method further includes: the knowledge base updating device calculating a weighted keyword similarity between the first keyword and the second keyword based on the first keyword of the user query information and the second keyword of the service query information, wherein the weighted keyword similarity is used to characterize the degree of overlap between the first keyword and the second keyword; the knowledge base updating device calculating a cosine similarity between the first semantic vector and the second semantic vector based on the first semantic vector of the user query information and the second semantic vector of the service query information, wherein the cosine similarity is used to characterize the degree of closeness between the first semantic vector and the second semantic vector at the semantic level; and the knowledge base updating device performing a weighted summation of the weighted keyword similarity and the cosine similarity to obtain the mixed similarity between the question records of the user query information and the question records of the service query information.

8. The method as described in request item 1, wherein, The knowledge base updating device uses a large language model to verify the matching between the user query information and the target service query information, and obtains a verification result, including: the knowledge base updating device inputs the user query information, the target service query information, and a second prompt word into the large language model, and obtains a result from the large language model indicating whether the user query information and the target service query information match; if the large language model outputs a result indicating that the user query information and the target service query information match, the knowledge base updating device determines the verification result as the result of the matching between the user query information and the target service query information passing the verification; if the large language model outputs a result indicating that the user query information and the target service query information do not match, the knowledge base updating device determines the verification result as the result of the matching between the user query information and the target service query information failing the verification. The large language model is used to determine whether the matching between the user query information and the target service query information passes the verification according to the second prompt word. The second prompt word includes prompt words that guide the large language model to analyze at least one of the following: whether the user query information and the target service query information match in terms of operation process and / or service intent, whether the sentence structure of the user query information is complete, and whether the semantic scope of the user query information and the target service query information matches.

9. The method as described in claim 1, wherein, The target service query information includes target service standard query information or target service similar query information corresponding to the target service standard query information; the knowledge base updating device updates the knowledge base according to the user query information, including: when the target service query information includes the target service standard query information, the knowledge base updating device generates a data log corresponding to the user query information based on the user query information, the target service query information matching the user query information, and the verification result of the user query information and the target service query information; when the target service query information includes target service similar query information, the knowledge base updating device generates a data log corresponding to the user query information based on the user query information, the target service query information matching the user query information, the verification result of the user query information and the target service standard query information corresponding to the target service query information; the knowledge base updating device adds the data log corresponding to the user query information to a pending corpus to prompt the operations expert for review; If the data log corresponding to the user query information is approved, the knowledge base updating device will update the knowledge base by treating the user query information as service-similar query information corresponding to the target service standard query information.

10. The method as described in claim 1, wherein, The method further includes: the knowledge base updating device training the service assistant based on the updated knowledge base, so that the service assistant does not reject the user's query information.

11. A knowledge base updating device, characterized in that it comprises: The system comprises the following modules: an acquisition module for acquiring query information, including user query information rejected by the service assistant and service query information from the knowledge base; a determination module for determining target service query information matching the user query information based on the mixed similarity between the question records of the user query information and the question records of the service query information, wherein the question records record the query text, semantic vector, and keywords of the query information; a verification module for verifying the matching between the user query information and the target service query information using a large language model to obtain a verification result; and an update module for updating the knowledge base based on the user query information if the verification result indicates that the matching between the user query information and the target service query information has passed the verification. A segmentation module is used to segment all candidate service query information in the knowledge base using N service standard query information as the segmentation dimension when the service query information includes service standard query information and service similar query information corresponding to the service standard query information, to obtain a first independent data layer corresponding to each service standard query information. The first independent data layer includes each service standard query information and the candidate service similar query information corresponding to each service standard query information. An adjustment module is used to adjust the candidate service-similar query information in the first independent data layer according to the sampling benchmark number and the first number of candidate service-similar query information in the first independent data layer, to obtain a second independent data layer corresponding to each of the service standard query information; wherein, when the sampling benchmark number is less than the first number, the knowledge base updating device removes a second number of candidate service-similar query information from the first independent data layer to obtain the second independent data layer, the second number being the difference between the first number and the sampling benchmark number; when the sampling benchmark number is greater than the first number, the knowledge base updating device expands the candidate service-similar query information in the first independent data layer to obtain expanded service-similar query information; the expanded service-similar query information is added to the first independent data layer to obtain the second independent data layer, wherein the number of expanded service-similar query information is a third number, the third number being the difference between the sampling benchmark number and the first number; the determination module is used to determine the query information in the second independent data layer corresponding to each of the service standard query information as the service query information.

12. A computer device, characterized in that the computer device comprises: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the knowledge base update method as described in any one of requests 1 to 10.

13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the knowledge base update method as described in any one of claims 1 to 10.

Images