Knowledge base expansion method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a knowledge base expansion method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: inputting artificial dialogue data into a large language model, and extracting question and answer pairs from the artificial dialogue data through a two-stage prompt project; acquiring a plurality of question and answer pairs extracted from the plurality of groups of artificial dialogue data, and distributing the similar question and answer pairs to the same data set; and selecting a core question and answer pair from each data set, and adding the core question and answer pair into a knowledge base. According to the embodiment of the invention, the knowledge base can be effectively and transversely expanded, data redundancy is reduced, and storage resources are saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a knowledge base expansion method and device, equipment and a storage medium. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, intelligent customer service systems are gradually applied to various industries. Intelligent customer service systems can effectively reduce the intervention of manual customer service, thereby reducing the workload of staff and the investment cost of enterprises.

[0003] Intelligent customer service systems usually need to maintain a professional knowledge base. The knowledge base contains a large amount of domain knowledge, which can guide the intelligent customer service system to match and recommend correct answers. Therefore, the quality of the knowledge base and the coverage of user questions directly affect the intelligent level of the customer service system.

[0004] In order to improve the quality and accuracy of the knowledge base, the knowledge base can be continuously expanded. For example, the knowledge base can be expanded by constructing similar questions. However, constructing similar questions only increases the depth of the knowledge base for similar questions and cannot essentially improve the coverage of the knowledge base for different user questions. SUMMARY

[0005] The embodiments of the present application provide a knowledge base expansion method, device, equipment and storage medium, which can solve the technical problem that constructing similar questions cannot essentially improve the coverage of the knowledge base for different user questions. To achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0006] In a first aspect, the embodiments of the present application provide a knowledge base expansion method, comprising:

[0007] inputting artificial dialogue data into a large language model, and extracting question and answer pairs from the artificial dialogue data through a two-stage prompting process;

[0008] obtaining a plurality of question and answer pairs extracted from a plurality of groups of artificial dialogue data, and distributing similar question and answer pairs to the same data set;

[0009] selecting core question and answer pairs from each data set, and adding the core question and answer pairs to the knowledge base.

[0010] In a second aspect, the embodiments of the present application provide a knowledge base expansion device, comprising:

[0011] a question and answer pair extraction module configured to input artificial dialogue data into a large language model, and extract question and answer pairs from the artificial dialogue data through a two-stage prompting process;

[0012] The question and answer pair assignment module is configured to obtain a plurality of question and answer pairs extracted from a plurality of sets of artificial dialogue data respectively, and assign similar question and answer pairs to the same data set.

[0013] The core question and answer pair selection module is configured to select core question and answer pairs from each data set, and add the core question and answer pairs to the knowledge base.

[0014] In a third aspect, an embodiment of the present application provides a knowledge base expansion device, which comprises a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the method of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a plurality of instructions, and the instructions are adapted to be loaded and executed by a processor to perform the method of the first aspect.

[0016] In the embodiments of the present application, the powerful knowledge summarization capability of the large language model can accurately extract question and answer pairs from complex artificial dialogue data, and the two-stage prompt engineering can reduce the extraction difficulty of the large model, thereby realizing the rapid and effective horizontal expansion of the knowledge base, and essentially improving the coverage of the knowledge base for different user problems. Moreover, only the core question and answer pairs are selected and added to the knowledge base, which reduces data redundancy and saves storage resources.

[0017] In addition, in some embodiments provided by the present application, effective data, i.e., artificial dialogue data, can be first screened out from historical dialogue data as candidate data for extracting question and answer pairs. Then, sensitive information in the historical dialogue is identified and replaced by BERT-CRF and regularization methods, to avoid the problem of information leakage. Next, the large language model is used to extract question and answer pairs from the candidate historical dialogue, and the powerful knowledge summarization capability of the large model is used to extract the corresponding questions and answers from the complex historical dialogue question and answer, to realize the expansion of the existing knowledge base, thereby improving the interception efficiency of the robot. Finally, the clustering algorithm is used to cluster the question and answer pairs, and the core question and answer pairs are extracted and added to the knowledge base, which can filter question and answer pairs with different expressions but the same semantics, reduce redundant information, and improve the quality of the knowledge base. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 A flowchart of a knowledge base expansion method provided for an embodiment of the present application is shown in FIG. 1.

[0020] Figure 2 A training diagram of a text labeling model provided for an embodiment of the present application is shown in FIG. 2.

[0021] Figure 3 A flowchart of a question and answer pair distribution processing provided for an embodiment of the present application is shown in FIG. 3.

[0022] Figure 4 A flowchart of clustering question and answer pairs provided for an embodiment of the present application is shown in FIG. 4.

[0023] Figure 5 A flowchart of another knowledge base expansion method provided for an embodiment of the present application is shown in FIG. 5.

[0024] Figure 6 A structural diagram of a knowledge base expansion device provided for an embodiment of the present application is shown in FIG. 6.

[0025] Figure 7 A structural diagram of a knowledge base expansion device provided for an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0026] With the continuous development of artificial intelligence technology, intelligent customer service systems are gradually applied to various industries. More and more enterprises and institutions begin to establish their own websites and intelligent customer service systems to provide more personalized services for users. For example, in the e-commerce industry, intelligent customer service systems can answer user questions, provide shopping guidance, and handle after-sales problems. For example, in the banking and financial industry, intelligent customer service systems can provide around-the-clock service and handle business inquiries, account information inquiries, and other services. Intelligent customer service systems can effectively reduce the involvement of human customer service, thereby reducing the workload of staff and the investment cost of enterprises.

[0027] A knowledge base is an important part of an intelligent customer service system. The knowledge base contains a large amount of domain knowledge, which can guide the intelligent customer service system to match and recommend correct answers, so that the intelligent customer service system can provide more accurate and efficient services than human. It can be seen that an intelligent customer service system usually needs to maintain a professional knowledge base, and the quality of the knowledge base and the coverage of user questions directly affect the intelligence level of the customer service system.

[0028] Improving the coverage of the knowledge base to user questions can effectively improve the interception rate of the customer service robot, thereby reducing the intervention of manual customer service. The general knowledge base construction method is manual construction. Specifically, some frequently asked questions can be preset and stored in the knowledge base after structured processing. However, the efficiency of constructing the knowledge base is low, and the input cost is large.

[0029] In order to improve the expansion efficiency of the knowledge base, similar questions can also be constructed by methods such as translation enhancement and synonym replacement of questions in the knowledge base, and the similar questions are added to the knowledge base. However, constructing similar questions only increases the depth of the knowledge base to similar questions, and cannot essentially improve the coverage of the knowledge base to different user questions. In other words, the above example can only expand the depth of the knowledge base, but cannot increase the coverage field and knowledge points of the knowledge base.

[0030] In addition, the intelligent customer service system can also update and optimize the knowledge base according to the user's questions and answers, including extracting user questions from the historical dialogue records of the intelligent customer service system, distinguishing keywords and non-keywords from the questions, and processing the non-keywords by replacing or deleting, etc. After processing, the user questions and corresponding answers are stored in the knowledge base. However, in the multi-round dialogue customer service scene, it is difficult to extract the core questions from the complex historical dialogue.

[0031] Based on this, the embodiment of the present application provides a knowledge base expansion method. The powerful knowledge summarization capability of a large language model (LLM or large model for short) is used, and the question-answer pairs (question-answer pairs for short) are extracted from the historical dialogue data through two-stage Prompt engineering, and the core question-answer pairs are selected from the extracted many question-answer pairs and added to the knowledge base. The existing knowledge base is quickly and effectively expanded horizontally, and the coverage of the knowledge base to different user questions is essentially improved.

[0032] It is worth mentioning that the core question-answer pair can be any one of multiple similar question-answer pairs with different expressions but the same semantics, and the present application does not limit this. Only the core question-answer pair is selected and added to the knowledge base, which can reduce data redundancy and save storage resources.

[0033] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings.

[0034] It should be clear that the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the scope of protection of the present application.

[0035] The following description refers to the accompanying drawings. Unless otherwise noted, same or similar components in different drawings have same or similar reference numerals. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are simply examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0036] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily represent a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. "And / or", which describes the association between objects, means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0037] In one embodiment, referring to Figure 1 The knowledge base expansion method provided by the embodiments of the present application can specifically include the following steps:

[0038] S102, inputting the artificial dialogue data into a large language model, and extracting question and answer pairs from the artificial dialogue data through a two-stage prompting engineering.

[0039] In implementation, the historical dialogue data generated by the customer service system includes many types. The large language model can be any one of chatgpt, wenxin yiyang, xunfei xinghuo, etc.

[0040] For example, the first type of historical dialogue data is dialogue data in which the user asks a question and the customer service robot directly replies. Since the user's question involved in this type of dialogue data has been solved by the current customer service system, it means that the current knowledge base can match the correct answer to the question raised by the user, so extracting question and answer pairs from this data for knowledge base expansion has little gain in intercept rate improvement.

[0041] For another example, the second type of historical dialogue data is data when the user closes the customer service system directly after asking a question. Since this type of dialogue data does not include the answer of the customer service reply, it belongs to invalid dialogue data.

[0042] For example, the third type of historical dialogue data is: after the user asks a question, the answer of the customer service robot is not satisfactory, and the user transfers to the manual customer service, and finally the manual customer service completes the answer to the user's question. Since this type of dialogue data contains complete user questions and manual customer service answers, if the question-answer pair can be extracted from this dialogue data and added to the knowledge base, the customer service robot can correctly answer the user's question the next time the user asks a similar question. Therefore, before step S102 is performed, valid data can be selected from historical dialogue data. That is, in implementation, manual dialogue data can be selected as candidate data for extracting question-answer pairs to subsequently expand the knowledge base.

[0043] In one embodiment, the specific processing of step S102 can include: generating a first prompt sentence based on the manual dialogue data, and inputting the first prompt sentence into the large language model to obtain a core question output by the large language model; generating a second prompt sentence based on the manual dialogue data and the core question, and inputting the second prompt sentence into the large language model to obtain a core answer output by the large language model; and generating a question-answer pair corresponding to the manual dialogue data based on the core question and the core answer.

[0044] In implementation, a set of manual dialogue data can include multiple rounds of dialogue data generated by multiple dialogues between a user and a customer service. Such dialogue data contains continuous questions, answers and feedback, and some also contain sentences unrelated to user questions, such as chatting, repeated content, off-topic content, and typos in the dialogue, which are more complex than single question-answer interaction. Therefore, the large language model can be used to extract question-answer pairs from each set of manual dialogue data.

[0045] The present application divides the extraction task into two steps: first, use a large model to extract user questions, and then input the historical dialogue and user questions into the large model to extract questions and answers, which can reduce the difficulty of the large model. For example, if the large language model uses chatgpt, the prompt engineering of the first stage is to extract the core question of the user's question by constructing the following prompt:

[0046] question_prompt='Help me extract the core question of the user's question: \n' + dialogue.

[0047] Where \n is a line break, usually used as a format control in output; dialogue represents the specific dialogue record content.

[0048] The prompt engineering of the second stage is to obtain the corresponding question answer by constructing the following prompt:

[0049] answer_prompt = dialogue +'The question asked by the user is:'+ question +'Answer the question in the tone of a customer service representative: '

[0050] wherein question represents the user question extracted in the previous stage.

[0051] In one embodiment, the manual dialogue data collected in the customer service system includes:

[0052] Manual customer service: Welcome to enter the customer service of Xiaoxi, I am your exclusive customer service, I will help you solve the problem

[0053] User: Hello

[0054] User: How to cast a notebook computer to an all-in-one machine

[0055] User: Urgent

[0056] Manual customer service: You can cast through live or software here

[0057] Manual customer service: Can you please provide the serial number of the large screen on your side

[0058] Manual customer service: I will provide you with relevant solutions

[0059] User: 12345678912345

[0060] Manual customer service: You can cast through HDMI cable on your device

[0061] Manual customer service: Connect one end to the HDMI interface of your large screen and the other end to your notebook computer

[0062] User: I forgot

[0063] Manual customer service: Xi image screen website (sending end) Xi image screen receiving end

[0064] Manual customer service: Please install the sending end on your external computer

[0065] Manual customer service: Install the receiving end on your large screen computer system

[0066] Manual customer service: Connect both sides to the same Wi-Fi

[0067] Manual customer service: The receiving end requires three file packages to be downloaded and installed on the large screen

[0068] Manual customer service: This session is over, thank you for your attention to Xiaoxi. If the current problem is not solved, you can input "manual" to connect again.

[0069] For the above user and artificial customer service dialogue data, first use dialogue to represent the above dialogue content, and the first stage prompt is constructed as: "question_prompt='Help me extract the key question asked by the user: \n'+dialogue", and the prompt is input into the large language model chatgpt. The key question output by chatgpt is as follows: "How to project a notebook computer to an all-in-one computer?".

[0070] Then, the dialogue content and user question are spliced into a new prompt to obtain the answer corresponding to the key question. The second stage prompt is constructed as: "answer_prompt=dialogue+'The question asked by the user is:'+question+'Answer the question in the customer service tone for the user: '". The constructed prompt is input into the large language model chatgpt again, and the answer corresponding to the key question extracted in the first stage is shown in Table 1.

[0071] Table 1: Question and answer pairs extracted based on two-stage prompt engineering

[0072]

[0073] Through the above method, the key question and answer pair can be extracted from a multi-round dialogue data. By executing the same process on all dialogue data in the historical data, a large number of question and answer pairs can be obtained to expand the knowledge base.

[0074] In an embodiment, during the conversation between the user and the customer service, some sensitive data may be mentioned. For example, the user's name, address, phone number, bank account number, etc. In order to avoid data leakage and ensure personal privacy and data security, before inputting the artificial dialogue data into the large language model, the sensitive data in each group of the artificial dialogue data can be identified and desensitized.

[0075] In implementation, different types of sensitive data can be identified based on different methods, and the identified sensitive data can be desensitized, and the knowledge base can be expanded based on the desensitized data. Desensitization can be masking part or all of the sensitive data with specified characters, or encrypting part or all of the sensitive data with specified encoding format, or other desensitization methods, which are not limited by the present application. For example, the user's name "Li Xiaoming" can be replaced with "XXX" or "Li XX". For another example, the phone number "12345678910" can be replaced with "123****8910".

[0076] In one embodiment, the sensitive data includes sensitive numbers, and correspondingly, the process of identifying the sensitive data in each group of artificial dialogue data can specifically include: identifying the sensitive numbers in each group of the artificial dialogue data by using a regular expression.

[0077] In implementation, the sensitive numbers usually have regularity, and thus can be directly identified and replaced by using a regular expression. For example, the mobile phone numbers in China are mostly composed of 11 digits, usually starting with 1, the second digit being any one of 3-9, and the last 9 digits being any number, and correspondingly, the regular expression for matching the Chinese mobile phone numbers can be pattern = re.compile(r'\b1[3-9]\d{9}\b'). Among them, \b is a word boundary, which ensures that the phone number is an independent word, rather than part of other number strings; \d{9} means matching 9 consecutive numbers. For another example, the Chinese patent application numbers are usually composed of the country code CN and 13 digits, and correspondingly, the regular expression for matching the Chinese patent application numbers can be pattern = re.compile(r'CN\d{13}').

[0078] In another embodiment, the sensitive data includes sensitive text, and correspondingly, the process of identifying the sensitive data in each group of artificial dialogue data can specifically include: identifying the sensitive text in each group of the artificial dialogue data by using a named entity recognition algorithm.

[0079] In implementation, the Named Entity Recognition (NER) method can be used for identification and replacement. Among them, the BERT-CRF, BiLSTM-CRF, BERT-BiLSTM-CRF and other models can be used for identification of sensitive information.

[0080] In addition, the type of sensitive text can be specified in advance. Taking the sensitive text including specific product model, user's name and address as an example, first, a batch of dialogue data can be collected in advance, and the corresponding product model, user's name and address are labeled in the form of BIO (Begin Inside Outside, three-bit sequence labeling method). The candidate labels of each character position can include: B-Person, I-Person, B-Location, I-Location, B-Product, I-Product, O. Among them, B-Person represents the beginning position of the name; I-Person represents the middle position of the name; B-Location represents the beginning position of the address; I-Location represents the middle position of the address; B-Product represents the beginning position of the product model; I-Product represents the middle position of the product model; O represents other categories. For example, seeFigure 2 The pre-collected dialogue data includes the text "Zhang San, Guangzhou City, ". In BIO form, "Zhang San, Guangzhou City, " can be annotated as "B-Person", "I-Person", "O", "B-Location", "I-Location", "I-Location", and "O" word by word. Then, the annotated data can be used as training data to train the BERT-CRF model.

[0081] It is worth mentioning that the pre-collected dialogue data can come from the current customer service system or from a public data set, and the present application does not limit this.

[0082] After training the BERT-CRF model, for artificial dialogue data input into a large language model, the BERT-CRF model can be used to predict whether each group of artificial dialogue data contains product models, names, addresses, and other sensitive text. If there is, the corresponding character position is masked. For example, the specified character "XXX" is used instead of the user's name.

[0083] It should be noted that, due to the limitation of the length, all the embodiments are not enumerated in the present application, and any features that are not mutually contradictory can be freely and randomly combined to become optional embodiments of the present application. For example, for any group of artificial dialogue data, various types of sensitive numbers and various types of sensitive text can be identified together.

[0084] S104, obtaining multiple question and answer pairs extracted from multiple groups of artificial dialogue data, and distributing similar question and answer pairs to the same data set.

[0085] In implementation, all question and answer pairs extracted from multiple groups of artificial dialogue data can be distributed to multiple data sets according to the distribution of the question and answer pairs. Among them, more similar question and answer pairs are divided into the same data set. Among them, the question and answer pairs can be matched by keywords, semantic analysis and other methods. By comparing the text content of the question and answer pairs, the same or similar question and answer pairs are found out; the key information of the question and answer pairs can be extracted by natural language processing technology, so as to judge the similarity of the question and answer pairs; the similar question and answer pairs are automatically identified by using clustering algorithm, deep learning and other machine learning algorithms. Of course, the present application can select any one of the ways to identify similar question and answer pairs according to actual needs, and the present application does not limit this.

[0086] In an embodiment, a clustering algorithm can be employed to assign similar question-answer pairs to the same data set. Accordingly, the specific processing of step S104 can include: selecting a preset number of question-answer pairs from the plurality of question-answer pairs as initial clustering centers, and assigning each question-answer pair to the same data set with the nearest clustering center; updating the clustering centers according to the data distribution of each data set, and reassigning each question-answer pair according to the updated clustering centers until the clustering centers are stable.

[0087] wherein the preset number can be any value, which is usually set based on the characteristics of sample data and business requirements. In other words, the preset number can be different for different data and different businesses.

[0088] For example, in an embodiment, the technical personnel sets the preset number, i.e., the number of initial clustering centers, to 1000 based on the understanding of the data. Accordingly, the specific processing of step S104 can refer to Figure 3 , including:

[0089] S302, selecting a preset number of question-answer pairs from the plurality of question-answer pairs as initial clustering centers.

[0090] In implementation, referring to Figure 4 , for question-answer pairs extracted by large models such as chatgpt, a pre-training model can be used to vectorize each question-answer pair first, and then any one of Kmeans, Kmeans++, etc. clustering algorithm is used to cluster all question-answer pairs, and then the core question-answer pair is obtained based on the calculation of the similarity algorithm. Representing text as a vector can improve computational efficiency on the one hand, and on the other hand, it can facilitate distance calculation processing in subsequent steps.

[0091] wherein the model used for vectorization processing can be any one of roformer (Rotary Transformer, Rotary Positional Encoder), bert (Bidirectional Encoder Representations from Transformers, Bidirectional Semantic Encoding Representation Model), roberta (Robustly Optimized BERT Pre-training Approach, Robustly Optimized BERT Pre-training Approach), etc. The present application does not make any limitation.

[0092] In an embodiment, the Kmeans clustering algorithm can be adopted, and 1000 question-answer pairs are randomly selected from all the extracted question-answer pairs as initial clustering centers. The Kmeans++ clustering algorithm can also be adopted, 1 question-answer pair is randomly selected as the initial first clustering center, then the shortest distance between each question-answer pair and all current clustering centers is calculated, and the probability of each question-answer pair being selected as the next clustering center is calculated according to the shortest distance, the sample point corresponding to the maximum probability value is selected as the next clustering center, and the above step is repeated until 1000 initial clustering centers are selected. Of course, other arbitrary clustering algorithms can also be selected to select clustering centers and perform distribution processing on each question-answer pair, which is not limited in the present application.

[0093] In S304, the target question-answer pair is obtained, and the distance between the target question-answer pair and each clustering center is calculated.

[0094] For ease of description, any one question-answer pair can be referred to as a target question-answer pair.

[0095] In S306, the target question-answer pair and the nearest clustering center are assigned to the same data set.

[0096] In S308, whether all question-answer pairs are assigned?

[0097] In S310, the clustering centers are updated according to the data distribution of each data set.

[0098] In S312, whether the clustering center is stable?

[0099] In implementation, after multiple iterations of steps S304-S312, the clustering center will no longer change or change very little, at this time the clustering center is stable, and the clustering center can no longer be updated.

[0100] In S314, a preset number of data sets are obtained.

[0101] In implementation, the data sets where the final 1000 clustering centers are located are the 1000 distributed data sets. For each data set, only the core question-answer pairs are imported into the knowledge base.

[0102] For the question-answer pairs extracted in S102, Figure 3The embodiment shown uses a clustering algorithm to cluster the question-answer pairs. In the questions asked by the user history, there are many questions asking the same question. If all these question-answer pairs are stored in the knowledge base, there will be a large number of repeated questions in the knowledge base, which will reduce the efficiency of querying the knowledge base. Therefore, the embodiment filters question-answer pairs that express different but have the same semantics, which can reduce redundant information and improve the quality of the knowledge base. And using a clustering algorithm to assign similar question-answer pairs to the same data set (i.e., the same category), on the one hand, it can have high computational efficiency and can run quickly on large data sets; on the other hand, the clustering algorithm has no strict requirements for the category and distribution of data, so it can process various types of data and has high flexibility. In short, the embodiment can flexibly and efficiently classify a large number of question-answer pairs.

[0103] S106, selecting core question-answer pairs from each data set and adding the core question-answer pairs to the knowledge base.

[0104] In implementation, the core question-answer pairs can be selected based on any one of cosine similarity (Cosine Similarity), Euclidean distance (Euclidean Distance), Hamming distance (Hamming Distance), etc. Correspondingly, the process of selecting core question-answer pairs from each data set can specifically include: calculating the similarity of each question-answer pair in each data set with the cluster center in each data set; selecting the question-answer pair with the highest similarity with the cluster center in each data set as the core question-answer pair.

[0105] In one embodiment, the similarity algorithm can be used to select the core question-answer pairs. First, the cosine similarity of each sentence (core question-answer pair) with the cluster center of the category it belongs to is calculated, and the calculation formula can be:

[0106]

[0107] wherein S i represents the i-th element of the sentence S, and T i represents the i-th element of the sentence T.

[0108] Then, the sentence s with the highest similarity with the cluster center T in all question-answer pairs (S 1 , S 2 , …, S m ) is taken as the core question-answer pair, and the calculation formula can be:

[0109] s = argmax (cos (S 1 , T), cos (S 2 , T), …, cos (S m , T))

[0110] wherein T represents a cluster center of a certain category, S in represents a certain question-answer pair in the category.

[0111] Based on the above method, only the core question-answer pairs of each data set are added to the knowledge base, which can effectively filter similar question-answer pairs, reduce data redundancy, and improve the quality of the knowledge base.

[0112] Each embodiment provided in the present application can accurately extract question-answer pairs from complex artificial dialogue data by using the powerful knowledge summarization capability of the large language model, and can reduce the extraction difficulty of the large model by using the two-stage prompt engineering, thereby realizing the rapid and effective horizontal expansion of the knowledge base, and essentially improving the coverage of the knowledge base for different user problems. Moreover, only the core question-answer pairs are selected and added to the knowledge base, which reduces data redundancy and saves storage resources.

[0113] In addition, reference can be made to Figure 5 In some embodiments provided in the present application, effective data, i.e. artificial dialogue data, can be first screened from historical dialogue data as candidate data for extracting question-answer pairs. Then, sensitive information in the historical dialogue is identified and replaced by BERT-CRF and regularization methods to avoid information leakage. Next, the large language model is used to extract question-answer pairs from the candidate historical dialogue, and the powerful knowledge summarization capability of the large model is used to extract the corresponding questions and answers from the complex historical dialogue question-answer pairs through the two-stage prompt engineering, to realize the expansion of the existing knowledge base, thereby improving the interception efficiency of the robot. Finally, the clustering algorithm is used to cluster the question-answer pairs, and the core question-answer pairs are extracted and added to the knowledge base, which can filter question-answer pairs with different expressions but the same semantics, reduce redundant information, and improve the quality of the knowledge base.

[0114] It is worth mentioning that after the large language model extracts the question-answer pairs from the dialogue data, the subsequent clustering can be based only on the user questions in the question-answer pairs, and different types of user questions are divided into different data sets. Then, the core questions are selected from each data set, and the core questions and the corresponding question answers are added to the knowledge base. In this way, compared with clustering based on question-answer pairs, the computational complexity can be reduced and the computational efficiency can be improved.

[0115] Based on the same technical concept, the present application also provides a knowledge base expansion device, which can be seen from Figure 6 The device can include:

[0116] A question-answer pair extraction module for inputting artificial dialogue data into a large language model and extracting question-answer pairs from the artificial dialogue data through a two-stage prompt engineering;

[0117] The question and answer pair distribution module is configured to obtain a plurality of question and answer pairs extracted from a plurality of sets of artificial dialogue data, and distribute similar question and answer pairs to the same data set.

[0118] The core question and answer pair selection module is configured to select a core question and answer pair from each data set and add the core question and answer pair to the knowledge base.

[0119] Optionally, the question and answer pair extraction module can be specifically configured to:

[0120] generate a first prompt sentence based on the artificial dialogue data, input the first prompt sentence into the large language model, and obtain a core question output by the large language model;

[0121] generate a second prompt sentence based on the artificial dialogue data and the core question, input the second prompt sentence into the large language model, and obtain a core answer output by the large language model;

[0122] generate a question and answer pair corresponding to the artificial dialogue data based on the core question and the core answer.

[0123] Optionally, the question and answer pair distribution module can be specifically configured to:

[0124] select a preset number of question and answer pairs from the plurality of question and answer pairs as initial clustering centers, and distribute each question and answer pair to the same data set with the nearest clustering center;

[0125] update the clustering centers according to the data distribution of each data set, and re-distribute each question and answer pair according to the updated clustering centers until the clustering centers are stable.

[0126] Optionally, the core question and answer pair selection module can be specifically configured to:

[0127] calculate the similarity of each question and answer pair in each data set to the clustering center in each data set;

[0128] select the question and answer pair with the highest similarity to the clustering center in each data set as the core question and answer pair.

[0129] Optionally, the similarity can be a cosine similarity.

[0130] Optionally, the knowledge base expansion device can further include:

[0131] The desensitization module is configured to identify sensitive data in each set of artificial dialogue data and perform desensitization processing on the sensitive data.

[0132] Optionally, the sensitive data includes sensitive numbers; correspondingly, the desensitization module is specifically configured to identify the sensitive numbers in each group of the artificial dialogue data by using a regular expression.

[0133] Optionally, the sensitive data includes sensitive text; correspondingly, the desensitization module is specifically configured to identify the sensitive text in each group of the artificial dialogue data by using a named entity recognition algorithm.

[0134] It should be noted that the knowledge base expansion device provided in the above embodiments only divides the above functions for example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the knowledge base expansion device and the knowledge base expansion method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0135] The knowledge base expansion device provided in each of the above embodiments can accurately extract question and answer pairs from complex artificial dialogue data by using the powerful knowledge summarization capability of the large language model, and can reduce the extraction difficulty of the large model by using the two-stage prompt engineering, thereby realizing the rapid and effective horizontal expansion of the knowledge base, and essentially improving the coverage of the knowledge base for different user problems. Moreover, only the core question and answer pairs are selected and added to the knowledge base, which reduces data redundancy and saves storage resources.

[0136] Based on the same technical concept, the embodiments of the present application also provide a knowledge base expansion device. Referring to Figure 7 The device includes a processor and a memory, the memory stores a computer program, the computer program is suitable for being loaded and executed by the processor, and the computer program is suitable for being loaded and executed by the processor to implement the knowledge base expansion method provided in any of the above embodiments.

[0137] It should be noted that the knowledge base expansion device provided in the above embodiments can be any electronic device such as a server, and it belongs to the same concept as the knowledge base expansion method, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0138] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary universal hardware platforms through the description of the above embodiments, and the various embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing an electronic device to execute the method described in each embodiment or some parts of the embodiment.

[0139] The above merely describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A knowledge base expansion method, characterized in that: The method comprises: Inputting human dialogue data into a large language model, generating a first prompt sentence based on the human dialogue data, and inputting the first prompt sentence into the large language model to obtain a core question output by the large language model; generating a second prompt sentence based on the manual dialogue data and the core question, and inputting the second prompt sentence into the large language model to obtain a core answer output by the large language model; Based on the core question and the core answer, generating a question-answer pair corresponding to the artificial dialogue data; Obtaining multiple question-answer pairs extracted from multiple sets of manual conversation data, and assigning similar question-answer pairs to the same data set; Core question-answer pairs are selected from each of the data sets, and the core question-answer pairs are added to the knowledge base.

2. The method according to claim 1, wherein The obtaining of multiple question-answer pairs extracted from multiple groups of manual conversation data, and assigning similar question-answer pairs to the same data set, specifically includes: Selecting a preset number of question-answer pairs from the plurality of question-answer pairs as initial cluster centers, and assigning each of the question-answer pairs and the cluster center closest to them to the same data set; The cluster centers are updated according to the data distribution of each data set, and each question-answer pair is redistributed according to the updated cluster centers until the cluster centers are stable.

3. The method according to claim 1, wherein The selecting of core question-answer pairs from each of the data sets specifically includes: Calculating the similarity between each question-answer pair in each of the data sets and the cluster center in each of the data sets respectively; The question-answer pair with the highest similarity to the cluster center in each of the data sets is selected as the core question-answer pair.

4. The method according to claim 3, wherein The similarity is cosine similarity.

5. The method according to claim 1, wherein Before inputting the artificial dialogue data into the large language model, the method further includes: Identify sensitive data in each group of manual conversation data and perform desensitization processing on the sensitive data.

6. The method according to claim 5, wherein The sensitive data includes sensitive numbers; The identifying of sensitive data in each group of manual conversation data specifically includes: using regular expressions to identify the sensitive numbers in each group of manual conversation data.

7. The method according to claim 5, wherein The sensitive data includes sensitive text; The identifying of sensitive data in each group of the manual conversation data specifically includes: using a named entity recognition algorithm to identify the sensitive text in each group of the manual conversation data.

8. A knowledge base expansion device, characterized in that: The device comprises: A question-answer pair extraction module, which is used to input human conversation data into a large language model and extract question-answer pairs from the human conversation data through a two-stage prompting process; A question-answer pair assignment module is used to obtain multiple question-answer pairs extracted from multiple groups of manual conversation data, and assign similar question-answer pairs to the same data set; The core question-answer pair selection module is used to select core question-answer pairs from each of the data sets and add the core question-answer pairs to the knowledge base.

9. A knowledge base expansion device, characterized in that: The device comprises: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

Images