Question and answer method and device and storage medium

Abstract

The invention provides a question-answering method and device and a storage medium, relates to the technical field of artificial intelligence, and is used for solving the problem that answers generated by a traditional knowledge base question-answering technology are not accurate enough. The method comprises the following steps: acquiring a current question; based on the current problem, a target similar problem is obtained, and the similarity between the target similar problem and the current problem is larger than a preset similarity threshold value. And based on all questions, determining a target knowledge unit from a preset knowledge base, the all questions including a current question and a target similar question. And obtaining a target answer based on the target knowledge unit.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a question-answering method, device, and storage medium. Background Art

[0002] Knowledge base question answering technology is a natural language processing technology based on knowledge base, which can understand and answer questions raised by users in natural language. However, the documents retrieved by traditional knowledge base question answering technology are weakly related to the questions, resulting in inaccurate answers.

[0003] Therefore, how to improve the accuracy of the generated answers has become a problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a question-answering method, device, and storage medium for solving the problem that answers generated by traditional knowledge base question-answering technology are not accurate enough.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a question-answering method. In the method, a current question is obtained. Based on the current question, a target similar question is obtained, and the similarity between the target similar question and the current question is greater than a preset similarity threshold. Based on all questions, a target knowledge unit is determined from a preset knowledge base, and all questions include: the current question and the target similar question. Based on the target knowledge unit, a target answer is obtained.

[0007] Based on the above technical solution, the current question can be obtained. Afterwards, a target similar question can be obtained based on the current question, and the similarity between the target similar question and the current question is greater than a preset similarity threshold. In this way, the target similar question has a high similarity with the current question, and the matching knowledge unit will have a stronger relevance to the current question. Afterwards, the target knowledge unit can be determined from a preset knowledge base based on all questions, and all questions include: the current question and the target similar question. Afterwards, the target answer can be obtained based on the target knowledge unit. In this way, by combining the current question and the target similar question, more target knowledge units that are more closely related to the current question can be obtained, thereby generating more accurate and richer answers.

[0008] In a possible design, based on all questions, a target knowledge document is determined from a preset knowledge base, including: obtaining a first answer to each question in all questions, and determining a target knowledge unit matching all the first answers from the preset knowledge base.

[0009] In a possible design, based on all the first answers, a target knowledge unit is determined from a preset knowledge base, including: fusing all the first answers to obtain a second answer. Based on the second answer, a target knowledge unit is determined from a preset knowledge base.

[0010] In a possible design, all first answers are merged to obtain a second answer, including: for each first answer, obtaining the semantic similarity between the first answer and the corresponding question to obtain the semantic similarity corresponding to each first answer. Based on the semantic similarity corresponding to each first answer, a weight value of each first answer is determined. Based on the weight value of each first answer, all first answers are merged to obtain a second answer.

[0011] In one possible design, based on the second answer, determining the target knowledge unit from the preset knowledge base includes: processing the second answer and the preset knowledge unit in the preset knowledge base based on the first information retrieval algorithm to obtain a first score of the preset knowledge unit, the first score being used to indicate the degree of association between the preset knowledge unit and the second answer. Processing the second answer and the preset knowledge unit based on the second information retrieval algorithm to obtain a second score of the preset knowledge unit, the second score being used to indicate the degree of association between the preset knowledge unit and the second answer. Determining the target knowledge unit from the preset knowledge base based on the first score and the second score of the preset knowledge unit.

[0012] In a possible design, data processing is performed on the second answer and the knowledge unit in the preset knowledge base based on a keyword retrieval algorithm to obtain a first score for the preset knowledge unit, including: obtaining document information and a target adjustment factor of the preset knowledge unit, the document information including an inverse document frequency, a document length, and a target word frequency, the target word frequency being the frequency of occurrence of each word in the preset knowledge unit in the preset knowledge unit, and the target adjustment factor being used to indicate the influence of the words shared by each question and the preset knowledge unit on the first score. Based on the target adjustment factor, the document length, and the target word frequency, an objective function is generated. The objective function is multiplied by the inverse document frequency to obtain the first score.

[0013] In a second aspect, the present application provides a question-answering device, which includes an acquisition module and a processing module.

[0014] The acquisition module is used to acquire the current question. The processing module is used to obtain a target similar question based on the current question, where the similarity between the target similar question and the current question is greater than a preset similarity threshold. The processing module is also used to determine a target knowledge unit from a preset knowledge base based on all questions, where all questions include: the current question and the target similar question. The processing module is also used to obtain a target answer based on the target knowledge unit.

[0015] In a possible design, the acquisition module is used to acquire the first answer of each question in all the questions, and the processing module is used to determine the target knowledge unit matching all the first answers from the preset knowledge base.

[0016] In a possible design, the processing module is used to merge all the first answers to obtain a second answer. The processing module is also used to determine a target knowledge unit from a preset knowledge base based on the second answer.

[0017] In a possible design, the acquisition module is used to acquire the semantic similarity between each first answer and the corresponding question to obtain the semantic similarity corresponding to each first answer. The processing module is used to determine the weight value of each first answer based on the semantic similarity corresponding to each first answer. The processing module is also used to merge all the first answers based on the weight value of each first answer to obtain the second answer.

[0018] In one possible design, the processing module is used to process the second answer and the preset knowledge unit in the preset knowledge base based on the first information retrieval algorithm to obtain a first score of the preset knowledge unit, and the first score is used to indicate the association between the preset knowledge unit and the second answer. The processing module is also used to perform data processing on the second answer and the preset knowledge unit based on the second information retrieval algorithm to obtain a second score of the preset knowledge unit, and the second score is used to indicate the association between the preset knowledge unit and the second answer. The processing module is also used to determine the target knowledge unit from the preset knowledge base based on the first score and the second score of the preset knowledge unit.

[0019] In a possible design, the acquisition module is used to acquire document information and a target adjustment factor of a preset knowledge unit, the document information includes an inverse document frequency, a document length, and a target word frequency, the target word frequency is the frequency of occurrence of each word of the preset knowledge unit in the preset knowledge unit, and the target adjustment factor is used to indicate the influence of the words shared by each question and the preset knowledge unit on the first score. The processing module is used to generate an objective function based on the target adjustment factor, the document length, and the target word frequency. The processing module is also used to multiply the objective function by the inverse document frequency to obtain the first score.

[0020] In a third aspect, the present application provides a question-and-answer device, comprising: a processor and a memory; the processor and the memory are coupled; the memory is used to store one or more programs, the one or more programs comprising computer-executable instructions, and when the question-and-answer device is running, the processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method described in the above-mentioned first aspect and any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the method described in the first aspect and any possible implementation of the first aspect.

[0023] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to execute the method described in the above-mentioned first aspect and any possible implementation manner of the first aspect.

[0024] In the above scheme, the technical problems that can be solved by the question-and-answer device, computer equipment, computer storage medium, chip or computer program product and the technical effects achieved can be referred to the technical problems and technical effects solved by the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A system architecture diagram of a question-answering system provided in an embodiment of the present application;

[0026] Figure 2 A flowchart of a question-and-answer method provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of another question-and-answer method provided in an embodiment of the present application;

[0028] Figure 4 A flowchart of another question-and-answer method provided in an embodiment of the present application;

[0029] Figure 5 A flowchart of another question-and-answer method provided in an embodiment of the present application;

[0030] Figure 6 An example schematic diagram of a merging and deduplication method provided in an embodiment of the present application;

[0031] Figure 7 A flowchart of another question-and-answer method provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of a question-and-answer device provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of the structure of another question-and-answer device provided in an embodiment of the present application;

[0034] Fig.10 A conceptual partial view of a computer program product provided for an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] The terms “first” and “second” in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.

[0037] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include other steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0038] In addition, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present concepts in a specific way.

[0039] Knowledge base question answering technology is a natural language processing technology based on knowledge base, which can understand and answer questions raised by users in natural language. Knowledge base question answering technology includes three steps: question understanding, knowledge base query, answer generation and return.

[0040] Question understanding includes natural language processing and semantic parsing. Natural language processing can perform word segmentation, part-of-speech tagging, and syntactic analysis on natural language questions input by users to extract key information and understand the semantics of the questions. Semantic parsing can convert natural language questions into machine-understandable semantic representations, such as logical forms and graph representations, so as to facilitate queries in the knowledge base.

[0041] Knowledge base query includes knowledge base construction, query construction and result acquisition. Knowledge base construction can build a structured knowledge base containing information such as entities, attributes, and relationships. Query construction can build query statements in the knowledge base based on the semantic representation of the problem understanding, such as simple protocol and RDF query language (SPARQL). Result acquisition can execute query statements to obtain candidate answers related to the question from the knowledge base.

[0042] Answer generation and return include answer sorting, answer generation and answer return. Answer sorting can be based on the relevance, accuracy and other factors of candidate answers. Answer generation can select the most appropriate answer from the sorted candidate answers. Answer return can return the generated answer to the user in natural language.

[0043] Knowledge base question-answering technology combines technologies from multiple fields such as natural language processing, information retrieval and knowledge representation. It can parse and understand natural language questions input by users, query and reason in the structured information of the knowledge base, and generate and return accurate and concise answers.

[0044] However, traditional knowledge base question answering technology mainly relies on some small and medium-sized models with limited language understanding capabilities, such as knowledge base question answering or knowledge base question answering (KBQA) question answering models. In addition, the length of the document content is relatively long, which may miss similar content or contextual information. In addition, the knowledge of traditional knowledge bases is usually static and does not contain the latest information. As a result, the documents retrieved by traditional knowledge base question answering technology are less relevant to the questions, and the answers generated are not accurate enough.

[0045] Therefore, how to improve the accuracy of the generated answers has become a problem that needs to be solved urgently.

[0046] In order to solve the above technical problems, an embodiment of the present application provides a question-answering method. In this method, a current question can be obtained. Afterwards, a target similar question can be obtained based on the current question, and the similarity between the target similar question and the current question is greater than a preset similarity threshold. In this way, the similarity between the target similar question and the current question is high, and the matching knowledge unit will be more closely related to the current question. Afterwards, the target knowledge unit can be determined from a preset knowledge base based on all questions, and all questions include: the current question and the target similar question. Afterwards, the target answer can be obtained based on the target knowledge unit. In this way, by combining the current question and the target similar question, more target knowledge units with a strong correlation with the current question can be obtained, thereby generating more accurate and richer answers.

[0047] The implementation environment of the embodiments of the present application is introduced below.

[0048] like Figure 1 As shown, a question-answering system provided in an embodiment of the present application includes: a knowledge base construction module 101, a knowledge base query module 102 and a knowledge base management module 103.

[0049] The knowledge base construction module 101 is used to obtain knowledge information. The knowledge base construction module 101 is also used to store the knowledge information to construct a preset knowledge base. The knowledge base construction module 101 is also used to update the preset knowledge base.

[0050] It is understandable that by updating and expanding the preset knowledge base, it is possible to ensure that the answers obtained by users are the latest, thereby making the generated answers more accurate.

[0051] In the embodiment of the present application, the knowledge base construction module 101 can retrieve information in various fields to obtain raw data. The knowledge base construction module 101 can also pre-process, analyze and extract the raw data to obtain knowledge information. The knowledge base construction module 101 can also segment the knowledge information to obtain multiple text segments, obtain knowledge units based on the text segments, and store the knowledge units in a preset knowledge base.

[0052] It should be noted that the original data includes data in various document formats, including but not limited to pdf, docx, doc, html and markdown. The knowledge units include but are not limited to business policy information, product introduction information, troubleshooting information, and FAQ information.

[0053] Exemplarily, the knowledge base construction module 101 may retrieve information on the operator's business domain from external data sources such as the operator's historical knowledge base, the Internet, and professional forums to obtain original data on the operator's business domain.

[0054] In this way, by retrieving information in various fields from multiple data sources, the comprehensiveness and timeliness of information can be improved, so that the question-answering system can have good scalability and customizability to meet the business needs and application scenarios in different fields.

[0055] Optionally, the knowledge base construction module 101 may also add metadata to each text segment, convert the text segment into a vector and store it in a preset knowledge base, and construct an index structure to obtain a structured preset knowledge base.

[0056] It should be noted that the present application does not limit the index structure. For example, the index structure may be an inverted index structure.

[0057] In this way, by building a structured preset knowledge base, efficient query and retrieval services can be provided.

[0058] The knowledge base query module 102 is used to obtain the current question and obtain a target similar question based on the current question. The knowledge base query module 102 is also used to obtain the first answer to the current question and the first answer to the target similar question, and determine the target knowledge unit that matches all the first answers from the preset knowledge base. The knowledge base query module 102 is also used to generate a target answer.

[0059] Optionally, the knowledge base query module 102 may search the preset knowledge base based on a multi-way recall algorithm to obtain a plurality of candidate knowledge units. The knowledge base query module 102 may also sort the plurality of candidate knowledge units to obtain a target knowledge unit.

[0060] The knowledge base management module 103 is used to manage and maintain the preset knowledge base. For example, the knowledge base management module 103 can add, modify, and delete knowledge units. The knowledge base management module 103 is also used to back up and restore the preset knowledge base.

[0061] In this way, data loss and damage can be prevented by backing up and restoring the preset knowledge base.

[0062] It is understandable that the administrator can flexibly manage and maintain the knowledge base through the knowledge base management module 103 to ensure the accuracy and completeness of the knowledge base.

[0063] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0064] like Figure 2 As shown, a question-answering method provided in an embodiment of the present application includes:

[0065] S201. Get the current question.

[0066] S202: Based on the current problem, a target similarity problem is obtained.

[0067] Among them, the similarity between the target similar question and the current question is greater than a preset similarity threshold.

[0068] In a possible implementation, the current question may be input into a large language model to obtain multiple similar questions, and then the semantic similarity between each similar question and the current question may be calculated to determine a target similar question.

[0069] Optional, such as Figure 3 As shown, before the current question is input into the large language model, the current question can be subjected to noise reduction processing. After obtaining multiple similar questions, the multiple similar questions can be subjected to noise reduction processing.

[0070] In the embodiment of the present application, the semantic similarity may be compared with a preset similarity threshold, and similar questions having a semantic similarity greater than the preset similarity threshold may be determined as target similar questions.

[0071] It should be noted that the present application does not limit the preset similarity threshold. For example, the preset similarity threshold may be 0.6, 0.65, 0.7, 0.8, or 0.9.

[0072] Optionally, each similar question may be ranked based on semantic similarity, and the semantic similarity of similar questions ranked at a preset position may be used as a preset similarity threshold.

[0073] That is to say, the semantic similarities ranked before the preset ranking are determined as target similarity issues.

[0074] Exemplarily, the preset ranking is fifth. If the semantic similarity of similar question a is 0.75 and the ranking is fifth, the semantic similarity of similar question a is used as the preset similarity threshold. If the semantic similarity of similar question b is 0.7 and the ranking is sixth, it is determined that similar question b is not the target similar question; if the semantic similarity of similar question c is 0.8 and the ranking is fourth, it is determined that similar question c is the target similar question.

[0075] In another possible implementation, historical dialogue information may be obtained. Then, based on the historical dialogue information and the current question, a target similar question is obtained, wherein the historical dialogue information includes questions and answers of a preset round.

[0076] It is understandable that by obtaining historical conversation information, we can better understand the current question based on contextual information, thereby generating an answer that is more in line with the semantics of the current question, thereby greatly improving the subsequent recall quality of the model and improving the accuracy and fluency of question and answering.

[0077] S203. Based on all the questions, determine the target knowledge unit from the preset knowledge base.

[0078] Among them, all problems include: current problems and target similar problems.

[0079] In a possible implementation, knowledge units can be matched from a preset knowledge base based on all questions. Then, a target question can be determined based on the number of matched knowledge units, where the target question is a question whose number of matched knowledge units meets a preset number threshold. Then, a target knowledge unit can be determined based on the target question.

[0080] It should be noted that the present application does not limit the preset quantity threshold. For example, the preset quantity threshold may be 5, 10, 15, 20, or 25.

[0081] Optionally, the preset quantity threshold may be the maximum quantity among the quantities of matched knowledge units.

[0082] In this way, knowledge units can be deeply integrated with current questions to build rich contextual information, thereby providing sufficient information support for the generated answers.

[0083] S204. Obtain a target answer based on the target knowledge unit.

[0084] Among them, the target answer is the answer to the current question.

[0085] In a possible implementation, the target knowledge unit can be input into a large language model in combination with the prompt word template technology. Afterwards, the target knowledge unit can be processed by the large language model based on natural language generation technology to obtain the target answer.

[0086] In another possible implementation, the target knowledge unit may be used as the target answer.

[0087] Based on the above technical solution, the current question can be obtained. Afterwards, a target similar question can be obtained based on the current question, and the similarity between the target similar question and the current question is greater than a preset similarity threshold. In this way, the target similar question has a high similarity with the current question, and the matching knowledge unit will have a stronger relevance to the current question. Afterwards, the target knowledge unit can be determined from a preset knowledge base based on all questions, and all questions include: the current question and the target similar question. Afterwards, the target answer can be obtained based on the target knowledge unit. In this way, by combining the current question and the target similar question, more target knowledge units that are more closely related to the current question can be obtained, thereby generating more accurate and richer answers.

[0088] like Figure 4 As shown, another question-answering method provided in an embodiment of the present application, S203 also includes:

[0089] S401. Obtain the first answer to each question among all the questions.

[0090] In a possible implementation, all questions can be input into a large language model in combination with the prompt word template technology to obtain the first answer to each question.

[0091] In a possible design, there is similarity between the first answer and the question corresponding to the first answer. All questions can be input into the large language model to obtain the initial first answer for each question. Then, for each initial first answer, the semantic similarity between the initial first answer and the corresponding question can be obtained to obtain the semantic similarity corresponding to each initial first answer. Then, each initial first answer is sorted based on the semantic similarity corresponding to each initial first answer to obtain the first answer for each question.

[0092] Exemplarily, the initial first answers are sorted from high to low based on the semantic similarity corresponding to each initial first answer. The semantic similarity of the initial first answer a is 0.5, the semantic similarity of the initial first answer b is 0.7, the semantic similarity of the initial first answer c is 0.6, and the semantic similarity of the initial first answer d is 0.8. Then the initial first answer d is the first, the initial first answer b is the second, the initial first answer c is the third, and the initial first answer a is the first. If the initial first answer ranked in the top three is the first answer, then the initial first answer d, the initial first answer b and the initial first answer c are all the first answers.

[0093] S402: Determine target knowledge units that match all first answers from a preset knowledge base.

[0094] In a possible implementation, for each first answer, the degree of association between the first answer and a preset knowledge unit in a preset knowledge base may be determined. Then, the preset knowledge unit in the preset knowledge base whose degree of association with each first answer is greater than a preset association threshold is used as the target knowledge unit.

[0095] Exemplarily, the preset correlation threshold b corresponding to the first answer a and the first answer b is 0.7. Among them, the correlation between knowledge unit a and the first answer a is 0.8, and the correlation between knowledge unit a and the first answer b is 0.65. The correlation between knowledge unit b and the first answer a is 0.5, and the correlation between knowledge unit b and the first answer b is 0.5. The correlation between knowledge unit c and the first answer a is 0.8, and the correlation between knowledge unit c and the first answer b is 0.8. Determine knowledge unit c as the target knowledge unit.

[0096] In another possible implementation, all first answers may be merged to obtain a second answer. Then, based on the second answer, a target knowledge unit may be determined from a preset knowledge base.

[0097] In one possible design, Figure 3 As shown in FIG. 1 , the first answer is subjected to denoising. Then, the first answer is input into the embedding model to obtain the transformed first answer. Then, all the transformed first answers are fused to obtain the second answer.

[0098] Optionally, the embedding model can be a fine-tuned embedding model.

[0099] In an embodiment of the present application, for each first answer, the semantic similarity between the first answer and the corresponding question can be obtained to obtain the semantic similarity corresponding to each first answer. The weight value of each first answer can be determined based on the semantic similarity corresponding to each first answer. Afterwards, all the first answers can be fused based on the weight value of each first answer to obtain the second answer.

[0100] That is, the first answer after conversion is a dense vector. The weighted average of all dense vectors can be calculated to obtain the second answer.

[0101] Optionally, the semantic similarity corresponding to each first answer is a weight value of each first answer. All first answers and the current question may be merged based on the semantic similarity corresponding to each first answer to obtain a second answer.

[0102] Exemplarily, the second answer satisfies Formula 1.

[0103]

[0104] Where v is the second answer, N is the number of first answers, and w k is the semantic similarity between the first answer and the corresponding question, k represents each number in the integer sequence from 1 to N, f(d k ) is the vector transformed from the kth first answer, and f(q) is the vector transformed from the current question.

[0105] It is understandable that since the second answer contains information about the current question and the first answer, and since the semantics of the first answer is closer to the semantics of the knowledge unit in the preset knowledge base, searching the preset knowledge base based on the second answer can improve the recall rate of the search.

[0106] In one possible design, Figure 5 As shown, the second answer and the preset knowledge unit in the preset knowledge base can be processed based on the first information retrieval algorithm to obtain a first score of the preset knowledge unit, and the first score is used to indicate the association between the preset knowledge unit and the second answer. Afterwards, the second answer and the preset knowledge unit can be processed based on the second information retrieval algorithm to obtain a second score of the preset knowledge unit, and the second score is used to indicate the association between the preset knowledge unit and the second answer. Afterwards, based on the first score and the second score of the preset knowledge unit, the target knowledge unit is determined from the preset knowledge base.

[0107] It should be noted that the first information retrieval algorithm and the second information retrieval algorithm are two different information retrieval algorithms, and the present application does not limit the first information retrieval algorithm and the second retrieval algorithm. For example, the first information retrieval algorithm may be a keyword retrieval algorithm, and the second information retrieval algorithm may be a vector retrieval algorithm. For another example, the first information retrieval algorithm may be a hash algorithm, and the second information retrieval algorithm may be a graph retrieval algorithm. Regarding the first information retrieval algorithm and the second information retrieval algorithm, reference may be made to the introduction of information retrieval algorithms in conventional technology, which will not be elaborated here.

[0108] Exemplarily, taking the first information retrieval algorithm as the improved BM25 algorithm as an example, the process of obtaining the first score is introduced.

[0109] In a possible design, document information and a target adjustment factor of a preset knowledge unit may be obtained, wherein the document information includes an inverse document frequency, a document length, and a target word frequency, wherein the target word frequency is the frequency of occurrence of each word of the preset knowledge unit in the preset knowledge unit, and the target adjustment factor is used to indicate the influence of the words shared by each question and the preset knowledge unit on the first score. Afterwards, an objective function may be generated based on the target adjustment factor, the document length, and the target word frequency. Afterwards, the objective function may be multiplied by the inverse document frequency to obtain the first score.

[0110] In an embodiment of the present application, the target adjustment factor may be the ratio of the words that each question has in common with the preset knowledge unit to all the words in the question.

[0111] Optionally, a first adjustment factor and a second adjustment factor may also be obtained, wherein the first adjustment factor is used to indicate the influence of the target word frequency on the first score, and the second adjustment factor is used to indicate the influence of the document length on the first score. Afterwards, an objective function may be generated based on the first adjustment factor, the second adjustment factor, the target adjustment factor, the document length, and the target word frequency. Afterwards, the objective function may be multiplied by the inverse document frequency to obtain the first score.

[0112] Exemplarily, the first score may satisfy Formula 2.

[0113]

[0114] Among them, Score(Q, d) is the first score, n is, IDF(q i ) is the inverse document frequency, w i is the target adjustment factor, k1 is the first adjustment factor, f i is the target word frequency, b is the second adjustment factor, dl is the document length of the preset knowledge unit, and avgdl is the average length of all documents in the preset knowledge base.

[0115] In the embodiment of the present application, the inverse document frequency satisfies Formula 3.

[0116]

[0117] Where N represents the number of preset knowledge units in the preset knowledge base, q i is the word in the preset knowledge unit, n(qi) is the q contained in the preset knowledge unit i The number of

[0118] Specifically, the first score and the second score of the preset knowledge unit can be added to obtain the target score of the preset knowledge unit. If the target score of the preset knowledge unit is greater than the preset score threshold, the preset knowledge unit can be determined as the target knowledge unit.

[0119] Optionally, the preset knowledge units can be sorted based on the sorting algorithm and the target score to obtain the target knowledge unit. It should be noted that the present application does not limit the sorting algorithm. For example, the sorting algorithm can be a basegraph embedding (BGE) algorithm. For another example, the sorting algorithm can be a selection sorting algorithm.

[0120] Exemplarily, the ranking is performed from high to low according to the size of the target score. The target score 1 of the preset knowledge unit a is 60, the target score 2 of the preset knowledge unit b is 70, the target score 3 of the preset knowledge unit c is 80, and the target score 4 of the preset knowledge unit d is 90. Then the preset knowledge unit d is ranked first, the preset knowledge unit c is ranked second, the preset knowledge unit b is ranked third, and the preset knowledge unit a is ranked fourth. If the preset knowledge units ranked in the top three are the target knowledge units, then the preset knowledge units b, the preset knowledge units c, and the preset knowledge units d are the target knowledge units.

[0121] Alternatively, the present application may sort the preset knowledge units based on the sorting model to obtain the target knowledge units.

[0122] Optionally, the present application may fine-tune the BGE sorting model, and sort the preset knowledge units based on the fine-tuned sorting model.

[0123] It is understandable that by fine-tuning the BGE ranking model, more precise target knowledge units can be obtained, thereby improving the accuracy of the target answer.

[0124] Based on the above technical solution, the first answer of each question in all questions can be obtained. Afterwards, based on all the first answers, the target knowledge unit can be determined from the preset knowledge base, and the target knowledge unit is the knowledge unit in the preset knowledge base whose correlation with each first answer is greater than the preset correlation threshold. In this way, the semantic space of the query and the document can be aligned through the first answer, and the semantic mismatch between the question and the knowledge unit can be improved, so as to improve the accuracy and recall rate of the retrieval.

[0125] In some embodiments, after obtaining the target knowledge units, the target knowledge units need to be merged and deduplicated to remove redundant information.

[0126] For example, Figure 6 As shown, there are two knowledge units 1, one knowledge unit 2, one knowledge unit 3 and one knowledge unit 4, then the two knowledge units 1 are deduplicated to obtain one knowledge unit 1. Afterwards, knowledge unit 1, knowledge unit 2 and knowledge unit 3 are merged to obtain knowledge unit 5.

[0127] The following is an introduction to the embodiments of the present application with reference to specific examples. Figure 7 As shown, the original data can be obtained. Afterwards, the original data is preprocessed, analyzed and extracted to obtain knowledge information. Afterwards, the knowledge information is stored to construct a preset knowledge base. Afterwards, the second answer and the preset knowledge unit in the preset knowledge base can be data processed based on the first information retrieval algorithm to obtain a first score of the preset knowledge unit, and the first score is used to indicate the association between the preset knowledge unit and the second answer. Afterwards, the second answer and the preset knowledge unit can be data processed based on the second information retrieval algorithm to obtain a second score of the preset knowledge unit, and the second score is used to indicate the association between the preset knowledge unit and the second answer. Afterwards, based on the first score and the second score of the preset knowledge unit, the target knowledge unit is determined from the preset knowledge base. Afterwards, the current question and the target knowledge unit can be input into the large language model in combination with the prompt word template technology to obtain the target answer.

[0128] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It can be understood that in order to realize the above functions, the question-and-answer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the question-and-answer method steps of each example described in the embodiment disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] The embodiment of the present application also provides a question-answering device, which can be a server, a CPU in the server, a module for question-answering in the server, or a client for question-answering in the server.

[0130] The embodiment of the present application can divide the question-and-answer device into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic, which is only a logical functional division. There may be other division methods in actual implementation.

[0131] The present application embodiment provides a question-answering device. Figure 8 As shown, the question-answering device may include: an acquisition module 801 and a processing module 802 .

[0132] The acquisition module 801 is used to acquire the current question. The processing module 802 is used to obtain a target similar question based on the current question, and the similarity between the target similar question and the current question is greater than a preset similarity threshold. The processing module 802 is also used to determine a target knowledge unit from a preset knowledge base based on all questions, and all questions include: the current question and the target similar question. The processing module 802 is also used to obtain a target answer based on the target knowledge unit.

[0133] Fig. 9 9 is a schematic diagram of another question-answering device according to an exemplary embodiment. The question-answering device may include a processor 902, and the processor 902 is used to execute application code to implement the question-answering method in the present application.

[0134] The processor 902 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0135] like Fig. 9 As shown, the question-answering device may further include a memory 903. The memory 903 is used to store application program codes for executing the solution of the present application, and the execution is controlled by the processor 902.

[0136] The memory 903 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 903 may exist independently and be connected to the processor 902 via a bus 904. The memory 903 may also be integrated with the processor 902.

[0137] like Fig. 9 As shown, the question-answering device may further include a communication interface 901, wherein the communication interface 901, the processor 902, and the memory 903 may be coupled to each other, for example, via a bus 904. The communication interface 901 is used to exchange information with other devices, for example, to support information exchange between the question-answering device and other devices.

[0138] It should be pointed out that Fig. 9 The device structure shown in the figure does not constitute a limitation on the question-answering device, except Fig. 9 In addition to the components shown, the question-and-answer device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0139] In actual implementation, the functions implemented by the processing unit can be Fig. 9 The processor 902 shown calls the program code in the memory 903 to implement.

[0140] The present application also provides a computer-readable storage medium having instructions stored thereon, and when the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer is enabled to execute the question-answering method provided in the above-mentioned embodiment. For example, the computer-readable storage medium may be a memory 903 including instructions, and the above instructions may be executed by a processor 902 of a computer device to complete the above method. Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0141] Fig.10 A conceptual partial view of a computer program product provided by an embodiment of the present application is schematically shown, where the computer program product includes a computer program for executing a computer process on a computing device.

[0142] In one embodiment, the computer program product is provided using a signal bearing medium 1000. The signal bearing medium 1000 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 , Figure 4 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S201 to S204 may be undertaken by one or more instructions associated with the signal bearing medium 1000. In addition, Fig.10 The program instructions in also describe example instructions.

[0143] In some examples, the signal bearing medium 1000 may include a computer readable medium 1001, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM) or a random access memory (RAM), and the like.

[0144] In some implementations, signal bearing medium 1000 may include computer recordable medium 1002 such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, or the like.

[0145] In some implementations, signal bearing medium 1000 may include communication medium 1003 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0146] The signal bearing medium 1000 may be communicated by a wireless form of communication medium 1003. The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0147] In some examples, the question-and-answer device may be configured to provide various operations, functions, or actions in response to one or more program instructions in computer-readable medium 1001 , computer-recordable medium 1002 , and / or communication medium 1003 .

[0148] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute the full classification part or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes various media that can store program codes, such as-U disk, mobile hard disk, ROM, RAM, disk or CD.

[0153] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A question-answering method, characterized in that: The method comprises: Get the current issue; Based on the current question, a target similar question is obtained, wherein the similarity between the target similar question and the current question is greater than a preset similarity threshold; Determine a target knowledge unit from a preset knowledge base based on all questions, wherein all questions include: the current question and the target similar question; Based on the target knowledge unit, a target answer is obtained.

2. The method according to claim 1, characterized in that Based on all the questions, the target knowledge document is determined from the preset knowledge base, including: Obtaining a first answer to each of the questions; The target knowledge unit matching all the first answers is determined from the preset knowledge base.

3. The method according to claim 2, characterized in that The step of determining the target knowledge unit from the preset knowledge base based on all the first answers includes: Merge all of the first answers to obtain a second answer; Based on the second answer, the target knowledge unit is determined from the preset knowledge base.

4. The method according to claim 3, characterized in that The step of fusing all of the first answers to obtain a second answer includes: For each of the first answers, obtaining the semantic similarity between the first answer and the corresponding question to obtain the semantic similarity corresponding to each of the first answers; Determining a weight value of each of the first answers based on the semantic similarity corresponding to each of the first answers; All of the first answers are merged based on the weight value of each of the first answers to obtain a second answer.

5. The method according to claim 3, characterized in that: The step of determining the target knowledge unit from the preset knowledge base based on the second answer includes: Processing the second answer and the preset knowledge unit in the preset knowledge base based on a first information retrieval algorithm to obtain a first score for the preset knowledge unit, where the first score is used to indicate a degree of association between the preset knowledge unit and the second answer; Performing data processing on the second answer and the preset knowledge unit based on a second information retrieval algorithm to obtain a second score of the preset knowledge unit, where the second score is used to indicate the degree of association between the preset knowledge unit and the second answer; The target knowledge unit is determined from the preset knowledge base based on the first score and the second score of the preset knowledge unit.

6. The method according to claim 5, characterized in that The step of performing data processing on the second answer and the knowledge unit in the preset knowledge base based on the first information retrieval algorithm to obtain a first score of the preset knowledge unit includes: Obtaining document information and a target adjustment factor of the preset knowledge unit, wherein the document information includes an inverse document frequency, a document length, and a target word frequency, wherein the target word frequency is the frequency of occurrence of each word of the preset knowledge unit in the preset knowledge unit, and the target adjustment factor is used to indicate the influence of the words shared by each question and the preset knowledge unit on the first score; Generate a target function based on the target adjustment factor, the document length and the target word frequency; The objective function is multiplied by the inverse document frequency to obtain the first score.

7. A question-answering device, characterized in that: The device comprises an acquisition module and a processing module: The acquisition module is used to acquire the current question; The processing module is used to obtain a target similar question based on the current question, wherein the similarity between the target similar question and the current question is greater than a preset similarity threshold; The processing module is further used to determine a target knowledge unit from a preset knowledge base based on all questions, wherein all questions include: the current question and the target similar question; The processing module is also used to obtain a target answer based on the target knowledge unit.

8. A question-answering device, characterized in that: include: Processor and memory; The processor is coupled to the memory; The memory is used to store one or more programs, and one or more of the programs include computer-executable instructions. When the question-and-answer device is running, the processor executes the computer-executable instructions stored in the memory to enable the question-and-answer device to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When a computer executes the instruction, the computer performs the method according to any one of claims 1 to 6.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 6.

Images