Question and answer method and apparatus, device, and storage medium

Abstract

The present application relates to the technical field of computers, and discloses a question and answer method and apparatus, a device, and a storage medium. The method comprises: generating a second problem on the basis of a first question of a user and a context of the first question; recalling in a knowledge base set on the basis of the second problem to obtain N knowledge points, wherein N is a positive integer, the knowledge base set comprises a plurality of knowledge bases of different field types, the plurality of knowledge bases comprise at least one knowledge base corresponding to the field type to which the second question belongs, and each knowledge base is used for storing knowledge points related to the corresponding field type; performing content correlation extraction on the N knowledge points on the basis of the second problem, and determining knowledge point content related to the second question from the N knowledge points to obtain knowledge point content; and on the basis of the second problem and the knowledge point content, generating a target answer by means of a language model. A language model and an external knowledge base are effectively combined, and the advantages of the language model and the external knowledge base are fully utilized, so that the accuracy of question and answer results is improved.

Description

Question-answering method, device, equipment, and storage medium Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a question-answering method, apparatus, device, and storage medium.

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 5, 2023, with application number 202311660567.7 and invention name “Question-and-Answer Method, Device, Equipment and Storage Medium”, the entire contents of which are incorporated by reference into this application. Background Art

[0003] The rise of ultra-large-scale pre-trained language models is considered a paradigm in the field of artificial intelligence (AI). Future AI applications will rely heavily on these large-scale language models to fully leverage their superior language modeling capabilities and the vast amount of pre-trained knowledge they contain.

[0004] In related technologies, user questions are input into a language model, which then generates answers and returns them to the user. However, language models acquire knowledge during a pre-training phase, while human knowledge is constantly evolving. Language models often fail to acquire and utilize the latest knowledge in a timely manner, resulting in poor performance when answering questions related to current events, emerging fields, or updated knowledge points. Therefore, relying directly on language models for knowledge question answering has certain limitations, reducing the accuracy of the results. Technical issues

[0005] This application provides a question-answering method, apparatus, device, and storage medium that improves the accuracy of question-answering results. The technical solution is as follows:

[0006] In a first aspect, a question-answering method is provided, the method comprising: generating a second question based on a user's first question and the context of the first question; recalling a knowledge base set based on the second question to obtain N knowledge points, where N is a positive integer, the knowledge base set comprising multiple knowledge bases of different domain types, the multiple knowledge bases comprising at least one knowledge base corresponding to the domain type to which the second question belongs, and each knowledge base being used to store knowledge points related to the corresponding domain type; performing content relevance extraction on the N knowledge points based on the second question, determining the knowledge point content related to the second question from the N knowledge points, and obtaining the knowledge point content; and generating a target answer through a language model based on the second question and the knowledge point content.

[0007] In a second aspect, a question-and-answer device is provided, comprising: a question generation module for generating a second question based on a user's first question and the context of the first question; a recall module for recalling a knowledge base set based on the second question to obtain N knowledge points, where N is a positive integer, and the knowledge base set includes multiple knowledge bases of different domain types, and the multiple knowledge bases include at least one knowledge base corresponding to the domain type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding domain type; an extraction module for performing content relevance extraction on the N knowledge points based on the second question, determining the knowledge point content related to the second question from the N knowledge points, and obtaining the knowledge point content; and an answer generation module for generating a target answer through a language model based on the second question and the knowledge point content.

[0008] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method described in the first aspect when executed by the processor.

[0009] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0010] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0011] The embodiment of the present application provides a question-answering method, apparatus, device, and storage medium. According to the technical solution provided by the present application, the method includes generating a second question based on the user's first question and the context of the first question; recalling N knowledge points from a knowledge base set based on the second question, where N is a positive integer, and the knowledge base set includes multiple knowledge bases of different domain types, the multiple knowledge bases including at least one knowledge base corresponding to the domain type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding domain type; the knowledge base can pre-integrate and store a large amount of knowledge data. The present solution uses an external knowledge base to recall knowledge points related to the second question from knowledge bases of different sources from at least one knowledge base, thereby achieving more flexible use of the latest external knowledge points to answer questions, thereby improving the effectiveness of question-answering. Content relevance extraction is performed on the N knowledge points based on the second question, and the knowledge point content related to the second question is determined from the N knowledge points to obtain the knowledge point content; the knowledge point content with greater relevance to the second question is selected from the N knowledge points as a reference, and the knowledge point content with less relevance is filtered out, thereby reducing interference with the language model. Then, based on the second question and the knowledge point, the language model is used to generate the target answer. Specifically, the language model, combined with the second question, comprehensively considers at least one reference knowledge point to provide a concise and clear response. This solution organically combines the language model with an external knowledge base, leveraging the strengths of both. This approach offers greater flexibility and accuracy, meeting user needs for diverse and up-to-date knowledge in diverse fields while continuously improving the intelligence of the question-answering system as the external knowledge base is updated. This improves the accuracy of the Q&A results compared to solutions that directly use large language models for Q&A. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG1 is a flow chart of a question-and-answer method provided in an embodiment of the present application;

[0013] FIG2 is a flow chart of another question-and-answer method provided in an embodiment of the present application;

[0014] FIG3 is a flow chart of another question-and-answer method provided in an embodiment of the present application;

[0015] FIG4 is a flowchart of another question-and-answer method provided in an embodiment of the present application;

[0016] FIG5 is a flowchart of another question-and-answer method provided in an embodiment of the present application;

[0017] FIG6 is a flowchart of another question-and-answer method provided in an embodiment of the present application;

[0018] FIG7 is a schematic diagram of the structure of a question-answering device provided in an embodiment of the present application;

[0019] FIG8 is a schematic structural diagram of a computer device provided in an embodiment of the present application. Modes for Carrying Out the Invention

[0020] The present application embodiment provides a question-answering method, as shown in FIG1 , which is a flow chart of a question-answering method provided by the present application embodiment. The question-answering method includes:

[0021] S101. Generate a second question based on the user's first question and the context of the first question.

[0022] In this embodiment of the present application, a first question is input into a language model, which then generates a second question based on the first question and its context. The language model has the ability to learn context, which reflects the contextual information of the first question. It can generate a complete question (i.e., the second question) based on the current question (i.e., the first question) and the context. The second question better reflects the user's query intent.

[0023] The first question can also be called the current question. This can be the first question a user enters. The context of the first question doesn't include contextual information, meaning there's no historical conversation. The language model restates the first question to generate the second question, which can also be understood as using the first question as the second question. The first question can also be a question a user enters again after multiple inquiries. The context of the first question includes contextual information, meaning there's historical conversation. The language model learns the first question and its context to generate the second question.

[0024] The question-answering method provided in the embodiment of the present application can be applied to social software. For example, the question-answering method can be used in chat software to implement multiple rounds of question-answering.

[0025] It should be noted that the language model used to generate the second question uses the pre-provided template (the question sample input by the user and its context, and the corresponding generated complete question sample) as a prompt for the language model when using the language model to generate the second question, so that the language model can generate the second question based on the first question and its context.

[0026] S102. Recall the knowledge base set according to the second question to obtain N knowledge points, where N is a positive integer. The knowledge base set includes multiple knowledge bases of different field types, and the multiple knowledge bases include at least one knowledge base corresponding to the field type to which the second question belongs. Each knowledge base is used to store knowledge points related to the corresponding field type.

[0027] In an embodiment of the present application, the knowledge base set includes multiple knowledge bases of different field types, and each knowledge base stores multiple knowledge points related to its corresponding field type. The field types include but are not limited to government affairs, finance, news, shopping, home, health, history, entertainment, pets, games, vehicles, music, painting, video and travel, etc. The knowledge points can be question-answer pairs or text information. There can be one or more field types to which the second question belongs. At least one knowledge base corresponding to the field type to which the second question belongs is selected from the knowledge base set to reduce the number of knowledge bases used during recall and improve processing efficiency. The knowledge base is relatively flexible. When recalling (i.e., searching) knowledge points, the knowledge base to be retrieved can also be specified. It can be one or more knowledge bases. Recall is performed in each knowledge base according to the second question, and a total of N knowledge points are recalled. The knowledge base stores knowledge points and their corresponding vectors. When recalling knowledge points, the second question is vectorized. For each knowledge base, the vector similarity (which can be a cosine distance) between the second question and each knowledge point in the knowledge base is calculated. Knowledge points with vector similarity greater than a preset threshold are selected, or a preset number of knowledge points with a high ranking (larger vector similarity) are selected, thereby obtaining the knowledge points returned by each knowledge base. The above operations are performed on each knowledge base in at least one knowledge base to obtain the knowledge points returned by at least one knowledge base.

[0028] Because knowledge bases can pre-integrate and store large amounts of knowledge data, and because the knowledge data stored in these knowledge bases can be updated and maintained in a timely manner, they provide the question-answering system (the device used to execute the question-answering method) with the latest knowledge points in various fields. This solution leverages external knowledge bases to recall knowledge points related to the second question from different knowledge bases, enabling more flexible use of the latest external knowledge points to answer questions, thereby improving the effectiveness of question-answering.

[0029] S103. Extract content relevance of N knowledge points according to the second question, determine the knowledge point content related to the second question from the N knowledge points, and obtain the knowledge point content.

[0030] In an embodiment of the present application, among the N knowledge points screened, there may be knowledge points that are not very relevant to the question. This example uses a language model to extract the content relevance of each of the N knowledge points according to the second question, and further screens out the knowledge point content related to the second question. For knowledge points that are not very relevant to the second question, the language model outputs "none", indicating that the knowledge point has no relevance to the second question. For knowledge points that are relevant to the second question, the language model outputs "knowledge point content", indicating that the knowledge point has relevance to the second question. A knowledge point can be a question-answer pair or text information, and the knowledge point content is part of the knowledge point. For example, it can be a fragment in the knowledge point, or it can be multiple characters in the knowledge point. If the knowledge point content is multiple characters in the knowledge point, it is necessary to further match the multiple characters in the corresponding knowledge points, input the multiple characters into the language model, and output one or more fragments through the language model.

[0031] It should be noted that when using the language model to extract content relevance, the pre-provided examples (question samples and knowledge point samples, as well as the corresponding content samples extracted by content relevance) are used as prompts for the language model, so that the language model can extract the content relevance of each knowledge point according to the second question, thereby obtaining the knowledge point content.

[0032] The language model is used to extract the knowledge points that are more relevant to the second question from N knowledge points so that they can be used as a reference when generating the target answer later. The knowledge points with less relevance are filtered out, reducing interference with the language model (used to generate the target answer) and improving the accuracy of the question-answering results.

[0033] S104. Generate a target answer through a language model based on the second question and the knowledge point content.

[0034] In an embodiment of the present application, the knowledge point content obtained above is used as a reference, and a language model is used in conjunction with the second question to comprehensively consider at least one reference knowledge point to generate a target answer. When extracting the knowledge point content above, if the language model outputs "none" for all N knowledge points, the number of knowledge point contents obtained is 0, that is, there is no reference knowledge point content. Furthermore, the target answer generated by the language model based on the second question and the knowledge point content is an answer that cannot be answered.

[0035] It should be noted that when using the language model to generate answers, the pre-provided examples (question samples and known reference answer samples, as well as the corresponding generated answer samples) are used as prompts for the language model, so that the language model can generate the target answer based on the second question and knowledge point content.

[0036] The language model used to generate the second question, the language model used to extract content relevance, and the language model used to generate the answer can be the same large language model, or they can be independent language models with different functions, which is not limited in this embodiment of the present application. To facilitate the distinction between the language models that implement different functions, the language model used to generate the second question can be referred to as the first language model, the language model used to extract content relevance can be referred to as the second language model, and the language model used to generate the answer can be referred to as the third language model.

[0037] According to the technical solution provided by the present application, the method includes generating a second question based on the user's first question and the context of the first question; recalling N knowledge points from a knowledge base set based on the second question, where N is a positive integer, and the knowledge base set includes multiple knowledge bases of different field types, and the multiple knowledge bases include at least one knowledge base corresponding to the field type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding field type; the knowledge base can pre-integrate and store a large amount of knowledge data, and this solution uses an external knowledge base to recall knowledge points related to the second question from at least one knowledge base from knowledge bases of different sources, thereby achieving more flexible use of the latest external knowledge points to answer questions, thereby improving the effect of question and answering. Content relevance is extracted from the N knowledge points based on the second question, and the knowledge point content related to the second question is determined from the N knowledge points to obtain the knowledge point content; the knowledge point content with greater relevance to the second question is selected from the N knowledge points as a reference, and the knowledge point content with less relevance is filtered out, thereby reducing interference with the language model. Then, based on the second question and the knowledge point, the language model is used to generate the target answer. Specifically, the language model, combined with the second question, comprehensively considers at least one reference knowledge point to provide a concise and clear response. This solution organically combines the language model with an external knowledge base, leveraging the strengths of both. This approach offers greater flexibility and accuracy, meeting user needs for diverse and up-to-date knowledge in diverse fields while continuously improving the intelligence of the question-answering system as the external knowledge base is updated. This improves the accuracy of the Q&A results compared to solutions that directly use large language models for Q&A.

[0038] In some embodiments, S101 in Figure 1 can also be implemented in the following manner: As shown in Figure 2, Figure 2 is a flow chart of another question-answer generation method provided in an embodiment of the present application.

[0039] S1011. If there is no historical conversation before the first question, generate a second question based on the first question.

[0040] In this example, if there is no historical conversation before the first question, that is, the first question is the first question input by the user, and the context of the first question does not include contextual information, the language model restates the first question and generates the second question output. It can be understood that the first question is directly output as the second question.

[0041] S1012: If there is a historical conversation before the first question, generate a second question based on the first question and the historical conversation.

[0042] In this example, if there is a previous conversation before the first question—that is, the first question is a question the user enters again after multiple inquiries—and the context of the first question includes contextual information, the language model learns the context of the first question and its context to generate the second question. This previous conversation includes the previous answer, the previous question the user entered, and the previous conversation before the previous question.

[0043] After this round of inquiry, the historical dialogue before the first question, the first question and the target answer are used as the historical dialogue for the next question, and the next target question is generated based on the next question input by the user and the historical dialogue for the next question; according to the next target question, recall is performed in the preset knowledge base to obtain N next knowledge points, and the preset knowledge base includes at least one knowledge base corresponding to the field type to which the next target question belongs; according to the next target question, content relevance is extracted for the N next knowledge points, and the knowledge point content related to the next target question is determined from the N next knowledge points to obtain at least one next knowledge point content; according to the next target question and the at least one next knowledge point content, the next answer is generated through the language model.

[0044] In this example, the user's input question (i.e., the first question) is restated based on the conversation history to generate a semantically complete question, which assists in subsequent knowledge base retrieval, thereby enabling multi-round dialogue in a knowledge base scenario. As an example, the following example uses Assistant to represent the question-answering system and User to represent the user, listing a conversation history between the user and the question-answering system.

[0045] Assistant: Hello, how can I help you?

[0046] User: Hello, I would like to inquire about the contact information of the library in District B, City A.

[0047] Assistant: OK, the contact number for the library in District B, City A is 1234567.

[0048] User: What about area D?

[0049] In the above conversation, the context indicates that the user actually wants to inquire about the contact information for the library in District D, City A. However, if the user's question, "What about District D?" is directly searched in the knowledge base, the key information, "library contact information," would be missing. Therefore, a language model needs to be used to combine the context of the conversation and restate the user's question to regenerate a complete question.

[0050] In this example, we can use in-context learning to allow the language model to help the user generate a complete question (i.e., the second question) based on the conversation history. For example, in the above generation of the complete question, we can add prompt information to the language model so that it can generate a complete question (i.e., the second question) based on the question input by the user. Here, the input for constructing the language model includes:

[0051] "You are a question-and-answer robot. The following is a conversation between a user and the robot. Please use the conversation history to fully rephrase the user's question.

[0052] Here are some examples (also called templates), including Example 1, Example 2, Example 3, etc.

[0053] The following is the conversation record that you need to retell: it includes the history of the current conversation and the latest user questions. The history of the current conversation is as follows: "Assistant: Hello, how can I help you? User: Hello, I would like to inquire about the contact information of the library in District B of City A. Assistant: OK, the contact number of the library in District B of City A is 1234567." The latest user question is as follows: "User: What about District D?"

[0054] Through this contextual learning approach, prompt information is added to the language model (the examples listed above), so that the language model can generate a complete statement (i.e., the second question) for the question entered by the user, for example, "I would like to inquire about the contact information of the library in District D of City A."

[0055] It should be noted that the above conversation is a multi-round process. In the first round of inquiries, there is no historical conversation. The question-answering system provided by this solution repeats the above question "I would like to inquire about the contact information of the library in District B of City A." Then it performs the steps of recalling knowledge points and extracting relevant content, and outputs the answer "Assistant: OK, the contact number of the library in District B of City A is XXXX." In the second round of inquiries, the user continues to ask "What about D?" The question-answering system generates a complete question based on the above latest question "What about D?" and the historical conversation "User: Hello, I would like to inquire about the contact information of the library in District B of City A. Assistant: OK, the contact number of the library in District B of City A is XXXX." Then it performs the steps of recalling knowledge points and extracting relevant content, and outputs the answer.

[0056] In an embodiment of the present application, if the first question is asked for the first time, it can be used as the second question, that is, the first question is restated to generate the second question. If the first question is a question asked again after multiple rounds of dialogue, the first question and the multiple rounds of dialogue are integrated to regenerate the second question. In this way, combined with a large-scale language model, the question-answering system used to execute the question-answering method can generate more accurate questions based on the context of the dialogue, thereby having the ability of multiple rounds of dialogue, and improving the richness of application scenarios.

[0057] In some embodiments, S102 in Figure 1 can also be implemented in the following manner: As shown in Figure 3, Figure 3 is a flow chart of another question and answer generation method provided in an embodiment of the present application.

[0058] S1021. Perform semantic expansion on the second question to obtain at least one expanded question.

[0059] This example expands the second question to generate multiple questions with different wordings but similar semantics (i.e., expanded questions) to help improve recall. For example, the question "I want to inquire about the contact information of the library in District D, City A" in the above example can be expanded to generate "Inquiry about the contact number of the library in District D, City A" and "Inquiry about the phone number of the library in District D" after similar questions. Diverse question wording can significantly improve recall accuracy. This similar question expansion process can use contextual learning to enable the language model to generate similar questions.

[0060] It should be noted that when using a language model to generate extended questions, a pre-provided template (a sample question and the corresponding sample extended question) is used as a prompt for the language model, allowing the language model to generate multiple extended questions based on the first question. The language model used for similar question expansion and the language model used to generate the second question, the language model used for content relevance extraction, and the language model used to generate the answer can be the same large language model, or they can be independent language models with different functions, and this is not limited in this embodiment of the present application.

[0061] The similar question expansion method can generate more relevant questions to help improve the subsequent knowledge recall effect, so that knowledge related to the question can be quickly retrieved from knowledge bases from different sources during multi-channel knowledge recall.

[0062] S1022. Recall the knowledge base set according to at least one extended question to obtain K candidate knowledge points, where K is an integer greater than or equal to N.

[0063] For each extended question, there may be one or more domain types to which the extended question belongs. At least one knowledge base corresponding to the domain type to which the extended question belongs is selected from the knowledge base set, and the extended question is recalled in the at least one corresponding knowledge base. Since the extended question is obtained after the language model performs a similar extension on the first question, these extended questions have the same domain type, that is, different extended questions have the same knowledge base. Based on this, knowledge points are recalled for each knowledge base to avoid recalling duplicate knowledge points. Each extended question is vectorized, and the vector similarity between each extended question and each knowledge point in the knowledge base can be calculated. The knowledge points whose vector similarity is greater than a preset threshold are screened out, or a preset number of knowledge points with a high ranking (larger vector similarity) are screened out, thereby obtaining the knowledge points returned by each knowledge base. Each knowledge base in at least one knowledge base performs the above operation to obtain the knowledge points returned by at least one knowledge base, and the knowledge points returned by all knowledge bases are used as K candidate knowledge points.

[0064] By expanding the second question and recalling the knowledge base corresponding to the technical field to which each expanded question belongs, the comprehensiveness of the recalled candidate knowledge points is improved.

[0065] In some embodiments, S1022 in FIG3 above can also be implemented in the following manner. Using a first vectorization method, vectorize the knowledge points included in at least one extended question and at least one knowledge base respectively to obtain the first vector of each extended question and the first vector of the knowledge points included in each knowledge base; based on the first vector of the knowledge points included in each knowledge base and the first vector of at least one extended question, calculate the vector similarity between the knowledge points included in each knowledge base and the at least one extended question respectively to obtain multiple first vector similarities corresponding to each knowledge base; sort the multiple first vector similarities corresponding to each knowledge base to obtain the first sorting result corresponding to each knowledge base; based on the first sorting result corresponding to each knowledge base, screen out at least one candidate knowledge point corresponding to each knowledge base; and use the at least one candidate knowledge point corresponding to at least one knowledge base as K candidate knowledge points.

[0066] In an embodiment of the present application, a first vectorization method is used to vectorize each extended question and the knowledge points included in each knowledge base, respectively, to obtain the first vector of each extended question and the first vector of the knowledge points included in each knowledge base. For each knowledge base, based on the first vector of the knowledge points included in the knowledge base and the first vector of each extended question, the vector similarity between the knowledge points included in the knowledge base and each extended question is calculated to obtain multiple first vector similarities corresponding to the knowledge base; the multiple first vector similarities corresponding to the knowledge base are sorted to obtain the first sorting result corresponding to the knowledge base. Based on the first sorting result corresponding to the knowledge base, knowledge points with vector similarity greater than a preset threshold are screened, or a preset number of knowledge points with a high ranking (the first sorting result is sorted from high to low using similarity) are screened as at least one candidate knowledge point corresponding to the knowledge base. Each knowledge base in at least one knowledge base performs the above operation, thereby obtaining at least one candidate knowledge point returned by each of the at least one knowledge base, and the at least one candidate knowledge point corresponding to all knowledge bases is used as K candidate knowledge points.

[0067] In this example, when recalling knowledge points, the multiple questions expanded in the previous step are each vectorized using a vectorized model to obtain corresponding vector representations. The system then searches multiple knowledge bases related to the expanded questions to identify knowledge points within the knowledge base that are similar to each expanded question. Each knowledge base can be configured to return the top M (TopM) recall results, meaning the top M knowledge points.

[0068] When performing multi-way recall in at least one knowledge base related to the type of domain to which the extended question belongs, the recall results can include the following two methods: Method 1: For each extended question, each knowledge base returns the TopM recall results. In this method, since there are multiple knowledge bases and extended questions, when each knowledge base returns the TopM recall results for each extended question, there is still a situation where the same knowledge points are returned. Therefore, it is also necessary to deduplicate all knowledge points returned by the same knowledge base. Method 2: Regardless of whether it is for one or multiple extended questions, each knowledge base only returns the TopM recall results.

[0069] By calculating the vector similarity between the knowledge points and the extended questions, K candidate knowledge points are screened out after sorting the calculated vector similarities, thereby improving the accuracy of the screened candidate knowledge points.

[0070] In some embodiments, before step S1022 in FIG. 3 , the question-answering method further includes determining at least one knowledge base. The knowledge base associated with the at least one domain type input by the user is used as the at least one knowledge base; or, the domain to which each extended question in the at least one extended question belongs is classified to obtain the domain type to which each extended question belongs; and the knowledge base corresponding to the domain type to which the at least one extended question belongs is used as the at least one knowledge base.

[0071] In this embodiment of the present application, the knowledge base is constructed based on domain types, such as government affairs, finance, news, shopping, home, health, history, entertainment, pets, games, vehicles, music, painting, video, and travel. When entering a question, the user can specify at least one domain type, and the knowledge base related to the at least one domain type will be used as at least one knowledge base. A large language model can also be used to classify the domain type of the expanded question based on the knowledge base's profile (including the domain type) to determine which knowledge base of the domain type should be called.

[0072] The knowledge base to be searched can be pre-specified by the user, meaning they select the domain type of the knowledge base when entering the question. Alternatively, the extended question can be categorized using a classification model to determine the domain type of the extended question. The knowledge base corresponding to the domain type of the extended question is then selected as the knowledge base to be searched, thereby generating different knowledge bases.

[0073] In the embodiment of the present application, by classifying the extended questions through user pre-specified or classified models, the knowledge base that needs to be recalled multiple times can be reduced, thereby improving the recall efficiency.

[0074] S1023. Determine N knowledge points based on the K candidate knowledge points.

[0075] In this example, the K candidate knowledge points can be used as the N knowledge points, or the K candidate knowledge points can be further sorted to screen out the N knowledge points.

[0076] In some embodiments, S1023 in FIG3 can also be implemented in the following manner. A second vectorization method is used to vectorize at least one extended question and K candidate knowledge points respectively to obtain a second vector for each extended question and a second vector for each candidate knowledge point; the second vectorization method is different from the first vectorization method; based on the second vector of each extended question and the second vector of each candidate knowledge point, the vector similarity between each extended question and each candidate knowledge point is calculated to obtain multiple second vector similarities; the multiple second vector similarities are sorted to obtain a second sorting result; and N knowledge points are screened out based on the second sorting result.

[0077] In an embodiment of the present application, a second vectorization method is adopted to vectorize at least one extended question and K candidate knowledge points respectively to obtain a second vector for each extended question and a second vector for each candidate knowledge point; the vector similarity between each extended question and each candidate knowledge point is calculated to obtain multiple second vector similarities; the multiple second vector similarities are sorted to obtain a second sorting result; according to the second sorting result, if the second sorting result is sorted from high to low using similarity, the top N knowledge points are filtered.

[0078] In this example, because the previous recall step used multi-way recall, many knowledge points were obtained from different knowledge bases. Therefore, the recall results need to be uniformly sorted again to ensure that knowledge points related to the expanded question are prioritized. The sorting stage also uses a vectorization model (which can be different from the vectorization model used in the recall stage) to extract vectors for all recalled knowledge points and expanded questions. These vectors are then sorted based on cosine similarity to further filter out knowledge points that are similar to the expanded question. The sorting stage can be set to retain only the top N results with the highest similarity.

[0079] It's important to note that using a different vectorization model than the one used in the recall phase (the second vectorization method differs from the first) allows for encoding the textual semantic information of the extended question and knowledge points from a different dimension. This is then sorted based on vector similarity to select knowledge points with high relevance. Further vectorization and sorting of knowledge points can reduce the number of knowledge points, thereby reducing data processing for subsequent question-related content extraction and improving processing efficiency.

[0080] In an embodiment of the present application, the relevant knowledge sorting method can ensure that the knowledge points most relevant to the expanded question have a higher priority, and further screen out N knowledge points with greater relevance to the expanded question, which reduces the amount of data compared to the K candidate knowledge points, so as to improve processing efficiency when subsequently extracting relevant content.

[0081] In some embodiments, the question-answering method further includes a process for constructing a knowledge base set. The process involves obtaining text information samples and question-answer pair samples; vectorizing the question-answer pair samples and text information samples to obtain question-answer pair vectors and text information vectors; using the question-answer pair vectors and text information vectors as knowledge point samples, and constructing a knowledge base corresponding to the domain type of the knowledge point samples based on the knowledge point samples and the domain type to which the knowledge point samples belong.

[0082] In an embodiment of the present application, the construction of the knowledge base mainly includes two steps, namely knowledge normalization and vectorization. The input of the knowledge base allows two standardized formats, one is a question-answer pair, that is, a question-answer format, and the other is a plain text description (i.e., text information). For the question-answer pair format, no normalization is required, and the question can be directly used for vectorization and stored in a vector database. For the plain text description, each text information is vectorized and stored in a vector database. The vectorized question-answer pair and the vectorized text information are stored as knowledge points in the knowledge base corresponding to the field type to which the knowledge point belongs, thereby constructing a knowledge base of multiple field types, each knowledge base including a plurality of knowledge points related to its corresponding field type for knowledge point recall. The vectorization process can use an open source sentence transformer or SimBert model, which can encode a piece of text into a vector of fixed length (e.g., 768 dimensions). The vector encodes the semantic information of the text. By calculating and comparing the cosine distance of the corresponding vectors of different texts, texts with similar semantics can be found.

[0083] In an embodiment of the present application, by constructing a knowledge base, the question-answering system used to execute the question-answering method can pre-integrate and store a large amount of knowledge data, which is more flexible. It can not only meet the user's needs for knowledge in different fields and the latest knowledge, but also continuously improve the intelligence level of the question-answering system as the external knowledge base is updated. Compared with the technical solution of directly using a large language model for question answering, the accuracy of the question-answering results is improved.

[0084] In some embodiments, when constructing a knowledge base set, the question-answering method further includes: when the length of a text information sample exceeds a preset threshold, segmenting the text information sample to obtain at least two segmented samples, where adjacent segmented samples include overlapping characters. Correspondingly, the vectorization step can also be implemented by separately vectorizing the question-answer pair sample and the at least two segmented samples to obtain a question-answer pair vector and at least two segmented information vectors; wherein the text information vector includes the at least two segmented information vectors.

[0085] For plain text descriptions, when the length of the text information is greater than a preset threshold, the normalization stage also requires the text information to be split or segmented, dividing a text message into at least two segmented texts. For example, each segmented text does not exceed 256 characters, and there is a certain overlap of characters between two adjacent segmented texts to ensure the integrity of the text. Each segmented text is then vectorized and stored in a vector database. The vectorized question-answer pairs and the vectorized segmented texts are stored as knowledge points in the knowledge base corresponding to the field type to which the knowledge point belongs, thereby constructing a knowledge base of multiple field types. By segmenting the text, the subsequent extraction of related content can be accelerated and the processing efficiency can be improved.

[0086] In some embodiments, S103 in Figure 1 can also be implemented in the following manner: As shown in Figure 4, Figure 4 is a flow chart of another question and answer generation method provided in an embodiment of the present application.

[0087] S1031. Perform content relevance extraction on each of the N knowledge points according to the second question to obtain an extraction result corresponding to each knowledge point. The extraction result is a result representing a result unrelated to the second question or text content related to the second question.

[0088] In an embodiment of the present application, some of the N knowledge points that have been sorted and screened are still irrelevant to the second question. In this step, a large language model is used to extract relevant content from the N knowledge points. That is, the language model is used to extract the content relevance of each knowledge point according to the second question to obtain the extraction results corresponding to each knowledge point. There are two types of extraction results. One is a result that is irrelevant to the second question. The language model can output "none" to indicate that no knowledge content related to the second question has been extracted, that is, the knowledge point has no relevance to the second question. The other is text content related to the second question, and the language model can output the text content.

[0089] Exemplarily, the second question and each knowledge point (the knowledge point can be a recalled related question-answer pair, or a recalled related text information or segmented text) are input into the language model together. The prompt word used for the language model (that is, the input for constructing the language model) can be "Please judge whether the given question and the reference content are relevant. The given question: XXX, the given reference content: YYY. If the question and the content are not relevant, please output "none". If relevant, please output the first 5 characters and the last 5 characters of the relevant content". This prompt word is input into the language model. Let the language model determine whether the second question and the knowledge point are relevant. If not relevant, the language model outputs "none". If relevant, the language model outputs the first 5 characters and the last 5 characters of the relevant paragraph (that is, the extraction result corresponding to each knowledge point), and then uses the first 5 characters and the last 5 characters to do the matching.

[0090] S1032. Determine the content of the knowledge point based on the extraction result corresponding to at least one knowledge point among the N knowledge points, wherein the extraction result corresponding to each knowledge point among the at least one knowledge point includes text content related to the second question.

[0091] Since the extraction results can be unrelated to the second question, after obtaining the extraction results corresponding to each knowledge point, it is necessary to filter out the results unrelated to the second question from the N knowledge point extraction results, determine the text content related to the second question, and thus obtain the knowledge point content. If the extraction results of all N knowledge points represent results unrelated to the second question, the number of knowledge point contents obtained is 0, that is, there is no reference knowledge point content. Then, the target answer subsequently generated by the language model based on the second question and the knowledge point content is an answer that cannot be answered.

[0092] By extracting content relevance, we extract the knowledge point content from N knowledge points, reducing the amount of data used as reference content when generating answers, improving processing efficiency. Furthermore, by extracting text content related to the second question, we also improve the accuracy of the reference content used when generating answers.

[0093] In some embodiments, the text content is a text segment, and S1032 in FIG4 can also be implemented in the following manner: determining the text segment related to the second question included in the extraction result corresponding to each knowledge point in the at least one knowledge point as the knowledge point content.

[0094] In an embodiment of the present application, for each extraction result, if the text content related to the second text included in the extraction result is a text fragment, the text fragment is used as the knowledge point content corresponding to the extraction result.

[0095] By extracting text fragments, there is no need to match them again in the knowledge points, which improves processing efficiency.

[0096] In some embodiments, the text content is a plurality of characters in a text segment, and at least one knowledge point includes a first knowledge point. S1032 in FIG4 can also be implemented in the following manner. A plurality of characters in a text segment related to the second question included in the extraction result corresponding to the first knowledge point are matched with the first knowledge point to determine a text segment matching the plurality of characters from the first knowledge point; if there is one text segment matching the plurality of characters, the text segment matching the plurality of characters is used as the knowledge point content; if there are multiple text segments matching the plurality of characters, the text segment with the longest text matching length with the plurality of characters is determined from the plurality of text segments, and the determined text segment is used as the knowledge point content.

[0097] In an embodiment of the present application, the first knowledge point is any one of the at least one knowledge point. For the extraction result corresponding to the first knowledge point, if the text content related to the second question included in the extraction result is multiple characters, and the multiple characters are characters in one or more text fragments in the first knowledge point, then the multiple characters need to be matched with the first knowledge point to match the text fragment, and the matched text fragment is used as the knowledge point content. When matching multiple characters with the first knowledge point, if there is one matching text fragment, then the text fragment is used as the knowledge point content; if there are multiple matching text fragments (that is, multiple text fragments all include these multiple characters), then the multiple text fragments are all used as the knowledge point content, or the text fragment with the longest text matching length among the multiple text fragments is used as the knowledge point content, or a preset number (for example, 2 or 3) of text fragments with the longest text matching length among the multiple text fragments are all used as the knowledge point content. When there are multiple matching text segments and the number of text segments with the longest text matching length is multiple (that is, the lengths of multiple text matching segments are equal and the longest), the multiple text segments with the longest text matching length are all used as knowledge point content.

[0098] For each knowledge point and the second question, if the knowledge point is not related to the second question, the language model outputs "none"; if it is related, the language model outputs several characters of the text fragment related to the second question in the knowledge point, which can be the first P characters, the last P characters, or a combination of the above two. P can be 3, 4, 5, etc., and this embodiment of the present application does not limit this. Based on the principle of string matching, the text fragment is extracted from the knowledge point. If multiple text fragments are matched, the longest matched text fragment is taken. Since the language model is equivalent to a transformer decoder, if the entire segment is output during the decoding process, it will take a lot of time. By allowing the language model to output only the first P characters and the last P characters of the text fragment, the decoding can be accelerated and the processing efficiency can be improved.

[0099] In some embodiments, S104 in Figure 1 can also be implemented in the following manner: The number of knowledge point contents is at least one, as shown in Figure 5 , which is a flow chart of another question and answer generation method provided in an embodiment of the present application.

[0100] S1041. Use the second question and at least one knowledge point content as input to the language model, and output candidate answers through the language model.

[0101] In an embodiment of the present application, when generating an answer, at least one knowledge point content is used as a reference, and the second question is input into a language model, and a candidate answer is output through the language model.

[0102] For example, in the summary answer, the relevant content extracted in the previous step (i.e., at least one knowledge point) is used as a reference, along with the user's question (i.e., the second question), and is input into the language model, which then responds (i.e., outputs a candidate answer). Here, the language model can use the following template (i.e., construct the language model input):

[0103] "Based on the following known information, answer the user's question concisely and professionally. The question is: {query}. If you cannot get the answer from it, please say {The question cannot be answered based on the known information}. Fabricated elements are not allowed in the answer. Please use Chinese to answer. The known content is in quotation marks as follows: {content} answer."

[0104] Here, query is the generated user question (i.e., the second question), and content is the extracted reference content related to the question (i.e., at least one knowledge point content).

[0105] S1042. Determine the longest common substring between the candidate answer and each knowledge point content in at least one knowledge point content, and obtain the longest common substring corresponding to each knowledge point content.

[0106] S1043. Determine the target answer based on the length of the longest common substring corresponding to the candidate answer and at least one knowledge point content.

[0107] Since the candidate answers may be fabricated by the language model without reference to the knowledge point content, the candidate answers need to be verified. For each knowledge point content, the longest common substring between the candidate answer and the knowledge point content is calculated to obtain multiple longest common substrings. By comparing the relationship between the substring length of the candidate answer and the length of the multiple longest common substrings, it is determined whether the candidate answer is output by the language model without reference to the knowledge point content, or after the language model has referenced at least one knowledge point content. If it is determined that the candidate answer is output by the language model without reference to the knowledge point content, the candidate answer is discarded and the answer that cannot be answered is fed back to the user as the target answer; if it is determined that the candidate answer is output by the language model after reference to at least one knowledge point content, the candidate answer is fed back to the user as the target answer.

[0108] By summarizing the answers, we can integrate the multiple extracted knowledge points, provide concise and clear answers, output candidate answers, and verify the results (candidate answers) generated by the language model through result review to ensure the accuracy of the generated content. In other words, it ensures that the answer of the language model refers to at least one given knowledge point content to reduce the interference of hallucination phenomena (that is, the phenomenon of the language model talking nonsense seriously) and improve the accuracy of the target answer.

[0109] In some embodiments, S1043 in FIG. 5 can also be implemented in the following manner: if the length of the longest common substring corresponding to at least one knowledge point content includes a longest common substring greater than a preset length threshold, the candidate answer is used as the target answer; if the length of the longest common substring corresponding to at least one knowledge point content is less than or equal to the preset length threshold, the answer indicating that the answer cannot be answered is used as the target answer.

[0110] In an embodiment of the present application, the longest common substring of the generated candidate answer and at least one knowledge point content (as reference content) is calculated one by one. If the length of the longest common substring between the generated candidate answer and all reference contents is less than or equal to the preset length threshold (for example, less than 5 characters), it means that the candidate answer is likely to be output by the language model without reference to the knowledge point content, and the answer is directly discarded, and the answer that cannot be answered is used as the target answer. Correspondingly, if the length of the longest common substring between the generated candidate answer and all reference contents contains a substring greater than the preset length threshold, it means that the candidate answer is output by the language model based on the reference content, and the candidate answer is used as the target answer.

[0111] It should be noted that the preset length threshold can be appropriately set by those skilled in the art according to actual conditions and can be determined based on a large amount of experimental data, for example, 5 characters, 4 characters, 7 characters, etc., and this embodiment of the present application does not limit this.

[0112] By comparing the longest common substring with a preset length threshold, the candidate answers generated by the language model are verified, thereby improving the accuracy of the target answer.

[0113] In some embodiments, S1043 in FIG5 above can also be implemented in the following manner. Determine the length ratio between the length of the longest common substring corresponding to each knowledge point content in at least one knowledge point content and the corresponding reference length to obtain at least one length ratio; wherein, the longest common substring corresponding to each knowledge point content refers to the longest common substring between each knowledge point content and the candidate answer, and the reference length corresponding to each knowledge point content refers to the minimum substring length between the substring corresponding to each knowledge point content and the substring of the candidate answer; in the case where there is a value greater than a preset ratio in at least one length ratio, the candidate answer is used as the target answer; in the case where at least one length ratio is less than or equal to the preset ratio, the answer representing an unanswerable answer is used as the target answer.

[0114] In an embodiment of the present application, the longest common substring between each knowledge point content and the candidate answer is calculated, and the minimum substring length in the substring corresponding to each knowledge point content and the substring of the candidate answer is used as the reference substring corresponding to the knowledge point content, and then the length ratio between the length of each longest common substring and the reference length is calculated, and the length ratio range is 0-1. If at least one of the length ratios is greater than the preset ratio, it means that the candidate answer is output by the language model based on the reference content, and the candidate answer is used as the target answer. If at least one of the length ratios is less than or equal to the preset ratio, it means that the candidate answer is likely to be output by the language model without reference to the knowledge point content, and the answer is directly discarded, and the answer that cannot be answered is used as the target answer.

[0115] Exemplarily, the longest common substring / min (reference answer length, generated candidate answer length) is calculated, where the longest common substring refers to the longest common substring between the knowledge point content and the candidate answer, the reference answer refers to the knowledge point content, and min (reference answer length, generated candidate answer length) represents the length of the smallest substring between the substring corresponding to the knowledge point content and the substring of the candidate answer. This length ratio is a value between 0 and 1. If it is less than the preset ratio, it can be considered that the candidate answer output by the language model cannot be fed back to the user.

[0116] It should be noted that the preset ratio can be appropriately set by those skilled in the art according to actual conditions and can be determined based on a large amount of experimental data, for example, 0.5, 0.4, 0.7, 0.9, etc., and this embodiment of the present application does not limit this.

[0117] This solution offers an alternative judgment method that considers the overall length of substrings in real-world scenarios and uses a ratio calculation method to determine whether a candidate answer can be used as the target answer, increasing the diversity of judgment methods. By calculating the ratio between the length of the longest common substring and a reference length and comparing this length ratio with a preset ratio, candidate answers generated by the language model are verified, improving the accuracy of the target answer.

[0118] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0119] This application proposes an intelligent question-answering system that combines an external knowledge base and a large-scale language model, which is used to utilize the reading comprehension and contextual learning capabilities of the large-scale language model, combined with the external knowledge base, to achieve a more intelligent question-answering function. The question-answering system includes eight key modules: knowledge base construction, question generation combined with contextual dialogue, similar question expansion, multi-channel knowledge recall, relevant knowledge sorting, question-related content extraction, summary answer, and result generation review. This application systematically designs multiple key modules, organically combining large-scale language models with external knowledge bases, and fully utilizing the advantages of both, making the question-answering system more flexible and accurate, which can not only meet the user's needs for knowledge in different fields, but also continuously improve the intelligence level of the question-answering system as the external knowledge base is updated. The following is an introduction.

[0120] As shown in FIG6 , FIG6 is a flowchart of another question-and-answer method provided in an embodiment of the present application, including S201 - S208 .

[0121] S201. Knowledge base construction.

[0122] In this example, the plain text description is normalized and the text information is segmented or divided into at least two segments, with a certain amount of character overlap between adjacent segments. Each segment is then vectorized. Question-answer pairs are directly vectorized. The plain text and its vector, as well as the question-answer pairs and their vectors, are stored in knowledge bases of corresponding domain types, thus constructing a knowledge base collection.

[0123] S202: Generate questions based on the context of the dialogue.

[0124] In this example, the user's input question (i.e., the first question) is restated based on the conversation history to generate a semantically complete question, assisting in subsequent knowledge base retrieval, thus enabling multi-turn conversations within the knowledge base. Here, contextual learning can be used to enable the language model to help the user generate a complete question (i.e., the second question) based on the conversation history. During this complete question generation process, prompts can be added to the language model to enable it to generate a complete question (i.e., the second question) based on the user's input question.

[0125] S203. Extension of similar questions.

[0126] In this example, the second question is expanded to generate multiple questions with different wordings but similar semantics (i.e., expanded questions) to help improve recall. This similar question expansion process uses contextual learning to enable the language model to generate similar questions. Pre-provided templates (sample questions and corresponding expanded question samples) are used as prompts for the language model, allowing it to generate multiple expanded questions based on the first question.

[0127] S204, multi-channel knowledge recall.

[0128] In this example, when recalling knowledge points, the multiple questions expanded in the previous step are each vectorized using a vectorized model to obtain corresponding vector representations. Then, a search is performed from multiple knowledge bases related to the expanded questions to identify knowledge points within the knowledge base that are similar to each expanded question. Each knowledge base can be configured to return the top M recall results, meaning the top M knowledge points.

[0129] S205. Sorting of related knowledge.

[0130] In this example, because the previous recall step used multi-way recall, many knowledge points were obtained from different knowledge bases. Therefore, the recall results need to be uniformly sorted again to ensure that knowledge points related to the expanded question are prioritized. The sorting stage also uses a vectorization model (which can be different from the vectorization model used in the recall stage) to extract vectors for all recalled knowledge points and expanded questions. These vectors are then sorted based on cosine similarity to further filter out knowledge points that are similar to the expanded question. The sorting stage can be set to retain only the top N results with the highest similarity.

[0131] S206: Extracting question-related content.

[0132] In this example, some of the sorted results are still irrelevant to the question. In this step, a large language model is used to extract relevant content from the sorted results. For each knowledge point and the second question, if the knowledge point is irrelevant to the second question, the language model outputs "none." If relevant, the language model outputs a few characters of the text fragment from the knowledge point that is relevant to the second question. Based on the principle of string matching, this text fragment is then extracted from the knowledge point. If multiple text fragments are matched, the longest matching text fragment is used.

[0133] S207. Summarize the answers.

[0134] In this example, the relevant content extracted in the previous step is used as a reference, along with the user's question, and is input into the language model to answer the question.

[0135] S208: Review the generated results.

[0136] In this example, the generated results are verified to ensure that the language model's answers reference the given knowledge points, thereby reducing the risk of hallucination (i.e., the language model sounding serious and nonsensical). The longest common substring between the generated answer and the reference content extracted in step S206 is calculated one by one. If the length of the generated answer and all the longest common substrings is below a certain threshold (e.g., less than 5 characters), then the answer is likely generated by the language model without reference to the knowledge point content, and the answer is discarded.

[0137] It should be noted that in the above steps, a language model is used in steps S202, S203, S206, and S207. When executing these steps (for example, generating a complete question, question expansion, content extraction, and summary answer), some examples (also called templates) are input into the language model as prompts so that the model can output the desired content (for example, a complete question, an expanded question, N characters, and a final answer).

[0138] By building a knowledge base, the system can pre-integrate and store large amounts of knowledge data. Combined with a large-scale language model, the system can generate more accurate questions based on the context of the conversation, enabling the system to have multiple rounds of conversations. The similar question expansion module can generate more relevant questions to help improve the subsequent knowledge recall effect. The multi-channel knowledge recall module can quickly retrieve knowledge related to the question from knowledge bases from different sources. The relevant knowledge sorting module ensures that the most relevant knowledge has a higher priority, and the question-related content extraction module extracts knowledge related to the question from the sorted results. The summary answer module can synthesize multiple pieces of extracted knowledge and provide concise and clear answers. The final result review module conducts a second verification of the results generated by the language model to ensure the accuracy of the generated content.

[0139] Through the organic combination of these key modules, the intelligent question-answering system can better understand the questions and provide accurate and comprehensive answers based on the latest knowledge base. Compared with directly using a large language model for question-answering, this application introduces an external knowledge base. This method of combining the knowledge base enables the large language model to more flexibly utilize the latest external knowledge, thereby improving the effect of question-answering. By combining the external knowledge base, the intelligent question-answering system can obtain the latest knowledge in a timely manner and integrate it into the question-answering process to meet the user's demand for the latest knowledge, thereby improving the flexibility and accuracy of the question-answering system.

[0140] Based on the method provided in the above embodiment, FIG7 is a structural diagram of a question-answering device provided in an embodiment of the present application. The question-answering device can be implemented as part or all of a computer device by software, hardware, or a combination of both. Referring to FIG7, the question-answering device 70 (which can be applied to the above-mentioned question-answering system) includes: a question generation module 701, which is used to generate a second question based on the user's first question and the context of the first question; a recall module 702, which is used to recall in a knowledge base set according to the second question to obtain N knowledge points, where N is a positive integer, and the knowledge base set includes multiple knowledge bases of different domain types, and the multiple knowledge bases include at least one knowledge base corresponding to the domain type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding domain type; an extraction module 703, which is used to extract content relevance from the N knowledge points according to the second question, determine the knowledge point content related to the second question from the N knowledge points, and obtain the knowledge point content; an answer generation module 704, which is used to generate a target answer through a language model based on the second question and the knowledge point content.

[0141] Optionally, the extraction module 703 is also used to perform content relevance extraction on each of the N knowledge points according to the second question, and obtain an extraction result corresponding to each knowledge point, wherein the extraction result is a result representing a result that is irrelevant to the second question, or text content related to the second question; and determine the content of the knowledge point based on the extraction result corresponding to at least one knowledge point among the N knowledge points, wherein the extraction result corresponding to each knowledge point in the at least one knowledge point includes text content related to the second question.

[0142] Optionally, the text content is a text fragment;

[0143] The extraction module 703 is further configured to determine the text segment related to the second question included in the extraction result corresponding to each knowledge point in the at least one knowledge point as the knowledge point content.

[0144] Optionally, the text content is a plurality of characters in a text segment, and the at least one knowledge point includes a first knowledge point;

[0145] The extraction module 703 is further configured to match a plurality of characters in a text segment related to the second question included in the extraction result corresponding to the first knowledge point with the first knowledge point, so as to determine a text segment matching the plurality of characters from the first knowledge point;

[0146] If there is only one text segment matching the multiple characters, then the text segment matching the multiple characters is used as the knowledge point content;

[0147] If there are multiple text segments that match the multiple characters, a text segment with the longest text matching length with the multiple characters is determined from the multiple text segments, and the determined text segment is used as the knowledge point content.

[0148] Optionally, the number of the knowledge point content is at least one;

[0149] The answer generation module 704 is also used to take the second question and at least one knowledge point content as inputs of the language model, and output candidate answers through the language model; determine the longest common substring between the candidate answer and each knowledge point content in the at least one knowledge point content, and obtain the longest common substring corresponding to each knowledge point content; and determine the target answer based on the length of the longest common substring corresponding to the candidate answer and the at least one knowledge point content.

[0150] Optionally, the answer generation module 704 is further configured to use the candidate answer as the target answer if there is a longest common substring whose length is greater than a preset length threshold among the longest common substrings corresponding to the at least one knowledge point content;

[0151] When the length of the longest common substring corresponding to the at least one knowledge point content is less than or equal to the preset length threshold, the answer representing an unanswerable answer is used as the target answer.

[0152] Optionally, the answer generation module 704 is further used to determine the length ratio between the length of the longest common substring corresponding to each knowledge point content in the at least one knowledge point content and the corresponding reference length to obtain at least one length ratio; wherein, the longest common substring corresponding to each knowledge point content refers to the longest common substring between each knowledge point content and the candidate answer, and the reference length corresponding to each knowledge point content refers to the minimum substring length between the substring corresponding to each knowledge point content and the substring of the candidate answer; when there is a value greater than a preset ratio in the at least one length ratio, the candidate answer is used as the target answer; when the at least one length ratio is less than or equal to the preset ratio, the answer representing an unanswerable answer is used as the target answer.

[0153] Optionally, the question generation module 701 is further used to generate the second question based on the first question when there is no historical conversation before the first question; and to generate the second question based on the first question and the historical conversation when there is a historical conversation before the first question.

[0154] Optionally, the recall module 702 is further used to semantically expand the second question to obtain at least one expanded question; recall the knowledge base set based on the at least one expanded question to obtain K candidate knowledge points, where K is an integer greater than or equal to N; and determine the N knowledge points based on the K candidate knowledge points.

[0155] Optionally, the recall module 702 is further used to adopt a first vectorization method to vectorize the at least one extended question and the knowledge points included in the at least one knowledge base, respectively, to obtain the first vector of each extended question and the first vector of the knowledge points included in each knowledge base; based on the first vector of the knowledge points included in each knowledge base and the first vector of the at least one extended question, respectively calculate the vector similarity between the knowledge points included in each knowledge base and the at least one extended question, to obtain multiple first vector similarities corresponding to each knowledge base; sort the multiple first vector similarities corresponding to each knowledge base, to obtain the first sorting result corresponding to each knowledge base; based on the first sorting result corresponding to each knowledge base, screen out at least one candidate knowledge point corresponding to each knowledge base; and use the at least one candidate knowledge point corresponding to the at least one knowledge base as the K candidate knowledge points.

[0156] Optionally, the recall module 702 is also used to adopt a second vectorization method to vectorize the at least one extended question and the K candidate knowledge points respectively to obtain a second vector of each extended question and a second vector of each candidate knowledge point; the second vectorization method is different from the first vectorization method; based on the second vector of each extended question and the second vector of each candidate knowledge point, the vector similarity between each extended question and each candidate knowledge point is calculated to obtain multiple second vector similarities; the multiple second vector similarities are sorted to obtain a second sorting result; and the N knowledge points are screened out according to the second sorting result.

[0157] Optionally, the recall module 702 is further used to use the knowledge base involved in at least one field type input by the user as the at least one knowledge base; or, classify the field to which each extended question in the at least one extended question belongs to obtain the field type to which each extended question belongs; and use the knowledge base corresponding to the field type to which the at least one extended question belongs as the at least one knowledge base.

[0158] Optionally, the question-answering device further includes a construction module 705;

[0159] Construction module 705 is also used to obtain text information samples and question-answer pair samples; vectorize the question-answer pair samples and the text information samples respectively to obtain question-answer pair vectors and text information vectors; use the question-answer pair vectors and the text information vectors as knowledge point samples, and construct a knowledge base corresponding to the field type to which the knowledge point samples belong based on the knowledge point samples and the field type to which the knowledge point samples belong.

[0160] Optionally, the construction module 705 is also used to split the text information sample when the length of the text information sample is greater than a preset threshold, to obtain at least two segmented samples, and the two adjacent segmented samples include overlapping characters; vectorize the question-answer pair sample and the at least two segmented samples respectively to obtain a question-answer pair vector and at least two segmented information vectors; wherein the text information vector includes the at least two segmented information vectors.

[0161] It should be noted that when the question-answering device provided in the above embodiment executes the question-answering method, it only uses the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0162] The functional units and modules in the above embodiments may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The above integrated units may be implemented in the form of hardware or software functional units. In addition, the specific names of the functional units and modules are only for the purpose of distinguishing them from each other and are not intended to limit the scope of protection of the embodiments of this application.

[0163] The question-and-answer device and method embodiment provided in the above embodiments belong to the same concept. The question-and-answer device and method embodiment provided in the above embodiments belong to the same concept. The specific working process of the units and modules in the above embodiments and the technical effects brought about can be found in the method embodiment part and will not be repeated here.

[0164] Based on the method provided in the above embodiment, Figure 8 is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 8, the computer device 80 (also referred to as a question-and-answer system) includes: a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, the steps in the question-and-answer method in the above embodiment are implemented.

[0165] Computer device 80 can be a general-purpose computer device or a dedicated computer device. In a specific implementation, computer device 80 can be a desktop computer, a portable computer, a network server, a PDA, a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device 80. Those skilled in the art will understand that FIG8 is merely an example of computer device 80 and does not constitute a limitation of computer device 80. Computer device 80 may include more or fewer components than shown, or may combine certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0166] The processor 801 may be a central processing unit (CPU). The processor 801 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0167] In some embodiments, the memory 802 may be an internal storage unit of the computer device 80, such as a hard drive or memory of the computer device 80. In other embodiments, the memory 802 may also be an external storage device of the computer device 80, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 80. Furthermore, the memory 802 may include both an internal storage unit of the computer device 80 and an external storage device. The memory 802 is used to store an operating system, application programs, a boot loader, data, and other programs. The memory 802 may also be used to temporarily store data that has been output or is about to be output.

[0168] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0169] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0170] An embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the steps in the above-mentioned various method embodiments.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, an executable file, or some intermediate form. The computer-readable medium can include at least any entity or device capable of carrying the computer program code to a camera / terminal device, a recording medium, computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, a non-transitory storage medium.

Claims

1. A question-answering method, characterized in that: The method comprises: Generate a second question based on the user's first question and the context of the first question; Recalling in a knowledge base set according to the second question to obtain N knowledge points, where N is a positive integer, the knowledge base set includes a plurality of knowledge bases of different field types, the plurality of knowledge bases include at least one knowledge base corresponding to the field type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding field type; Extracting content relevance of the N knowledge points according to the second question, determining knowledge point content related to the second question from the N knowledge points, and obtaining knowledge point content; According to the second question and the content of the knowledge point, a target answer is generated through a language model.

2. The method according to claim 1, characterized in that The extracting content relevance of the N knowledge points according to the second question, determining knowledge point content related to the second question from the N knowledge points, and obtaining knowledge point content includes: Performing content relevance extraction on each of the N knowledge points according to the second question to obtain an extraction result corresponding to each knowledge point, wherein the extraction result is a result representing nothing to do with the second question or text content related to the second question; The content of the knowledge point is determined according to an extraction result corresponding to at least one knowledge point among the N knowledge points, and the extraction result corresponding to each knowledge point among the at least one knowledge point includes text content related to the second question.

3. The method according to claim 2, characterized in that The text content is a text fragment; The determining the content of the knowledge point according to the extraction result corresponding to at least one knowledge point among the N knowledge points includes: A text segment related to the second question included in the extraction result corresponding to each knowledge point in the at least one knowledge point is determined as the knowledge point content.

4. The method according to claim 2, characterized in that The text content is a plurality of characters in a text segment, and the at least one knowledge point includes a first knowledge point; The determining the content of the knowledge point according to the extraction result corresponding to at least one knowledge point among the N knowledge points includes: Matching a plurality of characters in a text segment related to the second question included in the extraction result corresponding to the first knowledge point with the first knowledge point to determine a text segment matching the plurality of characters from the first knowledge point; If there is only one text segment matching the multiple characters, then the text segment matching the multiple characters is used as the knowledge point content; If there are multiple text segments matching the multiple characters, a text segment with the longest text matching length with the multiple characters is determined from the multiple text segments, and the determined text segment is used as the knowledge point content.

5. The method according to claim 1, characterized in that The number of the knowledge point content is at least one; Generating a target answer by using a language model according to the second question and the knowledge point content includes: using the second question and at least one knowledge point content as inputs of the language model, and outputting candidate answers through the language model; Determine the longest common substring between the candidate answer and each knowledge point content in the at least one knowledge point content, and obtain the longest common substring corresponding to each knowledge point content; The target answer is determined according to the length of the longest common substring corresponding to the candidate answer and the at least one knowledge point content.

6. The method according to claim 5, characterized in that The determining the target answer according to the length of the longest common substring corresponding to the candidate answer and the at least one knowledge point content includes: In the case where there is a longest common substring whose length is greater than a preset length threshold among the longest common substrings corresponding to the at least one knowledge point content, taking the candidate answer as the target answer; In the case where the length of the longest common substring corresponding to the at least one knowledge point content is less than or equal to the preset length threshold, the answer representing an unanswerable answer is taken as the target answer.

7. The method according to claim 5, characterized in that The determining the target answer according to the length of the longest common substring corresponding to the candidate answer and the at least one knowledge point content includes: Determine a length ratio between a length of a longest common substring corresponding to each knowledge point content in the at least one knowledge point content and a corresponding reference length, and obtain at least one length ratio; The longest common substring corresponding to each knowledge point content refers to the longest common substring between each knowledge point content and the candidate answer, and the reference length corresponding to each knowledge point content refers to the minimum substring length between the substring corresponding to each knowledge point content and the substring of the candidate answer; In the case where a value greater than a preset ratio exists in the at least one length ratio, taking the candidate answer as the target answer; In the case that at least one of the length ratios is less than or equal to the preset ratio, the answer indicating that the answer cannot be answered is taken as the target answer.

8. The method according to any one of claims 1 to 7, characterized in that: The generating the second question according to the first question of the user and the context of the first question includes: In the case that there is no historical dialogue before the first question, generating the second question according to the first question; In the case where there is a historical conversation before the first question, the second question is generated according to the first question and the historical conversation.

9. The method according to any one of claims 1 to 7, characterized in that: The step of recalling the knowledge base set according to the second question to obtain N knowledge points includes: semantically expanding the second question to obtain at least one expanded question; Recalling the knowledge base set according to the at least one extended question to obtain K candidate knowledge points, where K is an integer greater than or equal to N; The N knowledge points are determined based on the K candidate knowledge points.

10. The method according to claim 9, characterized in that The recalling in the at least one knowledge base according to the at least one extended question to obtain K candidate knowledge points includes: Using a first vectorization method, respectively vectorize the at least one extended question and the knowledge points included in the at least one knowledge base to obtain a first vector of each extended question and a first vector of each knowledge point included in the knowledge base; According to the first vectors of the knowledge points included in the respective knowledge bases and the first vector of the at least one extended question, respectively calculating the vector similarities between the knowledge points included in the respective knowledge bases and the at least one extended question, to obtain a plurality of first vector similarities corresponding to the respective knowledge bases; Sorting the similarities of the multiple first vectors corresponding to the respective knowledge bases to obtain first sorting results corresponding to the respective knowledge bases; Filtering out at least one candidate knowledge point corresponding to each knowledge base according to the first ranking result corresponding to each knowledge base; At least one candidate knowledge point corresponding to the at least one knowledge base is used as the K candidate knowledge points.

11. The method according to claim 9, characterized in that The determining the N knowledge points according to the K candidate knowledge points includes: Using a second vectorization method, respectively vectorizing the at least one extended question and the K candidate knowledge points to obtain a second vector of each extended question and a second vector of each candidate knowledge point; the second vectorization method is different from the first vectorization method; Calculating the vector similarities between each of the extended questions and each of the candidate knowledge points according to the second vectors of each of the extended questions and the second vectors of each of the candidate knowledge points to obtain a plurality of second vector similarities; Sorting the plurality of second vector similarities to obtain a second sorting result; The N knowledge points are screened out according to the second sorting result.

12. The method according to claim 9, characterized in that Before recalling the knowledge base set according to the at least one extended question to obtain K candidate knowledge points, the method further includes: Using the knowledge base involved in at least one field type input by the user as the at least one knowledge base; Alternatively, the field to which each extended question in the at least one extended question belongs is classified to obtain the field type to which each extended question belongs; and the knowledge base corresponding to the field type to which the at least one extended question belongs is used as the at least one knowledge base.

13. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtaining text information samples and question-answer pair samples; Segmenting the text information sample to obtain at least two segmented text samples, wherein two adjacent segmented text samples include overlapping characters; Vectorizing the question-answer pair sample and the text information sample respectively to obtain a question-answer pair vector and a text information vector; The question-answer pair vector and the text information vector are used as knowledge point samples, and according to the knowledge point samples and the field types to which the knowledge point samples belong, a knowledge base corresponding to the field type to which the knowledge point samples belong is constructed.

14. The method according to claim 13, characterized in that After obtaining the text information sample and the question-answer pair sample, the method further includes: When the length of the text information sample is greater than a preset threshold, segmenting the text information sample to obtain at least two segment samples, wherein two adjacent segment samples include overlapping characters; The question-answer pair sample and the text information sample are respectively vectorized to obtain a question-answer pair vector and a text information vector, including: The question-answer pair sample and the at least two segmented samples are respectively vectorized to obtain a question-answer pair vector and at least two segmented information vectors; wherein the text information vector includes the at least two segmented information vectors.

15. A question-answering device, characterized in that: The device comprises: A question generation module, used to generate a second question according to a first question of a user and a context of the first question; A recall module, configured to recall in a knowledge base set according to the second question to obtain N knowledge points, where N is a positive integer, the knowledge base set includes a plurality of knowledge bases of different domain types, the plurality of knowledge bases include at least one knowledge base corresponding to the domain type to which the second question belongs, and each knowledge base is used to store knowledge points related to the corresponding domain type; An extraction module, configured to extract content relevance of the N knowledge points according to the second question, determine knowledge point content related to the second question from the N knowledge points, and obtain knowledge point content; An answer generation module is used to generate a target answer through a language model according to the second question and the content of the knowledge point.

16. A computer device, characterized in that: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 14 when executed by the processor.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images