Question and answer processing method and device based on document knowledge base and medium

Abstract

The invention relates to the technical field of electric digital data processing, in particular to a question and answer processing method and device based on a document knowledge base and a medium. The method comprises the steps of receiving a question input by a user; obtaining a plurality of slice texts obtained by slicing the document in the document knowledge base; obtaining the similarity between the semantic vector of each slice text in the plurality of slice texts and the semantic vector of the question input by the user; according to the similarity, selecting first M slice texts with the maximum similarity from the plurality of slice texts; inputting the first M slice texts with the maximum similarity and the question input by the user into a large language model; and generating a final answer of the question input by the user by the large language model according to the first M slice texts with the maximum similarity, and outputting the final answer. According to the invention, the accuracy of obtaining the required information from the document knowledge base can be improved.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a question-and-answer processing method, device, and medium based on a document knowledge base. Background Technology

[0002] In today's rapidly developing information technology landscape, various organizations have accumulated massive amounts of documents, forming vast document knowledge bases. How to efficiently and accurately extract the required information from these unstructured documents has become a significant technical challenge. Traditional document question-answering systems typically employ keyword-matching-based retrieval. This method returns relevant paragraphs by matching user questions with keywords in the document. However, this method heavily relies on literal matching and cannot understand semantics. For questions involving synonyms, polysemous words, or words expressed differently but with the same meaning, its retrieval performance is poor, resulting in low accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide a question-answering method based on a document knowledge base, so as to improve the accuracy of obtaining the required information from the document knowledge base.

[0004] According to a first aspect of the present invention, a question-answering processing method based on a document knowledge base is provided, comprising the following steps: S100, the issue of receiving user input.

[0005] S200: Obtain several sliced ​​texts obtained after slicing documents in the document knowledge base.

[0006] S300, obtain the similarity between the semantic vector of each of the several slice texts and the semantic vector of the question input by the user.

[0007] S400, based on the similarity, select the top M slice texts with the highest similarity from the plurality of slice texts; M is a preset selection number.

[0008] S500, the top M most similar text slices and the user-input question are input into the large language model.

[0009] S600, the large language model generates and outputs the final answer to the user's input question based on the top M similarity text slices.

[0010] According to a second aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the above-described question-and-answer processing method based on a document knowledge base.

[0011] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described question-answering method based on a document knowledge base.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects: This invention calculates the similarity between the semantic vectors of user questions and document slices, enabling a deep understanding of semantic relationships and accurately selecting the top M most relevant slices. Subsequently, a large language model generates answers based on these highly relevant source documents, effectively avoiding the illusion phenomenon and ensuring that the answer content is based on evidence, greatly improving the accuracy and credibility of the answer. Moreover, by providing concise, highly relevant top M slices as context, the noise information that the large language model needs to process is greatly reduced. This allows the large language model to focus more on key information, resulting in higher quality and more concise answers, while also reducing the model's computational overhead and processing time, thus achieving a dual optimization of efficiency and effectiveness.

[0013] Furthermore, this invention, through a recursive judgment method based on semantic center offset, can dynamically and intelligently merge semantically related text units (such as sentences and paragraphs) into the same segment. This effectively avoids the semantic fragmentation problem that may be caused by fixed-length segments, ensuring the coherence and integrity of the semantics within each segment. Since each segment is a self-contained semantic unit, its semantic vector can more accurately represent the core content of the segment. When performing similarity matching with user questions, it can more accurately retrieve truly relevant and complete segments, reducing the omission of important information (improving recall) and the mixing of irrelevant information (improving precision). This is beneficial to improving the accuracy of the documents finally input into the large language model, thereby improving the accuracy of the large language model's output answer, enabling users to accurately obtain the information they need. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a question-answering method based on a document knowledge base provided in Embodiment 1 of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1:

[0018] According to this embodiment, as Figure 1 As shown, a question-answering method based on a document knowledge base is provided, including the following steps: S100, the issue of receiving user input.

[0019] In this embodiment, the question input by the user refers to the text entered by the user in natural language form, which is obtained through the user interaction interface.

[0020] In this embodiment, the question input by the user is one that requires finding the answer from a document knowledge base. As a specific implementation, the user interface includes a second specific input channel. When the user inputs a question through this input channel, it is determined that the input question requires finding the answer from a document knowledge base.

[0021] S200: Obtain several sliced ​​texts obtained after slicing documents in the document knowledge base.

[0022] In this embodiment, the document knowledge base includes several documents, which are unstructured or semi-structured text data, and the documents are natural language text.

[0023] S300, obtain the similarity between the semantic vector of each of the several slice texts and the semantic vector of the question input by the user.

[0024] In this embodiment, the semantic vector of the user-input question and the semantic vector of any slice of text have the same dimension. Those skilled in the art will understand that the process of obtaining semantic vectors and the process of obtaining the similarity between two vectors are existing technologies and will not be described further here. As an optional specific implementation, the similarity is cosine similarity.

[0025] S400, based on the similarity, select the top M slice texts with the highest similarity from the plurality of slice texts; M is a preset selection number.

[0026] As a specific implementation, M is an empirical value, such as M being 3, 5, or 10, etc.

[0027] S500, the top M most similar text slices and the user-input question are input into the large language model.

[0028] In this embodiment, the large language model can be a large language model in the prior art, such as GPT-4.

[0029] In one specific implementation, the top M most similar text segments, along with the user-inputted question, are combined to form a carefully crafted prompt, which is then fed into the large language model. The prompt template might be similar to: Please answer the question based on the following information: Information: [The top M most similar text segments]. Question: [The user-inputted question].

[0030] S600, the large language model generates and outputs the final answer to the user's input question based on the top M similarity text slices.

[0031] In this embodiment, the large language model, based on its powerful language understanding and generation capabilities, reads and understands the provided document content, extracts, integrates, and summarizes information from it, and directly generates a natural, fluent, and accurate text answer, which is then returned to the user.

[0032] This embodiment calculates the similarity between the semantic vectors of the user's question and document slices, enabling a deep understanding of semantic relationships and accurately selecting the top M most relevant slices. Subsequently, the large language model generates answers based on these highly relevant source documents, effectively avoiding the illusion phenomenon and ensuring that the answer content is based on evidence, greatly improving the accuracy and credibility of the answer. Moreover, by providing concise, highly relevant top M slices as context, the noise information that the large language model needs to process is greatly reduced. This allows the large language model to focus more on key information, resulting in higher quality and more concise answers, while also reducing the model's computational overhead and processing time, thus achieving a dual optimization of efficiency and effectiveness.

[0033] As a preferred embodiment, the process of obtaining the plurality of sliced ​​texts includes: S201, set the first text unit of the specified document as the current segment text; the specified document is any document in the document knowledge base.

[0034] S202, for each subsequent text unit in the specified document, perform the following operations: a) Obtain the semantic center of the current text segment.

[0035] As a specific implementation, a) includes: inputting the entire text of the current segmented text into a trained language model, and determining the vector output by the trained language model as the semantic center of the current segmented text; the trained language model is used to obtain the semantic vector of the input text.

[0036] b) Simulate adding subsequent text units to the new semantic center after the current text segment.

[0037] In one specific implementation, the text unit is a sentence, a paragraph, or a text block containing a preset number of characters.

[0038] As a specific implementation, b) includes: merging the current text segment and subsequent text units to obtain the merged whole text, and inputting the merged whole text into a trained language model, and determining the vector output by the trained language model as the new semantic center; the trained language model is used to obtain the semantic vector of the input text.

[0039] c) Obtain the offset between the new semantic center and the semantic center of the current segmented text.

[0040] In one specific implementation, the offset between the new semantic center and the semantic center of the current text segment is the cosine distance between the new semantic center and the semantic center of the current text segment. The cosine distance is the difference between 1 and the cosine similarity, meaning the offset between the new semantic center and the semantic center of the current text segment is the difference between 1 and the cosine similarity between the two semantic centers.

[0041] It should be understood that the smaller the offset between the new semantic center and the semantic center of the current segmented text, the higher the semantic similarity between the subsequent text and the current segmented text.

[0042] d) If the offset is less than a preset offset threshold, then the subsequent text unit is merged into the current text segment.

[0043] In this embodiment, the preset offset threshold is a pre-set value, which can be an empirical value, for example, the preset offset threshold is 0.1 or 0.2.

[0044] e) If the offset is greater than or equal to a preset offset threshold, the current segment text is output as a complete segment text, and the subsequent text unit is set as the starting text unit of the new current segment text.

[0045] S203, repeat S202 until all text units in the specified document have been processed, and determine that the slicing of the specified document is complete.

[0046] In this embodiment, any document in the document knowledge base is sliced ​​according to the process of S201-S203, thereby completing the slicing process of the document knowledge base.

[0047] Based on this preferred implementation, this embodiment uses a recursive judgment method based on semantic center offset to dynamically and intelligently merge semantically related text units (such as sentences and paragraphs) into the same segment. This effectively avoids the semantic fragmentation problem that may be caused by fixed-length segments, ensuring the coherence and integrity of the semantics within each segment. Since each segment is a self-contained semantic unit, its semantic vector can more accurately represent the core content of the segment. When performing similarity matching with user questions, it can more accurately retrieve truly relevant and complete segments, reducing the omission of important information (improving recall) and the mixing of irrelevant information (improving precision). This is beneficial to improving the accuracy of the documents finally input to the large language model, thereby improving the accuracy of the large language model's output answer, enabling users to accurately obtain the information they need.

[0048] Example 2:

[0049] Compared with Embodiment 1, this embodiment further includes: between S100 and S200, a process of determining whether the user input question is a first type of question or a second type of question, wherein the first type of question refers to a question that requires online search and the second type of question refers to a question that requires search from a document knowledge base; if the user input question is determined to be a second type of question, then S200-S600 are executed; if the user input question is determined to be a first type of question, then F200-F600 are executed.

[0050] As a specific implementation, if the user inputs a question from a first specific input channel of the user interface, the user input question is determined to be a first type of question; if the user inputs a question from a second specific input channel of the user interface, the user input question is determined to be a second type of question.

[0051] The F200 performs an online search to retrieve several web pages and their corresponding summaries.

[0052] As a specific implementation method, the user-input question is submitted to a search engine (such as Google or Baidu). If the search engine can directly output a summary, the summary output by the search engine is determined as the summary of the corresponding webpage; if the search engine cannot output a summary, a lightweight text summarization model (such as TextRank) is called to crawl the webpage title and body text to generate a summary.

[0053] F300, obtain the similarity between the semantic vector of each webpage summary in the plurality of webpage summaries and the semantic vector of the question input by the user.

[0054] In this embodiment, the semantic vector of the user-input question and the semantic vector of any webpage summary have the same dimension. Those skilled in the art will understand that the process of obtaining semantic vectors and the process of obtaining the similarity between two vectors are existing technologies and will not be described further here. As an optional specific implementation, the similarity is cosine similarity.

[0055] F400, based on the similarity, select the top N web pages with the highest similarity from the plurality of web pages; N is a preset number of pages to be selected.

[0056] In one specific implementation, N is an empirical value, such as N being 5 or 10, etc.

[0057] F500 inputs the content of the top N most similar web pages and the user-input question into the large language model.

[0058] In this embodiment, the large language model can be a large language model in the prior art, such as GPT-4.

[0059] As a specific implementation, the complete text (not just the summary) of the top N most similar web pages, along with the user-inputted question, is constructed into a carefully crafted prompt and input into the large language model. The prompt template might be similar to: Please answer the question based on the following information: Information: [The complete text of the top N most similar web pages]. Question: [The user-inputted question].

[0060] F600 generates and outputs the final answer to the user's input question based on the content of the top N web pages with the highest similarity, using the large language model.

[0061] In this embodiment, the large language model, based on its powerful language understanding and generation capabilities, reads and understands the provided web page content, extracts, integrates, and summarizes information from it, and directly generates a natural, fluent, and accurate text answer, which is then returned to the user.

[0062] This embodiment performs an online search for the user-input question, overcoming the limitations of static training data in large language models. It can obtain the latest real-time information and generate answers, effectively overcoming the illusions or incorrect answers caused by outdated knowledge in large language models, ensuring the timeliness and authenticity of the final answer. Moreover, based on semantic similarity, this embodiment can accurately and automatically filter the content of the top N web pages most relevant to the user-input question from massive search results, filtering out irrelevant and low-relevance noise information. This provides high-quality and high-purity information input for the large model to answer the user-input question, ensuring the accuracy and credibility of the final answer generated by the large model from the source. It also reduces the amount of content that the large model needs to process (compared to inputting all search results into the large model), accelerating the reasoning speed and the speed of generating the final answer, and improving the efficiency of the large model in generating the final answer.

[0063] In a preferred embodiment, prior to F200, the method further includes the following steps: F110 analyzes the user-input question and determines whether the user-input question meets the first preset condition and the second preset condition; the first preset condition is used to determine whether the user-input question needs to be rewritten based on context information; the second preset condition is the same as determining whether the user-input question needs to be rewritten based on related words.

[0064] In this embodiment, the first preset condition is the condition for initiating context completion, and the second preset condition is the condition for initiating semantic expansion. As a specific implementation, the first preset condition includes the presence of pronouns in the question, and the second preset condition includes the presence of keywords of a preset type in the question. For example, the preset type of keywords includes technical terms.

[0065] As a specific implementation, the named entity recognition module in natural language processing analyzes the sentence structure of the user's input question to detect the presence of pronouns (such as he, it, this, there, etc.). If pronouns are found, the subsequent context completion step is triggered. The named entity recognition module is used to identify pronouns in the sentence.

[0066] As a specific implementation method, keywords are extracted from the user's question, and the extracted keywords are matched with a predefined list of keywords of preset types (such as a professional terminology dictionary) or a model specifically trained on the domain corpus is used to determine whether the user's input question contains professional terms. If the match is successful, the subsequent semantic expansion steps are triggered.

[0067] F120. According to the judgment result, execute the corresponding rewriting path: If the question input by the user meets the first preset condition, access the memory bank, retrieve the context information related to the question input by the user, and generate a query question with context completion; if the question input by the user meets the second preset condition, access the knowledge base, obtain the related words of the keywords of the preset type in the question input by the user, and generate a query question with semantic expansion; if both are met, access the memory bank and the knowledge base, and generate a query question with context completion and semantic expansion; if neither is met, directly use the original question as the query question; the memory bank is constructed based on historical conversations.

[0068] F130: Use the query question to replace the question input by the user for subsequent steps.

[0069] As a specific implementation manner, the accessing the memory bank, retrieving the context information related to the question input by the user, and generating a query question with context completion includes: F121. Obtain the semantic vector of the question input by the user.

[0070] F122. Calculate the similarity between the semantic vector of the question input by the user and the semantic vectors of each memory unit in the memory bank; the memory bank is constructed by converting the key entities in historical conversations into semantic vectors.

[0071] F123. Based on the similarity ranking, retrieve the most relevant memory unit.

[0072] As a preferred specific implementation manner, determine the priority value of each memory unit according to the similarity and the conversation turn. Among them, the conversation turn reflects the timeliness, and the memory unit corresponding to the later conversation turn has better timeliness (that is, the memory unit corresponding to the latest conversation turn has better timeliness and higher timeliness score). Thus, the conversation turn is positively correlated with the priority value. Optionally, use the weighted summation formula to obtain the priority value of any memory unit. For example, the priority value of any memory unit is: w×a+(1 - w)×b, where w is the preset weight of the similarity (which can be an empirical value), a is the similarity corresponding to this memory unit, b is the timeliness score corresponding to this memory unit, 0 < w < 1, 0 < b ≤ 1, and b is obtained by normalizing the conversation turn corresponding to this memory unit. It should be understood that the higher the priority value, the more relevant the corresponding memory unit. Those skilled in the art know that the normalization process is a prior art and will not be elaborated here.

[0073] In this embodiment, the memory unit with the highest priority value is determined as the most relevant memory unit retrieved.

[0074] F124. Complete the pronouns in the question input by the user according to the content in the most relevant memory unit, and obtain a query question with context completion.

[0075] As a specific implementation, replacing the pronouns in the user-input question with the entity names stored in the most relevant memory unit will yield a context-completed query question.

[0076] As a specific implementation method, the process of constructing the memory includes: F1201: For any round of historical dialogue, extract the core entities from the questions and answers of that round of historical dialogue.

[0077] As a specific implementation method, for each round of historical dialogue, information extraction technology is used to extract core entities (such as person names, organization names, and product names).

[0078] F1202 uses a semantic vector model to convert the extracted content into corresponding semantic vectors, and encapsulates the converted semantic vectors and related information into structured memory units; the related information includes the content before conversion and the dialogue rounds.

[0079] As a specific implementation, a semantic vector model (such as Sentence-BERT) is used to convert the extracted text content into a fixed-dimensional semantic vector. This vector mathematically represents the deep semantics of the text. Subsequently, the semantic vector, the original text content (i.e., the entity names before conversion), and information such as the dialogue turn are packaged into a structured memory unit.

[0080] F1203 stores the packaged memory cell into the memory bank.

[0081] Based on F1201-F1203, fragmented historical dialogues can be constructed into a structured database (i.e., a memory bank) that can be semantically retrieved.

[0082] As a specific implementation, the step of accessing the knowledge base, obtaining related words of preset types of keywords in the user-input question, and generating a semantically expanded query question includes: F11 identifies and extracts keywords of a preset type from the user's input question; the preset type of keywords includes technical terms.

[0083] In this embodiment, technical terms refer to words that have a clear meaning within a specific field and may have multiple expressions. As a specific implementation, a terminology recognition tool or a predefined domain dictionary is used to scan the user's question and identify words belonging to technical terms. For example, the user's input question might be: "What is the attention mechanism of Transformer?" where "Transformer" and "attention mechanism" are technical terms.

[0084] Press F12 to query the knowledge base and retrieve related words of the extracted keywords of the preset type; the related words include: synonyms or words with a relevance greater than or equal to the preset relevance threshold.

[0085] As a specific implementation method, the extracted keywords are submitted to a knowledge base (such as WordNet, a domain-specific thesaurus, or a relational graph constructed by experts) for querying. The obtained related words include synonyms or words with a relevance greater than or equal to a preset relevance threshold. Synonyms refer to words with the same meaning, and words with a relevance greater than or equal to the preset relevance threshold refer to words that are semantically closely related (such as Apple iPhone and Steve Jobs, machine learning and deep learning). Their relevance can be measured by predefined weights or co-occurrence frequencies in the knowledge base.

[0086] F13 combines the user-input question with the associated words to construct a semantically expanded query question.

[0087] As a specific implementation method, the user-input question and the obtained related words are combined using syntax supported by the search engine (such as OR, parentheses, etc.). For example, if a technical term (denoted as D) appears in the user-input question, and its corresponding related words include D1 and D2, then the semantically expanded query question obtained after combination is to replace D in the user-input question with D (or D1 or D2).

[0088] Based on F11-F13, the search scope can be expanded, avoiding missed detections caused by inconsistencies between user-defined terms and webpage terms, and improving information recall.

[0089] This embodiment also introduces a multi-path rewriting mechanism. For example, when faced with context-related questions involving pronoun references, this embodiment can automatically complete the context through a knowledge base, transforming the user's input question into a clear query question. This effectively solves the reference problem in multi-turn dialogues, ensuring that the output of the large model corresponds to the user's question. Furthermore, when the user's input question includes technical terms, this embodiment can perform semantic expansion through a knowledge base, generating richer query questions based on words with similar meanings and high relevance to the technical terms. This improves the search coverage and recall, resulting in a more comprehensive answer output by the large model.

[0090] Example 3:

[0091] Compared with Embodiment 1, this embodiment further includes: after receiving the user's input question, it also includes a process of obtaining the target question based on the user's input question; and replacing the user's input question with the target question to perform subsequent steps S200-S600.

[0092] In this embodiment, the process of obtaining the target question based on the user's input includes: H110 determines whether the user's input question contains words representing time. If it does, the user's input question is identified as the target question; otherwise, proceed to H120.

[0093] As a specific implementation, keyword matching or named entity recognition (NER) technology is used to identify whether the user's input question contains words representing time. When using keyword matching, a dictionary containing numerous time expressions can be pre-built. The user's input question is matched against entries in the dictionary. If the user's input question contains any entry from the dictionary, it is determined that the user's input question contains words representing time; otherwise, it is determined that the user's input question does not contain words representing time. When using named entity recognition technology to identify the user's input question, a named entity recognition model capable of recognizing words representing time needs to be pre-trained. Those skilled in the art will understand that the training process of a named entity recognition model is existing technology and will not be elaborated upon here.

[0094] For example, the user inputs the question: "What's the weather like next Monday?" Since the question includes a word representing time (i.e., next Monday), this embodiment directly identifies the user's input as the target question.

[0095] For example, if the user enters the question: "How's the weather?", and since the question does not contain any words indicating time, then option H120 is executed.

[0096] H120: Perform intent analysis on the user-inputted question to determine whether the user-inputted question needs to consider the time factor. If not, then identify the user-inputted question as the target question; otherwise, proceed to H130.

[0097] As a specific implementation, the intent analysis of the user-inputted question to determine whether the time factor needs to be considered includes: H121 utilizes a pre-trained text classification model to perform intent recognition on user-input questions, categorizing the user-input questions into specific question types in a preset mapping knowledge base; the preset mapping knowledge base includes several specific question types and corresponding time-sensitive labels; wherein, when the time-sensitive label is a first preset value, the corresponding specific question type is a question type that does not need to consider time factors; when the time-sensitive label is a second preset value, the corresponding specific question type is a question type that needs to consider time factors.

[0098] In this embodiment, the pre-trained text classification model refers to a machine learning model that has been trained on a large text dataset and fine-tuned for the question type classification task, such as BERT or TextCNN. The pre-defined mapping knowledge base refers to a structured database or configuration file that stores the correspondence between question types and time-sensitive labels.

[0099] As a specific implementation method, the user input question is fed into a pre-trained text classification model. The model outputs multiple question type labels and their confidence scores. The question type label with the highest confidence score is determined as the type label of the user input question (i.e., the specific question type to which the user input question is classified).

[0100] H122 determines whether the user-input question needs to consider the time factor based on the preset mapping knowledge base and the specific question type into which the user-input question is categorized.

[0101] In this embodiment, if the time sensitivity label corresponding to the specific question type categorized by the user input question in the preset mapping knowledge base is a first preset value (e.g., 0), then the time factor does not need to be considered when determining the user input question; if the time sensitivity label corresponding to the specific question type categorized by the user input question in the preset mapping knowledge base is a second preset value (e.g., 1), then the time factor needs to be considered when determining the user input question.

[0102] For example, if a user inputs the question "What is the speed of light?", the pre-trained text classification model outputs the question type label with the highest confidence as "scientific principle". The time sensitivity label corresponding to scientific principle in the preset mapping knowledge base is 0, indicating that the user's input question does not need to consider the time factor and can be searched directly.

[0103] For example, if the user inputs a question about the weather in Beijing, and the pre-trained text classification model outputs a question type label with the highest confidence as "weather," and the time sensitivity label corresponding to the scientific principle in the preset mapping knowledge base is 1, indicating that the user input question needs to consider the time factor, then the question will proceed to H130.

[0104] H130, obtain the target time range that matches the question input by the user, and construct the target question based on the target time range and the question input by the user.

[0105] As one specific implementation, obtaining the target time range that matches the user-input question includes: H131, obtain the preset time mapping table; the preset time mapping table includes the correspondence between different specific problem types and default time ranges.

[0106] In this embodiment, the preset time mapping table is a configuration file that stores the mapping relationship between problem types and default time ranges.

[0107] H132, based on the specific question type categorized by the user-input question, queries a preset time mapping table to obtain the default time range corresponding to the specific question type categorized by the user-input question.

[0108] In this embodiment, a preset time mapping table is queried based on the identified problem type to obtain a preset, reasonable default time range. For example, when the problem type is weather, the corresponding default time range is the next 24 hours; when the problem type is a movie release, the corresponding default time range is the next 7 days.

[0109] H133 determines the target time range as the default time range corresponding to the specific question type into which the user-input question is categorized.

[0110] Based on H131-H133, meaningful answers can be returned for time-sensitive questions.

[0111] As a specific implementation, prior to H131, obtaining the target time range matching the user-input question further includes: H1311, obtain the context information of the dialogue in which the user inputs the question; the context information includes historical question and answer records in the same dialogue.

[0112] H1312: Determine whether a target time range matching the user-input question can be obtained based on the context information. If not, execute H131; if yes, obtain the target time range matching the user-input question based on the context information.

[0113] As a specific implementation, the determination of whether a target time range matching the user-input question can be obtained based on the context information includes: H1301, determine whether time information exists in the context information. If not, determine that the target time range matching the user's input question cannot be obtained based on the context information; otherwise, proceed to H1302.

[0114] H1302, determine the semantic similarity between the context information and the user-input question. If the semantic similarity is greater than a preset similarity threshold, determine that a target time range matching the user-input question can be obtained based on the context information; otherwise, determine that a target time range matching the user-input question cannot be obtained based on the context information.

[0115] In this embodiment, if time information exists in the context information, and the semantic similarity between the context information and the user-input question is greater than a preset similarity threshold, then it is determined that a target time range matching the user-input question can be obtained based on the context information, and the time information existing in the context information is identified as the target time range matching the user-input question. Optionally, the preset similarity threshold is an empirical value, such as 0.7 or 0.8. As a specific implementation, if multiple time information exists in the context information, then the semantic similarity between the question-and-answer record corresponding to each time information and the user-input question is obtained, and the time information corresponding to the highest similarity is identified as the target time range matching the user-input question.

[0116] As a specific implementation, constructing the target question based on the target time range and the user-input question includes: appending the target time range to the user-input question text, and using the new question obtained after appending as the target question.

[0117] In addition to the advantages of Embodiment 1, this embodiment first determines whether the user-inputted question contains words representing time. Even if it does not contain such words, it still identifies unspoken but real time needs through intent analysis of the user's input question. This prevents the question-answering system from ignoring the time dimension when replying, avoiding the return of outdated or irrelevant information, thus improving the accuracy of the answer and enhancing user satisfaction. Furthermore, this embodiment considers the different time sensitivities of different questions and determines corresponding target time ranges for the user-inputted questions, enabling the question-answering system to search and query information more accurately and generate more precise answers.

[0118] Example 4:

[0119] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: S100, the issue of receiving user input.

[0120] S200: Obtain several sliced ​​texts obtained after slicing documents in the document knowledge base.

[0121] S300, obtain the similarity between the semantic vector of each of the several slice texts and the semantic vector of the question input by the user.

[0122] S400, based on the similarity, select the top M slice texts with the highest similarity from the plurality of slice texts; M is a preset selection number.

[0123] S500, the top M most similar text slices and the user-input question are input into the large language model.

[0124] S600, the large language model generates and outputs the final answer to the user's input question based on the top M similarity text slices.

[0125] Example 5:

[0126] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: S100, the issue of receiving user input.

[0127] S200: Obtain several sliced ​​texts obtained after slicing documents in the document knowledge base.

[0128] S300, obtain the similarity between the semantic vector of each of the several slice texts and the semantic vector of the question input by the user.

[0129] S400, based on the similarity, select the top M slice texts with the highest similarity from the plurality of slice texts; M is a preset selection number.

[0130] S500, the top M most similar text slices and the user-input question are input into the large language model.

[0131] S600, the large language model generates and outputs the final answer to the user's input question based on the top M similarity text slices.

[0132] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A question-answering processing method based on a document knowledge base, characterized in that, Includes the following steps: S100, the issue of receiving user input; S200, Obtain several sliced ​​texts obtained after slicing documents in the document knowledge base; S300, obtain the similarity between the semantic vector of each of the plurality of sliced ​​texts and the semantic vector of the question input by the user; S400, based on the similarity, select the top M most similar slice texts from the plurality of slice texts; M is a preset selection number; S500, input the top M most similar text slices and the user-inputted question into the large language model; S600, the large language model generates and outputs the final answer to the user's input question based on the top M similarity text slices.

2. The question-answering method based on a document knowledge base according to claim 1, characterized in that, The process of obtaining the aforementioned text slices includes: S201, set the first text unit of the specified document as the current segment text; the specified document is any document in the document knowledge base; S202, for each subsequent text unit in the specified document, perform the following operations: a) Obtain the semantic center of the current text segment; b) Simulate adding subsequent text units to the new semantic center after the current text segment; c) Obtain the offset between the new semantic center and the semantic center of the current text segment; d) If the offset is less than a preset offset threshold, then the subsequent text unit is merged into the current text segment; e) If the offset is greater than or equal to a preset offset threshold, the current segment text is output as a complete segment text, and the subsequent text unit is set as the starting text unit of the new current segment text; S203, repeat S202 until all text units in the specified document have been processed, and determine that the slicing of the specified document is complete.

3. The question-answering method based on a document knowledge base according to claim 2, characterized in that, The text unit can be a sentence, a paragraph, or a text block containing a preset number of characters.

4. The question-answering method based on a document knowledge base according to claim 2, characterized in that, a) includes: inputting the entire text of the current segmented text into a trained language model, and determining the vector output by the trained language model as the semantic center of the current segmented text; the trained language model is used to obtain the semantic vector of the input text.

5. The question-answering method based on a document knowledge base according to claim 2, characterized in that, b) includes: merging the current text segment and subsequent text units to obtain the merged whole text, and inputting the merged whole text into a trained language model, and determining the vector output by the trained language model as the new semantic center; the trained language model is used to obtain the semantic vector of the input text.

6. The question-answering method based on a document knowledge base according to claim 2, characterized in that, The offset between the new semantic center and the semantic center of the current segmented text is the cosine distance between the new semantic center and the semantic center of the current segmented text.

7. The question-answering method based on a document knowledge base according to claim 1, characterized in that, The similarity is cosine similarity.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the question-answering method based on a document knowledge base as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the question-answering method based on a document knowledge base as described in any one of claims 1 to 7.

Images