Document response method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a document response method and device, electronic equipment and a nonvolatile storage medium. The method comprises the steps that cue words input by a user are obtained, and the cue words at least comprise user questions to be answered and document files corresponding to the user questions; determining a first matching score of the user question and each paragraph in the document file by using a first mode; determining a second matching score of the user question and each paragraph in the document file by using a second mode; determining a third matching score of the user question and each paragraph in the document file by using a third mode; and determining a target answer corresponding to the user question according to the first matching score, the second matching score and the third matching score of each paragraph. According to the document response method and device, the technical problem that the document response accuracy is low due to the fact that a document response method in the related technology is limited in understanding ability for the document and cannot accurately recognize key information in the document is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly, to a document response method, apparatus, electronic device, and non-volatile storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the continuous innovation in the field of natural language processing, the document response systems in related technologies mainly rely on rule-based algorithms to parse document content and generate responses. Although this method has a certain efficiency in processing structured data and simple documents, its accuracy and intelligence are significantly insufficient when facing unstructured, long-length, or highly specialized documents. In recent years, intelligent document response systems integrating Retrieval Augmented Generation (RAG) technology have emerged. The RAG technology can generate more accurate and natural answers based on understanding the document content by combining document retrieval and the generation ability of language models, significantly improving the user experience and system performance. However, the response effect is often still not satisfactory. The current RAG-based document response systems still face a series of challenges in practical applications. First, the document processing and paragraph segmentation technologies are not yet mature, resulting in the difficulty of accurately capturing the semantic structure of document content. Especially when processing long documents, multilingual documents, or documents containing a large number of professional terms, the segmentation methods often cannot effectively retain the complete information and context association of paragraphs, with limited understanding ability of the documents, affecting the accuracy of subsequent retrieval and response. Second, the cooperation between the retrieval component and the generation model in the RAG technology is insufficient, and the calculation method for the relevance between the retrieved paragraphs and the user's question is relatively single, resulting in the possibility that the retrieved paragraphs may not match the deep semantic requirements of the question, unable to accurately identify the key information in the document, lacking flexibility and scalability in processing user questions, and being difficult to meet the diverse needs of users in different scenarios.

[0003] No effective solution has been proposed for the above problems. Summary of the Invention

[0004] Embodiments of this application provide a document response method, apparatus, electronic device, and non-volatile storage medium to at least solve the technical problem that the document response method in related technologies has limited understanding ability of documents, cannot accurately identify the key information in the documents, and results in low accuracy of document response.

[0005] According to one aspect of the embodiments of the present application, a document answering method is provided, including: obtaining a prompt word input by a user, where the prompt word at least includes a user question to be answered and a document file corresponding to the user question; using a first method to determine a first matching score between the user question and each paragraph in the document file, where the first method includes: calculating the first matching score based on keyword matching; using a second method to determine a second matching score between the user question and each paragraph in the document file, where the second method includes: calculating the second matching score based on question-answer pair matching; using a third method to determine a third matching score between the user question and each paragraph in the document file, where the third method includes: calculating the third matching score based on expanding the user question; determining a target answer corresponding to the user question according to the first matching score, the second matching score, and the third matching score of each paragraph.

[0006] In some embodiments of the present application, calculating the first matching score based on keyword matching includes: extracting a first preset number of question keywords from the user question to obtain a question keyword set; for each paragraph in the document file, extracting a second preset number of paragraph keywords to obtain a paragraph keyword set for each paragraph; determining the first matching score of each paragraph based on the question keyword set and the paragraph keyword set of each paragraph.

[0007] In some embodiments of the present application, determining the first matching score of each paragraph based on the question keywords and the paragraph keywords of each paragraph includes: determining the intersection of the question keyword set and the paragraph keyword set of each paragraph as the matching set of each paragraph; determining the total number of keywords in the matching set as the number of matching keywords, and determining the keyword matching score through the number of matching keywords, the total number of question keywords in the question keyword set, and the total number of paragraph keywords in the paragraph keyword set; determining the total number of matching characters corresponding to all keywords in the matching set, where repeatedly occurring characters are only recorded once in the total number of matching characters; determining the character matching score through the total number of matching characters, the total number of first characters, and the total number of second characters, where the total number of first characters is the total number of characters corresponding to all question keywords in the question keyword set, and the total number of second characters is the total number of characters corresponding to all paragraph keywords in the paragraph keyword set of each paragraph, and repeatedly occurring characters are only recorded once in the total number of first characters and the total number of second characters; determining the first matching score through the keyword matching score and the character matching score of each paragraph.

[0008] In some embodiments of the present application, calculating the second matching score based on the Q&A pair matching method includes: extracting the abstract information of the document file; determining the target audience of the document file based on the abstract information; determining a preset number of Q&A pairs corresponding to each paragraph based on the abstract information and the target audience, where one Q&A pair corresponds to a reference question and a reference answer; clustering the preset number of Q&A pairs of each paragraph to obtain a target Q&A pair set, and storing the target Q&A pair set corresponding to each paragraph in a vector library; calculating the similarity score between the user question and the reference questions in the target Q&A pair set corresponding to each paragraph in the vector library, and determining the maximum similarity score of each paragraph as the second matching score of each paragraph.

[0009] In some embodiments of the present application, determining a preset number of Q&A pairs corresponding to each paragraph based on the abstract information and the target audience includes: identifying entities, actions of the entities, and relationships between the entities in the abstract information, and determining the topics and context covered by each paragraph based on the entities, actions of the entities, and relationships between the entities; determining an initial Q&A pair list corresponding to the topics and context covered by each paragraph based on the attributes of the target audience, where the attributes at least include professional background, interest preference, and common context; performing semantic expansion on the initial Q&A pair list corresponding to each paragraph to obtain a preset number of Q&A pairs corresponding to each paragraph.

[0010] In some embodiments of the present application, calculating the third matching score based on the method of expanding the user question includes: extracting the paragraph abstract information of each paragraph in the document file; expanding the user question to obtain a third preset number of similar questions, and determining the user question and the similar questions as a question set; calculating the semantic similarity between the questions in the question set and the paragraph abstract information of each paragraph, and determining the maximum semantic similarity as the third matching score of each paragraph.

[0011] In some embodiments of the present application, expanding the user question to obtain a third preset number of similar questions and determining the user question and the similar questions as a question set includes: performing semantic parsing on the user question to extract the core intention; determining a third preset number of similar questions based on the core intention and question keywords, where the similar questions have the same core intention as the user question, and the similar questions cover different expressions and different inquiry angles of the user question; performing grammar verification on the similar questions, and forming a question set with the similar questions after grammar verification and the user question.

[0012] According to another aspect of the embodiments of the present application, a document answering device is further provided, including: an acquisition module, configured to acquire a prompt word input by a user, where the prompt word at least includes a user question to be answered and a document file corresponding to the user question; a first determination module, configured to determine a first matching score between the user question and each paragraph in the document file by using a first method, where the first method includes: calculating the first matching score based on keyword matching; a second determination module, configured to determine a second matching score between the user question and each paragraph in the document file by using a second method, where the second method includes: calculating the second matching score based on Q&A pair matching; a third determination module, configured to determine a third matching score between the user question and each paragraph in the document file by using a third method, where the third method includes: calculating the third matching score based on expanding the user question; a fourth determination module, configured to determine a target answer corresponding to the user question according to the first matching score, the second matching score, and the third matching score of each paragraph.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is further provided. A program is stored in the non-volatile storage medium, and when the program runs, it controls the device where the non-volatile storage medium is located to execute the document answering method of any one of the above.

[0014] According to another aspect of the embodiments of the present application, an electronic device is further provided, including: a memory and a processor, where the processor is configured to run a program stored in the memory, and when the program runs, it executes the document answering method of any one of the above.

[0015] According to another aspect of the embodiments of the present application, a computer program product is further provided, including computer instructions, and when the computer instructions are executed by a processor, the document answering method of any one of the above is implemented.

[0016] In the embodiments of the present application, a prompting word input by a user is obtained, where the prompting word at least includes a user question to be answered and a document file corresponding to the user question; a first matching score between the user question and each paragraph in the document file is determined by a first method, where the first method includes: calculating the first matching score based on keyword matching; a second matching score between the user question and each paragraph in the document file is determined by a second method, where the second method includes: calculating the second matching score based on question-and-answer pair matching; a third matching score between the user question and each paragraph in the document file is determined by a third method, where the third method includes: calculating the third matching score based on expanding the user question; a method for determining a target answer corresponding to the user question according to the first matching score, the second matching score, and the third matching score of each paragraph, calculates the matching scores of the document file corresponding to the user question through three different methods, and finally determines the target answer corresponding to the final user question based on the three matching scores, achieving the purpose of deeply understanding the document file and the user question, accurately identifying key information in the document file, thereby improving the accuracy of question answering, and further solving the technical problem that the document answering method in the related art has limited understanding ability of the document, cannot accurately identify key information in the document, and results in low accuracy of document answering. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a document answering method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of a document answering method according to an embodiment of the present application;

[0020] Figure 3 is a flowchart of determining a first matching score between a user question and each paragraph in a document file by a first method according to an embodiment of the present application;

[0021] Figure 4 is a flowchart of determining a second matching score between a user question and each paragraph in a document file by a second method according to an embodiment of the present application;

[0022] Figure 5 is a flowchart of determining a third matching score between a user question and each paragraph in a document file by a third method according to an embodiment of the present application;

[0023] Figure 6 It is a schematic structural diagram of a document response device provided according to an embodiment of the present application. Detailed implementation manners

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0025] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, complies with the relevant laws, regulations, and standards of the relevant regions, takes necessary confidentiality measures, does not violate public order and good customs, and provides corresponding operation entrances for users to choose to authorize or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0026] It should be noted that the terms "first", "second", etc. in the specification, claims, and the above-mentioned drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0027] To better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0028] Retrieval Augmented Generation (abbreviated as RAG): It is a natural language processing method that combines Information Retrieval (abbreviated as IR) and Neural Language Generation (abbreviated as NLG) technologies, mainly used to generate high-quality answers based on documents or a large amount of text data. The design of RAG aims to solve the knowledge limitation problems that may be encountered by generative AI relying solely on the knowledge base of pre-trained models during the answering process.

[0029] LLM (Large Language Model): It is an important technological breakthrough in the field of natural language processing (NLP). Its core feature is that the number of model parameters is extremely large, usually reaching billions or even trillions. Through pre-training on a vast amount of text data, models of this scale can learn extremely rich language patterns and knowledge, thus demonstrating performance beyond conventional models in various NLP tasks.

[0030] In related technologies, the document processing and paragraph segmentation technologies of document answering methods are not yet mature, resulting in the semantic structure of document content being difficult to accurately capture. Especially when dealing with long documents, multilingual documents, or documents containing a large number of professional terms, the segmentation methods often fail to effectively retain the complete information and context relevance of paragraphs, with limited understanding ability for documents, which affects the accuracy of subsequent retrieval and answering. Secondly, the synergy between the retrieval component and the generation model in the RAG technology is insufficient, and the calculation method for the relevance between the retrieved paragraphs and the user's question is relatively single, resulting in the retrieved paragraphs possibly not matching the deep semantic requirements of the question, being unable to accurately identify the key information in the document, lacking flexibility and scalability when dealing with user questions, and being difficult to meet the diverse needs of users in different scenarios. Therefore, there is a technical problem that the document answering methods in related technologies have limited understanding ability for documents, cannot accurately identify the key information in the document, and result in low accuracy of document answering. To solve this problem, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0031] According to the embodiments of the present application, an embodiment of a document answering method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0032] The method embodiments provided by the embodiments of the present application can be executed on a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal for implementing the document answering method is shown. As Figure 1As shown, the computer terminal 10 may include one or more processors 102 (shown as 102a, 102b, ……, 102n in the figure) (the processor 102 may include, but is not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown in, or have a different configuration from Figure 1 that shown.

[0033] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" herein. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10. As involved in the embodiments of the present application, the data processing circuit is a kind of processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the document response method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned document response method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10.

[0037] Under the above operating environment, an embodiment of a document response method is provided in an embodiment of the present application. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0038] As Figure 2 shown, it is a flowchart of a document response method provided according to an embodiment of the present application, including:

[0039] Step S202, obtain the prompt word input by the user, where the prompt word at least includes the user's question to be answered and the document file corresponding to the user's question.

[0040] In the technical solution provided in step S202, the large model of the document response system obtains the prompt word input by the user, and the prompt word at least includes the user's question to be answered and the document file corresponding to the user's question. For example, the following is a prompt word example: User question: "How can unemployed people withdraw provident funds?" The document file corresponding to the user's question is a PDF file about provident fund policies, or "provident fund policies" is entered as a keyword.

[0041] Step S204, use the first method to determine the first matching score between the user's question and each paragraph in the document file, where the first method includes: calculating the first matching score based on the keyword matching method.

[0042] In the technical solution provided in step S204, there are various implementation methods for calculating the first matching score based on keyword matching. For example: extract the first preset number of question keywords in the user's question to obtain a question keyword set; for each paragraph in the document file, extract the second preset number of paragraph keywords to obtain a paragraph keyword set for each paragraph; determine the first matching score for each paragraph based on the question keyword set and the paragraph keyword set for each paragraph. There are various implementation methods for determining the first matching score for each paragraph based on the question keywords and the paragraph keywords for each paragraph. For example: determine the intersection of the question keyword set and the paragraph keyword set for each paragraph as the matching set for each paragraph; determine the total number of keywords in the matching set as the number of matching keywords, and determine the keyword matching score through the number of matching keywords, the total number of question keywords in the question keyword set, and the total number of paragraph keywords in the paragraph keyword set; determine the total number of matching characters corresponding to all the keywords in the matching set, where duplicate characters are only recorded once in the total number of matching characters; determine the character matching score through the total number of matching characters, the total number of the first characters, and the total number of the second characters, where the total number of the first characters is the total number of characters corresponding to all the question keywords in the question keyword set, and the total number of the second characters is the total number of characters corresponding to all the paragraph keywords in the paragraph keyword set for each paragraph, and duplicate characters are only recorded once in the total number of the first characters and the total number of the second characters; determine the first matching score through the keyword matching score and the character matching score for each paragraph.

[0043] The following are specific embodiments:

[0044] The large model of the document response system receives a prompt word corresponding to the keyword matching method. For example, "Please summarize and extract 3 to 5 keywords that can express the core semantics of the document based on the content information I provide you", perform word segmentation on the user's question, remove stop words, and finally extract the first preset number (for example, 3 to 5) of question keywords in the user's question to obtain a question keyword set. For example, for the question "How can unemployed people withdraw provident funds?", finally extract three keywords, namely unemployed people, provident fund withdrawal, and provident fund, to form the question keyword set.

[0045] The user's question can be segmented and stop words removed in the following way: The question text corresponding to the user's question is segmented into a series of words according to semantic units, and stop words are removed. Then, based on the semantics and context of the question, a keyword extraction algorithm is used to find the most important and relevant words from the word list. Keyword extraction can be based on techniques such as term frequency, term frequency-inverse document frequency (TF-IDF), word embeddings weighted by term frequency-inverse document frequency, or deep learning-based methods can be used to score keywords. From the extracted keyword list, the first N keywords (N is a preset number, such as 3 to 5) are selected to form a question keyword set. The number of selections usually needs to be adjusted according to the complexity of the question and the application scenario to ensure that the set contains the words that best represent the core of the question. Similarly, for each paragraph in the document file, the second preset number (for example, 3 - 5) of paragraph keywords are extracted to obtain the paragraph keyword set for each paragraph. Then, the intersection between the question keyword set and each paragraph keyword set is calculated, that is, the keywords that appear in both the question keyword set and the paragraph keyword set are found to form the matching set for each paragraph. Calculate the number of matching keywords in the matching set. Determine the total number of matching keywords in the matching set as the number of matching keywords. Through the number of matching keywords, the total number of question keywords in the question keyword set, and the total number of paragraph keywords in the paragraph keyword set, determine the keyword matching score for each paragraph. For example, the keyword matching score is determined by the following formula:

[0046] score1 (i.e., the keyword matching score) = number of matching keywords / set(total number of question keywords + total number of paragraph keywords), where set refers to the sum of the total number of question keywords in the question keyword set and the total number of paragraph keywords in the paragraph keyword set, and duplicate elements are removed at the same time.

[0047] Then determine the total number of matching characters corresponding to all the keywords in the matching set. Among them, repeatedly appearing characters are only recorded once in the total number of matching characters to ensure that each character is only counted once. At the same time, calculate the total number of characters corresponding to all the question keywords in the question keyword set (the first total number of characters), and the total number of characters corresponding to all the paragraph keywords in the paragraph keyword set of each paragraph (the second total number of characters), and remove duplicate characters. Determine the character matching score for each paragraph through the total number of matching characters, the first total number of characters, and the second total number of characters. For example, it is determined by the following formula:

[0048] score2 (i.e., the character matching score) = the total number of matching characters / the number of characters in set (question keyword set + paragraph keyword set). The number of characters in set (question keyword set + paragraph keyword set) represents taking the union of the characters in the question keyword set + paragraph keyword set and removing duplicate characters, that is, the sum of the total number of the first characters and the total number of the second characters.

[0049] The first matching score for each paragraph is determined based on the character matching score, keyword matching score, and a preset weight for each paragraph. For example, through the following formula: score (the first matching score) = score1 * the first preset weight (e.g., 50%) + score2 * the second preset weight (e.g., 50%).

[0050] As Figure 3 shown, it is a flowchart for determining the first matching score between the user question and each paragraph in the document file according to the embodiment of the present application using the first method. First, the user question is tokenized and stop words are removed, then keyword 1 and keyword 2 are extracted (i.e., the first preset number of question keywords in the above-mentioned user question are extracted to obtain the question keyword set), and then for each paragraph in the document file (paragraph 1 to paragraph N, N is an integer greater than 1), keyword extraction is performed respectively (i.e., for each paragraph in the document file, the second preset number of paragraph keywords are extracted to obtain the paragraph keyword set for each paragraph), the co-occurrence frequency of keywords and keywords between the question and the paragraph is calculated (the co-occurrence frequency of keywords and keywords is the above-mentioned keyword matching score and character matching score), and finally the final score is generated (i.e., the first matching score is determined through the keyword matching score and character matching score of each paragraph).

[0051] Step S206, use the second method to determine the second matching score between the user question and each paragraph in the document file, where the second method includes: calculating the second matching score based on the method of question-answer pair matching.

[0052] In the technical solution provided in step S206, there are various implementation methods for calculating the second matching score based on the method of question-answer pair matching. For example: extracting the summary information of the document file; determining the target audience of the document file based on the summary information; determining a preset number of question-answer pairs corresponding to each paragraph based on the summary information and the target audience, where one question-answer pair corresponds to a reference question and a reference answer; clustering the preset number of question-answer pairs for each paragraph to obtain the target question-answer pair set, and storing the target question-answer pair set corresponding to each paragraph in the vector library; calculating the similarity score between the user question and the reference questions in the target question-answer pair set corresponding to each paragraph in the vector library, and determining the similarity score of each paragraph as the second matching score of each paragraph.

[0053] It should be noted that in the above steps, there are multiple ways to implement determining the preset number of Q&A pairs corresponding to each paragraph based on the abstract information and the target audience. For example: identifying entities, actions of entities, and relationships between entities in the abstract information, and determining the topics and context covered by each paragraph based on the entities, actions of entities, and relationships between entities; determining the initial Q&A pair list corresponding to the topics and context covered by each paragraph based on the attributes of the target audience, where the attributes at least include professional background, interest preferences, and common context; semantically expanding the initial Q&A pair list corresponding to each paragraph to obtain the preset number of Q&A pairs corresponding to each paragraph.

[0054] The following are specific embodiments: The large model (e.g., LLM) converts the document file (such as PDF, Word, etc.) into a plain text format. For non-text format documents, optical character recognition (OCR) technology can be used to scan and convert them into text. Remove irrelevant information in the text, such as headers, footers, annotations, picture descriptions, etc., and retain the main text content. Split the document text according to structures such as paragraphs and subsections for subsequent processing. Decompose the sentences in the text into words and label the part of speech of each word to provide a basis for subsequent semantic understanding. Identify the named entities in the text, such as person names, place names, organization names, etc., and these entities are often the information that needs to be highlighted in the abstract. Score each sentence according to the occurrence frequency, position information (such as title, beginning of the paragraph), syntactic structure, etc. of the paragraph keywords of each paragraph, and identify the sentences that contribute the most to the abstract. Select several sentences with the highest scores and splice them together in the original text order to form the abstract of the document. To ensure the coherence and integrity of the abstract, at the same time make necessary sentence adjustments to regenerate a general and coherent abstract text. The abstract does not have to be exactly the same as the sentences in the original text and is a condensed expression of the original meaning.

[0055] Identify entities (such as person names, organization names, place names, etc.) in the abstract information, the actions of the entities (the actions performed by the entities in the abstract information), and extract the relationships between these entities (such as belonging, influence, causality, etc.). And determine the topics and context covered by each paragraph based on the entities, the actions of the entities, and the relationships between the entities. The topic is the core theme of the paragraph, while the context includes the background information and detailed descriptions of the paragraph content. Audience population attribute analysis collects the attribute information of the audience population, including but not limited to professional background, interest preferences, common contexts, etc. This attribute information can be obtained through user registration information, historical interaction data, or questionnaires. According to the attributes of the audience population, generate an initial list of question-and-answer pairs related to the topic and context of each paragraph. For example, for an article about health insurance, for the "working class" population, possible initial question-and-answer pairs include: "What is the special significance of health insurance for the working class?", "How to choose a health insurance plan suitable for the working class?", etc. Finally, semantically expand the initial question-and-answer pairs to generate more relevant questions, ensuring that the questions cover multiple aspects of the paragraph. Obtain the preset number (for example, 10) of question-and-answer pairs corresponding to each paragraph. Then cluster all the questions for each paragraph, and the number of clusters is the total number of the preset number (for example, 10) of question-and-answer pairs / the total number of the audience population. Integrate through a large model (or called the integrated large model) to obtain the target question-and-answer pair set corresponding to each paragraph, and store the target question-and-answer pair set corresponding to each paragraph in the vector library.

[0056] Calculate the similarity score between the user's question and the reference questions in the target question-and-answer pair set corresponding to each paragraph in the vector library, and determine the maximum similarity score of each paragraph as the second matching score of each paragraph. For example, through the following formula:

[0057] Second matching score = max(cosin(bert_embedding(user question), bert_embedding(reference question))).

[0058] (cosin(bert_embedding(user question), bert_embedding(reference question)) represents calculating the similarity score between the user's question and the reference questions in the target question-and-answer pair set corresponding to each paragraph in the vector library. bert_embedding is a vector representation, and max(cosin(bert_embedding(user question), bert_embedding(reference question))) represents taking the similarity score of the reference question with the maximum similarity in each paragraph as the second matching score of this paragraph.

[0059] Such as Figure 4As shown in the figure, it is a flowchart of using a second method to determine the second matching score between a user's question and each paragraph in a document file according to an embodiment of the present application. Based on the LLM, the article abstract is extracted (i.e., the above-mentioned extraction of the abstract information of the document file), and the article population is extracted based on the LLM (i.e., the above-mentioned determination of the target population of the document file based on the abstract information). For each paragraph (paragraph 1 to paragraph N), question-answer pairs are extracted respectively based on the paragraph + role + abstract (i.e., the above-mentioned determination of a preset number of question-answer pairs corresponding to each paragraph based on the abstract information and the target population). Then, all the questions are clustered, and the large model is integrated to store the final question-answer pairs in the vector library (i.e., the above-mentioned clustering of the preset number of question-answer pairs for each paragraph to obtain a set of target question-answer pairs, and storing the set of target question-answer pairs corresponding to each paragraph in the vector library). Calculate the similarity score between the user's question and the reference questions in the set of target question-answer pairs corresponding to each paragraph in the vector library, and determine the maximum similarity score of each paragraph as the second matching score of each paragraph.

[0060] Step S208 is to determine the third matching score between the user's question and each paragraph in the document file using a third method according to an embodiment of the present application. Among them, the third method includes: calculating the third matching score based on the method of expanding the user's question.

[0061] In the technical solution provided in step S208, there are various implementation methods for calculating the third matching score based on the method of expanding the user's question. For example: extracting the paragraph abstract information of each paragraph in the document file; expanding the user's question to obtain a third preset number of similar questions, and determining the user's question and the similar questions as a set of questions; calculating the semantic similarity between the questions in the set of questions and the paragraph abstract information of each paragraph, and determining the maximum semantic similarity as the third matching score of each paragraph.

[0062] In the above steps, there are various implementation methods for expanding the user's question to obtain a third preset number of similar questions and determining the user's question and the similar questions as a set of questions. For example: performing semantic parsing on the user's question to extract the core intention; determining a third preset number of similar questions based on the core intention and question keywords, where the similar questions have the same core intention as the user's question, and the similar questions cover different expression forms and different inquiry angles of the user's question; performing grammar verification on the similar questions, and forming a set of questions with the user's question after grammar verification.

[0063] The following are specific implementation manners and embodiments:

[0064] Use NLP technology, such as the summary generation technology based on Bidirectional Encoder Representations from Transformers (BERT), to extract the summary of each paragraph in the document file, generate paragraph summary information, and use deep learning models (such as BERT, etc.) to semantically parse the user questions, identify and extract the core intent of the questions. Based on the core intent and keywords, use a pre-trained language model (such as BART) or a rule-based method to generate a third preset number (such as 5-10) of similar questions. These similar questions should maintain the same core intent as the original question, but cover different expressions and inquiry angles to improve the coverage and flexibility of the questions. Perform grammar check on the generated similar questions to ensure that each question is grammatically correct and fluent to avoid affecting the subsequent semantic similarity calculation due to grammatical errors. Combine the grammatically checked similar questions with the original user questions into a question set. Use the BERT model to convert the summary information of each question in the question set and each paragraph in the document into a vector representation. Calculate the semantic similarity between each question vector in the question set and each paragraph summary information vector. Summarize the semantic similarity scores of each question and the summary information of the same paragraph, and use the maximum value as the third matching score of the paragraph. The maximum semantic similarity is the maximum value among the semantic similarities. For example, the third matching score can be determined by the following formula:

[0065] The third matching score = max(cosin(bert_embedding(question set), bert_embedding(paragraph summary information))), cosin(bert_embedding(question set), bert_embedding(paragraph summary information)) means calculating the semantic similarity between each question vector in the question set and each paragraph summary information vector, and max(cosin(bert_embedding(question set), bert_embedding(paragraph summary information))) means determining the maximum semantic similarity (i.e. the maximum value of the semantic similarity) as the third matching score for each paragraph.

[0066] like Figure 5As shown in the figure, it is a flowchart of using the third method to determine the third matching score between the user's question and each paragraph in the document file. First, perform question expansion on the user's question using a large model (i.e., expand the user's question as described above to obtain a third preset number of similar questions). Then, for each paragraph (paragraph 1 to paragraph N), perform paragraph summarization using a large model (i.e., extract the paragraph summary information of each paragraph in the document file), calculate the vector distance between the expanded question and the paragraph summary (i.e., determine the user's question and the similar questions as a question set; calculate the semantic similarity between the questions in the question set and the paragraph summary information of each paragraph), and generate the final score (i.e., determine the maximum semantic similarity as the third matching score for each paragraph).

[0067] Step S210, determine the target answer corresponding to the user's question based on the first matching score, the second matching score, and the third matching score of each paragraph.

[0068] An embodiment of the present application also provides a document answering device, as Figure 6 shown, including:

[0069] An acquisition module 602, configured to acquire a prompt word input by a user, where the prompt word at least includes the user's question to be answered and the document file corresponding to the user's question.

[0070] A first determination module 604, configured to use the first method to determine the first matching score between the user's question and each paragraph in the document file, where the first method includes: calculating the first matching score based on a keyword matching method.

[0071] A second determination module 606, configured to use the second method to determine the second matching score between the user's question and each paragraph in the document file, where the second method includes: calculating the second matching score based on a question-and-answer pair matching method.

[0072] A third determination module 608, configured to use the third method to determine the third matching score between the user's question and each paragraph in the document file, where the third method includes: calculating the third matching score based on a method of expanding the user's question.

[0073] A fourth determination module 610, configured to determine the target answer corresponding to the user's question based on the first matching score, the second matching score, and the third matching score of each paragraph.

[0074] It should be noted that Figure 6 the shown document answering device is used to execute Figure 2 the shown document answering method. Therefore Figure 2 the relevant explanations in the document answering method also apply to this document answering device, and will not be elaborated here.

[0075] It should be noted that each module in the above document response device can be a program module (for example, a set of program instructions that implement a specific function), or a hardware module. For the latter, it can be presented in the following forms, but not limited to: the manifestation form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0076] The embodiment of the present application also provides a non-volatile storage medium. The non-volatile storage medium includes a stored program. When the program runs, it controls the device where the non-volatile storage medium is located to execute the above document response method. For example, obtain a prompt word input by a user, where the prompt word at least includes the user's question to be answered and the document file corresponding to the user's question; use a first method to determine the first matching score between the user's question and each paragraph in the document file, where the first method includes: calculating the first matching score based on a keyword matching method; use a second method to determine the second matching score between the user's question and each paragraph in the document file, where the second method includes: calculating the second matching score based on a question-and-answer pair matching method; use a third method to determine the third matching score between the user's question and each paragraph in the document file, where the third method includes: calculating the third matching score based on a method of expanding the user's question; determine the target answer corresponding to the user's question according to the first matching score, the second matching score, and the third matching score of each paragraph.

[0077] The embodiment of the present application also provides an electronic device. The electronic device includes a processor, and the processor is used to run a program. When the program runs, it executes the above document response method. For example, obtain a prompt word input by a user, where the prompt word at least includes the user's question to be answered and the document file corresponding to the user's question; use a first method to determine the first matching score between the user's question and each paragraph in the document file, where the first method includes: calculating the first matching score based on a keyword matching method; use a second method to determine the second matching score between the user's question and each paragraph in the document file, where the second method includes: calculating the second matching score based on a question-and-answer pair matching method; use a third method to determine the third matching score between the user's question and each paragraph in the document file, where the third method includes: calculating the third matching score based on a method of expanding the user's question; determine the target answer corresponding to the user's question according to the first matching score, the second matching score, and the third matching score of each paragraph.

[0078] According to another aspect of the embodiments of the present application, there is also provided a computer program product, including a computer program, which when executed by a processor implements the above-mentioned document answering method. For example, obtaining a prompt word input by a user, where the prompt word at least includes a user question to be answered and a document file corresponding to the user question; using a first method to determine a first matching score between the user question and each paragraph in the document file, where the first method includes: calculating the first matching score based on keyword matching; using a second method to determine a second matching score between the user question and each paragraph in the document file, where the second method includes: calculating the second matching score based on question-answer pair matching; using a third method to determine a third matching score between the user question and each paragraph in the document file, where the third method includes: calculating the third matching score based on the method of expanding the user question; determining a target answer corresponding to the user question according to the first matching score, the second matching score, and the third matching score of each paragraph.

[0079] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0080] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0081] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0083] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0084] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and modifications can still be made, and these improvements and modifications should also be regarded as the protection scope of this application.

Claims

1. A document response method, characterized in that, Including: Obtain a prompt word input by a user, where the prompt word at least includes a user question to be answered and a document file corresponding to the user question; Use a first method to determine a first matching score between the user question and each paragraph in the document file, where the first method includes: calculating the first matching score based on a keyword matching method; Use a second method to determine a second matching score between the user question and each paragraph in the document file, where the second method includes: calculating the second matching score based on a question-and-answer pair matching method; Use a third method to determine a third matching score between the user question and each paragraph in the document file, where the third method includes: calculating the third matching score based on a method of expanding the user question; Determine a target answer corresponding to the user question according to the first matching score, the second matching score, and the third matching score of each paragraph.

2. The method according to claim 1, wherein The calculating the first matching score based on the keyword matching method includes: Extract a first preset number of question keywords from the user question to obtain a question keyword set; For each paragraph in the document file, extract a second preset number of paragraph keywords to obtain a paragraph keyword set for each paragraph; Determine the first matching score of each paragraph based on the question keyword set and the paragraph keyword set of each paragraph.

3. The method according to claim 2, wherein The determining the first matching score of each paragraph based on the question keyword and the paragraph keyword of each paragraph includes: Determine the intersection of the question keyword set and the paragraph keyword set of each paragraph as the matching set of each paragraph; Determine the total number of keywords in the matching set as the number of matching keywords, and determine a keyword matching score through the number of matching keywords, the total number of question keywords in the question keyword set, and the total number of paragraph keywords in the paragraph keyword set; Determine the total number of matching characters corresponding to all keywords in the matching set, where repeated characters are only recorded once in the total number of matching characters; Determine a character matching score through the total number of matching characters, a first total number of characters, and a second total number of characters, where the first total number of characters is the total number of characters corresponding to all question keywords in the question keyword set, and the second total number of characters is the total number of characters corresponding to all paragraph keywords in the paragraph keyword set of each paragraph, where repeated characters are only recorded once in the first total number of characters and the second total number of characters; Determine the first matching score through the keyword matching score and the character matching score of each paragraph.

4. The method according to claim 1, wherein The calculating the second matching score based on the question-and-answer pair matching method includes: Extract the abstract information of the document file; Determine the target audience of the document file based on the abstract information; Determine a preset number of question-and-answer pairs corresponding to each paragraph based on the abstract information and the target audience, where one question-and-answer pair corresponds to a reference question and a reference answer; Cluster the preset number of Q&A pairs for each paragraph to obtain a target Q&A pair set, and store the target Q&A pair set corresponding to each paragraph in a vector library; Calculate the similarity score between the user question and the reference questions in the target Q&A pair set corresponding to each paragraph in the vector library, and determine the maximum similarity score of each paragraph as the second matching score of each paragraph.

5. The method according to claim 4, wherein The determining the preset number of Q&A pairs corresponding to each paragraph based on the summary information and the target audience includes: Identify the entities, actions of the entities, and relationships between the entities in the summary information, and determine the topics and context covered by each paragraph based on the entities, actions of the entities, and relationships between the entities; Based on the attributes of the target audience, determine an initial Q&A pair list corresponding to the topics and context covered by each paragraph, where the attributes at least include professional background, interest preference, and common context; Semantically expand the initial Q&A pair list corresponding to each paragraph to obtain the preset number of Q&A pairs corresponding to each paragraph.

6. The method according to claim 1, wherein The calculating the third matching score based on the way of expanding the user question includes: Extract the paragraph summary information of each paragraph in the document file; Expand the user question to obtain a third preset number of similar questions, and determine the user question and the similar questions as a question set; Calculate the semantic similarity between the questions in the question set and the paragraph summary information of each paragraph, and determine the maximum semantic similarity as the third matching score of each paragraph.

7. The method according to claim 6 or 2, characterized in that The expanding the user question to obtain a third preset number of similar questions and determining the user question and the similar questions as a question set includes: Semantically parse the user question to extract the core intention; Based on the core intention and the question keywords, determine the third preset number of similar questions, where the similar questions have the same core intention as the user question, and the similar questions cover different expression forms and different inquiry angles of the user question; Perform grammar verification on the similar questions, and form the question set with the similar questions after grammar verification and the user question.

8. A document response device, characterized in that, including: An acquisition module, configured to acquire a prompt word input by a user, where the prompt word at least includes a user question to be answered and a document file corresponding to the user question; A first determination module, configured to determine the first matching score between the user question and each paragraph in the document file by using a first method, where the first method includes: calculating the first matching score based on a keyword matching method; A second determination module, configured to determine the second matching score between the user question and each paragraph in the document file by using a second method, where the second method includes: calculating the second matching score based on a Q&A pair matching method; A third determination module, configured to determine a third matching score between the user question and each paragraph in the document file by using a third method, where the third method includes: calculating the third matching score based on a method of expanding the user question; A fourth determination module, configured to determine a target answer corresponding to the user question through the first matching score, the second matching score, and the third matching score of each paragraph.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, where when the program runs, it controls the device where the non-volatile storage medium is located to execute the document response method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Comprising: A memory and a processor, the processor is configured to run the program stored in the memory, where when the program runs, it executes the document response method according to any one of claims 1 to 7.

11. A computer program product, comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the document response method according to any one of claims 1 to 7 is implemented.

Images