Question answering method, device, equipment, system and program product

Abstract

The embodiment of the invention provides a question answering method. The method comprises the steps of obtaining a to-be-processed question initiated by a target object; then, a problem vector corresponding to the problem to be processed is determined, similar knowledge point vectors are retrieved in a vector database according to the problem vector, and the vector database is obtained based on document knowledge point configuration obtained through proposition clustering; and obtaining prompt information in combination with the problem vector and the knowledge point vector. And inputting the prompt information into the large language model to obtain answer information corresponding to the to-be-processed question. Therefore, a targeted knowledge expansion process is realized, the integrity of the expanded knowledge points is ensured through a proposition extraction and clustering mode, noise is reduced, the problem of knowledge point breakage caused by text block splitting is avoided, the retrieval hit rate is increased, and the accuracy of large language model reply is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to question answering methods, devices, equipment, systems and program products. Background Art

[0002] In recent years, the wave of Large Language Models (LLMs) has swept almost every industry and performed well in various industry tasks, but they also have inherent limitations, including hallucinations, lack of interpretability of generated texts, poor understanding of professional domain knowledge, and limited understanding of the latest knowledge.

[0003] In the face of the above problems, Retrieval Augmented Generation (RAG), referred to as RAG, has become one of the most popular solutions. RAG assists large language models in generating more accurate and richer text content by retrieving relevant information from external knowledge bases.

[0004] However, the existing RAG scheme adopts a fixed text segmentation method to expand the content. This process easily destroys the relevance of knowledge content, affects the effectiveness of prompt word expansion, and further affects the accuracy of large language model answers. Summary of the invention

[0005] In view of this, the present application provides a question answering method, which can improve the accuracy of question answering.

[0006] According to a first aspect of an embodiment of the present application, a method for answering a question is provided, which can be applied to a system or program including a question answering function in a terminal device, and specifically includes:

[0007] Get the pending issues initiated by the target object;

[0008] Determine a question vector corresponding to the problem to be processed, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model;

[0009] Combining the question vector with the knowledge point vector to obtain prompt information;

[0010] The prompt information is input into the large language model to obtain answer information corresponding to the question to be processed.

[0011] Optionally, in some possible implementations, the vector database is obtained by the following configuration:

[0012] Get document data;

[0013] Configuring a topic prompt word for the document data;

[0014] Extracting a topic from the document data by using the large language model and referring to the topic prompt words to obtain a document topic;

[0015] Configuring clustering prompt words for the document title;

[0016] Clustering the document propositions by referring to the clustering prompt words through the large language model to obtain document knowledge points;

[0017] Determine the knowledge point vector corresponding to the document knowledge point to configure the vector database.

[0018] Optionally, in some possible implementations, obtaining document data includes:

[0019] Acquire knowledge documents;

[0020] Performing layout analysis on the knowledge document to obtain key information corresponding to the knowledge document;

[0021] The key information is paradigm-aligned to obtain the document data.

[0022] Optionally, in some possible implementations, the performing paradigm alignment on the key information to obtain the document data includes:

[0023] Performing paradigm alignment on the key information to obtain parsed data;

[0024] Segmenting the parsed data according to the segmentation rules corresponding to the modality of the knowledge document to obtain document data blocks;

[0025] The document data is obtained based on the document data block configuration.

[0026] Optionally, in some possible implementations, extracting a proposition from the document data by referring to the proposition prompt word through the large language model to obtain a document proposition includes:

[0027] Performing object analysis on the document data by referring to the proposition prompt words through the large language model to obtain subject information and fact information;

[0028] Performing content segmentation based on the correspondence between the subject information and the fact information to obtain segmented content;

[0029] Performing content optimization on the pronouns in the segmented content to obtain optimized content;

[0030] The optimized content is classified according to the main information to obtain the document title.

[0031] Optionally, in some possible implementations, the method further includes:

[0032] The document proposition is input as a sample into the large language model for proposition extraction to refine the document proposition.

[0033] Optionally, in some possible implementations, clustering the document propositions with reference to the clustering prompt words by the large language model to obtain document knowledge points includes:

[0034] Classifying the document propositions by referring to the clustering prompt words through the large language model to obtain a proposition array;

[0035] Determining the title information indicating the association of the object in the document title;

[0036] Integrating the proposition array based on the title information to obtain an integrated array;

[0037] The core contents indicated by the integrated array are summarized to obtain the document knowledge points.

[0038] Optionally, in some possible implementations, the method further includes:

[0039] The document knowledge points are input into the large language model to repeatedly perform a clustering process to update the document knowledge points.

[0040] Optionally, in some possible implementations, the method further includes:

[0041] Get the response information of the target object;

[0042] identifying a negative evaluation indicated in the response message;

[0043] Calling a knowledge database associated with the evaluation object indicated in the negative evaluation;

[0044] The document data is supplemented based on the knowledge database.

[0045] According to a second aspect of an embodiment of the present application, there is provided a question answering device, comprising:

[0046] An acquisition unit, used to acquire pending issues initiated by a target object;

[0047] A determination unit, configured to determine a question vector corresponding to the problem to be processed, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model;

[0048] The determining unit is further used to combine the question vector and the knowledge point vector to obtain prompt information;

[0049] The answering unit is used to input the prompt information into the large language model to obtain answer information corresponding to the question to be processed.

[0050] Optionally, in some possible implementations, the determination unit is used to acquire document data; configure proposition prompt words for the document data; perform proposition extraction on the document data with reference to the proposition prompt words through the large language model to obtain document propositions; configure clustering prompt words for the document propositions; cluster the document propositions with reference to the clustering prompt words through the large language model to obtain document knowledge points; determine the knowledge point vectors corresponding to the document knowledge points to configure the vector database.

[0051] Optionally, in some possible implementations, the determination unit is used to obtain a knowledge document when obtaining the document data; perform layout analysis on the knowledge document to obtain key information corresponding to the knowledge document; and perform paradigm alignment on the key information to obtain the document data.

[0052] Optionally, in some possible implementations, the determination unit is used to perform paradigm alignment on the key information to obtain parsed data when the key information is paradigm aligned to obtain the document data; segment the parsed data according to the segmentation rules corresponding to the modality of the knowledge document to obtain document data blocks; and obtain the document data based on the document data block configuration.

[0053] Optionally, in some possible implementations, the determination unit is used to perform object analysis on the document data with reference to the proposition prompt words through the large language model to obtain subject information and factual information when the document data is subjected to proposition extraction with reference to the proposition prompt words to obtain the document proposition; perform content segmentation based on the correspondence between the subject information and the factual information to obtain segmented content; perform content optimization on the pronouns in the segmented content to obtain optimized content; and classify the optimized content according to the subject information to obtain the document proposition.

[0054] Optionally, in some possible implementations, the determination unit is used to input the document proposition as a sample into the large language model for proposition extraction to refine the document proposition.

[0055] Optionally, in some possible implementations, the determination unit is used to classify the document propositions by referring to the clustering prompt words through the large language model to obtain a proposition array when the document propositions are clustered by referring to the clustering prompt words to obtain document knowledge points; determine the title information indicating the object association in the document propositions; integrate the proposition array based on the title information to obtain an integrated array; and summarize the core content indicated by the integrated array to obtain the document knowledge points.

[0056] Optionally, in some possible implementations, the determining unit is used to input the document knowledge points into the large language model to repeatedly perform a clustering process to update the document knowledge points.

[0057] Optionally, in some possible implementations, the answer unit is used to obtain reply information of the target object's feedback; determine the negative evaluation indicated in the reply information; call the knowledge database associated with the evaluation object indicated in the negative evaluation; and supplement the document data based on the knowledge database.

[0058] According to a third aspect of an embodiment of the present application, there is provided a question answering device, including an input-output component and a processor;

[0059] The input and output components are used to obtain problems to be processed;

[0060] The processor is used to answer the pending question obtained by the input-output component by executing the question answering method as described in the first aspect or any implementation of the first aspect.

[0061] According to a fourth aspect of an embodiment of the present application, there is provided a question answering system, including an interactive client and a server;

[0062] The interactive client is used to obtain the pending questions and send the pending questions to the server, and to display the answer results of the questions output by the server;

[0063] The server is used to answer the pending questions obtained by the interactive client by executing the question answering method described in the first aspect or any implementation of the first aspect.

[0064] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, including: a computer program, which, when executed by a processor, implements the question answering method described in the first aspect or any implementation manner of the first aspect.

[0065] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0066] By obtaining the pending questions initiated by the target object, the question vector corresponding to the pending question is determined, and similar knowledge point vectors are retrieved from the vector database based on the question vector. The vector database is configured based on the document knowledge points corresponding to the document data, and the document knowledge points are clustered based on the document propositions extracted from the document data by the large language model. The prompt information is obtained by combining the question vector and the knowledge point vector. The prompt information is then input into the large language model to obtain the answer information corresponding to the pending question. In this way, a targeted knowledge expansion process is realized, and the integrity of the expanded knowledge points is ensured through proposition extraction and clustering, noise is reduced, and the problem of knowledge point fragmentation caused by text block splitting is avoided, the retrieval hit rate is improved, and the accuracy of the large language model reply is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0068] Figure 1 A diagram of the network architecture that runs the question answering system;

[0069] Figure 2 A process architecture diagram of answering questions provided in an embodiment of the present application;

[0070] Figure 3 A flowchart of a question answering method provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of a scenario execution of a question answering method provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of a scenario execution of another question answering method provided in an embodiment of the present application;

[0073] Figure 6 A schematic diagram of the structure of a question answering device provided in an embodiment of the present application;

[0074] Figure 7A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0076] It should be understood that the question answering method provided in the present application can be applied to a system or program including a question answering function in a terminal device, such as a teaching aid application. Specifically, the question answering system can be run on a computer such as a computer. Figure 1 In the network architecture shown in Figure 1 As shown in FIG. 1 , it is a network architecture diagram of the question answering system. As can be seen from the figure, the question answering system can provide a question answering process with multiple information sources, that is, the questions to be processed are determined through interactive operations on the terminal side, and then sent to the server for targeted answers to the questions based on the large language model; it can be understood that Figure 1 A variety of terminal devices are shown in FIG. 1 . The terminal devices may be computer devices. In actual scenarios, more or fewer types of terminal devices may participate in the process of answering questions. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may be involved, especially in multi-disciplinary answer scenarios, and the specific number of servers depends on the actual scenario.

[0077] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.

[0078] It can be understood that the above-mentioned question answering system can run on a personal mobile terminal, for example: as an application such as educational tutoring, it can also run on a server, and it can also run on a third-party device to provide answers to questions to obtain the question answering processing results of the information source; the specific question answering system can be run in the above-mentioned device in the form of a program, and it can also be run as a system component in the above-mentioned device, and it can also be used as a cloud service program. The specific operation mode depends on the actual scenario and is not limited here.

[0079] In recent years, the wave of Large Language Models (LLMs) has swept almost every industry and performed well in various industry tasks, but they also have inherent limitations, including hallucinations, lack of interpretability of generated texts, poor understanding of professional domain knowledge, and limited understanding of the latest knowledge.

[0080] In the face of the above problems, Retrieval Augmented Generation (RAG), referred to as RAG, has become one of the most popular solutions. RAG assists large language models in generating more accurate and richer text content by retrieving relevant information from external knowledge bases.

[0081] However, the current RAG scheme requires regular splitting of the document content to generate text blocks. However, such text blocks usually contain knowledge points of multiple different topics, and the integrity of the knowledge points cannot be guaranteed. Knowledge points of different topics interfere with each other, affecting the hit rate of RAG retrieval and directly affecting the accuracy of the large language model response.

[0082] In order to solve the above problems, this application proposes a question answering method, which is applied to Figure 2 In the process framework of question answering shown in Figure 2 As shown, it is a process architecture diagram of question answering provided by an embodiment of the present application, that is, in response to the interactive operation on the terminal side, the server is triggered to answer the question, and the large language model used in the answering process generates a reply after expanding the prompt words based on the vector database. The specific execution of the server includes a data preparation stage and an application stage, wherein the data preparation stage is to extract data from multimodal documents, including designing large language model prompt words to extract knowledge points, clustering the extracted knowledge points, and replacing text blocks with clustered knowledge points for vectorization and data storage; and for the application stage, the user asks a question, and data is retrieved to obtain the answer based on the question, and the prompt words are designed and then delivered to the large language model in combination with the answer to generate the answer.

[0083] It can be seen that this embodiment does not directly construct vectors after extracting and splitting the data, but instead extracts propositions through a large language model, and the acquired propositions are further refined as samples. The refined propositions are clustered to obtain knowledge points, which replace the text blocks in the general RAG solution and participate in the retrieval and recall of RAG and the response generation of the large language model.

[0084] It can be understood that the question answering method provided by the present application can be written as a program to serve as a processing logic in a hardware system, or as a question answering device to implement the above processing logic in an integrated or external manner. As an implementation method, the question answering device obtains the pending question initiated by the target object; then determines the question vector corresponding to the pending question, and retrieves similar knowledge point vectors in the vector database according to the question vector. The vector database is obtained based on the document knowledge point configuration corresponding to the document data, and the document knowledge point is obtained by clustering the document propositions extracted from the document data based on the large language model; and combines the question vector and the knowledge point vector to obtain prompt information; and then inputs the prompt information into the large language model to obtain the answer information corresponding to the pending question. Thereby, a targeted knowledge expansion process is realized, and the integrity of the expanded knowledge points is ensured by proposition extraction and clustering, noise is reduced, and the problem of knowledge point fragmentation caused by text block splitting is avoided, the retrieval hit rate is improved, and the accuracy of the large language model reply is improved.

[0085] In combination with the above process architecture, the following will introduce the method of answering questions in this application. Please refer to Figure 3 , Figure 3 A flowchart of a question answering method provided in an embodiment of the present application, the embodiment of the present application at least includes the following steps:

[0086] 301. Get the pending issues initiated by the target object.

[0087] In this embodiment, the target object may be a user, a terminal or other indicative object, and this embodiment is described by taking a user as an example. The problem to be processed may be a question initiated by a user, a sentence in a communication process or a problem generated by other media input forms, and the specific input form depends on the actual scenario.

[0088] 302. Determine a question vector corresponding to the problem to be processed, so as to retrieve similar knowledge point vectors in a vector database based on the question vector. The vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data using a large language model.

[0089] In this embodiment, the vector database is a database used to expand the knowledge dimension of the problem vector. The vector calculation method used is the same as the calculation method for determining the problem vector corresponding to the problem to be processed, so that the knowledge point expansion process for the problem can be performed.

[0090] In one possible scenario, for the determination of knowledge point vectors, that is, the matching process of similarity scores, after receiving a question, the question is vectorized using the same encoding model, and the similarity score between the question vector and the vector in the vector library is calculated, thereby extracting the knowledge point vector whose similarity score meets the threshold condition.

[0091] Specifically, the expansion process of knowledge points is carried out through proposition clustering; wherein, a proposition is the knowledge or content classification corresponding to the document data in the knowledge database, and a knowledge point is the summary and refinement of the knowledge or content of this category, that is, after extracting and splitting the data, this embodiment does not directly construct a vector, but extracts propositions through a large language model, and the obtained propositions are further refined as samples. The refined propositions are clustered to obtain knowledge points, and the knowledge points replace the text blocks separated by a fixed method to participate in the retrieval and recall of RAG and the response generation of the large language model.

[0092] The following is an explanation of the configuration process of the vector database, namely, first obtaining document data, which may be knowledge document data of different types from different data sources; then configuring proposition prompt words for the document data; and extracting propositions from the document data with reference to the proposition prompt words through a large language model to obtain document propositions; then configuring clustering prompt words for the document propositions; and clustering the document propositions with reference to the clustering prompt words through a large language model to obtain document knowledge points; and then determining the knowledge point vectors corresponding to the document knowledge points to configure the vector database.

[0093] It is understandable that the determination of document data can be done by obtaining knowledge documents of different data sources and formats; then performing layout analysis on the knowledge documents to obtain the key information corresponding to the knowledge documents; and then aligning the key information with the paradigm to obtain document data. For example, multi-format data loading or obtaining data from different data sources, such as word documents, pictures, pdf files, etc.; for the layout analysis process, the layout analysis can be performed on the pdf file alone to analyze the title, table, picture, etc. in the document, extract the key information in the data, such as the file name, title, time, etc., and process the data into the same paradigm, thereby arranging the document data to improve the recognizability of the document data.

[0094] In addition, since the size and format of document data are different, it can be segmented to facilitate parallel processing of the recognition process. That is, the key information is aligned in paradigm to obtain parsed data; then the parsed data is segmented according to the segmentation rules corresponding to the modality of the knowledge document to obtain document data blocks (chunks); and then the document data is obtained based on the document data block configuration. For example, the data generated by document parsing is segmented into chunks, and the data is segmented into chunks with reasonable rules and sizes using rule parsing; the segmentation rules are determined based on the integrity of the content, that is, to ensure that the segmented chunks contain complete key information, thereby avoiding the discreteness of knowledge points.

[0095] Furthermore, the proposition extraction process is carried out through a large language model. For the proposition extraction operation, proposition extraction prompt words are designed specifically, that is, independent propositions are extracted from a given text paragraph. Each proposition should represent an independent fact or information in the text and can be understood independently without additional context.

[0096] Therefore, the process of proposition extraction is carried out with reference to the above-mentioned proposition extraction prompt. First, the large language model is used to analyze the document data with reference to the proposition prompt to obtain the subject information and factual information, that is, by reading the text, each independent subject and fact is identified. Including people, objects, corresponding occupations or identities, and the assumptions or opinions they put forward; then the content is segmented based on the correspondence between the subject information and the factual information to obtain the segmented content. The segmentation process needs to ensure that each proposition reflects a single fact or opinion. If a sentence contains multiple facts or information, it is segmented into multiple propositions. For example, the names of people and their occupations should be extracted separately. Then, the pronouns in the segmented content are optimized to obtain optimized content. This optimization process is called pronoun optimization, which replaces pronouns or unclear pronouns in the proposition so that each proposition can be understood independently. Clearly point out important information such as the identity of the person, the location, and the time of the event; and then classify the optimized content according to the subject information to obtain the document proposition.

[0097] In one possible scenario, the document propositions can also be audited, that is, the extracted propositions are audited and optimized to ensure that they are concise, clear and independent. Ensure that each proposition expresses a complete and independent fact. As for the output process of the document propositions, it is necessary to ensure clear output, that is, to classify by subject, clearly and concisely output the final proposition list, and each proposition is separated by a line break.

[0098] Furthermore, after the designed prompt and chunk are simultaneously input into the large language model to extract propositions, the document propositions can also be input into the large language model as samples for proposition extraction to refine the document propositions. That is, the propositions extracted for the first time are input into the large language model again as a few shots to continue extraction, so as to achieve the effect of accurate proposition results.

[0099] After obtaining the document proposition, the proposition clustering process can be performed. For the clustering prompt words used in the clustering process, the design task is to classify the propositions extracted from the original text according to the original text, classify the propositions describing the same thing or attribute into the same array, and generate a classification summary. The summary description should include all aliases of the thing to ensure comprehensive information.

[0100] Therefore, for the process of proposition clustering, firstly, the document propositions are classified by the large language model with reference to the clustering prompt words to obtain the proposition array, that is, all propositions are reviewed, and propositions expressing the same theme, subject or attribute are assigned to an array, and the relevance between them is judged according to the proposition content. Then the title information indicating the object association in the document proposition is determined, that is, in the classification process, all the aliases involved are identified and explicitly mentioned in the classification summary. For example, if the same thing has different names (such as "Easter Bunny" and "Easter Hare"), all names need to be listed in the summary. Then, the proposition array is integrated based on the title information to obtain the integrated array, that is, each classification group is summarized, and the core content of the proposition of the group is briefly described to ensure that all key information and aliases are covered. Then, the core content indicated by the integrated array is summarized to obtain the document knowledge points, that is, for the process of result output, the classification results are output in the form of clear arrays, each array contains the proposition serial number under the category, and is accompanied by a concise summary description.

[0101] Furthermore, the document knowledge points may be input into the large language model to repeat the clustering process to update the document knowledge points, that is, repeat the above clustering operation to achieve a refined clustering effect. The specific number of repetitions depends on the actual scenario.

[0102] 303. Combining the question vector with the knowledge point vector, obtain prompt information.

[0103] In this embodiment, since the question vector and the knowledge point vector are vectorized using the same encoding model, the question and the retrieved knowledge points can be combined into a coherent prompt for large language model response.

[0104] 304. Input the prompt information into the large language model to obtain answer information corresponding to the question to be processed.

[0105] In this embodiment, through the integration of the above knowledge points, the large language model can use the knowledge points as the basis for content output as prompt words, thereby improving the accuracy of answering questions.

[0106] Specifically, the display of the answer information may be through the same interface or the same method as the user inputs the question to be processed, such as answering by voice. Figure 4 As shown, Figure 4 A schematic diagram of a scenario execution of a question answering method provided in an embodiment of the present application; the execution process of the scenario includes the following steps:

[0107] Step 1: Document parsing. The document includes data in multiple formats and from different data sources, such as word documents, pictures, pdf files, etc. Then, the layout of the file data is analyzed to analyze the title, table, picture, etc. in the document, extract key information from the data, such as file name, title, time, etc., and process the data into the same paradigm.

[0108] Step 2: Data block extraction. That is, chunk the data generated by document parsing, and use rule parsing to split the data into chunks with reasonable rules and sizes.

[0109] Step 3: Design the prompt words for question extraction.

[0110] 1. Design prompt words: For the document paragraph to be extracted, the design task is to extract independent propositions from the given text paragraph. Each proposition should represent an independent fact or information in the text and can be understood independently without additional context.

[0111] 2. Design workflow:

[0112] (1) Analyze the text: Read the text carefully and identify each independent subject and fact, including characters, objects, corresponding occupations or identities, and the assumptions or opinions they put forward.

[0113] (2) Sentence segmentation: Make sure each proposition reflects a single fact or opinion. If a sentence contains multiple facts or information, segment it into multiple propositions. For example, a person’s name and their occupation should be extracted separately.

[0114] (3) Reference optimization: Replace pronouns or unclear reference words in the proposition so that each proposition can be understood independently. Clearly indicate important information such as the identity of the person, the location, and the time of the event.

[0115] (4) Proposition review: Review and optimize the extracted propositions to ensure that they are concise, clear, and independent. Ensure that each proposition expresses a complete and independent fact.

[0116] (5) Clear output: The final list of propositions is output clearly and concisely by subject, with each proposition separated by a line break.

[0117] Step 4: Extract propositions. The designed prompt and chunk are simultaneously input into the large language model to extract propositions. The propositions extracted for the first time are used as few shots and then input into the large language model again to continue extraction and obtain accurate proposition results.

[0118] Step 5: Clustering.

[0119] That is, we use the large language model clustering method to design clustering prompts, which specifically include:

[0120] 1. Design Tips: As a proposition clustering expert, your task is to classify the propositions extracted from the original text according to the original text, classify the propositions describing the same thing or attribute into the same array, and generate a classification summary. The summary description should include all aliases of the thing to ensure comprehensive information.

[0121] 2. Design workflow:

[0122] (1) Proposition classification: Review all propositions and assign propositions that express the same theme, subject, or attribute to an array. Determine the relevance between the propositions based on their content.

[0123] (2) Alias ​​identification: During the classification process, identify all aliases involved and mention them clearly in the classification summary. For example, if the same thing has different names (such as "Easter Bunny" and "Easter Hare"), all names need to be listed in the summary.

[0124] (3) Summary description: Summarize each classification group and briefly describe the core content of the propositions in this group, ensuring that all key information and aliases are covered.

[0125] (4) Result output: Output the classification results in a clear array format. Each array contains the proposition number under the category and a concise summary description. Repeat the above clustering operation to refine the clustering results.

[0126] Step 6: Vectorization: vectorize the data obtained by the above clustering (select appropriate embedding) and store it in the vector database.

[0127] Step 7: Retrieval: After receiving the question, the same encoding model is used to vectorize the question and calculate the similarity score between the question vector and the vector in the vector library.

[0128] Step 8: Generate. Combine the question and the retrieved knowledge points into a coherent prompt for the large language model to respond.

[0129] Furthermore, after the user receives a reply from the large language model, feedback on the reply effect can be provided. For this scenario, Figure 5 As stated, Figure 5 A schematic diagram of a scenario execution of another question answering method provided in an embodiment of the present application; the diagram shows how, after obtaining the reply information of the target object's feedback, the negative evaluation indicated in the reply information can be determined, such as content associated with negative vocabulary, content with negative semantic understanding, etc.; then the knowledge database associated with the evaluation object indicated in the negative evaluation is called; the document data is supplemented based on the knowledge database, that is, the content that the user is dissatisfied with in the feedback is supplemented in a targeted manner, thereby further improving the accuracy of the large language model's response.

[0130] In combination with the above embodiments, it can be known that by obtaining the pending question initiated by the target object; then determining the question vector corresponding to the pending question, so as to retrieve similar knowledge point vectors in the vector database according to the question vector, the vector database is obtained based on the document knowledge point configuration corresponding to the document data, and the document knowledge point is obtained by clustering the document propositions extracted from the document data based on the large language model; and combining the question vector and the knowledge point vector to obtain prompt information; and then inputting the prompt information into the large language model to obtain the answer information corresponding to the pending question. In this way, a targeted knowledge expansion process is realized, and the integrity of the expanded knowledge points is ensured through proposition extraction and clustering, noise is reduced, and the problem of knowledge point fragmentation caused by text block splitting is avoided, the retrieval hit rate is improved, and the accuracy of the large language model reply is improved.

[0131] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 6 , Figure 6 The following is a schematic diagram of a question answering device provided in an embodiment of the present application. The question answering device 600 includes:

[0132] An acquisition unit 601 is used to acquire a pending issue initiated by a target object;

[0133] A determination unit 602 is used to determine a question vector corresponding to the to-be-processed question, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model;

[0134] The determining unit 602 is further configured to combine the question vector and the knowledge point vector to obtain prompt information;

[0135] The answering unit 603 is used to input the prompt information into the large language model to obtain answer information corresponding to the question to be processed.

[0136] Optionally, in some possible implementations, the determination unit 602 is used to obtain document data; configure proposition prompt words for the document data; perform proposition extraction on the document data with reference to the proposition prompt words through the large language model to obtain document propositions; configure clustering prompt words for the document propositions; cluster the document propositions with reference to the clustering prompt words through the large language model to obtain document knowledge points; determine the knowledge point vectors corresponding to the document knowledge points to configure the vector database.

[0137] Optionally, in some possible implementations, the determination unit 602 is used to obtain a knowledge document when obtaining the document data; perform layout analysis on the knowledge document to obtain key information corresponding to the knowledge document; and perform paradigm alignment on the key information to obtain the document data.

[0138] Optionally, in some possible implementations, the determination unit 602 is used to perform paradigm alignment on the key information to obtain parsed data when the key information is paradigm aligned to obtain the document data; segment the parsed data according to the segmentation rules corresponding to the modality of the knowledge document to obtain document data blocks; and obtain the document data based on the document data block configuration.

[0139] Optionally, in some possible implementations, the determination unit 602 is used to perform object analysis on the document data with reference to the proposition prompt words through the large language model to obtain subject information and factual information when the document data is subjected to proposition extraction with reference to the proposition prompt words through the large language model to obtain the document proposition; perform content segmentation based on the correspondence between the subject information and the factual information to obtain segmented content; perform content optimization on the pronouns in the segmented content to obtain optimized content; and classify the optimized content according to the subject information to obtain the document proposition.

[0140] Optionally, in some possible implementations, the determining unit 602 is configured to input the document proposition as a sample into the large language model for proposition extraction to refine the document proposition.

[0141] Optionally, in some possible implementations, the determination unit 602 is used to classify the document propositions by referring to the clustering prompt words through the large language model to obtain a proposition array when the document propositions are clustered by referring to the clustering prompt words to obtain document knowledge points; determine the title information indicating the object association in the document propositions; integrate the proposition array based on the title information to obtain an integrated array; and summarize the core content indicated by the integrated array to obtain the document knowledge points.

[0142] Optionally, in some possible implementations, the determining unit 602 is configured to input the document knowledge points into the large language model to repeatedly perform a clustering process to update the document knowledge points.

[0143] Optionally, in some possible implementations, the answer unit 603 is used to obtain reply information of the target object's feedback; determine the negative evaluation indicated in the reply information; call the knowledge database associated with the evaluation object indicated in the negative evaluation; and supplement the document data based on the knowledge database.

[0144] By obtaining the pending questions initiated by the target object, the question vector corresponding to the pending question is determined, and similar knowledge point vectors are retrieved from the vector database based on the question vector. The vector database is configured based on the document knowledge points corresponding to the document data, and the document knowledge points are clustered based on the document propositions extracted from the document data by the large language model. The prompt information is obtained by combining the question vector and the knowledge point vector. The prompt information is then input into the large language model to obtain the answer information corresponding to the pending question. In this way, a targeted knowledge expansion process is realized, and the integrity of the expanded knowledge points is ensured through proposition extraction and clustering, noise is reduced, and the problem of knowledge point fragmentation caused by text block splitting is avoided, the retrieval hit rate is improved, and the accuracy of the large language model reply is improved.

[0145] The question answering device provided in this embodiment belongs to the same application concept as the method provided in the above embodiments of this application, can execute the question answering method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the question answering method. For technical details not fully described in this embodiment, please refer to the specific processing content of the question answering method provided in the above embodiments of this application, which will not be repeated here.

[0146] The functions implemented by the above acquisition unit 601, determination unit 602 and answering unit 603 may be implemented by the same or different processors respectively, and the embodiment of the present application is not limited thereto.

[0147] It should be understood that each unit in the above device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units above are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all units above. All units of the above device can be implemented in the form of a processor calling software, or in the form of a hardware circuit, or in the form of a processor calling software, and the remaining part can be implemented in the form of a hardware circuit.

[0148] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0149] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0150] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0151] The embodiment of the present application also provides a question answering device; the question answering device includes a processor and an input-output component. The input-output component is used to obtain a question to be processed;

[0152] The processor is used to answer the pending question obtained by the input-output component by executing any one of the question answering methods in the above embodiments.

[0153] The above-mentioned interface circuit can be any interface circuit that can realize the data communication function, for example, it can be a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIE circuit, etc.

[0154] Optionally, the embodiment of the present application further provides a system, wherein the interactive client is used to obtain the pending questions and send the pending questions to the server, and to display the answer results of the questions output by the server;

[0155] The server is used to answer the pending questions obtained by the interactive client by executing the question answering method described in any implementation manner in the above embodiments.

[0156] Another embodiment of the present application also provides a question answering device, see Figure 7 As shown, the device includes:

[0157] Memory 700 and processor 710;

[0158] The memory 700 is connected to the processor 710 and is used to store programs;

[0159] The processor 710 is configured to implement the question answering method disclosed in any of the above embodiments by running the program stored in the memory 700 .

[0160] Specifically, the question answering device may further include: a bus, a communication interface 720 , an input device 730 and an output device 740 .

[0161] The processor 710, the memory 700, the communication interface 720, the input device 730 and the output device 740 are connected to each other via a bus.

[0162] A bus may include a pathway that transfers information between components of a computer system.

[0163] Processor 710 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0164] The processor 710 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0165] The memory 700 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 700 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0166] The input device 730 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0167] Output device 740 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0168] The communication interface 720 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0169] The processor 710 executes the program stored in the memory 700 and calls other devices, which can be used to implement each step of any question answering method provided in the above embodiments of the present application.

[0170] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the question answering method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0171] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0172] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the question answering method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0173] Get the pending issues initiated by the target object;

[0174] Determine a question vector corresponding to the problem to be processed, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model;

[0175] Combining the question vector with the knowledge point vector to obtain prompt information;

[0176] The prompt information is input into the large language model to obtain answer information corresponding to the question to be processed.

[0177] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0178] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0179] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0180] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0182] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0184] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0185] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0186] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0187] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A question answering method, characterized in that: include: Get the pending issues initiated by the target object; Determine a question vector corresponding to the problem to be processed, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model; Combining the question vector with the knowledge point vector to obtain prompt information; The prompt information is input into the large language model to obtain answer information corresponding to the question to be processed.

2. The method according to claim 1, characterized in that The vector database is obtained by the following configuration: Get document data; Configuring a topic prompt word for the document data; Extracting a topic from the document data by using the large language model and referring to the topic prompt words to obtain a document topic; Configuring clustering prompt words for the document title; Clustering the document propositions by referring to the clustering prompt words through the large language model to obtain the document knowledge points; The knowledge point vector corresponding to the document knowledge point is determined to configure the vector database.

3. The method according to claim 2, characterized in that The obtaining of document data includes: Acquire knowledge documents; Performing layout analysis on the knowledge document to obtain key information corresponding to the knowledge document; The key information is paradigm-aligned to obtain the document data.

4. The method according to claim 3, characterized in that The step of performing paradigm alignment on the key information to obtain the document data includes: Performing paradigm alignment on the key information to obtain parsed data; Segmenting the parsed data according to the segmentation rules corresponding to the modality of the knowledge document to obtain document data blocks; The document data is obtained based on the document data block configuration.

5. The method according to claim 2, characterized in that: The extracting the document data by referring to the proposition prompt words through the large language model to obtain the document proposition includes: Performing object analysis on the document data by referring to the proposition prompt words through the large language model to obtain subject information and fact information; Performing content segmentation based on the correspondence between the subject information and the fact information to obtain segmented content; Performing content optimization on the pronouns in the segmented content to obtain optimized content; The optimized content is classified according to the main information to obtain the document title.

6. The method according to claim 5, characterized in that The method further comprises: The document proposition is input as a sample into the large language model for proposition extraction to refine the document proposition.

7. The method according to claim 2, characterized in that The step of clustering the document propositions by using the large language model with reference to the clustering prompt words to obtain document knowledge points includes: Classifying the document propositions by referring to the clustering prompt words through the large language model to obtain a proposition array; Determining the title information indicating the association of the object in the document title; Integrating the proposition array based on the title information to obtain an integrated array; The core contents indicated by the integrated array are summarized to obtain the document knowledge points.

8. The method according to claim 7, characterized in that The method further comprises: The document knowledge points are input into the large language model to repeatedly perform a clustering process to update the document knowledge points.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Get the response information of the target object; identifying a negative evaluation indicated in the response message; Calling a knowledge database associated with the evaluation object indicated in the negative evaluation; The document data is supplemented based on the knowledge database.

10. A question answering device, characterized in that: include: An acquisition unit, used to acquire pending issues initiated by a target object; A determination unit, configured to determine a question vector corresponding to the problem to be processed, so as to retrieve a similar knowledge point vector in a vector database according to the question vector, wherein the vector database is configured based on document knowledge points corresponding to the document data, and the document knowledge points are clustered based on document propositions extracted from the document data by a large language model; The determining unit is further used to combine the question vector and the knowledge point vector to obtain prompt information; The answering unit is used to input the prompt information into the large language model to obtain answer information corresponding to the question to be processed.

11. A question answering device comprising an input-output component and a processor; The input and output components are used to obtain problems to be processed; The processor is used to answer the pending question obtained by the input-output component by executing the question answering method as described in any one of claims 1 to 9.

12. A question answering system, comprising an interactive client and a server; The interactive client is used to obtain the pending questions and send the pending questions to the server, and to display the answer results of the questions output by the server; The server is used to answer the pending questions obtained by the interactive client by executing the question answering method described in any one of claims 1 to 9.

13. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the question answering method according to any one of claims 1 to 9 is implemented.

Images