Question and answer method, electronic equipment and program product
Through the document collection divided by preset intent identification model and intention information, the problem of low adaptability to problems caused by text similarity retrieval in the prior art is solved, and the accuracy and user experience of question and answer are improved.
Patent Information
- Application Number
- CN202411086979.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, when performing questions and answers based on documents, the adaptability of questions and questions caused by text similarity retrieval is not high, which reduces the accuracy of questions and answers and affects the user experience.
The preset intent recognition model is used to obtain the model through training of multiple preset problem marking intent information, determine the intent information of the target problem, and divide the document-related sets based on the intent information for searching, so as to improve the adaptability of documents and problems.
Improve the accuracy and user experience of Q&A results, and enhance the relevance between questions and documents through more accurate document retrieval and matching.
Smart Images

Figure CN119025644A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a question-answering method, an electronic device, and a program product. Background Art
[0002] With the continuous development of the financial field and computer technology, it is necessary to conduct question and answer based on relevant documents in the financial field in order to extract the information required by users.
[0003] Currently, when answering questions based on documents, the question and all documents are generally processed to obtain vectorized questions and all vectorized documents. Then, based on the vectorized questions, text similarity retrieval is performed in all vectorized documents to obtain documents related to the question, and the question is answered based on these documents.
[0004] However, the degree of adaptability between the document and the question obtained by only considering text similarity is not high, which reduces the accuracy of question and answer and affects the user experience. Summary of the invention
[0005] The present application provides a question-and-answer method, an electronic device, and a program product to solve the technical problems in the prior art that the adaptability between documents and questions is not high, the accuracy of questions and answers is reduced, and the user experience is affected.
[0006] In a first aspect, the present application provides a question-and-answer method, comprising: receiving a target question-and-answer request sent by a terminal; the target question-and-answer request includes a target question;
[0007] Based on a preset intent recognition model, determine the intent information corresponding to the target question, and determine the document-related set corresponding to the target question according to the intent information; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence;
[0008] Searching in a corresponding document related collection based on the target question to obtain documents related to the target question;
[0009] An answer to the target question is determined in the document associated with the target question, and the answer is sent to the terminal.
[0010] In a second aspect, the present application provides a question-and-answer device, including: a transceiver module, configured to receive a target question-and-answer request sent by a terminal; the target question-and-answer request includes a target question;
[0011] A processing module is used to determine the intent information corresponding to the target question based on a preset intent recognition model, and determine the document-related set corresponding to the target question according to the intent information; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence; based on the target question, searching in the corresponding document-related set to obtain documents related to the target question; and determining the answer to the target question in the documents related to the target question;
[0012] The transceiver module is further used to send the reply to the terminal.
[0013] In a third aspect, the present application provides an electronic device, comprising: a processor, a memory and a transceiver communicatively connected to the processor;
[0014] The memory stores computer-executable instructions; the transceiver is used to send and receive data;
[0015] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.
[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.
[0017] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.
[0018] The question-and-answer method, electronic device and program product provided by the present application receive a target question-and-answer request sent by a terminal; the target question-and-answer request includes a target question; based on a preset intention recognition model, the intention information corresponding to the target question is determined, and the document-related set corresponding to the target question is determined according to the intention information; the preset intention recognition model is obtained by training a plurality of preset questions with corresponding intention information marked until convergence; based on the target question, the corresponding document-related set is retrieved to obtain a document associated with the target question; the answer to the target question is determined in the document associated with the target question, and the answer is sent to the terminal. Since the preset intention recognition model is obtained by training a plurality of preset questions with corresponding intention information marked in advance, the intention information corresponding to the target question can be determined based on the preset intention recognition model by receiving the target question-and-answer request sent by the terminal, and the document-related set corresponding to the target question can be further determined according to the intention information. Therefore, by retrieving the documents associated with the target question in the corresponding document-related set, the answer to the target question can be determined based on these documents, and the corresponding answer can be sent to the terminal, thereby improving the adaptability of the determined document to the question, improving the accuracy of the question-and-answer result, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0020] Figure 1 An application scenario diagram of the question-answering method provided in an embodiment of the present application;
[0021] Figure 2a Process of the question-and-answer method provided in the embodiment of the present application Figure 1 ;
[0022] Figure 2b A schematic diagram of performing preliminary parsing of a file in PDF format provided in an embodiment of the present application;
[0023] Figure 3a A complete flow chart of the question-and-answer method provided in the embodiment of the present application;
[0024] Figure 3b An architectural diagram of the question-and-answer method provided in an embodiment of the present application;
[0025] Figure 4 A schematic diagram of the structure of a question-and-answer device provided in an embodiment of the present application;
[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0027] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0029] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.
[0030] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0031] In the prior art, when conducting question-answering based on documents, generally all documents are first vectorized and uploaded to a preset knowledge base. Then, in response to a user triggering a question-answering request through a corresponding terminal, the question in the request is obtained and vectorized. Then, the preset knowledge base is accessed, and a text similarity search is performed on all vectorized documents contained in the vectorized question to obtain documents with the greatest similarity to at least one associated question, so as to answer the question based on these documents. However, the documents obtained by only considering text similarity are not highly compatible with the question, which reduces the accuracy of question-answering and affects the user experience.
[0032] In response to the above technical problems, the present application proposes the following technical concept: In order to improve the accuracy of questions and answers and improve the user experience, instead of directly performing text similarity retrieval between vectorized questions and all vectorized documents, all documents are pre-vectorized differently based on different intentions, and all documents corresponding to each intention and all vectorized documents are uploaded to the partition corresponding to the knowledge base. Then, in response to the user triggering a question and answer request through the terminal, the question in the request is obtained, and the intention of the question is determined. Then, based on the determined intention, the partition corresponding to the knowledge base is accessed to obtain all corresponding documents and all vectorized documents, thereby determining at least one document with the highest similarity based on the question and all corresponding documents and all vectorized documents, and then answering, thereby improving the adaptability between the document used for answering and the question, improving the accuracy of the question and answer results, and improving the user experience.
[0033] Figure 1 This is an application scenario diagram of the question-answering method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes: user terminal 1, electronic device 2, and database 3. Among them, user terminal 1 is a terminal where a user with a question-and-answer requirement is located, electronic device 2 is a device for performing question-and-answer, and database 3 is a database corresponding to electronic device 2, which can be a preset knowledge base. Among them, electronic device 2 is connected to user terminal 1 and database 3 for communication.
[0034] First, a user with a question-and-answer requirement inputs a target question through the user terminal 1, and triggers a target question-and-answer request after the output is completed. Then, the electronic device 2 obtains the target question in response to receiving the target question-and-answer request sent by the user terminal 1, and determines the intent information corresponding to the target question based on a preset intent recognition model. According to the intent information, a document-related set corresponding to the target question is determined in the database 3, and the corresponding document-related set is searched based on the target question to obtain a document associated with the target question. Furthermore, an answer to the target question is determined in the document associated with the target question, and the answer is sent to the user terminal 1. Among them, the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence.
[0035] The question-and-answer method provided in this application is intended to solve the above technical problems of the prior art.
[0036] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0037] Figure 2aProcess of the question-and-answer method provided in the embodiment of the present application Figure 1 ,like Figure 2a As shown, the execution subject of this embodiment is a question-answering device, and the question-answering device is located in an electronic device. Then the question-answering method includes:
[0038] S201. Receive a target question and answer request sent by a terminal; the target question and answer request includes a target question.
[0039] The terminal is a terminal where a user with question and answer needs is located.
[0040] The target question-answering request is a request for instructing the electronic device to output a response based on a question, and specifically includes a target question, which is the question that the user needs to ask.
[0041] Exemplarily, the target question may be "What is the main content disclosed by document X?" or "What industry trend does the document released by company A in January 2022 reflect?" or "What is the development history of company A?".
[0042] In this embodiment, the user may input the target question in advance based on the terminal, and after the input, the component where the target question and answer request is located is triggered. Then, in response to the component being triggered, the terminal generates a target question and answer request based on the target question. Based on this, the target question and answer request is received, and the target question may be further obtained in the request.
[0043] It should be understood that the above examples for the target problem are only examples and should not constitute any limitation to the present application.
[0044] S202. Based on a preset intent recognition model, determine the intent information corresponding to the target question, and determine a related set of documents corresponding to the target question based on the intent information; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence.
[0045] In this embodiment, a certain number of preset questions can be obtained in advance, and the corresponding intent information can be marked for each preset question to form a large number of training samples, and then the training samples are used to train until convergence to obtain a preset intent recognition model. The preset questions are pre-set questions, which can be historical questions or questions determined by the administrator based on the user's historical question and answer situation.
[0046] The intent information is used to characterize the intent of the target question, and specifically may be information about asking and answering questions based on a specified document, information about asking and answering questions based on documents within a specified time range, or information about asking and answering questions without specifying a document.
[0047] The specified document and the specified time range are the specific document and the specific time range indicated in the target question.
[0048] For example, if the target question is "What is the main content disclosed by document X?", the intent information corresponding to the target question can be determined as information for question-and-answer based on a specified document. If the target question is "What industry trends are reflected in the documents released by company A in January 2022?", the intent information corresponding to the target question can be determined as information for question-and-answer based on documents within a specified time range. If the target question is "What is the development history of company A?", the intent information corresponding to the target question can be determined as information for question-and-answer without specifying a document.
[0049] Based on this, after obtaining the target question, the target question is input into the preset intent recognition model. The preset intent recognition model identifies the field where the target question is located, and determines the intent information corresponding to the target question based on the training results. Further, the document-related set corresponding to the target question is determined based on the intent information.
[0050] The document-related collection is a collection of multiple files related to the document that are pre-uploaded to the knowledge base.
[0051] Specifically, all documents can be processed based on different intent information in advance to obtain document-related sets corresponding to different intent information, and corresponding partitions can be set in the knowledge base in advance, and then the corresponding document-related sets can be uploaded to the corresponding partitions of the knowledge base. Based on this, after determining the intent information corresponding to the target question, the partition corresponding to the intent information is determined in the knowledge base, and the document-related set of the partition is obtained, thereby determining the document-related set corresponding to the target question.
[0052] S203: Search in the corresponding document related collection based on the target question to obtain documents related to the target question.
[0053] It is understandable that the document-related set is obtained based on the processing of all documents. Therefore, it is necessary to retrieve the documents most relevant to the target question in the corresponding document-related set in order to conduct question and answer based on these documents and improve the accuracy of the question and answer results.
[0054] Based on this, after determining the document-related set corresponding to the target question, the specific demands of the target question are identified, and based on the identification result, the most relevant document for the target question is retrieved from the corresponding document-related set, thereby obtaining the document associated with the target question.
[0055] S204: Determine an answer to the target question in the document associated with the target question, and send the answer to the terminal.
[0056] In this embodiment, a large language model can be established in advance. After obtaining the document related to the target question, the obtained target question and the document related to the target question are input into the established large prediction model, and the large language model rewrites the input target question and the document related to the target question into prompt information, and then further determines the fields that can answer the target question based on the prompt information, and splices these fields to obtain the answer to the target question. Based on this, the answer is sent to the terminal where the user is located, so that the user can know the answer content through the terminal.
[0057] The question-answering method provided in this embodiment uses a preset intention recognition model to train a plurality of preset questions with corresponding intention information in advance, so by receiving a target question-answering request containing a target question sent by a terminal, the intention information corresponding to the target question can be determined based on the preset intention recognition model, and the document-related set corresponding to the target question can be further determined based on the intention information. Therefore, by retrieving documents related to the target question in the corresponding document-related set, the answer to the target question can be determined based on these documents, and the corresponding answer can be sent to the terminal, thereby improving the adaptability of the determined document to the question, improving the accuracy of the question-answering result, and improving the user experience.
[0058] As an optional embodiment, in this embodiment, based on the above-mentioned previous embodiment, the document-related set includes a document set and at least one vectorized document set. When searching in the corresponding document-related set based on the target question to obtain a document associated with the target question, S203 specifically includes the following steps:
[0059] Step b1: Determine a first number of documents in a corresponding document set according to corresponding intent information.
[0060] In this embodiment, the document-related set includes a document set and at least one vectorized document set, wherein the document set is a set formed by all documents, and the vectorized document set is a set obtained by vectorizing all documents.
[0061] In this embodiment, a document set may be generated in advance based on all documents, and each document in the document set may be vectorized using a specified vectorization strategy in advance to obtain at least one vectorized document set.
[0062] Among them, the specified vectorization strategy can be: BGE, M3E and other vectorization strategies. Correspondingly, a vectorized document set processed by BGE vectorization and a vectorized document set processed by M3E vectorization can be obtained, that is, at least one vectorized document set mentioned in this embodiment.
[0063] The first number is the number of documents finally obtained in the document set.
[0064] Based on this, after determining the relevant set of documents corresponding to the target question, a first number of documents are determined in the document set corresponding to the target question according to the specific requirements indicated by the intent information corresponding to the target question.
[0065] Step b2: Based on the first number of documents, respectively obtain the first number of vectorized documents in at least one corresponding vectorized document set.
[0066] It can be understood that each vectorized document in the vectorized document set is obtained by vectorizing each document in the document set. Therefore, for the first number of documents in the document set, each vectorized document set has a corresponding vectorized document.
[0067] Based on this, after determining the first number of documents, the first number of vectorized documents are respectively obtained from at least one vectorized document set corresponding to the target question to obtain the first number of vectorized documents.
[0068] Step b3: perform vectorization processing on the target question at least once, and determine and obtain documents related to the target question based on each processed target question and each first number of vectorized documents.
[0069] In this embodiment, after obtaining the first number of vectorized documents, the target question is vectorized at least once according to the specified vector processing strategy to obtain the target question after the vectorization. Based on this, the documents associated with the target question are determined and obtained according to the target question after the vectorization and the first number of vectorized documents.
[0070] Specifically, according to the vectorization strategies corresponding to each target question after vectorization processing and each first number of vectorized documents, the similarity between each target question after vectorization processing and the corresponding first number of vectorized documents is calculated, and the documents related to the target question are determined and obtained based on the calculation results.
[0071] Among them, when the target problem is vectorized at least once, the specific method is as follows:
[0072] In the question-answering method provided in this embodiment, since the document-related set includes a document set and at least one vectorized document set, the first number of documents corresponding to the target question can be determined in the document set corresponding to the target question through the intent information corresponding to the target question, and at least one first number of vectorized documents corresponding to the target question can be further determined. Therefore, by vectorizing the target question at least once according to the vectorization strategy of each vectorized document set, the documents associated with the target question can be determined and obtained based on each processed target question and each first number of vectorized documents. Thus, multiple vectorization processing methods are considered to determine the documents associated with the target question, further improving the adaptability of the determined documents to the question.
[0073] As an optional embodiment, based on the third embodiment, the intent information of this embodiment is information for questioning and answering based on a specified document. When a first number of documents are determined in a corresponding document set according to the corresponding intent information, step b1 specifically includes the following steps:
[0074] The document identifier of each document in the corresponding document set is identified, and a first number of documents specified by the target question is determined according to each document identifier.
[0075] The document identifier is any identifier that represents the identity of the document, such as a document title, document number, etc.
[0076] In this embodiment, when the intention information is information for question-answering based on a specified document, it indicates that there are certain fields in the target question indicating specific document identifiers, such as "X" in "What is the main content disclosed by file X?"
[0077] Based on this, the fields in the target question are identified, and the document identifiers reflected by the above fields are determined. At the same time, the document identifiers of each document in the document collection are identified. Then, among each document identifier, the document corresponding to the document identifier reflected by the target question is obtained, that is, the first number of documents.
[0078] For example, if the target question is “What is the main content disclosed by document X?”, it can be determined that the document identifier reflected is “X”. After the document identifier of each document is identified, all documents containing “X” can be obtained.
[0079] The question-and-answer method provided in this embodiment, since the specific document identifier will be indicated in the target question when the intention information is information for asking and answering based on a specified document, the first number of documents specified by the target question can be determined based on each document identifier by identifying the document identifier of each document in the document set corresponding to the target question. Therefore, when the intention information is information for asking and answering based on a specified document, the first number of documents can be accurately determined.
[0080] As an optional embodiment, in this embodiment, based on the above embodiment, the vectorized document is obtained by vectorizing the document text and the document identifier. When determining and acquiring the document associated with the target question according to each processed target question and each first number of vectorized documents, step b3 specifically includes the following steps:
[0081] Step b31: Based on the vectorized document text and document identifier, respectively calculate the similarity between each processed target question and the corresponding first number of vectorized documents.
[0082] The document text is the specific content text contained in the document.
[0083] In this embodiment, the intention-based information is information for question and answer based on a specified document, and the vectorized document is a vectorized document obtained by vectorizing the document text and the document identifier.
[0084] Based on this, after obtaining each first number of vectorized documents and performing at least one vectorization process on the target question, the vectorized document text and document identifier in the vectorized document are further obtained, and the cosine similarity between the vectorized document text and document identifier and the processed target question is calculated to obtain the similarity corresponding to each first number of vectorized documents.
[0085] When calculating the cosine similarity between the vectorized document text and document identifier and the processed target question, the calculation is specifically performed between the target question and the vectorized document text and document identifier that use the same vectorization strategy.
[0086] Step b32: sorting the first number of vectorized documents according to the similarity, and merging the sorted first number of vectorized documents to obtain a final sorting result for each document.
[0087] In this embodiment, after obtaining the similarities corresponding to the first number of vectorized documents, the first number of vectorized documents are sorted from large to small or from small to large according to the similarities to obtain the sorted first number of vectorized documents. Based on this, the sorted first number of vectorized documents are merged.
[0088] Specifically, the first number of sorted vectorized documents are used as lists, that is, each list is composed of the first number of sorted vectorized documents, and then the inverse sorting fusion method is used to merge the lists into a sorted list.
[0089] It can be understood that the vectorized documents in each list are obtained by vectorizing the documents using the specified vectorization strategy. Therefore, the vectorized documents in each list reflect the documents in the document collection. After merging the lists, the final sorting results of each document can be obtained.
[0090] Among them, when the inverse sorting fusion method is used to merge the lists into one sorted list, the specific calculation formula is as follows:
[0091]
[0092] in:
[0093] d is the document;
[0094] k is the number of lists that need to be merged;
[0095] ranki(d) is the rank of document d in the i-th list. If the document does not exist in the i-th list, its rank can be considered infinite.
[0096] c is a constant (usually set to 60) used to reduce the impact of low-ranked vectorized documents;
[0097] RRF(d) is the final ranking result of the documents.
[0098] Step b33: Obtain a second number of documents according to the final ranking results of each document, and determine and obtain the second number of documents as documents associated with the target question.
[0099] The second number is a pre-configured number of documents associated with the target question that need to be obtained.
[0100] In this embodiment, after the final ranking results of each document are obtained, the second number of highest-ranked documents are obtained according to the final ranking results, and these documents are taken as documents associated with the target question and obtained.
[0101] The question-and-answer method provided in this embodiment, when the intention of the target question is to ask and answer questions with a specified document, the corresponding vectorized document is obtained by vectorizing the document text and the document identifier. Therefore, by respectively calculating the similarity between each processed target question and the corresponding first number of vectorized document texts and document identifiers, the corresponding first number of vectorized documents can be sorted according to the similarity. Thus, by merging the sorted first number of vectorized documents, the final sorting result of each document can be obtained, and the second number of documents can be obtained according to the final sorting result of each document, thereby determining and obtaining the document associated with the target question. Thus, multiple similarity calculation results are merged, so that when it is determined that the intention information is information for asking and answering questions with documents within a specified time range, the adaptability of the document associated with the target question and the question is improved.
[0102] As an optional embodiment, based on the above embodiment or the previous embodiment, this embodiment further includes the following steps before receiving the target question and answer request sent by the terminal:
[0103] Step e1: Obtain a document collection, and obtain the document text and document identifier corresponding to each document.
[0104] It is understandable that a prerequisite for the implementation of this solution is that the document-related collection can be successfully obtained. Based on this, based on the intention information, the document collection is obtained, and the document text and document identifier corresponding to each document are obtained.
[0105] In this embodiment, the R&D personnel may collect the files in PDF format in advance and parse the files in PDF format, such as Figure 2b As shown in the figure, the bounding box bbox and label label are first parsed from the file layout, and then the table, title, text, picture and other information are further parsed, and the categories are filtered out. At the same time, the characters and the coordinates corresponding to the characters are parsed, and the characters and the coordinates corresponding to the characters are matched with the bbox to obtain the corpus information corresponding to the file in PDF format.
[0106] Based on this, each corpus information is used as each document to form a document set, and the document text and document identifier are obtained from the corpus content reflected by each document.
[0107] Step e2: Pack each document identifier into a code block, and use at least one vectorization strategy to vectorize the document text and document identifier corresponding to each document to obtain at least one vectorized document set.
[0108] A code block is an executable code set.
[0109] It is understandable that when the intent information is information for question and answer based on a specified document, it is necessary to determine the document indicated by the target question based on the document identifier. Therefore, it is necessary to package each document identifier to obtain the code block corresponding to each document identifier.
[0110] Based on this, the code blocks corresponding to each document identifier are divided into smaller blocks, and the document texts are segmented at the same time. At least one vectorization strategy is obtained, and each vectorization strategy is used to vectorize the document texts and document identifiers corresponding to each document after segmentation to obtain at least one vectorized document set.
[0111] Step e3: Generate a document-related set based on the document set and at least one vectorized document set.
[0112] In this embodiment, after obtaining a document set and at least one vectorized document set, a document-related set is generated based on the document set and at least one vectorized document set, and when the intent information is obtained as information for question and answer based on a specified document, the corresponding document-related set is generated.
[0113] Based on this, the HTTP interface of the knowledge base can be called to bind the document-related collection to the corresponding partition of the knowledge base.
[0114] The question-and-answer method provided in this embodiment is based on the premise that the document-related set has been generated and stored as the premise for the execution of the solution. Therefore, by obtaining the document set and the document text and document identifier corresponding to each document, each document identifier can be packaged into a code block, and at least one vectorization strategy is used to vectorize the document text and document identifier corresponding to each document to obtain at least one vectorized document set. Therefore, by generating a document-related set based on the document set and at least one vectorized document set, the corresponding document-related set can be obtained when the intent information is information for question-and-answer based on a specified document. When it is subsequently determined that the intent information of the target question is information for question-and-answer based on a specified document, the document-related set corresponding to the target question can be obtained sequentially, thereby improving the success rate of question-and-answer.
[0115] As an optional embodiment, based on the third embodiment, the intent information of this embodiment is information for question-answering based on documents within a specified time range. When a first number of documents are determined in a corresponding document set according to the corresponding intent information, step b1 specifically includes the following steps:
[0116] The publishing time of each document in the corresponding document set is identified, and a first number of documents within a time range specified by the target question is determined based on each publishing time.
[0117] The publishing time is used to represent the specific time when the document content is published.
[0118] For example, if the content of a document is published on January 31, 2022, the publication time of the document is January 31, 2022.
[0119] It should be understood that the above examples are only for illustrative purposes and should not constitute any limitation to the present application.
[0120] In this embodiment, when the intent information is information for asking and answering questions based on documents within a specified time range, it means that there are certain fields in the target question that indicate documents within a certain time range, such as "January 2022" in "What industry trends are reflected in the documents released by Company A in January 2022?"
[0121] Based on this, the fields in the target question are identified, and the publishing time reflected by the above fields is determined. At the same time, the publishing time of each document in the document collection is identified, and then in each publishing time, the document corresponding to the publishing time reflected by the target question is obtained, that is, the first number of documents.
[0122] For example, if the target question is "What industry trends are reflected in the documents released by Company A in January 2022?", it can be determined that the release time reflected is "January 2022". After identifying the release time of each document, all documents released in January 2022 can be obtained.
[0123] The question-and-answer method provided in this embodiment, since the publication time of the document content will be indicated in the target question when the intention information is information for asking and answering questions based on documents within a specified time range, by identifying the publication time of each document in the corresponding document collection, the first number of documents within the time range specified by the target question can be determined based on each publication time. Therefore, when the intention information is information for asking and answering questions based on documents within a specified time range, the first number of documents can be accurately determined.
[0124] As an optional embodiment, in this embodiment, based on the above embodiment, the vectorized document is obtained by vectorizing the document text, the document identifier and the document publishing time. When determining and acquiring the document associated with the target question according to each processed target question and each first number of vectorized documents, step b3 specifically includes the following steps:
[0125] Step b3a: Based on the vectorized document text, document identifier and document publishing time, respectively calculate the similarity between each processed target question and the corresponding first number of vectorized documents.
[0126] The document publishing time is the time when the document content is published.
[0127] In this embodiment, the intent-based information is information for question and answer based on documents within a specified time range, and the vectorized document is a vectorized document obtained by vectorizing the document text, document identifier, and document release time.
[0128] Based on this, after obtaining each first number of vectorized documents and performing at least one vectorization process on the target question, the vectorized document text, document identifier and document release time in the vectorized document are further obtained, and the cosine similarity between the vectorized document text, document identifier and document release time and the processed target question is calculated to obtain the similarity corresponding to each first number of vectorized documents. The specific execution method is similar to step b31 and will not be repeated here.
[0129] Step b3b: Determine the final ranking results of each document according to the similarity, and determine and obtain the documents related to the target question according to the final ranking results of each document.
[0130] In this embodiment, after obtaining the similarities corresponding to the first number of vectorized documents, the first number of vectorized documents are sorted according to the similarities, and the sorted first number of vectorized documents are merged to obtain the final sorting results of each document. Further, the second number of documents are obtained according to the final sorting results of each document, and the second number of documents are determined as documents associated with the target question and obtained. The specific execution method is similar to step b32 to step b33, and will not be repeated here.
[0131] In the question-answering method provided in this embodiment, when the intent of the target question is to ask and answer information based on documents within a specified time range, the corresponding vectorized document is obtained by vectorizing the document text, document identifier, and document release time. Therefore, by respectively calculating the similarity between each processed target question and the corresponding first number of vectorized document texts, document identifiers, and document release times, the final sorting result of each document can be determined according to the similarity, and the document associated with the target question can be determined and obtained according to the final sorting result of each document. Therefore, when it is determined that the intent information is to ask and answer information based on documents within a specified time range, the adaptability of the document associated with the target question and the question is improved.
[0132] As an optional embodiment, based on the above embodiment or the previous embodiment, this embodiment further includes the following steps before receiving the target question and answer request sent by the terminal:
[0133] Step f1: Get a document collection, and get the document text, document identifier and document publishing time corresponding to each document.
[0134] A prerequisite for executing this solution is that the document-related collection can be successfully obtained. When the intent information is information about questions and answers based on documents within a specified time range, the document collection is obtained, and the document text, document identifier, and document release time corresponding to each document are obtained. The specific execution method is similar to step e1 and will not be repeated here.
[0135] Step f2: Pack code blocks based on the document identifier and document publishing time corresponding to each document, and use at least one vectorization strategy to vectorize the document text, document identifier and document publishing time corresponding to each document to obtain at least one vectorized document set.
[0136] It can be understood that when the intent information is information for question and answer based on documents within a specified time range, it is necessary to determine the document indicated by the target question based on the specific release time of the document content. Therefore, it is necessary to package the document identifier corresponding to each document and the corresponding document release time into a code block.
[0137] Based on this, the above code block is divided into smaller blocks, and each document text is segmented at the same time. At least one vectorization strategy is obtained, and each vectorization strategy is used to vectorize the document text, document identifier and document release time corresponding to each document after segmentation to obtain at least one vectorized document set.
[0138] Step f3: Generate a document-related set based on the document set and at least one vectorized document set.
[0139] In this embodiment, after obtaining a document set and at least one vectorized document set, a document-related set is generated based on the document set and at least one vectorized document set, and when the intent information is obtained as information for question and answer based on documents within a specified time range, the corresponding document-related set is obtained.
[0140] Based on this, the HTTP interface of the knowledge base can be called to bind the document-related collection to the corresponding partition of the knowledge base.
[0141] The question-and-answer method provided in this embodiment is based on the premise that a document-related set has been generated and stored as a prerequisite for the execution of the solution. Therefore, by obtaining a document set and obtaining the document text, document identifier, and document release time corresponding to each document, each document identifier and the corresponding document release time can be packaged into a code block, and at least one vectorization strategy is used to vectorize the document text, document identifier, and document release time corresponding to each document to obtain at least one vectorized document set. Therefore, by generating a document-related set based on a document set and at least one vectorized document set, the corresponding document-related set can be obtained when the intent information is information for question-and-answer based on documents within a specified time range. When it is subsequently determined that the intent information of the target question is information for question-and-answer based on documents within a specified time range, the document-related set corresponding to the target question can be sequentially obtained, further improving the success rate of question-and-answer.
[0142] As an optional embodiment, based on the third embodiment, the intention information of this embodiment is information for questioning and answering without specifying a document. When a first number of documents are determined in a corresponding document set according to the corresponding intention information, step b1 specifically includes the following steps:
[0143] A first number of documents are randomly determined in the corresponding document set.
[0144] In this embodiment, when the intention information is information for questioning without specifying a document, indicating that the target question does not indicate any content, a first number of documents are randomly obtained from the document collection.
[0145] For example, if the target question is “What is the development history of Company A?”, a first number of documents may be randomly obtained from the document collection.
[0146] The question-and-answer method provided in this embodiment can successfully obtain the first number of documents by randomly determining a first number of documents in a document collection when the intention information is for asking questions without specifying documents, thereby further improving the success rate of the question-and-answer method.
[0147] As an optional embodiment, in this embodiment, based on the above embodiment, the vectorized document is obtained by vectorizing the document text. When determining and acquiring the document associated with the target question according to each processed target question and each first number of vectorized documents, step b3 specifically includes the following steps:
[0148] Step b3A: Based on the vectorized document text, respectively calculate the similarity between each processed target question and the corresponding first number of vectorized documents.
[0149] In this embodiment, the intention-based information is information for question-answering without specifying a document, and the vectorized document is a vectorized document obtained only from the document text.
[0150] Based on this, after obtaining each first number of vectorized documents and performing at least one vectorization process on the target question, the vectorized document text in the vectorized document is further obtained, and the cosine similarity between the vectorized document text and the processed target question is calculated to obtain the similarity corresponding to each first number of vectorized documents. The specific execution method is similar to step b31 and will not be repeated here.
[0151] Step b3B: Determine the final ranking results of each document according to the similarity, and determine and obtain the documents related to the target question according to the final ranking results of each document.
[0152] In this embodiment, after obtaining the similarities corresponding to the first number of vectorized documents, the first number of vectorized documents are sorted according to the similarities, and the sorted first number of vectorized documents are merged to obtain the final sorting results of each document. Further, the second number of documents are obtained according to the final sorting results of each document, and the second number of documents are determined as documents associated with the target question and obtained. The specific execution method is similar to step b32 to step b33, and will not be repeated here.
[0153] In the question-answering method provided in this embodiment, when the intention of the target question is to ask and answer questions without specifying a document, the corresponding vectorized document is obtained by vectorizing the document text. Therefore, by respectively calculating the similarity between each processed target question and the corresponding first number of vectorized document texts, the final sorting result of each document can be determined according to the similarity, and the document associated with the target question can be determined and obtained according to the final sorting result of each document. Therefore, when it is determined that the intention information is information for asking and answering questions without specifying a document, the adaptability of the document associated with the target question and the question is improved.
[0154] As an optional embodiment, based on the above embodiment or the previous embodiment, this embodiment further includes the following steps before receiving the target question and answer request sent by the terminal:
[0155] Step g1: Get a document collection and get the document text corresponding to each document.
[0156] Similarly, the document-related collection is successfully obtained based on the need. When the intention information is information for question-answering without specifying a document, the document collection is obtained, and the document text corresponding to each document is obtained. The specific execution method is similar to step e1 and will not be repeated here.
[0157] Step g2: Use at least one vectorization strategy to perform vectorization processing on the document text corresponding to each document to obtain at least one vectorized document set.
[0158] It can be understood that when the intent information is information for questioning without specifying a document, there is no need to consider the indicative content in the target question. Therefore, each document text is directly segmented and processed, and at least one vectorization strategy is obtained. Each vectorization strategy is used to vectorize the segmented document texts to obtain at least one vectorized document set.
[0159] Step g3: Generate a document-related set based on the document set and at least one vectorized document set.
[0160] In this embodiment, after obtaining a document set and at least one vectorized document set, a document-related set is generated based on the document set and at least one vectorized document set, and when the intention information is obtained as information for asking and answering questions without specifying a document, the corresponding document-related set is generated.
[0161] Based on this, the HTTP interface of the knowledge base can be called to bind the document-related collection to the corresponding partition of the knowledge base.
[0162] The question-and-answer method provided in this embodiment is based on the premise that a document-related set has been generated and stored as a prerequisite for the execution of the solution. Therefore, by obtaining a document set and obtaining the document text corresponding to each document, at least one vectorization strategy can be used to vectorize the document text corresponding to each document to obtain at least one vectorized document set. Thus, by generating a document-related set based on a document set and at least one vectorized document set, a corresponding document-related set can be obtained when the intent information is information for question-and-answer without specifying a document. This allows the document-related set corresponding to the target question to be obtained sequentially when it is determined that the intent information of the target question is information for question-and-answer without specifying a document, thereby improving the success rate of question-and-answer.
[0163] Figure 3a A complete flow chart of the question-answering method provided in the embodiment of the present application, such as Figure 3a As shown, the complete implementation process of this scheme is schematically illustrated.
[0164] S301a, obtaining a document set, and obtaining the document text and document identifier corresponding to each document.
[0165] S302a, packaging each document identifier into a code block, and using at least one vectorization strategy to vectorize the document text and document identifier corresponding to each document to obtain at least one vectorized document set.
[0166] S303a: Generate a document-related set based on the document set and at least one vectorized document set.
[0167] S304: Receive a target question and answer request sent by a terminal; the target question and answer request includes a target question.
[0168] S305. Determine the intent information corresponding to the target question based on a preset intent recognition model; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence.
[0169] S306: Determine a relevant set of documents corresponding to the target question based on the intent information.
[0170] The document-related set includes a document set and at least one vectorized document set.
[0171] S307a, in response to the intention information being information about question and answer based on a specified document, identifying the document identifier of each document in the corresponding document set, and determining a first number of documents specified by the target question based on each document identifier.
[0172] S308. Based on the first number of documents, respectively obtain the first number of vectorized documents in at least one corresponding vectorized document set.
[0173] S309: Perform vectorization processing on the target problem at least once.
[0174] S310a, based on the vectorized document text and document identifier, respectively calculate the similarity between each processed target question and the corresponding first number of vectorized documents.
[0175] S311a, sorting the first number of vectorized documents according to the similarity, and merging the sorted first number of vectorized documents to obtain the final sorting result of each document; obtaining the second number of documents according to the final sorting result of each document, and determining the second number of documents as documents associated with the target problem and obtaining them.
[0176] S312: Determine an answer to the target question in the document associated with the target question, and send the answer to the terminal.
[0177] Optional,
[0178] S301b, obtaining a document collection, and obtaining the document text, document identifier, and document publishing time corresponding to each document.
[0179] S302b, packaging code blocks based on the document identifiers and document publishing time corresponding to each document, and using at least one vectorization strategy to vectorize the document text, document identifiers and document publishing time corresponding to each document to obtain at least one vectorized document set.
[0180] S303b: Generate a document-related set based on the document set and at least one vectorized document set.
[0181] S304: Receive a target question and answer request sent by a terminal; the target question and answer request includes a target question.
[0182] S305. Determine the intent information corresponding to the target question based on a preset intent recognition model; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence.
[0183] S306: Determine a relevant set of documents corresponding to the target question based on the intent information.
[0184] The document-related set includes a document set and at least one vectorized document set.
[0185] S307b, in response to the intention information being information about asking and answering questions based on documents within a specified time range, identifying the release time of each document in the corresponding document collection, and determining a first number of documents within the time range specified by the target question based on each release time.
[0186] S308. Based on the first number of documents, respectively obtain the first number of vectorized documents in at least one corresponding vectorized document set.
[0187] S309: Perform vectorization processing on the target problem at least once.
[0188] S310b, based on the vectorized document text, document identifier and document publishing time, respectively calculate the similarity between each processed target question and the corresponding first number of vectorized documents.
[0189] S311b, determining the final ranking results of each document according to the similarity, and determining and acquiring the documents associated with the target question according to the final ranking results of each document.
[0190] S312: Determine an answer to the target question in the document associated with the target question, and send the answer to the terminal.
[0191] Optional,
[0192] S301c, obtaining a document collection, and obtaining the document text corresponding to each document.
[0193] S302c: Use at least one vectorization strategy to perform vectorization processing on the document text corresponding to each document to obtain at least one vectorized document set.
[0194] S303c: Generate a document-related set based on the document set and at least one vectorized document set.
[0195] S304: Receive a target question and answer request sent by a terminal; the target question and answer request includes a target question.
[0196] S305. Determine the intent information corresponding to the target question based on a preset intent recognition model; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence.
[0197] S306: Determine a relevant set of documents corresponding to the target question based on the intent information.
[0198] The document-related set includes a document set and at least one vectorized document set.
[0199] S307c: In response to the intention information that the question and answer is not to specify a document, randomly determine a first number of documents in the corresponding document set.
[0200] S308. Based on the first number of documents, respectively obtain the first number of vectorized documents in at least one corresponding vectorized document set.
[0201] S309: Perform vectorization processing on the target problem at least once.
[0202] S310c, based on the vectorized document text, respectively calculating the similarity between each processed target question and the corresponding first number of vectorized documents.
[0203] S311c, determining the final ranking results of each document according to the similarity, and determining and acquiring the documents associated with the target question according to the final ranking results of each document.
[0204] S312: Determine an answer to the target question in the document associated with the target question, and send the answer to the terminal.
[0205] Figure 3b The architecture diagram of the question-answering method provided in the embodiment of the present application is as follows: Figure 3bAs shown in the figure, the bottom layer is the storage component layer, which can specifically include Kafka, Minio, TiDB and the vector database Milvus, which is used to store knowledge base metadata, PDF-formatted files and corresponding metadata, document-related collections, etc.; the middle layer is the knowledge base engine layer, which mainly manages and retrieves document-related collections in the partitions corresponding to the knowledge base; the top layer is the business layer, which builds two core business flows by calling the HTTP interface of the knowledge base engine layer: warehousing and retrieval.
[0206] Among them, the warehousing process includes: uploading document-related collections, creating a knowledge base and binding the document-related collections to the knowledge base; the retrieval process includes: chatbot-based retrieval, that is, determining the file-related collection corresponding to the target question based on the intent information and further determining the retrieval of documents related to the target question, and summarization generation based on the large language model LLM, that is, the process of interacting the documents related to the target question with the large language model to generate a reply.
[0207] Figure 4 A schematic diagram of the structure of the question-answering device provided in the embodiment of the present application is shown in FIG. Figure 4 As shown, the execution subject of the present application is a question-and-answer device 40 , which is specifically located in an electronic device. The question-and-answer device provided in this embodiment includes: a transceiver module 41 and a processing module 42 .
[0208] Among them, the transceiver module 41 is used to receive a target question and answer request sent by the terminal; the target question and answer request includes a target question; the processing module 42 is used to determine the intent information corresponding to the target question based on a preset intent recognition model, and determine the document-related set corresponding to the target question according to the intent information; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence; based on the target question, searching in the corresponding document-related set to obtain documents related to the target question; determining the answer to the target question in the documents related to the target question; the transceiver module 41 is also used to send the answer to the terminal.
[0209] Optionally, the document-related set includes a document set and at least one vectorized document set;
[0210] Accordingly, the processing module 42, when searching in the corresponding document related set based on the target question to obtain the document associated with the target question, is specifically configured to:
[0211] Determine a first number of documents in a corresponding document collection according to corresponding intent information; based on the first number of documents, obtain a first number of vectorized documents in at least one corresponding vectorized document collection; perform vectorization processing on the target question at least once, and determine and obtain documents associated with the target question based on each processed target question and each first number of vectorized documents.
[0212] Optionally, the intent information is information for question and answer based on a specified document;
[0213] Accordingly, when determining the first number of documents in the corresponding document set according to the corresponding intention information, the processing module 42 is specifically configured to:
[0214] The document identifier of each document in the corresponding document set is identified, and a first number of documents specified by the target question is determined according to each document identifier.
[0215] Optionally, the vectorized document is obtained by vectorizing the document text and the document identifier;
[0216] Accordingly, the processing module 42, when determining and acquiring documents associated with the target question according to each processed target question and each first number of vectorized documents, is specifically configured to:
[0217] Based on the vectorized document text and document identifier, the similarity between each processed target question and the corresponding first number of vectorized documents is calculated respectively; the corresponding first number of vectorized documents are sorted according to the similarity, and the sorted first number of vectorized documents are merged to obtain the final sorting result of each document; according to the final sorting result of each document, a second number of documents are obtained, and the second number of documents are determined as documents associated with the target question and obtained.
[0218] Optionally, the processing module 42 is also used to obtain a document collection and obtain the document text and document identifier corresponding to each document before the transceiver module 41 receives the target question and answer request sent by the terminal; package each document identifier into a code block, and use at least one vectorization strategy to vectorize the document text and document identifier corresponding to each document to obtain at least one vectorized document collection; generate a document-related collection based on the document collection and at least one vectorized document collection.
[0219] Optionally, the intent information is information for question and answer based on documents within a specified time range;
[0220] Accordingly, when determining the first number of documents in the corresponding document set according to the corresponding intention information, the processing module 42 is specifically configured to:
[0221] The publishing time of each document in the corresponding document set is identified, and a first number of documents within a time range specified by the target question is determined based on each publishing time.
[0222] Optionally, the vectorized document is obtained by vectorizing the document text, the document identifier, and the document publishing time;
[0223] Accordingly, the processing module 42, when determining and acquiring documents associated with the target question according to each processed target question and each first number of vectorized documents, is specifically configured to:
[0224] Based on the vectorized document text, document identifier and document publishing time, the similarities between each processed target question and the corresponding first number of vectorized documents are calculated respectively; the final sorting result of each document is determined according to the similarity, and the documents associated with the target question are determined and obtained according to the final sorting result of each document.
[0225] Optionally, the processing module 42 is also used to obtain a document collection and obtain the document text, document identifier and document release time corresponding to each document before the transceiver module 41 receives the target question and answer request sent by the terminal; package the code blocks based on the document identifier and document release time corresponding to each document, and use at least one vectorization strategy to vectorize the document text, document identifier and document release time corresponding to each document to obtain at least one vectorized document collection; generate a document-related collection based on the document collection and at least one vectorized document collection.
[0226] Optionally, the intent information is information for questioning and answering without specifying a document;
[0227] Accordingly, when determining the first number of documents in the corresponding document set according to the corresponding intention information, the processing module 42 is specifically configured to:
[0228] A first number of documents are randomly determined in the corresponding document set.
[0229] Optionally, the vectorized document is obtained by vectorizing the document text;
[0230] Accordingly, the processing module 42, when determining and acquiring documents associated with the target question according to each processed target question and each first number of vectorized documents, is specifically configured to:
[0231] Based on the vectorized document text, the similarities between each processed target question and the corresponding first number of vectorized documents are calculated respectively; the final sorting result of each document is determined according to the similarity, and the documents associated with the target question are determined and obtained according to the final sorting result of each document.
[0232] Optionally, the processing module 42 is also used to obtain a document collection and a document text corresponding to each document before the transceiver module 41 receives the target question and answer request sent by the terminal; use at least one vectorization strategy to vectorize the document text corresponding to each document to obtain at least one vectorized document collection; and generate a document-related collection based on the document collection and at least one vectorized document collection.
[0233] The question-and-answer device provided in the embodiment of the present application can be used to execute the technical solution of the question-and-answer method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.
[0234] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the transceiver module 41 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above transceiver module 41. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
[0235] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 may include: a processor 51 , a memory 52 , and a transceiver 53 .
[0236] The processor 51 executes the computer execution instructions stored in the memory, so that the processor 51 executes the scheme in the above embodiment. The processor 51 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0237] The memory 52 is connected to the processor 51 through the system bus and completes the communication between them. The memory 52 is used to store computer program instructions. The transceiver 53 is used to send and receive data.
[0238] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.
[0239] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the question-and-answer method in the above embodiment.
[0240] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the question-and-answer method of the above embodiment.
[0241] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the question-and-answer method in the above embodiment.
[0242] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.
[0243] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.
[0244] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
[0245] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.
[0246] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
[0247] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.
[0248] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0249] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.
[0250] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.
[0251] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0252] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0253] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A question-answering method, characterized in that: The method comprises: Receiving a target question-and-answer request sent by a terminal; the target question-and-answer request includes a target question; Based on a preset intent recognition model, determine the intent information corresponding to the target question, and determine the document-related set corresponding to the target question according to the intent information; the preset intent recognition model is obtained by training a plurality of preset questions marked with corresponding intent information until convergence; Searching in a corresponding document related collection based on the target question to obtain documents related to the target question; An answer to the target question is determined in the document associated with the target question, and the answer is sent to the terminal.
2. The method according to claim 1, characterized in that The document-related set includes a document set and at least one vectorized document set; The step of searching in a corresponding document related set based on the target question to obtain a document associated with the target question includes: Determining a first number of documents in a corresponding document set according to the corresponding intent information; Based on the first number of documents, respectively obtain a first number of vectorized documents in at least one corresponding vectorized document set; The target question is vectorized at least once, and documents associated with the target question are determined and acquired according to each processed target question and each first number of vectorized documents.
3. The method according to claim 2, characterized in that The intention information is information for question and answer based on a specified document; The determining a first number of documents in a corresponding document set according to the corresponding intent information includes: The document identifier of each document in the corresponding document set is identified, and a first number of documents specified by the target question is determined according to each document identifier.
4. The method according to claim 3, characterized in that The vectorized document is obtained by vectorizing the document text and the document identifier; The step of determining and acquiring documents related to the target questions according to the processed target questions and the first number of vectorized documents includes: Based on the vectorized document text and the document identifier, respectively calculating the similarity between each processed target question and the corresponding first number of vectorized documents; Sorting the first number of vectorized documents according to the similarity, and merging the sorted first number of vectorized documents to obtain a final sorting result of each document; A second number of documents is obtained according to the final ranking result of each document, and the second number of documents are determined as documents associated with the target question and obtained.
5. The method according to claim 3 or 4, characterized in that: Before receiving the target question-answer request sent by the terminal, the method further includes: Get a document collection, and get the document text and document identifier corresponding to each document; Packing each document identifier into a code block, and using at least one vectorization strategy to vectorize the document text and the document identifier corresponding to each document to obtain at least one vectorized document set; A document related set is generated based on the document set and the at least one vectorized document set.
6. The method according to claim 2, characterized in that The intention information is information for question and answer based on documents within a specified time range; The determining a first number of documents in a corresponding document set according to the corresponding intent information includes: The publishing time of each document in the corresponding document set is identified, and a first number of documents within a time range specified by the target question is determined based on each publishing time.
7. The method according to claim 6, characterized in that The vectorized document is obtained by vectorizing the document text, document identifier and document publishing time; The step of determining and acquiring documents related to the target questions according to the processed target questions and the first number of vectorized documents includes: Based on the vectorized document text, document identifier, and document publishing time, respectively calculating the similarity between each processed target question and the corresponding first number of vectorized documents; The final ranking result of each document is determined according to the similarity, and the document associated with the target question is determined and obtained according to the final ranking result of each document.
8. The method according to claim 6 or 7, characterized in that: Before receiving the target question-answer request sent by the terminal, the method further includes: Get the document collection, and get the document text, document ID and document publishing time corresponding to each document; Packing the code blocks based on the document identifiers and document publishing time corresponding to each document, and using at least one vectorization strategy to vectorize the document text, document identifier, and document publishing time corresponding to each document to obtain at least one vectorized document set; A document related set is generated based on the document set and the at least one vectorized document set.
9. The method according to claim 2, characterized in that: The intention information is information for questioning without specifying a document; The determining a first number of documents in a corresponding document set according to the corresponding intent information includes: A first number of documents are randomly determined in the corresponding document set.
10. The method according to claim 9, characterized in that The vectorized document is obtained by vectorizing the document text; The step of determining and acquiring documents related to the target questions according to the processed target questions and the first number of vectorized documents includes: Based on the vectorized document text, respectively calculating the similarity between each processed target question and the corresponding first number of vectorized documents; The final ranking result of each document is determined according to the similarity, and the document associated with the target question is determined and obtained according to the final ranking result of each document.
11. The method according to claim 9 or 10, characterized in that: Before receiving the target question-answer request sent by the terminal, the method further includes: Get a document collection and get the document text corresponding to each document; Using at least one vectorization strategy to vectorize the document text corresponding to each document to obtain at least one vectorized document set; A document related set is generated based on the document set and the at least one vectorized document set.
12. An electronic device comprising: A processor, a memory and a transceiver communicatively connected to the processor; The memory stores computer-executable instructions; the transceiver is used to send and receive data; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 11.
13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 11 when being executed by a processor.
Citation Information
Patent Citations
Question answer retrieval processing method and device
CN115544230A
Question reply method and device based on AI big language model, equipment and medium
CN118350465A
Cited By
Retrieval-augmented generation method and computing device
WO2026129656A1
