Intelligent question answering system based on large language model
By adopting the double tower semantic model and discriminant model in the intelligent question-and-answer system, the problem of insufficient word order information and long text matching needs is solved, and the semantic recall and correlation judgment are improved, and the user experience and reply accuracy are improved.
Patent Information
- Application Number
- CN202411219725.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-13
AI Technical Summary
The existing intelligent question and answer system has insufficient word order information in semantic understanding and generation, and cannot effectively distinguish the same word semantics in different contexts, and is not effective in the long text matching needs, especially in the automotive field.
Using an intelligent question-and-answer system based on large language models, including search recall module, correlation sorting module and large language model summary module, through question-and-answer query and document paragraph-level similarity model and discriminant model, Bert+PairWise and Bert+PointWise are used Training, build a double tower semantic model, and optimize semantic recall and correlation judgment.
It improves semantic recall effect and correlation judgment, can effectively recall and sort out relevant document paragraphs, improves the accuracy and user experience of answering questions, and meets the performance and effect requirements of the online system.
Smart Images
Figure CN119128094A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of intelligent question answering, and specifically relates to an intelligent question answering system based on a large language model (Large Language Model). Background Art
[0002] At present, the technology of large language models is becoming more and more mature, such as "GPT3.5" and "GPT4" proposed by OpenAI, "ChatGLM" of Tsinghua Zhipu AI, "Wenxin Yiyan" of Baidu, "MOSS" of Fudan University, "HunYuan Big Model" of Tencent, "Tongyi Big Model" of Alibaba, "Pangu Big Model" of Huawei, etc. They are mainly based on key technologies such as natural language understanding (NLU) and natural language generation (NLG), and the effect of non-factual question and answer depends on the quality of the question and answer library and the effect of the semantic retrieval model. They are mainly based on encyclopedia data, question and answer content generated by users on the platform, and unstructured data mining to generate frequently occurring questions and answers to construct FAQ index content.
[0003] The current intelligent question-answering system will recall the answer title based on the question query and semantics. The method for judging the semantic similarity is implemented by Word2Vec and the dual-tower model.
[0004] However, the Word2Vec model cannot fully consider the word order information of a sentence, and the text vectors of the same words in different contexts are fixed. For example, the word "apple" can refer to both an Apple phone and an Apple fruit depending on the context. In the vertical search field of Autohome, sentences that appear in the same level of car series have similar patterns, such as "BMW x3" and "BMW x5", which represent different car series, and the Word2Vec model cannot make semantic distinctions.
[0005] RNN can alleviate the long text dependency problem to a certain extent, mainly using the LSTM network structure. However, LSTM does not perfectly solve the long dependency problem at the sentence order level. In the automotive field, it cannot meet the long text matching needs, so it needs to be improved and optimized. Summary of the invention
[0006] The purpose of the present invention is to provide an intelligent question answering system based on a large language model (Large Language Model) to solve the problems raised in the above background technology.
[0007] In order to achieve the above object, the present invention provides the following technical solution: an intelligent question-answering system based on a large language model (Large Language Model), wherein the intelligent question-answering system is composed of a retrieval and recall module, a relevance ranking module and a large language model summarization module:
[0008] The retrieval and recall module includes a similarity model between the question-answer query and the document paragraph level; the relevance ranking module includes a discriminant model between the question-answer query and the document paragraph level; and the large language model summarization module is implemented based on the AutohomeChat large language model.
[0009] Preferably, in the retrieval recall module, the training method of the similarity model between the question-answer query and the document paragraph level is:
[0010] A1, obtain the user exposure click information from the search log table of Autohome. This information includes the query information searched by the user, the returned doc information and the location information;
[0011] A2, stores the searched information in table A;
[0012] A3, preliminarily screen the positive sample data and each query from Table A, extract the first 5 docs as positive samples according to the doc position, and store the docs in Table B by paragraph;
[0013] A4, sends the data in B to the annotator for annotation, and stores the annotated data in table C for future use;
[0014] A5, read the samples in Table C, for the positive sample (query, doc+), randomly select doc information from other queries as the negative sample doc- of the current query;
[0015] A6, construct PairWise training sample pairs (query, doc+, doc-), marked as dataset dataset_1;
[0016] A7, construct PointWise training pairs, (query, doc+, 1) and (query, doc-, 0), and mark them as dataset dataset_2;
[0017] A8 uses the dataset dataset_1 in A6 and inputs it into the similarity model between question and answer query and document paragraph level to train the question and answer query and document paragraph similarity model model_1.
[0018] A9: Output the semantic vector of the query through step A8, and then generate the semantic vector of the doc.
[0019] Preferably, the vector formation formula of the semantic vector is:
[0020] h 1 =tanh(W[h 0 ,y 0, c]+b)
[0021] o 1 =softmax(Vh 1 +c)
[0022] h 2 =tanh(W[h 1 ,y 1 , c]+b)
[0023] o 2 =softmax(Vh 2 +c)
[0024] …
[0025] h t =tanh(W[h t-1 ,y t-1 , c]+b)
[0026] o t =softmax(Vh t +c)
[0027]
[0028] Preferably, in the relevance ranking module, the training method of the discriminant model at the question-answer query and document paragraph level is:
[0029] B1, using the dataset dataset_2 in A7, train the question-answering query and document paragraph discrimination models;
[0030] B2, taking the query and doc paragraph as the input of model_2, and judging whether the current query and doc paragraph are related;
[0031] B3, obtains docs similar to the current query through information acquisition steps A1, A2 and A3;
[0032] B4, by setting score>single threshold, filter irrelevant doc paragraphs and obtain doc paragraphs similar to the query;
[0033] B5 then uses the query and similar doc paragraph pairs as inputs to B2's discriminant model model_2, outputs the score, and then performs fine ranking based on the score.
[0034] Preferably, the AutohomeChat language model can receive similar doc paragraphs uploaded in the relevance sorting module, and summarize the answers that meet the user's query requirements.
[0035] Preferably, the retrieval recall module recalls relevant doc paragraphs by training a dual-tower semantic model, and a sample consists of three parts, namely, question and answer query, answer doc+, and answer doc-.
[0036] Preferably, the relevance ranking module trains the discriminant model at the question-answer query and document paragraph level, and then finely ranks the recalled relevant doc paragraphs, and a sample consists of three parts, namely, question-answer query, answer doc+, answer doc-, label (0 / 1).
[0037] Preferably, the similarity model between the question-answer query and the document paragraph level is trained based on Bert+PairWise, and the discrimination model between the question-answer query and the document paragraph level is trained based on Bert+PointWise.
[0038] The beneficial effects of the present invention are as follows:
[0039] 1. The present invention constructs a similarity model between the question-answer query and the document paragraph level, and optimizes the semantic model in the automotive field by training the dual-tower semantic model. It can effectively recall the relevant document paragraphs of the query, improve the effect of semantic recall, and aggregate and summarize answers that meet user needs, meeting the dual requirements of online system performance and effect, thereby improving user experience.
[0040] 2. The present invention trains the discriminant model at the question-answer query and document paragraph level to make relevance judgment on the recalled documents, filter out irrelevant documents, and thus improve the relevance of the documents. It can judge the relevance of the question query and the entire answer content, and sort the recalled relevant doc paragraphs, so that the AutohomeChat language model can summarize the correct answers from several relevant document paragraphs, thereby improving the accuracy of answering questions and ensuring the relevance of answers.
[0041] 3. The present invention trains the AutohomeChat language model to achieve good results in content understanding and generation. It can summarize concise and clear summary information based on the contents of several documents provided by users, thereby ensuring user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a flow chart of the intelligent question-answering system of the present invention;
[0043] Figure 2 This is a framework diagram of the similarity model between the question-answering query and the document paragraph level of the present invention;
[0044] Figure 3This is a network structure diagram of the double-tower model of the present invention. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] like Figures 1 to 3 As shown, the embodiment of the present invention provides an intelligent question answering system based on a large language model (Large Language Model), and the intelligent question answering system is composed of a retrieval and recall module, a relevance sorting module and a large language model induction and summary module:
[0047] The retrieval and recall module includes a similarity model between the question and answer query and the document paragraph level; the relevance ranking module includes a discriminant model between the question and answer query and the document paragraph level; and the large language model induction and summary module is implemented based on the AutohomeChat large language model.
[0048] By constructing a similarity model between question-answering queries and document paragraph levels, we can effectively recall relevant document paragraphs of the query. Then, by training the discriminant model between question-answering queries and document paragraph levels, we can effectively rank the relevant paragraphs. Finally, by sending the relevant paragraphs into the AutohomeChat language model for aggregation and induction, we can effectively summarize the correct answers and provide users with a satisfactory experience.
[0049] Among them, in the retrieval recall module, the training method of the similarity model between the question-answer query and the document paragraph level is:
[0050] A1, obtain the user exposure click information from the search log table of Autohome. This information includes the query information searched by the user, the returned doc information and the location information;
[0051] A2, stores the searched information in table A;
[0052] A3, preliminarily screen the positive sample data and each query from Table A, extract the first 5 docs as positive samples according to the doc position, and store the docs in Table B by paragraph.
[0053] A4, sends the data in B to the annotator for annotation, and stores the annotated data in table C for future use;
[0054] A5, read the samples in Table C, for the positive sample (query, doc+), randomly select doc information from other queries as the negative sample doc- of the current query;
[0055] A6, construct PairWise training sample pairs (query, doc+, doc-), marked as dataset dataset_1;
[0056] A7, construct PointWise training pairs, (query, doc+, 1) and (query, doc-, 0), and mark them as dataset dataset_2;
[0057] A8 uses the dataset dataset_1 in A6 and inputs it into the similarity model between question and answer query and document paragraph level to train the question and answer query and document paragraph similarity model model_1.
[0058] A9: Output the semantic vector of the query through step A8, and then generate the semantic vector of the doc.
[0059] By building a similarity model between question-and-answer queries and document paragraphs, and optimizing the semantic model in the automotive field by training the dual-tower semantic model, we can effectively recall relevant document paragraphs of the query, improve the effect of semantic recall, and aggregate and summarize answers that meet user needs, meeting the dual requirements of online system performance and effect, thereby improving user experience.
[0060] Among them, the vector formation formula of the semantic vector is:
[0061] h 1 =tanh(W[h 0 ,y 0 , c]+b)
[0062] o 1 =softmax(Vh 1 +c)
[0063] h 2 =tanh(W[h 1 ,y 1 , c]+b)
[0064] o 2 =softmax(Vh 2 +c)
[0065] …
[0066] h t =tanh(W[h t-1 ,y t-1 , c]+b)
[0067] o t =softmax(Vh t +c)
[0068]
[0069] Among them, in the relevance ranking module, the training method of the discriminant model at the question-answer query and document paragraph level is:
[0070] B1, use the dataset dataset_2 in A7 to train the question-answering query and document paragraph discrimination model;
[0071] B2, take the query and doc paragraph as the input of model_2 to determine whether the current query and doc paragraph are related;
[0072] B3, obtains docs similar to the current query through information acquisition steps A1, A2 and A3;
[0073] B4, by setting score>single threshold, filter irrelevant doc paragraphs and obtain doc paragraphs similar to the query;
[0074] B5 then uses the query and similar doc paragraph pairs as inputs to the discriminant model model_2 of B2, outputs the score, and then performs fine ranking based on the score.
[0075] By training the discriminant model at the question-answer query and document paragraph level, it can judge the relevance of the recalled documents, filter out irrelevant documents, and thus improve the relevance of the documents. It can judge the relevance of the question query and the entire answer content, and sort the recalled relevant doc paragraphs, so that the AutohomeChat language model can summarize the correct answers from several relevant document paragraphs, thereby improving the accuracy of answering questions and ensuring the relevance of answers.
[0076] Among them, the home language model AutohomeChat can receive similar doc paragraphs uploaded in the relevance sorting module, and summarize the answers that meet the user's query requirements.
[0077] By training the AutohomeChat language model, it has achieved good results in content understanding and generation. It can summarize concise and clear summary information based on the contents of several documents provided by users, ensuring user satisfaction.
[0078] Among them, the retrieval recall module recalls relevant doc paragraphs by training the dual-tower semantic model, and a sample consists of three parts: question and answer query, answer doc+, and answer doc-.
[0079] In the dual-tower model structure, each tower can use a different network model. Text A is input into single tower A, and text B is input into single tower B. Finally, the output of each tower represents the vector representation of the text, and then the text similarity is calculated to obtain the similarity score. Representative dual-tower models include DSSM and SimNet. DSSM is divided into CNN-DSSM and LSTM-DSSM, mainly with different representation layer structures; SimNet representation layer can use network structures such as BOW, CNN, and RNN.
[0080] The retrieval and recall module can effectively recall the relevant document paragraphs of the query and construct similarity samples, which can effectively distinguish, for example, questions like "How about Hongqi H9" and "How about Hongqi H5".
[0081] Among them, the relevance ranking module trains the discriminant model at the question-answer query and document paragraph level, and then finely ranks the recalled relevant doc paragraphs. A sample consists of three parts: question-answer query, answer doc+, answer doc-, and label (0 / 1).
[0082] The relevance sorting module is used to precisely sort the recalled relevant doc paragraphs, solving the problem of users obtaining information on long-tail queries and thus improving the user experience.
[0083] Among them, the similarity model between question and answer query and document paragraph level is trained based on Bert+PairWise, and the discrimination model between question and answer query and document paragraph level is trained based on Bert+PointWise.
[0084] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0085] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. Intelligent question-answering system based on large language model, Features: The intelligent question answering system consists of a retrieval and recall module, a relevance ranking module, and a large language model summarization module: The retrieval and recall module includes a similarity model between the question-answer query and the document paragraph level; the relevance ranking module includes a discriminant model between the question-answer query and the document paragraph level; and the large language model summarization module is implemented based on the AutohomeChat large language model.
2. The intelligent question answering system based on a large language model according to claim 1, Features: In the retrieval recall module, the training method of the similarity model between the question-answer query and the document paragraph level is: A1, obtain the user exposure click information from the search log table of Autohome. This information includes the query information searched by the user, the returned doc information and the location information; A2, stores the searched information in table A; A3, preliminarily screen the positive sample data and each query from Table A, extract the first 5 docs as positive samples according to the doc position, and store the docs in Table B by paragraph; A4, sends the data in B to the annotator for annotation, and stores the annotated data in table C for future use; A5, read the samples in Table C, for the positive sample (query, doc+), randomly select doc information from other queries as the negative sample doc- of the current query; A6, construct PairWise training sample pairs (query, doc+, doc-), marked as dataset dataset_1; A7, construct PointWise training pairs, (query, doc+, 1) and (query, doc-, 0), and mark them as dataset dataset_2; A8 uses the dataset dataset_1 in A6 and inputs it into the similarity model between question and answer query and document paragraph level to train the question and answer query and document paragraph similarity model model_1. A9: Output the semantic vector of the query through step A8, and then generate the semantic vector of the doc.
3. The intelligent question answering system based on a large language model according to claim 2, Features: The vector formation formula of the semantic vector is: h 1 =tanh(W[h 0 ,y 0 ,c]+b) o 1 =softmax(Vh 1 +c) h 2 =tanh(W[h 1 ,y 1 ,c]+b) o 2 =softmax(Vh 2 +c) … h t =tanh(W[h t-1 ,y t-1 ,c]+b) o t =softmax(Vh t +c) 4. The intelligent question answering system based on a large language model according to claim 1, Features: In the relevance ranking module, the training method of the discriminant model at the question-answer query and document paragraph level is: B1, using the dataset dataset_2 in A7, train the question-answering query and document paragraph discrimination models; B2, takes the query and doc paragraphs as the input of model_2 to determine the current query y with do c Is the paragraph relevant? B3, obtains docs similar to the current query through information acquisition steps A1, A2 and A3; B4, by setting score>single threshold, filter out irrelevant doc paragraphs and get y Similar do c paragraph; B5 then uses the query and similar doc paragraph pairs as inputs to B2's discriminant model model_2, outputs the score, and then performs fine ranking based on the score.
5. The intelligent question answering system based on a large language model according to claim 1, Features: The AutohomeChat language model can receive similar doc paragraphs uploaded in the relevance sorting module, and summarize the answers that meet the user's query requirements.
6. The intelligent question answering system based on a large language model according to claim 1, Features: The retrieval recall module recalls relevant doc paragraphs by training a dual-tower semantic model, and a sample consists of three parts, namely, question and answer query, answer doc+, and answer doc-.
7. The intelligent question answering system based on a large language model according to claim 1, Features: The relevance ranking module trains the discriminant model at the question-answer query and document paragraph level, and then finely ranks the recalled relevant doc paragraphs. A sample consists of three parts, namely, question-answer query, answer doc+, answer doc-, and label (0 / 1).
8. The intelligent question answering system based on a large language model according to claim 1, Features: The similarity model between the question-answer query and the document paragraph level is trained based on Bert+PairWise, and the discrimination model between the question-answer query and the document paragraph level is trained based on Bert+PointWise.