Question and answer and quotation generation method and system based on large language model

Abstract

The invention discloses a question and answer and quotation generation method and system based on a large language model, and belongs to the technical field of large language models, and the method comprises the following steps: S1, training an intelligent question and answer large language model, and training an abstract to generate a large language model; s2, constructing a knowledge base; s3, segmenting a long text generated in the flow type answer according to a comma to obtain a plurality of short texts, and inputting the long text and the plurality of short texts into a language model to obtain corresponding sentence vectors; s4, calculating the similarity between the sentence vector of the long text and the sentence vector of the abstract in the knowledge base, and taking the abstract with the similarity TOP20 as a candidate; s5, calculating the weighted similarity of the plurality of short texts and the abstract of the TOP20 as the score of each abstract; and S6, selecting the abstract with the highest score which is greater than the threshold value, and inserting the abstract into the backmost of the long text. According to the method and the system, the user can quickly position the article which the user wants to watch while browsing the answer content, and more efficient and smooth user experience is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a question-answering and citation generation method and system based on a large language model. Background Art

[0002] With the emergence and development of deep learning technology, especially the Transformer architecture, the field of natural language processing (NLP) has made breakthrough progress. At the same time, the popularization of the Internet has generated a huge amount of language data, providing rich raw materials for training large-scale language models. The dual promotion of these technologies and data has laid the foundation for the birth and development of large language models. The intelligent question-answering system based on the large language model can provide uninterrupted service 24 hours a day, respond to user queries quickly, improve customer satisfaction, and reduce labor costs. Therefore, it has become one of the key technologies for the digital transformation of many enterprises. With the advancement of AI technology, users' requirements for interactive experience are also increasing. The intelligent question-answering system based on the large language model focuses on providing a more natural, smooth, and personalized interactive experience, striving to reach or approach the level of human communication. The explainability of answers is an important direction to improve users' interactive experience. For some intelligent question-answering system answers, providing reference sources or explaining their reasoning process can increase users' recognition and trust in the answers.

[0003] The current intelligent question-and-answer system based on a large language model is a generative AI model. Most intelligent question-and-answer systems can only provide links related to the answer at the end of the answer, and the related links are isolated from the content of the answer. Users can only judge whether there is the content they need based on the title of each linked article, or click on these links one by one to read the entire article to find the content they want to read. Some intelligent question-and-answer systems can insert the serial number of the link into the content of the answer. Users can only guess what the linked article talks about and whether it contains the content they need based on the context of the insertion position. This greatly reduces the user's interactive experience. If a brief description of these articles can be inserted at the appropriate location of the answer, users can quickly locate the article they want to read while browsing the answer content, providing a more efficient and smooth user experience.

[0004] Therefore, the existing intelligent question-answering systems based on large language models have poor user experience. Users can only search for questions of interest in many related links after browsing the answers, and cannot quickly locate articles of interest while browsing the answers. Summary of the invention

[0005] The technical task of the present invention is to address the above shortcomings and provide a question-answering and citation generation method and system based on a large language model, so that users can quickly locate the articles they want to read while browsing the answer content, providing a more efficient and smooth user experience.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A question-answering and citation generation method based on a large language model comprises the following steps:

[0008] S1. Train the intelligent question-answering language model and persist it locally. Train the summary generation language model and persist it locally.

[0009] S2. Build a knowledge base, which contains the abstracts and sentence vectors of various standards, articles, encyclopedias or blogs;

[0010] S3. Input a question and give an answer; split the long text generated in the streaming answer according to commas to obtain multiple short texts, and input the long text and the multiple short texts into the language model to obtain corresponding sentence vectors;

[0011] S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates;

[0012] S5, calculating the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary;

[0013] S6, select the summary with the highest score and greater than the threshold and insert it at the end of the long text;

[0014] Repeat the above operation for all the long texts generated in the streaming answer and output the final answer.

[0015] This method is applied in the scenario of intelligent question and answer. It is a question and answer and citation generation method based on a large language model and a summary matching algorithm. It can use the sentence vectors of long texts to determine possible candidates and use the sentence vectors of short texts to score each candidate. A high threshold is set for the score to ensure that only candidates that are closely related to the previous text logic can be finally inserted into the appropriate position in the answer. Users can quickly locate the article they want to read while browsing the answer content, providing a more efficient and smooth user experience.

[0016] Further, in the step S1, it includes:

[0017] Select ChatGLM3-6b as the base model for the intelligent question-answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. Train the intelligent question-answering large language model and persist it locally. Train the summary generation large language model and persist it locally.

[0018] For the summary generation large language model, the LCSTS dataset is used for fine-tuning training; LCSTS is a large-scale, high-quality Chinese short text summary dataset. The average number of words in the LCSTS summary is 18, the standard deviation of the number of words is 5, the maximum number of words is 30, and the minimum number of words is 4; the average number of words in the LCSTS text is 104, the standard deviation of the number of words is 10, the maximum number of words is 152, and the minimum number of words is 69; in order to ensure that the answer with the inserted summary still has a high reading fluency, this method modifies the summary of LCSTS, for example, for "Some provinces exceed the annual water use red line indicators, and water withdrawal projects will be restricted.", it will be modified to "This article summarizes that some provinces exceed the annual water use red line indicators, and water withdrawal projects will be restricted.".

[0019] Furthermore, in step S2, the Chinese Bert-base model is used as the embedding model of the natural language.

[0020] BERT-base is the most basic BERT model, which contains 12 Transformer encoder layers with a total of 110 million parameters. The input embedding vector dimension of BERT-base is 768, and the dimension of the hidden layer is also 768. For each input natural text, BERT-base can convert it into a 768-dimensional sentence vector.

[0021] Furthermore, in step S3, the long text and the multiple short texts are input into the language model to obtain corresponding sentence vectors, as shown in formula (1) and formula (2):

[0022] λ long =f(x long ,θ) (1)

[0023]

[0024] Among them, λ long Sentence vectors representing long texts, represents the sentence vector of the i-th short text, f(·) represents the language model, θ represents the parameters of the language model, and x long Represents long text, Represents the i-th short text.

[0025] Further, in the step S4, the following steps are included:

[0026] Input the summary in the knowledge base into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0027] λ abstr =f(x abstr ,θ) (3)

[0028] Among them, λ astr The sentence vector representing the summary, x abstr natural text representing the abstract;

[0029] Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, take the top 20 similarity summaries as candidates, and choose to use Pearson similarity as the similarity calculation method, as shown in formula (4):

[0030]

[0031] in, is the Pearson similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, cov(·) calculates the covariance, represents the standard deviation of the long text sentence vector, represents the standard deviation of the summary sentence vector, and E(·) calculates the expectation.

[0032] Further, in the step S5, the following steps are included:

[0033] For the jth abstract, the score of the i-th short text is 100 / n, where n is the number of short texts, and the maximum score of each abstract is 100 points; the weight of the jth abstract is:

[0034] in represents the Pearson similarity between the jth abstract and the first short text;

[0035] The weight of the j-th abstract is the sum of its Pearson similarities with all short texts; therefore, an abstract that is similar in meaning to all short texts should be given a higher weight, as such an abstract contains more information and is more logically connected to the long text and each short text; multiply the abstract weight by the short text score respectively, and the sum is the score of the j-th abstract.

[0036] Furthermore, in step S6, the threshold is set to 80.

[0037] Select the summary with the highest score and greater than the threshold and insert it at the end of the long text. After experimental testing, when the threshold is set to 80, the quantity and quality of the screened summaries can reach a good level. Perform the above operation on all the long texts generated in the streaming answer and output the final answer.

[0038] The present invention also claims a question-answering and citation generation system based on a large language model, comprising:

[0039] Model training module, used to train the intelligent question-answering language model and persist the intelligent question-answering language model locally; train the summary generation language model and persist the summary generation language model locally;

[0040] The knowledge base module contains the summaries and sentence vectors of various standards, articles, encyclopedias, or blogs;

[0041] The question-answering module is used to input questions and answer them; the long text generated in the streaming answer is segmented according to commas to obtain multiple short texts, and the long text and the multiple short texts are input into the language model to obtain corresponding sentence vectors;

[0042] The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates;

[0043] The summary score calculation module is used to calculate the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary;

[0044] The output module selects the summary with the highest score and greater than the threshold and inserts it at the end of the long text; operates on all the long texts generated in the streaming answer and outputs the final answer;

[0045] The system specifically realizes question-answering and citation generation through the above-mentioned large language model-based question-answering and citation generation method.

[0046] The present invention also claims a question-answering and citation generation device based on a large language model, comprising: at least one memory and at least one processor;

[0047] The at least one memory is used to store a machine-readable program;

[0048] The at least one processor is used to call the machine-readable program to implement the above method.

[0049] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which implement the above method when executed by a processor.

[0050] Compared with the prior art, the question-answering and citation generation method and system based on a large language model of the present invention has the following beneficial effects:

[0051] The present invention proposes a question-answer and citation generation method based on a large language model and a summary matching algorithm, which compares and analyzes the context of the answer and the articles in the knowledge base, and inserts the summaries of the articles in the knowledge base that are logically closely related to the previous text into the appropriate position in the answer.

[0052] The insertion of citations is determined based on the sentence vectors of long texts and short texts. The sentence vectors of long texts are used to determine the top 20 similarity summaries in the knowledge base as candidates. Then, the sentence vectors of several short texts are used to score each candidate. A high threshold α is set for the score to ensure that only candidates with close logic to the previous text can be finally inserted into the answer, thus realizing a method for determining the insertion of citations.

[0053] Each candidate is weighted and scored using the sentence vector of the short text. The similarity between each abstract and all the short texts is calculated, and the normalized similarity is used as the weight of the abstract. Therefore, abstracts that are similar in meaning to all the short texts should be given higher weights. Such abstracts contain more information and are more closely aligned with the logic of the long text and each short text. This improves the scientific nature of the scoring, so that the more consistent the abstract is with the logic of the long text, the higher the score. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a diagram showing the training and prediction process of the question-answering and citation generation method based on a large language model provided by an embodiment of the present invention;

[0055] Figure 2 4 is a weighted sum principle diagram of summary scores provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0057] A question answering and citation generation method based on a large language model, such as Figure 1 As shown, the steps include:

[0058] S1. Train the intelligent question-answering language model and persist it locally. Train the summary generation language model and persist it locally.

[0059] S2. Build a knowledge base, which contains the abstracts and sentence vectors of various standards, articles, encyclopedias or blogs;

[0060] S3. Input a question and answer it; split the long text generated in the streaming answer according to commas to obtain a number of short texts, and input the long text and the multiple short texts into the language model to obtain corresponding sentence vectors;

[0061] S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates;

[0062] S5, calculating the weighted similarity between several short texts and the TOP20 abstracts as the score of each abstract;

[0063] S6. Select the summary with the highest score and greater than the threshold and insert it to the end of the long text; perform the above operation on all the long texts generated in the streaming answer and output the final answer.

[0064] In the step S1, it includes:

[0065] Select ChatGLM3-6b as the base model for the intelligent question-answering big language model and the summary generation big language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. Train the intelligent question-answering big language model and persist the intelligent question-answering big language model locally. Train the summary generation big language model and persist the summary generation big language model locally.

[0066] For the large language model for abstract generation, the LCSTS dataset is used for fine-tuning training. LCSTS is a large-scale, high-quality Chinese short text abstract dataset. The average word count of LCSTS abstracts is 18, the standard deviation of the word count is 5, the maximum word count is 30, and the minimum word count is 4; the average word count of the LCSTS body is 104, the standard deviation of the word count is 10, the maximum word count is 152, and the minimum word count is 69. In order to ensure that the answers with inserted abstracts still have a high reading fluency, this method modifies the abstract of LCSTS. For example, for "Some provinces exceed the annual water use red line indicators, and water withdrawal projects will be restricted.", it will be modified to "This article summarizes that some provinces exceed the annual water use red line indicators, and water withdrawal projects will be restricted."

[0067] In the step S2, it includes:

[0068] Build a knowledge base, which contains the abstracts and sentence vectors of many standards, articles, encyclopedias, or blogs, such as the IEEE802.11 series of Wi-Fi standards and Baidu Encyclopedia articles on Wi-Fi.

[0069] The Chinese Bert-base model is used as the natural language embedding model. BERT-base is the most basic BERT model, which contains 12 Transformer encoder layers and a total of 110 million parameters. The input embedding vector dimension of BERT-base is 768, and the dimension of the hidden layer is also 768. For each input natural text, BERT-base can convert it into a 768-dimensional sentence vector.

[0070] The step S3 includes:

[0071] Input the question and answer it. The long text generated in the streaming answer is segmented according to commas to obtain several short texts. This long text and several short texts are input into the language model to obtain the corresponding sentence vector, as shown in formula (1) and formula (2):

[0072] λ long =f(x long ,θ) (1)

[0073]

[0074] Among them, λ long Sentence vectors representing long texts, represents the sentence vector of the i-th short text, f(·) represents the language model, θ represents the parameters of the language model, and x long Represents long text, Represents the i-th short text.

[0075] In step S4, the following steps are included:

[0076] Input the summary in the knowledge base into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0077] λ abstr =f(x abstr ,θ) (3)

[0078] Among them, λ astr The sentence vector representing the summary, x abstr natural text representing the abstract;

[0079] Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, take the top 20 similarity summaries as candidates, and choose to use Pearson similarity as the similarity calculation method, as shown in formula (4):

[0080]

[0081] in, is the Pearson similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, cov(·) calculates the covariance, represents the standard deviation of the long text sentence vector, represents the standard deviation of the summary sentence vector, and E(·) calculates the expectation.

[0082] In the step S5, the following steps are included:

[0083] For the jth abstract, the score of the i-th short text is 100 / n, where n is the number of short texts, and the maximum score of each abstract is 100 points; the weight of the jth abstract is:

[0084] in represents the Pearson similarity between the jth abstract and the first short text;

[0085] The weight of the j-th abstract is the sum of its Pearson similarities with all short texts. Therefore, abstracts that are similar in meaning to all short texts should be given higher weights. Such abstracts contain more information and are more closely connected to the long text and each short text. Multiply the abstract weight and the short text score respectively, and the sum is the score of the j-th abstract.

[0086] like Figure 2 As shown, the score of the i-th short text is 100 / 5=20 points, and the weight of the j-th abstract is The score of the jth abstract is

[0087] 20×0.8+20×0.4+20×0.1+20×0.2+20×0.3=36.

[0088] In the step S6, it includes:

[0089] Select the summary with the highest score greater than the threshold α and insert it at the end of the long text. After experimental testing, when α is set to 80, the quantity and quality of the screened summaries can reach a good level. Perform the above operation on all the long texts generated in the streaming answer and output the final answer.

[0090] This method first uses the sentence vector of the long text to determine the possible candidates, and then uses the sentence vector of several short texts to score each candidate. When scoring, the weighted idea is used to give higher weights to short texts with similar meanings, which improves the scientific nature of the scoring, so that the candidates that are more consistent with the logic of the long text have higher scores. A high threshold is set for the score to ensure that only candidates that are closely related to the logic of the previous text can be finally inserted into the answer.

[0091] The embodiment of the present invention also provides a question-answering and citation generation system based on a large language model, including:

[0092] Model training module, used to train the intelligent question-answering language model and persist the intelligent question-answering language model locally; train the summary generation language model and persist the summary generation language model locally;

[0093] The knowledge base module contains the summaries and sentence vectors of various standards, articles, encyclopedias, or blogs;

[0094] The question-answering module is used to input questions and answer them; the long text generated in the streaming answer is segmented according to commas to obtain multiple short texts, and the long text and the multiple short texts are input into the language model to obtain corresponding sentence vectors;

[0095] The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates;

[0096] The summary score calculation module is used to calculate the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary;

[0097] The output module selects the summary with the highest score and greater than the threshold and inserts it at the end of the long text; operates on all the long texts generated in the streaming answer and outputs the final answer;

[0098] The system specifically implements question-answering and citation generation through the question-answering and citation generation method based on a large language model described in the above embodiment.

[0099] The model training module selects ChatGLM3-6b as the base model for the intelligent question-answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction of a single V100 graphics card; trains the intelligent question-answering large language model and persists the intelligent question-answering large language model locally; trains the summary generation large language model and persists the summary generation large language model locally.

[0100] For the large language model for summary generation, the LCSTS dataset is used for fine-tuning training. LCSTS is a large-scale, high-quality Chinese short text summary dataset. The average number of words in the LCSTS summary is 18, the standard deviation of the number of words is 5, the maximum number of words is 30, and the minimum number of words is 4; the average number of words in the LCSTS text is 104, the standard deviation of the number of words is 10, the maximum number of words is 152, and the minimum number of words is 69.

[0101] The knowledge base module includes summaries of various standards, articles, encyclopedias or blogs and their sentence vectors, such as articles on the IEEE 802.11 series of Wi-Fi standards and Baidu Encyclopedia on Wi-Fi.

[0102] The Chinese Bert-base model is used as the natural language embedding model. BERT-base is the most basic BERT model, which contains 12 Transformer encoder layers and a total of 110 million parameters. The input embedding vector dimension of BERT-base is 768, and the dimension of the hidden layer is also 768. For each input natural text, BERT-base can convert it into a 768-dimensional sentence vector.

[0103] The question-answering module inputs questions and gives answers. The long text generated in the streaming answer is segmented according to commas to obtain several short texts. The long text and several short texts are input into the language model to obtain the corresponding sentence vectors, as shown in formulas (1) and (2):

[0104] λ long =f(x long ,θ) (1)

[0105]

[0106] Among them, λ long Sentence vectors representing long texts, represents the sentence vector of the i-th short text, f(·) represents the language model, θ represents the parameters of the language model, and x long Represents long text, Represents the i-th short text.

[0107] The similarity calculation module inputs the summary in the knowledge base into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0108] λ abstr =f(x abstr ,θ) (3)

[0109] Among them, λ astr The sentence vector representing the summary, x abstr natural text representing the abstract;

[0110] Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, take the top 20 similarity summaries as candidates, and choose to use Pearson similarity as the similarity calculation method, as shown in formula (4):

[0111]

[0112] in, is the Pearson similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, cov(·) calculates the covariance, represents the standard deviation of the long text sentence vector, represents the standard deviation of the summary sentence vector, and E(·) calculates the expectation.

[0113] The summary score calculation module is used to calculate the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary; for the jth summary, the score of the i-th short text is 100 / n, where n is the number of short texts, and the highest score of each summary is 100 points; the weight of the jth summary is:

[0114] in represents the Pearson similarity between the jth abstract and the first short text;

[0115] The weight of the j-th abstract is the sum of its Pearson similarities with all short texts. Therefore, abstracts that are similar in meaning to all short texts should be given higher weights. Such abstracts contain more information and are more closely connected to the long text and each short text. Multiply the abstract weight and the short text score respectively, and the sum is the score of the j-th abstract.

[0116] The output module selects the summary with the highest score and greater than the threshold and inserts it at the end of the long text. After experimental testing, when the threshold is set to 80, the quantity and quality of the screened summaries can reach a good level. The above operation is performed on all the long texts generated in the streaming answer, and the final answer is output.

[0117] The embodiment of the present invention also provides a question-answering and citation generation device based on a large language model, comprising: at least one memory and at least one processor;

[0118] The at least one memory is used to store a machine-readable program;

[0119] The at least one processor is used to call the machine-readable program to implement the question-answering and citation generation method based on the large language model described in the above embodiment.

[0120] The embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the question-answering and citation generation method based on a large language model described in the above embodiment is implemented. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0121] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0122] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0123] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0124] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0125] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A question-answering and citation generation method based on a large language model, characterized in that: The following steps are involved: S1. Train the intelligent question-answering language model and persist it locally. Train the summary generation language model and persist it locally. S2. Build a knowledge base, which contains the abstracts and sentence vectors of various standards, articles, encyclopedias or blogs; S3. Input a question and give an answer; split the long text generated in the streaming answer according to commas to obtain multiple short texts, and input the long text and the multiple short texts into the language model to obtain corresponding sentence vectors; S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates; S5, calculating the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary; S6, select the summary with the highest score and greater than the threshold and insert it at the end of the long text; Repeat the above operation for all the long texts generated in the streaming answer and output the final answer.

2. According to claim 1, a question-answering and citation generation method based on a large language model is characterized in that: In the step S1, it includes: ChatGLM3-6b is selected as the base model for the intelligent question-answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. For the summary generation large language model, the LCSTS dataset is used for fine-tuning training.

3. According to claim 1, a question-answering and citation generation method based on a large language model is characterized in that: In step S2, the Chinese Bert-base model is used as the embedding model of the natural language.

4. The question-answering and citation generation method based on a large language model according to claim 1, characterized in that: In step S3, the long text and the multiple short texts are input into the language model to obtain the corresponding sentence vectors, as shown in formula (1) and formula (2): l long =f(x long ,i) (1) Among them, λ long Sentence vectors representing long texts, represents the sentence vector of the i-th short text, f(·) represents the language model, θ represents the parameters of the language model, and x long Represents long text, Represents the i-th short text.

5. A question-answering and citation generation method based on a large language model according to claim 4, characterized in that: In step S4, The following steps are involved: Input the summary in the knowledge base into the language model to obtain the corresponding sentence vector, as shown in formula (3): l abstr =f(x abstr ,i) (3) Among them, λ astr The sentence vector representing the summary, x abstr natural text representing the abstract; Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, take the top 20 similarity summaries as candidates, and choose to use Pearson similarity as the similarity calculation method, as shown in formula (4): in, is the Pearson similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, cov(·) calculates the covariance, represents the standard deviation of the long text sentence vector, represents the standard deviation of the summary sentence vector, and E(·) calculates the expectation.

6. A question-answering and citation generation method based on a large language model according to claim 5, characterized in that: In step S5, The following steps are involved: For the jth abstract, the score of the i-th short text is 100 / n, where n is the number of short texts, and the maximum score of each abstract is 100 points; the weight of the jth abstract is: in represents the Pearson similarity between the jth abstract and the first short text; The weight of the j-th abstract is the sum of its Pearson similarities with all short texts; multiply the abstract weight by the short text score respectively, and the sum is the score of the j-th abstract.

7. A question-answering and citation generation method based on a large language model according to claim 1 or 6, characterized in that: In step S6, the threshold is set to 80.

8. A question-answering and citation generation system based on a large language model, characterized in that: include: Model training module, used to train the intelligent question-answering language model and persist the intelligent question-answering language model locally; Train the large language model for summarization and persist the large language model for summarization to the local computer. The knowledge base module contains the summaries and sentence vectors of various standards, articles, encyclopedias, or blogs; The question-answering module is used to input questions and answer them; the long text generated in the streaming answer is segmented according to commas to obtain multiple short texts, and the long text and the multiple short texts are input into the language model to obtain corresponding sentence vectors; The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and take the summary with the top 20 similarities as candidates; The summary score calculation module is used to calculate the weighted similarity between multiple short texts and the TOP20 summaries as the score of each summary; The output module selects the summary with the highest score and greater than the threshold and inserts it at the end of the long text; operates on all the long texts generated in the streaming answer and outputs the final answer; The system specifically realizes question and answer and citation generation through the question and answer and citation generation method based on a large language model as described in any one of claims 1 to 7.

9. A question-answering and citation generation device based on a large language model, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images