A Question Answering and Citation Generation Method and System Based on a Large Language Model

Abstract

This invention discloses a question-answering and citation generation method and system based on a large language model, belonging to the field of large language model technology. The method includes the following steps: S1, training an intelligent question-answering large language model and training a summary generation large language model; S2, constructing a knowledge base; S3, segmenting the long text generated in the streaming response according to commas to obtain multiple short texts, and inputting the long text and multiple short texts into the language model to obtain corresponding sentence vectors; S4, calculating the similarity between the sentence vectors of the long text and the sentence vectors of the summaries in the knowledge base, and selecting the top 20 similar summaries as candidates; S5, calculating the weighted similarity between the multiple short texts and the top 20 summaries as the score for each summary; S6, selecting the summary with the highest score and greater than a threshold and inserting it at the end of the long text. This invention enables users to quickly locate the articles they want to read while browsing the answer content, providing a more efficient and smooth user experience.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, specifically a question-answering and citation generation method and system based on a large language model. Background Technology

[0002] With the emergence and development of deep learning technology, especially the Transformer architecture, the field of Natural Language Processing (NLP) has made groundbreaking progress. Simultaneously, the widespread adoption of the internet has generated massive amounts of language data, providing abundant raw materials for training large-scale language models. This dual impetus of technology and data has laid the foundation for the birth and development of large language models. Intelligent question-answering systems based on large language models can provide 24 / 7 uninterrupted service, quickly respond to user queries, improve customer satisfaction, and reduce labor costs, thus becoming one of the key technologies for the digital transformation of many enterprises. With the advancement of AI technology, users' demands for interactive experiences are also increasing. Intelligent question-answering systems based on large language models focus on providing a more natural, fluent, and personalized interactive experience, striving to reach or approach the level of human communication. The explainability of answers is an important direction for improving the user's interactive experience. For some intelligent question-answering systems, providing citations or explanations of their reasoning processes can increase users' acceptance and trust in the answers.

[0003] Current intelligent question-answering systems based on large language models are generative AI models. Most of these systems can only provide links related to the answer at the end, and these links are isolated from the answer content. Users can only judge whether the linked articles contain the content they need based on their titles, or click on each link to read the entire article to find the content they want. Some intelligent question-answering systems can insert the link numbers into the answer content, but users can only guess the content of the linked articles and whether they contain the content they need based on the context of the insertion point. This greatly reduces the user's interactive experience. If short descriptions of these articles could be inserted at appropriate positions in the answer, users could quickly locate the articles they want to read while browsing the answer content, providing a more efficient and smooth user experience.

[0004] Therefore, existing intelligent question-answering systems based on large language models have poor user experience. Users can only search for questions of interest among many related links after browsing the answers, and cannot quickly locate articles of interest while browsing the answers. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a question-answering and citation generation method and system based on a large language model, enabling users to quickly locate the articles they want to read while browsing the answer content, thus providing a more efficient and smooth user experience.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A question-answering and citation generation method based on a large language model includes the following steps:

[0008] S1. Train the intelligent question answering large language model, persist the intelligent question answering large language model to the local machine, train the summary generation large language model, and persist the summary generation large language model to the local machine.

[0009] S2. Construct a knowledge base containing summaries and sentence vectors from various standards, articles, encyclopedias, or blogs.

[0010] S3. Input a question and provide an answer; the long text generated in the streaming answer is segmented according to commas to obtain multiple short texts, and the long text and multiple short texts are input into the language model to obtain the corresponding sentence vectors;

[0011] S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and select the top 20 similar summaries as candidates.

[0012] S5. Calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary;

[0013] S6. Select the summary with the highest score that is greater than the threshold and insert it at the end of the long text;

[0014] Perform the above operations on all long texts generated in the streaming response to output the final response.

[0015] This method, applied in intelligent question answering scenarios, is a question answering and citation generation approach based on a large language model and a summary matching algorithm. It can use sentence vectors of long texts to identify potential candidates and use sentence vectors of short texts to score each candidate. A high threshold is set for the scores to ensure that only candidates that are logically closely related to the preceding text can be inserted into the appropriate position in the answer. Users can quickly locate the articles they want to read while browsing the answer content, providing a more efficient and smooth user experience.

[0016] Furthermore, step S1 includes:

[0017] ChatGLM3-6b was chosen as the base model for both the intelligent question answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. The intelligent question answering large language model was trained and persisted locally, and the summary generation large language model was trained and persisted locally.

[0018] For the large language model for abstract generation, the LCSTS dataset was used for fine-tuning training. LCSTS is a large-scale, high-quality Chinese short text abstracting dataset. The average word count of LCSTS abstracts is 18, the standard deviation of word count is 5, the maximum word count is 30, and the minimum word count is 4. The average word count of the main text in LCSTS is 104, the standard deviation of word count is 10, the maximum word count is 152, and the minimum word count is 69. To ensure that the answers with inserted abstracts still have high readability, this method modifies the LCSTS abstracts. For example, "Some provinces have exceeded the annual water consumption red line indicators, and water withdrawal projects will be restricted." will be modified to "This article summarizes that some provinces have exceeded the annual water consumption red line indicators, and water withdrawal projects will be restricted."

[0019] Furthermore, in step S2, the Chinese Bert-base model is used as the natural language embedding model.

[0020] BERT-base is the most basic BERT model, containing 12 Transformer encoder layers and a total of 110 million parameters. The input embedding vector of BERT-base has a dimension of 768, and the hidden layers also have a dimension of 768. For each input natural text, BERT-base can transform it into a 768-dimensional sentence vector.

[0021] Furthermore, in step S3, the long text and multiple short texts are input into the language model to obtain the corresponding sentence vectors, as shown in formulas (1) and (2):

[0022] λ long =f(x) long ,θ) (1)

[0023]

[0024] Where, λ long Sentence vectors representing long texts Let f(·) represent the sentence vector of the i-th short text, f(·) represent the language model, θ represent the parameters of the language model, and x represent the sentence vector of the i-th short text. long Represents long text, This represents the i-th short text.

[0025] Furthermore, step S4 includes the following steps:

[0026] The summary from the knowledge base is input into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0027] λ abstr =f(x) abstr ,θ) (3)

[0028] Where, λ astr The sentence vector representing the summary, x abstr Natural text representing the abstract;

[0029] The similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base is calculated. The top 20 similar summaries are selected as candidates, and Pearson similarity is chosen as the method for similarity calculation, as shown in formula (4):

[0030]

[0031] in, The Pearson similarity is calculated between the sentence vectors of the long text and the sentence vectors of the summary in the knowledge base. The covariance is calculated using cov(·). This represents the standard deviation of the sentence vectors in a long text. E(·) represents the standard deviation of the abstract sentence vector, and E(·) is used to calculate the expectation.

[0032] Furthermore, step S5 includes the following steps:

[0033] For the j-th summary, the score of the i-th short text is 100 / n, where n is the number of short texts. Therefore, the highest score for each summary is 100. The weight of the j-th summary is:

[0034] in Pearson similarity between the j-th abstract and the first short text;

[0035] The weight of the j-th summary is the sum of its Pearson similarity to all the short texts; therefore, a summary that is similar in meaning to all the short texts should be given a higher weight, as such a summary contains more information and is more logically connected to the long text and each short text; the sum of the summary weights and the scores of the short texts is the score of the j-th summary.

[0036] Furthermore, in step S6, the threshold is set to 80.

[0037] The highest-scoring summary with a score greater than the threshold is selected and inserted at the end of the long text. Experimental testing showed that setting the threshold to 80 yielded a good balance of quantity and quality of selected summaries. This process is repeated for all long texts generated in the streaming response to output the final answer.

[0038] This invention also claims a question-answering and citation generation system based on a large language model, comprising:

[0039] The model training module is used to train the intelligent question answering large language model and persist the intelligent question answering large language model to the local machine; it also trains the summary generation large language model and persists the summary generation large language model to the local machine.

[0040] The knowledge base module contains summaries and sentence vectors from various standards, articles, encyclopedias, or blogs;

[0041] The question-and-answer module is used to input questions and provide answers. The long text generated in the streaming response is split into multiple short texts according to commas. The long text and multiple short texts are then input into the language model to obtain the corresponding sentence vectors.

[0042] The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and selects the top 20 similar summaries as candidates.

[0043] The module for calculating the summary score is used to calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary;

[0044] The output module selects the summary with the highest score that is greater than a threshold and inserts it at the end of the long text; it operates on all the long texts generated in the streaming response and outputs the final response.

[0045] Specifically, the system achieves question-answering and citation generation through the aforementioned question-answering and citation generation method based on a large language model.

[0046] The present invention also claims a question-answering and citation generation apparatus based on a large language model, comprising: at least one memory and at least one processor;

[0047] The at least one memory is used to store a machine-readable program;

[0048] The at least one processor is used to call the machine-readable program to implement the above method.

[0049] The present invention also claims a computer-readable medium storing computer instructions that, when executed by a processor, implement the above-described method.

[0050] Compared with existing technologies, the question-answering and citation generation method and system based on a large language model of the present invention have the following advantages:

[0051] This invention proposes a question-answering and citation generation method based on a large language model and a summary matching algorithm. It compares and analyzes the context of the answer with articles in the knowledge base, and inserts the summaries of articles in the knowledge base that are logically close to the preceding text into the appropriate position in the answer.

[0052] The method for determining the inserted citation is based on the sentence vectors of long and short texts. The sentence vectors of long texts are used to determine the top 20 similarity summaries in the knowledge base as candidates. Then, the sentence vectors of several short texts are used to score each candidate. A high threshold α is set for the score to ensure that only candidates that are logically close to the preceding text can be finally inserted into the answer, thus realizing the method for determining the inserted citation.

[0053] Each candidate is scored using sentence vectors from the short texts, and the similarity between each summary and all short texts is calculated. The normalized similarity is used as the weight of the summary. Therefore, summaries that are similar in meaning to all short texts should be given higher weights. Such summaries contain more information and are more logically connected to the long text and each short text. This improves the scientific nature of the scoring, so that summaries that are more consistent with the logic of the long text score higher. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the training and prediction process of the question-answering and citation generation method based on a large language model provided in an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of the weighted sum of summary scores provided in the embodiments of the present invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0057] A question-answering and citation generation method based on a large language model, such as Figure 1 As shown, the steps include:

[0058] S1. Train the intelligent question answering large language model, persist the intelligent question answering large language model to the local machine, train the summary generation large language model, and persist the summary generation large language model to the local machine.

[0059] S2. Construct a knowledge base containing summaries and sentence vectors from various standards, articles, encyclopedias, or blogs.

[0060] S3. Input a question and provide an answer; the long text generated in the streaming response is segmented according to commas to obtain several short texts, and the long text and multiple short texts are input into the language model to obtain the corresponding sentence vectors;

[0061] S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and select the top 20 similar summaries as candidates.

[0062] S5. Calculate the weighted similarity between several short texts and the top 20 summaries as the score for each summary;

[0063] S6. Select the summary with the highest score that is greater than the threshold and insert it at the end of the long text; perform the above operation on all the long texts generated in the streaming response, and output the final response.

[0064] Step S1 includes:

[0065] ChatGLM3-6b was chosen as the base model for both the intelligent question-answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. The intelligent question-answering large language model was trained and persisted locally, and the summary generation large language model was trained and persisted locally.

[0066] For the large language model used in the summary generation, the LCSTS dataset was used for fine-tuning training. LCSTS is a large-scale, high-quality Chinese short text summarization dataset. The average word count of LCSTS summaries is 18, the standard deviation of word count is 5, the maximum word count is 30, and the minimum word count is 4. The average word count of the main text in LCSTS is 104, the standard deviation of word count is 10, the maximum word count is 152, and the minimum word count is 69. To ensure that the answers with inserted summaries still have high readability, this method modifies the LCSTS summaries. For example, "Some provinces have exceeded the annual water consumption red line indicators, and water withdrawal projects will be restricted." will be modified to "This article summarizes that some provinces have exceeded the annual water consumption red line indicators, and water withdrawal projects will be restricted."

[0067] Step S2 includes:

[0068] A knowledge base is built, which contains summaries and sentence vectors of many standards, articles, encyclopedias or blogs, such as the IEEE 802.11 series of standards for Wi-Fi and articles such as the Baidu Encyclopedia entry on Wi-Fi.

[0069] We use the BERT-base model for Chinese as the natural language embedding model. BERT-base is the most basic BERT model, containing 12 Transformer encoder layers with a total of 110 million parameters. The input embedding vector of BERT-base has a dimension of 768, and the hidden layers also have a dimension of 768. For each input natural text, BERT-base can transform it into a 768-dimensional sentence vector.

[0070] Step S3 includes:

[0071] Input a question and get an answer. The long text generated in the streaming response is segmented by commas to get several short texts. Input this long text and several short texts into the language model to get the corresponding sentence vectors, as shown in formulas (1) and (2):

[0072] λ long =f(x) long ,θ) (1)

[0073]

[0074] Where, λ long Sentence vectors representing long texts Let f(·) represent the sentence vector of the i-th short text, f(·) represent the language model, θ represent the parameters of the language model, and x represent the sentence vector of the i-th short text. long Represents long text, This represents the i-th short text.

[0075] Step S4 includes the following steps:

[0076] The summary from the knowledge base is input into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0077] λ abstr =f(x) abstr ,θ) (3)

[0078] Where, λ astr The sentence vector representing the summary, x abstr Natural text representing the abstract;

[0079] The similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base is calculated. The top 20 similar summaries are selected as candidates, and Pearson similarity is chosen as the method for similarity calculation, as shown in formula (4):

[0080]

[0081] in, It is the Pearson similarity between the sentence vectors of the long text and the sentence vectors of the summary in the knowledge base, and cov(·) calculates the covariance. This represents the standard deviation of the sentence vectors in a long text. E(·) represents the standard deviation of the summary sentence vector, and E(·) is used to calculate the expectation.

[0082] Step S5 includes the following steps:

[0083] For the j-th summary, the score of the i-th short text is 100 / n, where n is the number of short texts. Therefore, the highest score for each summary is 100. The weight of the j-th summary is:

[0084] in Pearson similarity between the j-th abstract and the first short text;

[0085] The weight of the j-th summary is the sum of its Pearson similarity to all the short texts. Therefore, summaries that are similar in meaning to all the short texts should be given higher weights; such summaries contain more information and are more logically connected to the long text and each short text. Multiplying the summary weight by the scores of the short texts respectively, the sum is the score of the j-th summary.

[0086] like Figure 2 As shown, the score of the i-th short text is 100 / 5 = 20 points, and the weight of the j-th summary is... The score for the j-th abstract is then...

[0087] 20×0.8+20×0.4+20×0.1+20×0.2+20×0.3=36.

[0088] Step S6 includes:

[0089] The highest-scoring summary with a score greater than the threshold α is selected and inserted at the end of the long text. Experimental testing showed that setting α to 80 resulted in a good balance between the quantity and quality of the selected summaries. This process is repeated for all long texts generated in the streaming response to output the final answer.

[0090] This method first uses the sentence vectors of the long text to identify potential candidates, and then uses the sentence vectors of several short texts to score each candidate. A weighted scoring approach is used, assigning higher weights to short texts with similar meanings, thus improving the scientific rigor of the scoring and ensuring that candidates whose logic aligns more closely with the long text receive higher scores. A high threshold is set for the scores to guarantee that only candidates with strong logical connections to the preceding text can ultimately be included in the answer.

[0091] This invention also provides a question-answering and citation generation system based on a large language model, comprising:

[0092] The model training module is used to train the intelligent question answering large language model and persist the intelligent question answering large language model to the local machine; it also trains the summary generation large language model and persists the summary generation large language model to the local machine.

[0093] The knowledge base module contains summaries and sentence vectors from various standards, articles, encyclopedias, or blogs;

[0094] The question-and-answer module is used to input questions and provide answers. The long text generated in the streaming response is split into multiple short texts according to commas. The long text and multiple short texts are then input into the language model to obtain the corresponding sentence vectors.

[0095] The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and selects the top 20 similar summaries as candidates.

[0096] The module for calculating the summary score is used to calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary;

[0097] The output module selects the summary with the highest score that is greater than a threshold and inserts it at the end of the long text; it operates on all the long texts generated in the streaming response and outputs the final response.

[0098] Specifically, the system achieves question-answering and citation generation through the question-answering and citation generation method based on a large language model as described in the above embodiments.

[0099] The model training module selects ChatGLM3-6b as the base model for the intelligent question answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. The intelligent question answering large language model is trained and persisted locally, and the summary generation large language model is trained and persisted locally.

[0100] For the large language model for summarizing, the LCSTS dataset was used for fine-tuning training. LCSTS is a large-scale, high-quality Chinese short text summarizing dataset. The average word count of LCSTS summaries is 18, the standard deviation of word count is 5, the maximum word count is 30, and the minimum word count is 4. The average word count of LCSTS body text is 104, the standard deviation of word count is 10, the maximum word count is 152, and the minimum word count is 69.

[0101] The knowledge base module contains summaries and sentence vectors of various standards, articles, encyclopedias, or blogs, such as the IEEE 802.11 series of standards for Wi-Fi and articles such as the Baidu Encyclopedia entry on Wi-Fi.

[0102] We use the BERT-base model for Chinese as the natural language embedding model. BERT-base is the most basic BERT model, containing 12 Transformer encoder layers with a total of 110 million parameters. The input embedding vector of BERT-base has a dimension of 768, and the hidden layers also have a dimension of 768. For each input natural text, BERT-base can transform it into a 768-dimensional sentence vector.

[0103] The question-and-answer module takes a question as input and provides an answer. The long text generated in the streaming response is segmented by commas to obtain several short texts. These long texts and short texts are then input into the language model to obtain the corresponding sentence vectors, as shown in formulas (1) and (2).

[0104] λ long =f(x) long ,θ) (1)

[0105]

[0106] Where, λ long Sentence vectors representing long texts Let f(·) represent the sentence vector of the i-th short text, f(·) represent the language model, θ represent the parameters of the language model, and x represent the sentence vector of the i-th short text. long Represents long text, This represents the i-th short text.

[0107] The similarity calculation module inputs the summary from the knowledge base into the language model to obtain the corresponding sentence vector, as shown in formula (3):

[0108] λ abstr =f(x) abstr ,θ) (3)

[0109] Where, λ astr The sentence vector representing the summary, x abstr Natural text representing the abstract;

[0110] The similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base is calculated. The top 20 similar summaries are selected as candidates, and Pearson similarity is chosen as the method for similarity calculation, as shown in formula (4):

[0111]

[0112] in, It is the Pearson similarity between the sentence vectors of the long text and the sentence vectors of the summary in the knowledge base, and cov(·) calculates the covariance. This represents the standard deviation of the sentence vectors in a long text. E(·) represents the standard deviation of the summary sentence vector, and E(·) is used to calculate the expectation.

[0113] The module for calculating the summary score is used to calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary; for the j-th summary, the score of the i-th short text is 100 / n, where n is the number of short texts, and the maximum score for each summary is 100 points; the weight of the j-th summary is:

[0114] in Pearson similarity between the j-th abstract and the first short text;

[0115] The weight of the j-th summary is the sum of its Pearson similarity to all the short texts. Therefore, summaries that are similar in meaning to all the short texts should be given higher weights; such summaries contain more information and are more logically connected to the long text and each short text. Multiplying the summary weight by the scores of the short texts respectively, the sum is the score of the j-th summary.

[0116] The output module selects the highest-scoring summary that exceeds a threshold and inserts it at the end of the long text. Experimental testing showed that setting the threshold to 80 resulted in a good quantity and quality of the selected summaries. This process is repeated for all long texts generated in the streaming response to output the final answer.

[0117] This invention also provides a question-answering and citation generation device based on a large language model, comprising: at least one memory and at least one processor;

[0118] The at least one memory is used to store a machine-readable program;

[0119] The at least one processor is used to call the machine-readable program to implement the question-answering and citation generation method based on a large language model as described in the above embodiments.

[0120] This invention also provides a computer-readable medium storing computer instructions. When executed by a processor, these computer instructions implement the question-answering and citation generation method based on a large language model described in the above embodiments. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0121] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0122] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0123] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0124] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0125] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A question-answering and citation generation method based on a large language model, characterized in that, Includes the following steps: S1. Train the intelligent question answering large language model, persist the intelligent question answering large language model to the local machine, train the summary generation large language model, and persist the summary generation large language model to the local machine. S2. Construct a knowledge base containing summaries and sentence vectors from various standards, articles, encyclopedias, or blogs. S3. Input a question and provide an answer; the long text generated in the streaming answer is segmented according to commas to obtain multiple short texts, and the long text and multiple short texts are input into the language model to obtain the corresponding sentence vectors; S4. Calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and select the top 20 similar summaries as candidates. S5. Calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary; S6. Select the summary with the highest score that is greater than the threshold and insert it at the end of the long text; Perform the above operations on all long texts generated in the streaming response to output the final response.

2. The question-answering and citation generation method based on a large language model according to claim 1, characterized in that, Step S1 includes: ChatGLM3-6b was chosen as the base model for the intelligent question answering large language model and the summary generation large language model. The ChatGLM3-6b model supports efficient fine-tuning training and prediction on a single V100 graphics card. For the large language model for summarization, fine-tuning training was performed using the LCSTS dataset.

3. The question-answering and citation generation method based on a large language model according to claim 1, characterized in that, In step S2, the Chinese Bert-base model is used as the natural language embedding model.

4. The question-answering and citation generation method based on a large language model according to claim 1, characterized in that, In step S3, the long text and multiple short texts are input into the language model to obtain the corresponding sentence vectors, as shown in formulas (1) and (2): l long =f(x long ,i) (1) Where, λ long Sentence vectors representing long texts Let f(·) represent the sentence vector of the i-th short text, f(·) represent the language model, θ represent the parameters of the language model, and x represent the sentence vector of the i-th short text. long Represents long text, This represents the i-th short text.

5. The question-answering and citation generation method based on a large language model according to claim 4, characterized in that, In step S4, Includes the following steps: The summary from the knowledge base is input into the language model to obtain the corresponding sentence vector, as shown in formula (3): l abstr =f(x abstr ,i) (3) Where, λ abstr The sentence vector representing the summary, x abstr Natural text representing the abstract; The similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base is calculated. The top 20 similar summaries are selected as candidates, and Pearson similarity is chosen as the method for similarity calculation, as shown in formula (4): in, The Pearson similarity is calculated between the sentence vectors of the long text and the sentence vectors of the summary in the knowledge base. The covariance is calculated using cov(·). This represents the standard deviation of the sentence vectors in a long text. E(·) represents the standard deviation of the abstract sentence vector, and E(·) is used to calculate the expectation.

6. The question-answering and citation generation method based on a large language model according to claim 5, characterized in that, In step S5, Includes the following steps: For the j-th summary, the score of the i-th short text is 100 / n, where n is the number of short texts. Therefore, the highest score for each summary is 100. The weight of the j-th summary is: in Pearson similarity between the j-th abstract and the first short text; The weight of the j-th summary is the sum of its Pearson similarity to all the short texts; the sum of the summary weights and the scores of the short texts is the score of the j-th summary.

7. A question-answering and citation generation method based on a large language model according to claim 1 or 6, characterized in that, In step S6, the threshold is set to 80.

8. A question-answering and citation generation system based on a large language model, characterized in that, include: The model training module is used to train the intelligent question answering large language model and persist the intelligent question answering large language model to the local machine. Train a large language model for summarizing and persist the large language model locally. The knowledge base module contains summaries and sentence vectors from various standards, articles, encyclopedias, or blogs; The question-and-answer module is used to input questions and provide answers. The long text generated in the streaming response is split into multiple short texts according to commas. The long text and multiple short texts are then input into the language model to obtain the corresponding sentence vectors. The similarity calculation module is used to calculate the similarity between the sentence vector of the long text and the sentence vector of the summary in the knowledge base, and selects the top 20 similar summaries as candidates. The module for calculating the summary score is used to calculate the weighted similarity between multiple short texts and the top 20 summaries as the score for each summary; The output module selects the summary with the highest score that is greater than a threshold and inserts it at the end of the long text; it operates on all the long texts generated in the streaming response and outputs the final response. Specifically, the system achieves question-answering and citation generation through the question-answering and citation generation method based on a large language model as described in any one of claims 1 to 7.

9. A question-answering and citation generation device based on a large language model, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to implement the method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that, when executed by a processor, implement the method described in any one of claims 1 to 7.

Images