Document conversation method and related device based on self-reflection generation of large language model

Abstract

The present application provides a document conversation method and related device based on self-reflection generation of a large language model, which relates to the field of knowledge question-answering technology, and obtains the target question in the document conversation to be processed; based on the target question, a retrieval ranking model is used to perform multiple rounds of self-reflection generation based on question correction on the large language model, and the retrieval ranking model is pre-trained using matrix decomposition technology; the answer to the target question is obtained by reasoning the large language model after multiple rounds of self-reflection generation. The present application introduces matrix decomposition technology to train the retrieval ranking model, improves the knowledge recall ability of retrieving relevant text fragments from the index library, uses the step-by-step reasoning and context learning ability of the large language model in the self-reflection generation based on question correction, uses the retrieval ranking model to gradually expand the question and related reference information in a question correction manner, gives self-reflection of generating answers, and improves the reliability of document conversation answers.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge question answering, and in particular to a document conversation method and related devices based on self-reflection generation of a large language model. Background Art

[0002] Document conversation is an important task to reduce costs and increase efficiency. It answers questions raised by users by retrieving documents in specific fields, helping users to achieve document retrieval, question answering and document management. This task requires not only the ability to understand the content of the document, but also the ability to understand the intent of the question and give an accurate answer.

[0003] Traditional document conversation methods mainly rely on keyword matching and probabilistic retrieval models for information retrieval and sorting, and then use generative models to construct answers. However, during the retrieval process, it is necessary to rely on manual feature extraction and document sorting functions, which makes the retrieval accuracy dependent on human subjective ideas and experience, increasing labor costs. Summary of the invention

[0004] In view of the above problems, this application provides a document conversation method and related devices based on self-reflection generation of a large language model to achieve the purpose of improving the reliability of answers in the document conversation system. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a document conversation method based on self-reflection generation of a large language model, the document conversation method based on self-reflection generation of a large language model comprising:

[0006] Get the target question in the document session to be processed;

[0007] Based on the target question, a retrieval ranking model is used to generate multiple rounds of self-reflection based on question correction for the large language model; wherein the retrieval ranking model is pre-trained using matrix decomposition technology;

[0008] The answer to the target question is obtained by reasoning the large language model generated through multiple rounds of self-reflection.

[0009] In a possible implementation, the process of pre-training the retrieval ranking model using matrix decomposition technology includes:

[0010] Retrieving a basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer;

[0011] Acquire training samples for this iterative training; wherein the training samples include a first text segment with a label, a search question, and a search answer;

[0012] Performing keyword extraction on the first text segment, the search question, and the search answer, respectively, to obtain text segment keywords, search question keywords, and search answer keywords; and performing vector embedding processing on the first text segment, the search question, and the search answer, respectively, to obtain a text segment vector, a search question vector, and a search answer vector;

[0013] Performing matrix alignment on the text segment keywords, the search question keywords and the search answer keywords through the generalized matrix decomposition layer; and performing matrix alignment on the text segment vector, the search question vector and the search answer vector through the multi-layer perceptron layer;

[0014] The matrix alignment result of the generalized matrix decomposition layer and the matrix alignment result of the multi-layer perceptron layer are jointly solved by the matrix decomposition layer;

[0015] Calculate the loss function value of this iterative training according to the joint solution result of the matrix decomposition layer and the label;

[0016] If the loss function value does not meet the corresponding end condition, respectively adjust the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer, and return to execute the step of obtaining the training samples for this iterative training;

[0017] If the loss function value meets the corresponding end condition, the training is terminated, and the trained basic model is used as the retrieval ranking model.

[0018] In a possible implementation, the step of using the retrieval ranking model to perform multiple rounds of self-reflection generation based on question correction on the large language model based on the target question includes:

[0019] Get the second text segment;

[0020] In the process of generating the current round of self-reflection, the retrieval ranking model is used to retrieve the first target text segment associated with the target question and the second target text segment associated with the self-reflection question generated in the previous round of self-reflection in the second text segment;

[0021] Calculating the difference between the first target text segment and the second target text segment;

[0022] When the difference is greater than the corresponding difference threshold or the difference is equal to 0, the pre-built self-reflection generation prompt template is input into the large language model, so that the large language model outputs the self-reflection question generated in this round of self-reflection, and enters the next round of self-reflection generation;

[0023] When the difference is less than the corresponding difference threshold, or the round of self-reflection generation of the large language model reaches the corresponding round threshold, the self-reflection generation of the large language model is terminated.

[0024] In a possible implementation, the calculating the difference between the first target text segment and the second target text segment includes:

[0025] taking the intersection of the first target text segment and the second target text segment as the intersection text segment;

[0026] taking the union of the first target text segment and the second target text segment as a union text segment;

[0027] The difference degree is calculated according to the intersection text segment and the union text segment.

[0028] In a possible implementation, before the method uses the retrieval ranking model to perform multiple rounds of self-reflection generation based on question correction on the large language model based on the target question, the document conversation method based on self-reflection generation of the large language model also includes:

[0029] The large language model is subjected to enhanced alignment.

[0030] A second aspect of the present application provides a document conversation device based on self-reflection generation of a large language model, the document conversation device based on self-reflection generation of a large language model comprising:

[0031] A model training module is used to pre-train a retrieval ranking model using matrix decomposition technology;

[0032] A question acquisition module is used to acquire target questions in the document session to be processed;

[0033] A self-reflection generation module, used to perform multiple rounds of self-reflection generation based on problem correction on a large language model using the retrieval ranking model based on the target problem;

[0034] The answer reasoning module is used to obtain the answer to the target question through reasoning of the large language model generated after multiple rounds of self-reflection.

[0035] In a possible implementation, the model training module is specifically used to:

[0036] Retrieve a basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer; obtain training samples for this iterative training; wherein the training samples include a first text segment with a label, a search question and a search answer; perform keyword extraction on the first text segment, the search question and the search answer, respectively, to obtain text segment keywords, search question keywords and search answer keywords; and perform vector embedding processing on the first text segment, the search question and the search answer, respectively, to obtain a text segment vector, a search question vector and a search answer vector; perform matrix alignment on the text segment keywords, the search question keywords and the search answer keywords through the generalized matrix decomposition layer; and, The text segment vector, the retrieval question vector and the retrieval answer vector are matrix-aligned through the multi-layer perceptron layer; the matrix alignment results of the generalized matrix decomposition layer and the multi-layer perceptron layer are jointly solved through the matrix decomposition layer; the loss function value of this iterative training is calculated according to the joint solution result of the matrix decomposition layer and the label; if the loss function value does not meet the corresponding end condition, the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer are adjusted respectively, and the step of obtaining the training samples for this iterative training is returned to execute; if the loss function value meets the corresponding end condition, the training is terminated, and the trained basic model is used as the retrieval ranking model.

[0037] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the document conversation method based on self-reflection generation of a large language model according to the above-mentioned first aspect or any implementation of the first aspect.

[0038] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0039] The memory is used to store computer programs;

[0040] The processor is used to execute the computer program so that the electronic device can implement the document conversation method based on self-reflection generation of a large language model according to the above-mentioned first aspect or any implementation method of the first aspect.

[0041] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the document conversation method based on self-reflection generation of a large language model according to the above-mentioned first aspect or any implementation method of the first aspect.

[0042] By means of the above technical scheme, the document conversation method provided by the present application is based on the self-reflection generation of a large language model, which obtains the target question in the document conversation to be processed; based on the target question, the large language model is subjected to multiple rounds of self-reflection generation based on question correction using a retrieval ranking model, and the retrieval ranking model is pre-trained using matrix decomposition technology; the answer to the target question is obtained by reasoning with the large language model after multiple rounds of self-reflection generation. The present application introduces matrix decomposition technology to train the retrieval ranking model, improves the knowledge recall capability of retrieving relevant text fragments from the index library, uses the step-by-step reasoning and context learning capabilities of the large language model in the self-reflection generation based on question correction, uses the retrieval ranking model to gradually expand the question and related reference information in a question correction manner, gives self-reflection of the generated answer, and improves the reliability of the document conversation answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0044] Figure 1 A flowchart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application;

[0045] Figure 2 A partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a basic model provided in an embodiment of the present application;

[0047] Figure 4 Another partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application;

[0048] Figure 5 Another partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application;

[0049] Figure 6 An example of a self-reflection generation prompt template provided in an embodiment of the present application;

[0050] Figure 7 An example of a question and answer prompt template provided in an embodiment of the present application;

[0051] Figure 8A document conversation device based on self-reflection generation of a large language model provided in an embodiment of the present application;

[0052] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0054] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0055] The terms "first", "second" etc. in the specification of the application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, and this is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0056] See also Figure 1 , Figure 1 A flowchart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application. Figure 1 As shown, a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application may include steps S101 to S103, and these steps are described in detail below.

[0057] S101, obtaining a target question in a document session to be processed.

[0058] In the embodiment of the present application, the document session to be processed is a round of document session most recently input by the user, and the question therein, namely the target question, is obtained.

[0059] S102, based on the target question, use the retrieval ranking model to generate multiple rounds of self-reflection based on question correction for the large language model; wherein the retrieval ranking model is pre-trained using matrix decomposition technology.

[0060] In document dialogue tasks, large language model-driven methods leverage the contextual learning and reasoning capabilities of large language models, segmenting document content and embedding it using a vector space model to enhance the understanding and generation capabilities of the dialogue system. Popular applications or frameworks such as LangChain, Auto-GPT, and AutoChain are based on this approach. LLM can also be used as a question-answering decoder, combining CoT prompts and retrieval knowledge to improve performance.

[0061] However, although large language model-driven methods dominate the current market, since large language models often produce text that is meaningless or unfaithful to the provided source input, this undesirable generation is called "hallucination". The use of large language models requires complementary methods to ensure the credibility of their generation.

[0062] In this regard, in the document conversation task, in order to ensure the credibility of the model, the embodiment of the present application uses self-reflection generation to alleviate or even correct the hallucination of the large language model in order to address the problem of hallucinations in the large language model. Specifically, the retrieval ranking model is trained in advance using matrix decomposition technology to improve the knowledge recall ability of retrieving relevant text fragments from the index library. In the self-reflection generation based on question correction, the large language model's step-by-step reasoning and context learning capabilities are used. Based on the target question, the retrieval ranking model is used to gradually expand the question and related reference information in a question correction manner, and self-reflection of the generated answer is given to improve the reliability of the document conversation answer.

[0063] See also Figure 2 , Figure 2 A partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application. Figure 2 As shown, an embodiment of the present application provides a document conversation method based on self-reflection generation of a large language model, wherein the process of pre-training a retrieval ranking model using matrix decomposition technology may include steps S201 to S208, and these steps are described in detail below.

[0064] S201, retrieve a basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer.

[0065] See also Figure 3 , Figure 3 A schematic diagram of a basic model provided in an embodiment of the present application. Figure 3 As shown, the basic model for training the retrieval ranking model may include a generalized matrix decomposition layer A, a multi-layer perceptron layer B and a matrix decomposition layer C, wherein the output of the generalized matrix decomposition layer A and the output of the multi-layer perceptron layer B serve as the input of the matrix decomposition layer C.

[0066] S202, obtaining training samples for this iterative training; wherein the training samples include a first text segment with a label, a retrieval question, and a retrieval answer.

[0067] In the embodiment of the present application, in order to train and obtain the retrieval ranking model, the basic model needs to be iteratively trained multiple times. In each iterative training process, a training sample for this iterative training can be obtained, and the training sample includes multiple annotated data, each of which includes a first text segment with a label (positive label or negative label), a retrieval question and a retrieval answer. Taking the positive sample as an example, the label of the annotated data is a positive label, the first text segment is the document content based on which the large language model answers the retrieval, and the retrieval answer is the correct answer to the retrieval question.

[0068] In practical applications, the domain document can be segmented at the text representation layer to obtain a first text segment, and the first text segment, retrieval question, and retrieval answer can be annotated.

[0069] S203, perform keyword extraction on the first text segment, the search question and the search answer respectively to obtain text segment keywords, search question keywords and search answer keywords; and perform vector embedding processing on the first text segment, the search question and the search answer respectively to obtain a text segment vector, a search question vector and a search answer vector.

[0070] In the embodiment of the present application, keyword extraction is performed on the first text segment, the search question, and the search answer, respectively, to obtain a text segment keyword corresponding to the first text segment, a search question keyword corresponding to the search question, and a search answer keyword corresponding to the search answer. In addition, vector embedding processing is performed on the first text segment, the search question, and the search answer, respectively, to obtain a text segment vector corresponding to the first text segment, a search question vector corresponding to the search question, and a search answer vector corresponding to the search answer.

[0071] In practical applications, keyword extraction and vector embedding processing can be performed on the first text segment, retrieval question and retrieval answer respectively at the text feature layer.

[0072] S204, matrix alignment of text segment keywords, retrieval question keywords and retrieval answer keywords is performed through a generalized matrix decomposition layer; and matrix alignment of text segment vectors, retrieval question vectors and retrieval answer vectors is performed through a multi-layer perceptron layer.

[0073] In an embodiment of the present application, matrix decomposition technology is used to input text fragment keywords, retrieval question keywords and retrieval answer keywords into a generalized matrix decomposition layer A, so that the generalized matrix decomposition layer A aligns the text fragment keywords, retrieval question keywords and retrieval answer keywords into the same matrix; in addition, the text fragment vector, retrieval question vector and retrieval answer vector are input into a multi-layer perceptron layer B, so that the multi-layer perceptron layer B aligns the text fragment vector, retrieval question vector and retrieval answer vector into the same matrix.

[0074] S205, jointly solving the matrix alignment result of the generalized matrix decomposition layer and the matrix alignment result of the multi-layer perceptron layer through the matrix decomposition layer.

[0075] In an embodiment of the present application, the matrix alignment result of the generalized matrix decomposition layer A and the matrix alignment result of the multi-layer perceptron layer B are input to the matrix decomposition layer C, which merges and jointly solves them, and outputs the joint solution result through the Softmax layer.

[0076] In practical applications, the generalized matrix decomposition layer A, the multi-layer perceptron layer B and the matrix decomposition layer C can be deployed in the knowledge retrieval layer to perform matrix alignment and joint solution in the knowledge retrieval layer.

[0077] S206, calculating the loss function value of this iterative training based on the joint solution result and label of the matrix decomposition layer.

[0078] In the embodiment of the present application, the loss function value of this iterative training is calculated based on the joint solution result and label of the matrix decomposition layer C.

[0079] S207, if the loss function value does not meet the corresponding end condition, adjust the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer respectively, and return to execute step S202.

[0080] In an embodiment of the present application, if the loss function value does not tend to converge and the number of iterations does not reach the corresponding number threshold, the network parameters of the generalized matrix decomposition layer A, the multi-layer perceptron layer B and the matrix decomposition layer C are adjusted respectively, and the next iterative training is entered, and the execution of step S202 is returned.

[0081] S208: If the loss function value meets the corresponding end condition, the training is terminated and the trained basic model is used as the retrieval ranking model.

[0082] In the embodiment of the present application, if the loss function value tends to converge, or the number of iterations reaches the corresponding number threshold, the training is terminated, and the trained basic model is used as the retrieval ranking model. The retrieval ranking model can improve the knowledge recall ability of retrieving relevant text fragments from the index library.

[0083] In one possible implementation, in order to improve the large language model's ability to understand domain document knowledge and user questions and generate answers, in an embodiment of the present application, the large language model is subjected to enhanced alignment before multiple rounds of self-reflection generation based on question correction.

[0084] In the embodiment of the present application, the reinforced alignment of the large language model requires the use of domain document related data for supervised reinforced alignment to improve the large language model's understanding of domain document knowledge, user questions and answer generation capabilities. The idea of ​​reinforced alignment is adopted, and the answers output by the large language model are used as negative samples, and the answers annotated with domain documents are used as positive samples for reinforcement learning, so that the output of the model is closer to the annotated answers of the domain documents, thereby improving the performance of the large language model under domain knowledge.

[0085] Specifically, when performing enhanced alignment on a large language model, first prepare the labeled data, split the domain document into text segments, prepare questions and answers using several related text segments, and mark the answers as positive samples. Then use the large language model for reasoning, use the text segments of the domain document as the background of the question, and output the corresponding answers from the large language model, which are marked as negative samples. Finally, use the positive and negative samples to perform enhanced alignment training on the original large language model to obtain a domain-related large language model.

[0086] In a possible implementation, when the large language model prompts reasoning, the results obtained by the retrieval ranking model are used as background information to construct a prompt reasoning template, which is applied to self-reflection generation and final answer generation respectively. However, when facing ambiguous questions, the first round of predictions often cannot cover all possible explanations and answers. In this regard, the embodiment of the present application provides a self-reflection generation strategy, which covers three stages: question correction, self-reflection generation, and answer generation. Figure 4 , Figure 4 Another partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application. Figure 4 As shown, an embodiment of the present application provides a document conversation method based on self-reflection generation of a large language model, wherein step S102 "based on the target question, using the retrieval ranking model to perform multiple rounds of self-reflection generation based on problem correction on the large language model" may include steps S1021 to S1025, and these steps are described in detail below.

[0087] S1021: Obtain a second text segment.

[0088] In the embodiment of the present application, the domain document to which the large language model is attached is segmented to obtain a second text segment.

[0089] S1022, during the process of generating the current round of self-reflection, the retrieval ranking model is used to retrieve in the second text segment a first target text segment associated with the target question and a second target text segment associated with the self-reflection question generated in the previous round of self-reflection.

[0090] In the embodiment of the present application, the large language model performs multiple rounds of self-reflection generation, and in each round of self-reflection generation, the retrieval ranking model can be used to retrieve the first target text segment associated with the target question and the second target text segment associated with the self-reflection question generated by the previous round of self-reflection in the second text segment. Of course, in the first round of self-reflection generation, the self-reflection question generated by the previous round of self-reflection is empty, and accordingly, the second target text segment is empty.

[0091] Among them, the first target text segment is the topn text segment in the second text segment that is most relevant to the target question, and the second target text segment is the topn text segment in the second text segment that is most relevant to the self-reflection question generated by the previous round of self-reflection.

[0092] S1023: Calculate the difference between the first target text segment and the second target text segment.

[0093] In the embodiment of the present application, the first target text segment and the second target text segment are compared to calculate the difference between the two. In a possible implementation, the difference between the two can be calculated based on the intersection and union of the first target text segment and the second target text segment. Figure 5 , Figure 5 Another partial flow chart of a document conversation method based on self-reflection generation of a large language model provided in an embodiment of the present application. Figure 5 As shown, an embodiment of the present application provides a document conversation method based on self-reflection generation of a large language model, wherein step S1023 "calculates the difference between the first target text segment and the second target text segment" may include steps S10231 to S10233, and these steps are described in detail below.

[0094] S10231: Take the intersection of the first target text segment and the second target text segment as the intersection text segment.

[0095] In the embodiment of the present application, the first target text segment and the second target text segment may be intersected, and the intersection of the two may be used as the intersection text segment.

[0096] S10232: Take the union of the first target text segment and the second target text segment as the union text segment.

[0097] In the embodiment of the present application, the first target text segment and the second target text segment may be unioned, and the union of the two may be used as the union text segment.

[0098] S10233, calculating the difference degree according to the intersection text segment and the union text segment.

[0099] In the embodiment of the present application, the ratio of the intersection text segment to the union text segment can be used as the difference degree. Of course, when the first round of self-reflection generation is performed, the second target text segment is empty, and the difference degree is 0.

[0100] S1024, when the difference is greater than the corresponding difference threshold, or the difference is equal to 0, the pre-constructed self-reflection generation prompt template is input into the large language model, so that the large language model outputs the self-reflection questions generated by this round of self-reflection and enters the next round of self-reflection generation.

[0101] In the embodiment of the present application, if the difference is greater than the corresponding difference threshold, it means that the self-reflection questions generated by the large language model in the previous round of self-reflection are greatly different from the target questions, and if the difference is equal to 0, it means that this round of self-reflection generation is the first round of self-reflection generation. At this time, the pre-built self-reflection generation prompt template is input into the large language model, so that the large language model outputs the self-reflection questions generated by this round of self-reflection based on the self-reflection generation prompt template, and enters the next round of self-reflection generation, and returns to execute step S1022. See Figure 6 , Figure 6 An example of a self-reflection generation prompt template provided in an embodiment of the present application.

[0102] S1025, when the difference is less than the corresponding difference threshold, or the round of self-reflection generation of the large language model reaches the corresponding round threshold, the self-reflection generation of the large language model is terminated.

[0103] Taking into account that the first target text segment obtained based on the target question retrieval and the second target text segment obtained based on the self-reflection question retrieval may be exactly the same (the difference is 1 in this case) or completely different (the difference is 0 in this case), a round restriction for self-reflection generation is introduced in the embodiment of the present application.

[0104] Specifically, if the difference is less than the corresponding difference threshold, it means that the self-reflection question generated by the previous round of self-reflection of the large language model deviates slightly from the target question, and the round of self-reflection generation of the large language model reaches the corresponding round threshold, indicating that the large language model enters a cycle of self-reflection generation, and the self-reflection generation of the large language model is ended at this time.

[0105] S103, obtain the answer to the target question through reasoning of the large language model generated through multiple rounds of self-reflection.

[0106] In the embodiment of the present application, the second target text segment associated with the reflection question of the last round of self-reflection retrieved by the search and ranking model is used as background information, and the pre-built question and answer prompt template is input into the large language model to obtain the answer to the target question inferred by the large language model. Figure 7 , Figure 7 An example of a question and answer prompt template provided in an embodiment of the present application.

[0107] Through the above description, the embodiment of the present application provides a document conversation method based on self-reflection generation of a large language model, which introduces matrix decomposition technology to combine text vectors and keywords, improves the ability to retrieve and analyze relevant text fragments from the index library according to user questions, and improves the knowledge recall ability; in the large language model reinforcement alignment and reasoning, the large language model is reinforced learning to obtain a domain large language model, and in the self-reflection generation based on problem correction, the large language model is used for step-by-step reasoning and context learning capabilities, and the problem and related reference information are gradually expanded using the problem correction method, and self-reflection for generating answers is given, thereby improving the reliability of the answers of the document conversation system.

[0108] See also Figure 8 , Figure 8 A document conversation device based on self-reflection generation of a large language model is provided in an embodiment of the present application. Figure 8 As shown, the embodiment of the present application provides a document conversation device based on self-reflection generation of a large language model, including:

[0109] A model training module 10 is used to pre-train a retrieval ranking model using matrix decomposition technology;

[0110] A question acquisition module 20 is used to acquire a target question in a document session to be processed;

[0111] A self-reflection generation module 30, for performing multiple rounds of self-reflection generation based on question correction on the large language model using a retrieval ranking model based on the target question;

[0112] The answer reasoning module 40 is used to obtain the answer to the target question through reasoning of a large language model generated after multiple rounds of self-reflection.

[0113] In a possible implementation, the model training module is specifically used to:

[0114] Retrieve the basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer; obtain training samples for this iterative training; wherein the training samples include a first text segment with a label, a retrieval question and a retrieval answer; perform keyword extraction on the first text segment, the retrieval question and the retrieval answer, respectively, to obtain text segment keywords, retrieval question keywords and retrieval answer keywords; and perform vector embedding processing on the first text segment, the retrieval question and the retrieval answer, respectively, to obtain a text segment vector, a retrieval question vector and a retrieval answer vector; perform matrix alignment on the text segment keywords, the retrieval question keywords and the retrieval answer keywords through the generalized matrix decomposition layer ; and, matrix-aligning the text segment vector, the retrieval question vector and the retrieval answer vector through the multi-layer perceptron layer; jointly solving the matrix alignment results of the generalized matrix decomposition layer and the multi-layer perceptron layer through the matrix decomposition layer; calculating the loss function value of this iterative training based on the joint solution result and the label of the matrix decomposition layer; if the loss function value does not meet the corresponding end condition, respectively adjust the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer, and return to execute the step of obtaining the training samples for this iterative training; if the loss function value meets the corresponding end condition, end the training, and use the trained basic model as the retrieval ranking model.

[0115] In a possible implementation, the self-reflection generation module 30 is specifically used to:

[0116] Obtain a second text segment; during this round of self-reflection generation, use the retrieval ranking model to retrieve in the second text segment a first target text segment associated with the target question and a second target text segment associated with the self-reflection question generated in the previous round of self-reflection; calculate the difference between the first target text segment and the second target text segment; when the difference is greater than the corresponding difference threshold, or the difference is equal to 0, input the pre-constructed self-reflection generation prompt template into the large language model, so that the large language model outputs the self-reflection question generated in this round of self-reflection, and enters the next round of self-reflection generation; when the difference is less than the corresponding difference threshold, or the round of self-reflection generation of the large language model reaches the corresponding round threshold, end the self-reflection generation of the large language model.

[0117] In a possible implementation, the self-reflection generation module 30 for calculating the difference between the first target text segment and the second target text segment is specifically used to:

[0118] The intersection of the first target text segment and the second target text segment is taken as the intersection text segment; the union of the first target text segment and the second target text segment is taken as the union text segment; and the difference degree is calculated according to the intersection text segment and the union text segment.

[0119] In a possible implementation, the document conversation device based on self-reflection generation of a large language model further includes:

[0120] Reinforcement learning module for reinforcement alignment of large language models.

[0121] It should be noted that the detailed functions of each module in the embodiment of the present application can be found in the corresponding public part of the above-mentioned document conversation method embodiment based on self-reflection generation of a large language model, which will not be repeated here.

[0122] The present application also provides an electronic device in an embodiment. Fig. 9 , Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0123] like Fig. 9 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0124] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a memory card, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0125] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any of the document conversation methods based on large language model self-reflection generation provided in the embodiments of the present application.

[0126] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the document conversation methods based on self-reflection generation of a large language model provided in the embodiments of the present application.

[0127] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0128] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0129] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0130] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A document conversation method based on self-reflection generation of a large language model, characterized in that: The document conversation method based on self-reflection generation of a large language model includes: Get the target question in the document session to be processed; Based on the target question, a retrieval ranking model is used to generate multiple rounds of self-reflection based on question correction for the large language model, wherein the retrieval ranking model is pre-trained using matrix decomposition technology; Obtaining the answer to the target question through reasoning of the large language model generated through multiple rounds of self-reflection; The process of pre-training the retrieval ranking model using matrix decomposition technology includes: Retrieving a basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer; Acquire training samples for this iterative training; wherein the training samples include a first text segment with a label, a search question, and a search answer; Performing keyword extraction on the first text segment, the search question, and the search answer, respectively, to obtain text segment keywords, search question keywords, and search answer keywords; and performing vector embedding processing on the first text segment, the search question, and the search answer, respectively, to obtain a text segment vector, a search question vector, and a search answer vector; Performing matrix alignment on the text segment keywords, the search question keywords and the search answer keywords through the generalized matrix decomposition layer; and performing matrix alignment on the text segment vector, the search question vector and the search answer vector through the multi-layer perceptron layer; The matrix alignment result of the generalized matrix decomposition layer and the matrix alignment result of the multi-layer perceptron layer are jointly solved by the matrix decomposition layer; Calculate the loss function value of this iterative training according to the joint solution result of the matrix decomposition layer and the label; If the loss function value does not meet the corresponding end condition, respectively adjust the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer, and return to execute the step of obtaining the training samples for this iterative training; If the loss function value meets the corresponding end condition, the training is terminated, and the trained basic model is used as the retrieval ranking model.

2. The document conversation method based on self-reflection generation of a large language model according to claim 1 is characterized in that: The method of using the retrieval ranking model to perform multiple rounds of self-reflection generation based on question correction on the large language model based on the target question includes: Get the second text segment; In the process of generating the current round of self-reflection, the retrieval ranking model is used to retrieve the first target text segment associated with the target question and the second target text segment associated with the self-reflection question generated in the previous round of self-reflection in the second text segment; Calculating the difference between the first target text segment and the second target text segment; When the difference is greater than the corresponding difference threshold or the difference is equal to 0, the pre-built self-reflection generation prompt template is input into the large language model, so that the large language model outputs the self-reflection question generated in this round of self-reflection, and enters the next round of self-reflection generation; When the difference is less than the corresponding difference threshold, or the round of self-reflection generation of the large language model reaches the corresponding round threshold, the self-reflection generation of the large language model is terminated.

3. The document conversation method based on self-reflection generation of a large language model according to claim 2 is characterized in that: The calculating the difference between the first target text segment and the second target text segment includes: taking the intersection of the first target text segment and the second target text segment as the intersection text segment; taking the union of the first target text segment and the second target text segment as a union text segment; The difference degree is calculated according to the intersection text segment and the union text segment.

4. The document conversation method based on self-reflection generation of a large language model according to claim 1 is characterized in that: Before the method of using the retrieval ranking model to perform multiple rounds of self-reflection generation based on question correction on the large language model based on the target question, the method of document conversation based on self-reflection generation of the large language model also includes: The large language model is subjected to enhanced alignment.

5. A document conversation device based on self-reflection generation of a large language model, characterized in that: The document conversation device based on self-reflection generation of a large language model includes: A model training module is used to pre-train a retrieval ranking model using matrix decomposition technology; A question acquisition module is used to acquire target questions in the document session to be processed; A self-reflection generation module, used to perform multiple rounds of self-reflection generation based on problem correction on a large language model using the retrieval ranking model based on the target problem; An answer reasoning module, used to obtain an answer to the target question through reasoning with the large language model generated after multiple rounds of self-reflection; Wherein, the model training module is specifically used for: Retrieve a basic model; wherein the basic model includes a generalized matrix decomposition layer, a multi-layer perceptron layer and a matrix decomposition layer; obtain training samples for this iterative training; wherein the training samples include a first text segment with a label, a search question and a search answer; perform keyword extraction on the first text segment, the search question and the search answer, respectively, to obtain text segment keywords, search question keywords and search answer keywords; and perform vector embedding processing on the first text segment, the search question and the search answer, respectively, to obtain a text segment vector, a search question vector and a search answer vector; perform matrix alignment on the text segment keywords, the search question keywords and the search answer keywords through the generalized matrix decomposition layer; and, The text segment vector, the retrieval question vector and the retrieval answer vector are matrix-aligned through the multi-layer perceptron layer; the matrix alignment results of the generalized matrix decomposition layer and the multi-layer perceptron layer are jointly solved through the matrix decomposition layer; the loss function value of this iterative training is calculated according to the joint solution result of the matrix decomposition layer and the label; if the loss function value does not meet the corresponding end condition, the network parameters of the generalized matrix decomposition layer, the multi-layer perceptron layer and the matrix decomposition layer are adjusted respectively, and the step of obtaining the training samples for this iterative training is returned to execute; if the loss function value meets the corresponding end condition, the training is terminated, and the trained basic model is used as the retrieval ranking model.

6. A computer program product, characterized in that It includes computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the document conversation method based on self-reflection generation of a large language model as described in any one of claims 1 to 4.

7. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the document conversation method based on self-reflection generation of a large language model as described in any one of claims 1 to 4.

8. A computer storage medium, characterized in that The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the document conversation method based on self-reflection generation based on a large language model as described in any one of claims 1 to 4.

Images