A self-reinforcement learning child question-answering method, system and device based on problem rewriting and prompt optimization and a storage medium

Abstract

The application discloses a self-reinforcement learning child question and answer method and system based on question rewriting and prompt optimization, a device and a storage medium, and belongs to the natural language processing field. The method rewrites the obtained fuzzy question into a complete semantic question through fact retrieval and question rewriting, combines a TOP-3 candidate relationship to construct a rewriting editing matrix to generate an initial prompt, a prompt optimization model optimizes the initial prompt to generate a knowledge perception prompt by using the above information, a question and answer model generates a natural language answer based on the knowledge perception prompt, and the parameters of the question rewriting model and the prompt optimization model are continuously iteratively optimized through reinforcement learning according to a reward signal of the answer quality. The application reconstructs the semantic of the fuzzy question to improve the understanding ability of the question and answer model to the question, and realizes the knowledge perception optimization of the prompt to narrow the preference gap between the question and answer model and enhance the knowledge guiding property of the prompt, thereby improving the answer accuracy and robustness of the question and answer model in the child interaction scene.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a self-reinforcing learning method, system, device, and storage medium for children's question answering based on question rewriting and prompt optimization. Background Technology

[0002] As a key application in digital education, intelligent question-answering systems for children are gradually becoming important tools to assist children's learning and promote the equitable distribution of educational resources, demonstrating enormous application potential and research value. Although intelligent question-answering technology is constantly advancing, it faces unique challenges in primary school education scenarios. Compared to adults, children's language expression involves semantic ambiguity, incomplete expressions, and unclear pronunciation. These phenomena affect the understanding of questions and the retrieval of answers, leading to irrelevant answers or unanswered questions. Given the unique characteristics of children's language expression, existing adult-oriented intelligent question-answering systems struggle to meet children's interactive needs. They lack the ability to semantically complete information by combining children's cognitive level with the context of the dialogue, cannot effectively transform fragmented expressions into standardized queries, and their understanding of children's ambiguous questions is inaccurate, resulting in insufficient accuracy in subsequent answer retrieval. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a self-reinforcing learning method, system, device and storage medium for children's question answering based on question rewriting and prompt optimization. By semantically reconstructing and knowledge-enhancing children's fuzzy questions, the invention improves the question answering model's ability to understand fuzzy questions, thereby improving the accuracy and robustness of the question answering model's answers in children's interactive scenarios.

[0004] This invention provides the following technical solution:

[0005] Firstly, a self-reinforcing learning method for children's question-and-answering based on question rewriting and prompt optimization is provided, including:

[0006] Obtain the original fuzzy question, and retrieve the relevant fact set based on the original fuzzy question;

[0007] The original fuzzy question and its related fact set are input into a pre-trained question rewriting model to obtain the rewritten question.

[0008] Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, a rewriting and editing matrix is ​​constructed, wherein the TOP-3 candidate relations are determined during the question rewriting process based on the semantic similarity between the original fuzzy statement and the relevant fact set;

[0009] The rewrite / edit matrix and the relevant fact set are input into the pre-acquired prompt framework to obtain the initial prompt;

[0010] The original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix, and the initial prompt are input into a pre-built prompt optimization model to optimize the initial prompt and obtain a knowledge-aware prompt.

[0011] The knowledge-aware prompts are input into a pre-acquired question-answering model to obtain a natural language answer;

[0012] Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using the reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism.

[0013] As a preferred embodiment of the present invention, the method for determining the TOP-3 candidate relationship includes:

[0014] Key entities and relational phrases in the original fuzzy question were extracted using named entity recognition and dependency parsing techniques.

[0015] Semantically match the relation phrases with the candidate relations contained in the relevant fact set;

[0016] Based on the semantic matching results, the top three matching degrees from the candidate relations are selected as the TOP-3 candidate relations;

[0017] The semantic matching of the relation phrase with the candidate relations contained in the relevant fact set includes: encoding the relation phrase and each candidate relation into a semantic vector, and calculating the similarity between the semantic vector of the relation phrase and the semantic vector of each candidate relation using the cosine similarity method.

[0018] As a preferred embodiment of the present invention, the training process of the problem rewriting model includes:

[0019] Training samples are constructed based on the original training questions, the TOP-3 training candidate relationships corresponding to the original training questions, and manually annotated questions that are semantically consistent with the original training questions.

[0020] The training objective of the problem rewriting model is defined as follows:

[0021] ;

[0022] in, This indicates that the problem involves rewriting all trainable parameters of the model. This indicates taking the parameter at the maximum value. , Represents conditional probability. This indicates manually tagged questions. Indicates based on the original fuzzy question. Relevant facts corresponding to the TOP-3 candidate relationship A function to generate rewritten questions. This indicates the total number of relevant facts.

[0023] As a preferred embodiment of the present invention, a rewrite-edit matrix is ​​constructed based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relationship, including:

[0024] Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relationship, the lexical differences between the original fuzzy question and the rewritten question are identified through sequence alignment, and corresponding difference tags are generated.

[0025] For the newly added content in the rewritten question due to the difference, the editing operation is classified into one of replacement, modification, insertion or no operation, depending on whether it originates from the TOP-3 candidate relationship and whether there is deleted content in the corresponding position in the original fuzzy question.

[0026] Based on the difference markers and editing operation types for all word positions, a rewrite-edit matrix is ​​constructed, which is used to characterize the transformation relationship from the original question to the rewritten question.

[0027] As a preferred embodiment of the present invention, the original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix, and the initial prompt are input into a pre-constructed prompt optimization model to optimize the initial prompt and obtain a knowledge-aware prompt, including:

[0028] The GloVe embedding model is used to obtain the embedding vectors of the original fuzzy question and the rewritten question.

[0029] The embedding vectors of each relevant fact in the relevant fact set are obtained using the TransE embedding model. The dot product similarity between the embedding vector of the rewritten question and the embedding vectors of each relevant fact is calculated. Based on the dot product similarity, the three relevant facts with the highest dot product similarity are selected as the TOP-3 relevant facts, and the embedding vectors of the TOP-3 relevant facts are obtained.

[0030] Based on the embedding vector of the original fuzzy question, the embedding vector of the rewritten question, and the editing operation type in the rewritten editing matrix, the embedding vector of the editing matrix is ​​constructed.

[0031] The embedding vectors of the TOP-3 related facts and the embedding vectors of the edit matrix are concatenated to obtain the comprehensive embedding;

[0032] Based on the rewritten question and the comprehensive embedding, a prompt generation mechanism is used to generate knowledge-aware prompts.

[0033] As a preferred embodiment of the present invention, the embedding vectors of the original fuzzy question and the rewritten question are obtained using the GloVe embedding model, including:

[0034] The rewritten question and the original fuzzy question are segmented into words. The word vectors of each word in the rewritten question and the original fuzzy question are obtained using the GloVe embedding model. Average pooling is then performed on the word vectors of each word in the rewritten question and the original fuzzy question to obtain the embedding vectors of the rewritten question and the original fuzzy question.

[0035] The TransE embedding model is used to obtain the embedding vectors of each relevant fact in the relevant fact set, including:

[0036] The relevant facts in the relevant fact set are converted into JSON dictionary format and segmented. The TransE embedding model is used to obtain the word vectors of the terms of each relevant fact. The word vectors are then averaged and pooled to obtain the embedding vector of each relevant fact.

[0037] As a preferred embodiment of the present invention, the question-answering model adopts the ChatGLM2-6B language model, and the prompting framework adopts the MOSS-003-SFT prompting framework.

[0038] Secondly, a self-reinforcing learning question-and-answer system for children based on question rewriting and prompt optimization is provided, including:

[0039] The data acquisition module is used to acquire the original fuzzy question and filter the relevant fact set from the pre-acquired knowledge graph based on the original fuzzy question;

[0040] The rewriting module inputs the original fuzzy question and its related fact set into a pre-trained question rewriting model to obtain the rewritten question.

[0041] The rewrite-edit matrix construction module is used to construct a rewrite-edit matrix based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, wherein the TOP-3 candidate relations are determined based on the semantic similarity between the original fuzzy statement and the relevant fact set during the question rewriting process;

[0042] An initial prompt generation module is used to input the rewrite and edit matrix and the relevant fact set into a pre-acquired prompt framework to obtain an initial prompt;

[0043] The knowledge-aware prompt generation module is used to input the original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix and the initial prompt into a pre-built prompt optimization model, and optimize the initial prompt to obtain the knowledge-aware prompt;

[0044] The question-answering module is used to input the knowledge-aware prompts into a pre-acquired question-answering model to obtain natural language answers;

[0045] Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using the reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism.

[0046] Thirdly, a self-reinforcing learning child question-and-answer device based on question rewriting and prompt optimization is provided, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method described in the first aspect.

[0047] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described in the first aspect.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] This invention provides a self-reinforcing learning question-answering method for children based on question rewriting and prompt optimization. By using a question rewriting model and a prompt optimization model to perform semantic clarification and format optimization on ambiguous questions, the accuracy of question understanding is improved, and prompts that are more in line with user intent are generated. This enhances the question-answering model's ability to understand ambiguous questions and improves the accuracy of question-answering results. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of a self-reinforcement learning question-answering method based on question rewriting and prompt optimization in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of a problem rewriting model based on knowledge graph relationship alignment in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the rewrite-edit matrix construction in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the fuzzy question rewriting and response generation method in an embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0055] Example 1

[0056] This embodiment provides a self-reinforcing learning method for children's question-and-answering based on question rewriting and prompt optimization. For example... Figure 1 As shown, the specific steps include the following:

[0057] Step S1: Obtain the original fuzzy question and retrieve the relevant fact set based on the original fuzzy question;

[0058] In step S1, the original fuzzy question is obtained, and knowledge graph triples related to the original fuzzy question are retrieved based on the Ground Truth Subgraph (GTS) retrieval method. As a set of relevant facts, it is used for subsequent question rewriting and prompt optimization tasks. In this embodiment, the original fuzzy question is specifically "Why does Sister Moon sometimes look fat and sometimes look thin?".

[0059] Step S2: Input the original fuzzy question and its related fact set into the pre-trained question rewriting model to obtain the rewritten question;

[0060] Step S2.1: Extract key entities and relational phrases from the original fuzzy question using named entity recognition and dependency parsing techniques; in this embodiment, the key entity is "Moon" and the relational phrase is "lookfatandlookthin".

[0061] Step S2.2: Semantically match the relation phrase with the candidate relations contained in the relevant fact set; specifically, encode the relation phrase and each candidate relation into a semantic vector, and calculate the similarity between the semantic vector of the relation phrase and the semantic vector of each candidate relation using the cosine similarity method. In this embodiment, candidate relations include "orbit", "lunarphase", "phasesofwaxingandwaning", "appearancechange", etc.

[0062] Step S2.3: Based on the semantic matching results, select the top three matching degrees from the candidate relations as TOP-3 candidate relations; the candidate relations and their similarities are as follows: "phasesofwaxingandwaning" has a similarity of 0.92, "lunarphases" has a similarity of 0.90, and "appearancechange" has a similarity of 0.85.

[0063] Step S2.4: The question rewriting model is the T5 model. The T5 model obtains the rewritten question based on the original fuzzy statement and the TOP-3 candidate relations: "Why do you think he has phases of sleeping and waning?".

[0064] The training process of the problem rewriting model includes:

[0065] Training samples are constructed based on the original training questions, the TOP-3 training candidate relationships corresponding to the original training questions, and manually annotated questions that are semantically consistent with the original training questions.

[0066] The training objective of the problem rewriting model is defined as follows:

[0067] ;

[0068] in, This indicates that the problem involves rewriting all trainable parameters of the model. This indicates taking the parameter at the maximum value. , Represents conditional probability. This indicates manually tagged questions. Indicates based on the original fuzzy question. Relevant facts corresponding to the TOP-3 candidate relationship A function to generate rewritten questions. This indicates the total number of relevant facts.

[0069] The above training process is combined with the original problem. and related facts Find the optimal parameters by maximizing likelihood estimation. Thus, the induced function In other words, the rewritten questions generated by the model can be as close as possible to the manually annotated questions, so as to generate semantically clear and fully expressed questions, thereby improving the accuracy and coherence of the question-answering system.

[0070] Step S3: Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, construct a rewrite editing matrix, wherein the TOP-3 candidate relations are determined during the question rewriting process based on the semantic similarity between the original fuzzy statement and the relevant fact set;

[0071] In step S3, a rewrite-edit matrix is ​​constructed based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, including:

[0072] Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relationship, the lexical differences between the original fuzzy question and the rewritten question are identified through sequence alignment, and corresponding difference tags are generated.

[0073] For the newly added content in the rewritten question due to the difference, the editing operation is classified into one of replacement, modification, insertion or no operation, depending on whether it originates from the TOP-3 candidate relationship and whether there is deleted content in the corresponding position in the original fuzzy question.

[0074] Based on the difference markers and editing operation types for all word positions, a rewrite-edit matrix is ​​constructed, which is used to characterize the transformation relationship from the original question to the rewritten question.

[0075] Specifically, such as Figure 3 The diagram illustrates the construction of the rewrite-edit matrix. It uses the input question "Why does Sister Moon sometimes look fat and sometimes look thin?" and its rewritten question "Why does Sister Moon have phases of sleeping and waning?" as an example.

[0076] Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relation, lexical differences between the original fuzzy question and the rewritten question are identified through sequence alignment, and corresponding difference markers are generated, specifically including:

[0077] Initialize the Longest Common Subsequence (LCS) as an empty list. Use the LCS algorithm to find the longest common subsequence between the two questions, i.e., ["Why", "does", "moon", "and", "?"]. Next, examine the words in the rewritten questions, marking words not in the LCS as "ADD", such as "the", "have", etc., and merging consecutive "ADD" tags into a span. Similarly, examine the words in the original questions, marking words not in the LCS as "DEL", such as "Sister", "fat", "thin", etc., and merging consecutive "DEL" tags into a span. "ADD" and "DEL" are the difference markers mentioned above.

[0078] For the newly added content in the rewritten question due to differences, the editing operation is classified into one of the following based on whether it originates from the TOP-3 candidate relationship and whether there is deleted content in the corresponding position in the original fuzzy question: replacement, modification, insertion, or no operation. Specifically, it includes:

[0079] Initialize an empty edit matrix that matches the dimensions of the original question and the TOP-3 candidate relationships ("phasesofwaxingandwaning", "lunarphases", "appearancechange"). Iterate through each marker, classifying the edit operation as "replace", "modify", "insert", or "no operation" based on the marker type and position information, and populate the edit matrix accordingly. Specifically, analyze each marker in the ADD span to determine its corresponding edit operation type: if the marker word is in the TOP-3 candidate relationship and the corresponding position in the original question has a DEL span, it is a "replace" operation; if the marker word is not in the TOP-3 candidate relationship, but modifies the corresponding word in the original question and the corresponding position in the original question has a DEL span, it is a "modify" operation; if the marker word is not in the TOP-3 candidate relationship and the corresponding position in the original question has no DEL span, it is an "insert" operation; otherwise, it is a "no operation".

[0080] These operation types are populated into the edit matrix, thus fully constructing the rewrite edit matrix.

[0081] This rewrite-edit matrix not only records the mapping relationship and editing operations between the original question and the rewritten question, but also accurately connects the ambiguous and inaccurate parts of the user's question with relevant information in the knowledge base by filtering out unimportant details and noise. This ensures that the system accurately grasps the user's intent and enhances the accuracy and relevance of the subsequent answer generation module.

[0082] Step S4: Input the rewrite-edit matrix and the relevant fact set into the pre-acquired hint frame to obtain the initial hint. The hint frame adopts the MOSS-003-SFT hint frame.

[0083] Step S5: Input the original fuzzy question, the relevant fact set, the rewritten question, the rewrite edit matrix, and the initial prompt into the pre-built prompt optimization model to optimize the initial prompt and obtain the knowledge-aware prompt;

[0084] In step S5, the original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix, and the initial prompt are input into a pre-built prompt optimization model to optimize the initial prompt and obtain a knowledge-aware prompt, including:

[0085] Obtain the embedding vector of the original fuzzy question using the GloVe embedding model. Embedded vectors of rewritten questions ;

[0086] Using the TransE embedding model to obtain the embedding vectors of each relevant fact in the relevant fact set. ;

[0087] Calculate the dot product similarity between the embedding vector of the rewritten question and the embedding vectors of each relevant fact. , represented as:

[0088] ;

[0089] Based on dot product similarity The three relevant facts with the highest dot product similarity are selected as the TOP-3 relevant facts, and the embedding vectors of the TOP-3 relevant facts are obtained, represented as follows:

[0090] ,in, , , These are the embedding vectors representing the first three facts, which are used to prepare for subsequent embedding concatenation.

[0091] Based on the embedding vector of the original fuzzy question, the embedding vector of the rewritten question, and the editing operation type in the rewritten editing matrix, construct the embedding vector Emb_matrix of the editing matrix;

[0092] The embedding vectors of the TOP-3 relevant facts and the embedding vectors of the edit matrix are concatenated to obtain the comprehensive embedding, which is represented as:

[0093] ;

[0094] in, Indicates comprehensive embedding, Indicates a splicing operation;

[0095] Based on the rewritten question and the aforementioned integrated embedding, a knowledge-aware prompt is generated using a prompt generation mechanism, represented as follows:

[0096] ;

[0097] in, This indicates a knowledge perception prompt. This indicates the mechanism for generating the prompt.

[0098] The embedding vectors of the original fuzzy question and the rewritten question are obtained using the GloVe embedding model, including:

[0099] The rewritten question and the original fuzzy question are segmented into words. The word vectors of each word in the rewritten question and the original fuzzy question are obtained using the GloVe embedding model. The word segmentation result of the rewritten question is represented as follows: ;

[0100] The word segmentation result of the original fuzzy question is: ;

[0101] Average pooling is performed on the word vectors of each word element in the rewritten question and the word vectors of each word element in the original fuzzy question to obtain the embedding vectors of the original fuzzy question and the rewritten question, respectively, as follows:

[0102] ;

[0103] ;

[0104] in, The embedding vector representing the rewritten question. Indicates the rewriting of the question's first part One portion, This indicates the total number of components in the rewritten question. Indicates GloVe embedding, The embedding vector representing the original fuzzy question. The first character representing the original fuzzy question Each word segmentation result This represents the total number of components in the original fuzzy question.

[0105] The TransE embedding model is used to obtain the embedding vectors of each relevant fact in the relevant fact set, including:

[0106] The relevant facts in the relevant fact set are converted into JSON dictionary format and segmented. The TransE embedding model is used to obtain the word vectors of each relevant fact. The word vectors are then averaged and pooled to obtain the embedding vector of each relevant fact, as shown below:

[0107] ;

[0108] in, An embedding vector representing any relevant fact. Indicates the first Each word segmentation result This represents the total number of components of the relevant facts. Indicates TransE embedding;

[0109] Knowledge-aware prompts retain the core semantics of the original question while integrating relevant factual information from the knowledge graph. They possess stronger knowledge relevance and semantic coherence, providing effective guidance for question-answering systems to generate more accurate and context-appropriate answers.

[0110] Step S6: Input the knowledge-aware prompts into the pre-acquired question-answering model to obtain a natural language answer;

[0111] Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using this reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism. The formula for calculating the reward signal is:

[0112] ;

[0113] in, As a reward signal, The weighting parameters are used to adjust the relative importance of F1 and ACC. The F1 score is the harmonic mean of the precision and recall of the question-answering results, which is used to comprehensively measure the accuracy and completeness of the question-answering results. The ACC score measures the proportion of samples in the total sample whose output of the question-answering model completely matches the standard answer.

[0114] The question-answering model uses the ChatGLM2-6B language model.

[0115] Figure 4 This is a schematic diagram illustrating the question rewriting and response generation method based on a specific example according to the present invention. (Refer to...) Figure 4The specific steps are as follows: First, based on the input fuzzy question "Who did Li Bai like to play with?", relevant fact sets are retrieved from the knowledge graph, as shown in the subgraph of the knowledge graph related to Li Bai, including information such as his date of birth, wife, birthplace, and close friends. Then, the retrieved relevant fact sets are pruned, filtering out facts highly relevant to the intent of the fuzzy question, such as "(Li Bai,confidant,Meng Haoran)" and "(Li Bai,confidant,Du Fu)". Knowledge graph pruning technology reduces interference from irrelevant information, significantly improving the accuracy and efficiency of question rewriting and answer generation. Second, the question rewriting module rewrites the original fuzzy question based on the fuzzy question and the most relevant set of triples, generating a more semantically clear question, "Who was the confidence of Li Bai?". This process aims to improve the semantic clarity of the question and the system's understanding ability. Secondly, the prompt optimization module generates an optimized prompt based on the rewritten question and pruned triples: "Your task is to answer the question based on the knowledge that might be relevant and there written the question. Try to use the original words from the given knowledge to answer the question." This prompt aims to better utilize information in the knowledge graph to guide the system in generating the answer. Finally, the dialogue question-answering module uses the Sparql query language, based on the optimized prompt and pruned triples, to generate the final answer: "The confidants of Li Bai include Meng Haoran and Du Fu, among others." The entire process demonstrates the innovation of this invention in question rewriting, prompt optimization, and knowledge graph-based answer generation. Through multi-step processing, it effectively improves the accuracy and intelligence of the question-answering system.

[0116] Example 2

[0117] This embodiment provides a self-reinforcing learning question-and-answer system for children based on question rewriting and prompt optimization, including:

[0118] The data acquisition module is used to acquire the original fuzzy question and filter the relevant fact set from the pre-acquired knowledge graph based on the original fuzzy question;

[0119] The rewriting module inputs the original fuzzy question and its related fact set into a pre-trained question rewriting model to obtain the rewritten question.

[0120] The rewrite-edit matrix construction module is used to construct a rewrite-edit matrix based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, wherein the TOP-3 candidate relations are determined based on the semantic similarity between the original fuzzy statement and the relevant fact set during the question rewriting process;

[0121] An initial prompt generation module is used to input the rewrite and edit matrix and the relevant fact set into a pre-acquired prompt framework to obtain an initial prompt;

[0122] The knowledge-aware prompt generation module is used to input the original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix and the initial prompt into a pre-built prompt optimization model, and optimize the initial prompt to obtain the knowledge-aware prompt;

[0123] The question-answering module is used to input the knowledge-aware prompts into a pre-acquired question-answering model to obtain natural language answers;

[0124] Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using the reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism.

[0125] Example 3

[0126] This embodiment provides a self-reinforcing learning question-and-answer device for children based on question rewriting and prompt optimization, including a processor and a storage medium. The storage medium is used to store instructions. The processor is used to operate according to the instructions to execute the steps of the self-reinforcing learning question-and-answer method for children based on question rewriting and prompt optimization described in Embodiment 1.

[0127] Example 4

[0128] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the self-reinforcing learning child question-and-answer method based on question rewriting and prompt optimization described in Embodiment 1.

[0129] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A self-reinforcing learning method for children's question-and-answering based on question rewriting and prompt optimization, characterized in that, include: Obtain the original fuzzy question, and retrieve the relevant fact set based on the original fuzzy question; The original fuzzy question and its related fact set are input into a pre-trained question rewriting model to obtain the rewritten question. Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, a rewriting and editing matrix is ​​constructed, wherein the TOP-3 candidate relations are determined during the question rewriting process based on the semantic similarity between the original fuzzy statement and the relevant fact set; The rewrite / edit matrix and the relevant fact set are input into the pre-acquired prompt framework to obtain the initial prompt; The original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix, and the initial prompt are input into a pre-built prompt optimization model to optimize the initial prompt and obtain a knowledge-aware prompt. The knowledge-aware prompts are input into a pre-acquired question-answering model to obtain a natural language answer; Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using the reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism.

2. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization according to claim 1, characterized in that, The method for determining the TOP-3 candidate relationship includes: Key entities and relational phrases in the original fuzzy question were extracted using named entity recognition and dependency parsing techniques. Semantically match the relation phrases with the candidate relations contained in the relevant fact set; Based on the semantic matching results, the top three matching degrees from the candidate relations are selected as the TOP-3 candidate relations; The semantic matching of the relation phrase with the candidate relations contained in the relevant fact set includes: encoding the relation phrase and each candidate relation into a semantic vector, and calculating the similarity between the semantic vector of the relation phrase and the semantic vector of each candidate relation using the cosine similarity method.

3. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization according to claim 1, characterized in that, The training process of the problem rewriting model includes: Training samples are constructed based on the original training questions, the TOP-3 training candidate relationships corresponding to the original training questions, and manually annotated questions that are semantically consistent with the original training questions. The training objective of the problem rewriting model is defined as follows: ； in, This indicates that the problem involves rewriting all trainable parameters of the model. This indicates taking the parameter at the maximum value. , Represents conditional probability. This indicates manually tagged questions. Indicates based on the original training question Relevant facts related to the relationship between TOP-3 training candidates A function to generate rewritten questions. This indicates the total number of relevant facts.

4. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization according to claim 1, characterized in that, Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relationship, a rewrite-edit matrix is ​​constructed, including: Based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relationship, the lexical differences between the original fuzzy question and the rewritten question are identified through sequence alignment, and corresponding difference tags are generated. For the newly added content in the rewritten question due to the difference, the editing operation is classified into one of replacement, modification, insertion or no operation, depending on whether it originates from the TOP-3 candidate relationship and whether there is deleted content in the corresponding position in the original fuzzy question. Based on the difference markers and editing operation types for all word positions, a rewrite-edit matrix is ​​constructed, which is used to characterize the transformation relationship from the original question to the rewritten question.

5. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization according to claim 4, characterized in that, The original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix, and the initial prompt are input into a pre-built prompt optimization model to optimize the initial prompt to obtain a knowledge-aware prompt, including: The GloVe embedding model is used to obtain the embedding vectors of the original fuzzy question and the rewritten question. The embedding vectors of each relevant fact in the relevant fact set are obtained using the TransE embedding model. The dot product similarity between the embedding vector of the rewritten question and the embedding vectors of each relevant fact is calculated. Based on the dot product similarity, the three relevant facts with the highest dot product similarity are selected as the TOP-3 relevant facts, and the embedding vectors of the TOP-3 relevant facts are obtained. Based on the embedding vector of the original fuzzy question, the embedding vector of the rewritten question, and the editing operation type in the rewritten editing matrix, the embedding vector of the editing matrix is ​​constructed. The embedding vectors of the TOP-3 related facts and the embedding vectors of the edit matrix are concatenated to obtain the comprehensive embedding; Based on the rewritten question and the comprehensive embedding, a prompt generation mechanism is used to generate knowledge-aware prompts.

6. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization according to claim 5, characterized in that, The embedding vectors of the original fuzzy question and the rewritten question are obtained using the GloVe embedding model, including: The rewritten question and the original fuzzy question are segmented into words. The word vectors of each word in the rewritten question and the original fuzzy question are obtained using the GloVe embedding model. Average pooling is then performed on the word vectors of each word in the rewritten question and the original fuzzy question to obtain the embedding vectors of the rewritten question and the original fuzzy question. The TransE embedding model is used to obtain the embedding vectors of each relevant fact in the relevant fact set, including: The relevant facts in the relevant fact set are converted into JSON dictionary format and segmented. The TransE embedding model is used to obtain the word vectors of the terms of each relevant fact. The word vectors are then averaged and pooled to obtain the embedding vector of each relevant fact.

7. The self-reinforcing learning child question-answering method based on question rewriting and prompt optimization as described in claim 1, characterized in that, The question-answering model uses the ChatGLM2-6B language model, and the prompting framework uses the MOSS-003-SFT prompting framework.

8. A self-reinforcing learning question-and-answer system for children based on question rewriting and prompt optimization, characterized in that, include: The data acquisition module is used to acquire the original fuzzy question and filter the relevant fact set from the pre-acquired knowledge graph based on the original fuzzy question; The rewriting module inputs the original fuzzy question and its related fact set into a pre-trained question rewriting model to obtain the rewritten question. The rewrite-edit matrix construction module is used to construct a rewrite-edit matrix based on the original fuzzy question, the rewritten question, and the TOP-3 candidate relations, wherein the TOP-3 candidate relations are determined based on the semantic similarity between the original fuzzy statement and the relevant fact set during the question rewriting process; An initial prompt generation module is used to input the rewrite and edit matrix and the relevant fact set into a pre-acquired prompt framework to obtain an initial prompt; The knowledge-aware prompt generation module is used to input the original fuzzy question, the relevant fact set, the rewritten question, the rewrite-edit matrix and the initial prompt into a pre-built prompt optimization model, and optimize the initial prompt to obtain the knowledge-aware prompt; The question-answering module is used to input the knowledge-aware prompts into a pre-acquired question-answering model to obtain natural language answers; Specifically, after generating a natural language answer, a quality assessment score is calculated based on the natural language answer and converted into a reward signal. Using the reward signal, the parameters of the question rewriting model and the prompt optimization model are iteratively optimized through a reinforcement learning mechanism.

9. A self-reinforcing learning question-and-answer device for children based on question rewriting and prompt optimization, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the self-reinforcing learning child question-and-answer method based on question rewriting and prompt optimization as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the self-reinforcing learning child question-and-answer method based on question rewriting and prompt optimization as described in any one of claims 1 to 7.

Images