Question and answer text resetting method and device of artificial intelligence large model

Abstract

The invention provides a question and answer text resetting method and device of an artificial intelligence large model, and belongs to the technical field of artificial intelligence large models, and the method comprises the following steps: obtaining a spliced text containing a mark according to a question and answer text of the artificial intelligence large model; performing dynamic semantic coding on the spliced text by utilizing a Transform model to obtain a text semantic vector; processing the spliced text and the text semantic vector by utilizing a cursor prediction model to obtain cursor addresses of a plurality of pieces of key information in the text semantic vector; according to the cursor address of the key information, resetting the text semantic vector to obtain a reset text semantic vector; and decoding the reset text semantic vector by using a decoder to obtain a reset question and answer text. According to the method, the question and answer text can be rapidly reset, and semantic integrity and accuracy of the reset text are ensured.

Description

Technical Field

[0001] The present invention relates to a method and device for resetting question and answer text of an artificial intelligence large model, belonging to the technical field of artificial intelligence large models. Background Art

[0002] Focusing on the underlying technology of AI big models, the quality of question-answering text often affects the output effect of big models. In the actual context of multi-round dialogues, question-answering texts (especially question texts) are often incomplete. It is particularly difficult to use the existing big model framework to process incomplete texts, which often leads to inaccurate output results due to the inability to accurately understand the text information.

[0003] Incompleteness in text, especially the use of pronouns and ellipsis, leads to semantic ambiguity, which has attracted widespread attention in the field of natural language processing. To this end, researchers have proposed two core subtasks: pronoun resolution and ellipsis completion. The goal of these two tasks is to convert text that is semantically missing or dependent on a specific context into semantically complete and independently expressed text to ensure that the machine can accurately understand the meaning of the text. In modern large-scale model question-answering systems, a mainstream approach is to improve the machine's semantic understanding ability through dialogue reset technology. This method makes full use of the historical information of the dialogue, that is, the multi-round interaction records between the user and the machine. In this way, complex multi-round dialogues can be simplified into single-round dialogues, allowing the model to process user input more effectively. Specifically, the dialogue reset model analyzes the user's current input text, identifies and supplements the pronouns and ellipsis information therein, so that the machine can fully understand the user's intentions and provide more accurate and relevant responses.

[0004] In the field of natural language processing, the task of resetting text question-answering dialogues involves the operation of text sequences. Its core design challenges mainly focus on two aspects: on the one hand, it is necessary to select a neural network that can effectively handle the long-term dependencies of the text; on the other hand, the model should also have excellent feature extraction capabilities to ensure that it can capture and understand complex semantic structures. Another major challenge is how to accurately identify and supplement text content that is semantically missing. At present, the method that is more respected in the industry is to use a generation technology based on a pointer network. This technology uses a full copy mechanism, that is, referencing the user's historical conversation content through a pointer address to construct a new, semantically complete reset text word by word. This method successfully simplifies complex multi-round dialogue scenarios into a single-round dialogue, but its generation process is relatively time-consuming and inefficient. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for resetting question and answer text of a large artificial intelligence model, which can achieve the effect of quickly resetting the question and answer text and ensuring the semantic integrity and accuracy of the reset text through the context understanding ability of the Transformer architecture and the rapid extraction technology of key information based on the cursor address.

[0006] In order to achieve the above purpose / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for resetting a question and answer text of an artificial intelligence large model, comprising the following steps:

[0008] According to the question and answer text of the artificial intelligence large model, a concatenated text containing tags is obtained;

[0009] Using a Transformer model to dynamically semantically encode the concatenated text to obtain a text semantic vector;

[0010] Processing the concatenated text and the text semantic vector using a cursor prediction model to obtain cursor addresses of multiple key information in the text semantic vector;

[0011] Resetting the text semantic vector according to the cursor address of the key information to obtain a reset text semantic vector;

[0012] The reset text semantic vector is decoded by using a decoder to obtain a reset question and answer text.

[0013] In combination with the first aspect, further, the question-answer text of the artificial intelligence large model is used to obtain a concatenated text containing a mark, including:

[0014] Generate document samples according to question-and-answer texts of two adjacent rounds in the artificial intelligence big model, wherein the format of the document samples is (a, b, current), wherein a represents the input text of the previous round of the artificial intelligence big model, b represents the output text of the previous round of the artificial intelligence big model, and current represents the text to be reset in the current round, and the text to be reset in the current round includes the input text and output text of the current round of the artificial intelligence big model;

[0015] The text data in the document sample are concatenated in order, and a head tag is set before a, a connection tag is set at the connection between a and b, and between b and current, and a tail tag is set after current to obtain a concatenated text containing tags.

[0016] In combination with the first aspect, further, the using of the Transformer model to dynamically semantically encode the concatenated text to obtain a text semantic vector includes:

[0017] The Transformer-based rbt3 pre-trained model is used to process all characters in the concatenated text in parallel to obtain multiple word vectors. The formula is as follows:

[0018] ;

[0019] in, Represents the i-th word vector in the concatenated text The input embedding of Represents the i-th word vector word embedding, Represents the i-th word vector The position of the embedding, , N is the length of the concatenated text;

[0020] Map each word vector into a query vector, a key vector, and a value vector, and calculate the similarity between different word vectors in the concatenated text. The calculation formula is as follows:

[0021] ;

[0022] in, represents the similarity score of the i-th word vector to the j-th word vector in the concatenated text, Q i represents the query vector of the i-th word vector, K j represents the key vector of the jth word vector, is the dimension of the key vector;

[0023] The similarity between different word vectors is normalized using the softmax function to obtain the normalized similarity probability distribution. The formula is as follows:

[0024] ;

[0025] in, represents the normalized similarity. is the query matrix, K is the key matrix, and V is the value matrix;

[0026] The value vectors of all word vectors are weighted and summed according to the normalized similarity probability distribution to obtain the text semantic vector.

[0027] In combination with the first aspect, further, the using of the cursor prediction model to process the concatenated text and the text semantic vector to obtain the cursor addresses of multiple key information in the text semantic vector includes:

[0028] The cursor prediction model is used to process the spliced ​​text to obtain the key information of the spliced ​​text;

[0029] According to the key information of the spliced ​​text, the cursor prediction model is used to obtain the probability that each character position in the text semantic vector is the cursor address of the key information. The formula is as follows:

[0030] ;

[0031] in, The cursor address prediction probability representing the key information of the i-th character in the text semantic vector, represents the i-th character in the text semantic vector, is the learning parameter of the cursor prediction model, b is the bias value, and L is the total length of the text semantic vector;

[0032] According to the probability that each character position in the text semantic vector is the cursor address of the key information, the character position with the largest probability is selected as the cursor address of the corresponding key information.

[0033] In combination with the first aspect, further, the key information includes keywords, pronouns and / or default positions, and the cursor address of the key information includes a keyword position start cursor address, a keyword position end cursor address, a default position cursor address, a pronoun position start cursor address and a pronoun position end cursor address.

[0034] In combination with the first aspect, further, the text semantic vector is reset according to the cursor address of the key information to obtain the reset text semantic vector, including:

[0035] The keywords of the previous round of question and answer text are extracted from the text semantic vector according to the keyword position start cursor address and the keyword position end cursor address; if there is a default position cursor address, the keywords are inserted into the default position cursor address; if there is a pronoun position start cursor address and a pronoun position end cursor address, the pronouns in the text semantic vector are replaced with keywords according to the pronoun position start cursor address and the pronoun position end cursor address.

[0036] In a second aspect, the present invention provides a question-answer text resetting device for a large artificial intelligence model, comprising:

[0037] A data processing module is used to obtain a concatenated text containing a tag based on the question and answer text of the artificial intelligence large model;

[0038] A semantic encoding module, used for dynamically semantically encoding the concatenated text using a Transformer model to obtain a text semantic vector;

[0039] A cursor prediction module, used for processing the concatenated text and the text semantic vector using a cursor prediction model to obtain cursor addresses of multiple key information in the text semantic vector;

[0040] A text resetting module, used for resetting the text semantic vector according to the cursor address of the key information to obtain a reset text semantic vector;

[0041] The text output module is used to decode the reset text semantic vector using a decoder to obtain a reset question and answer text.

[0042] In conjunction with the second aspect, further, the semantic encoding module is specifically used to:

[0043] The Transformer-based rbt3 pre-trained model is used to process all characters in the concatenated text in parallel to obtain multiple word vectors. The formula is as follows:

[0044] ;

[0045] in, Represents the i-th word vector in the concatenated text The input embedding of Represents the i-th word vector word embedding, Represents the i-th word vector The position of the embedding, , N is the length of the concatenated text;

[0046] Map each word vector into a query vector, a key vector, and a value vector, and calculate the similarity between different word vectors in the concatenated text. The calculation formula is as follows:

[0047] ;

[0048] in, represents the similarity score of the i-th word vector to the j-th word vector in the concatenated text, Q i represents the query vector of the i-th word vector, K j represents the key vector of the jth word vector, is the dimension of the key vector;

[0049] The similarity between different word vectors is normalized using the softmax function to obtain the normalized similarity probability distribution. The formula is as follows:

[0050] ;

[0051] in, represents the normalized similarity. is the query matrix, K is the key matrix, and V is the value matrix;

[0052] The value vectors of all word vectors are weighted and summed according to the normalized similarity probability distribution to obtain the text semantic vector.

[0053] In combination with the second aspect, further, the cursor prediction module is specifically used to:

[0054] The cursor prediction model is used to process the spliced ​​text to obtain the key information of the spliced ​​text;

[0055] According to the key information of the spliced ​​text, the cursor prediction model is used to obtain the probability that each character position in the text semantic vector is the cursor address of the key information. The formula is as follows:

[0056] ;

[0057] in, The cursor address prediction probability representing the key information of the i-th character in the text semantic vector, represents the i-th character in the text semantic vector, is the learning parameter of the cursor prediction model, b is the bias value, and L is the total length of the text semantic vector;

[0058] According to the probability that each character position in the text semantic vector is the cursor address of the key information, the character position with the largest probability is selected as the cursor address of the corresponding key information.

[0059] In combination with the second aspect, further, the text resetting module is specifically used for:

[0060] The keywords of the previous round of question and answer text are extracted from the text semantic vector according to the keyword position start cursor address and the keyword position end cursor address; if there is a default position cursor address, the keywords are inserted into the default position cursor address; if there is a pronoun position start cursor address and a pronoun position end cursor address, the pronouns in the text semantic vector are replaced with keywords according to the pronoun position start cursor address and the pronoun position end cursor address.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The present invention proposes a question and answer text resetting method and device for a large artificial intelligence model. First, a Transformer model is selected as a neural network, and the context understanding ability of the Transformer architecture is fully utilized to extract the contextual global features of the question and answer text, and the question and answer text is converted into a vector form. Then, a cursor prediction model is used to predict the cursor addresses of multiple key information, so as to extract the key information of the text according to the cursor address for reference resolution and information omission completion, and finally obtain the complete semantic text after resetting, which can significantly improve the accuracy of text resetting and greatly reduce processing delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The figure shows a schematic diagram of the steps of a method for resetting question and answer text of a large artificial intelligence model provided by the present invention;

[0064] Figure 2 It is a schematic diagram of the technical architecture of the question and answer text resetting method in an embodiment of the present invention;

[0065] Figure 3 Shown is a structural schematic diagram of a question and answer text resetting device for a large artificial intelligence model provided by the present invention. DETAILED DESCRIPTION

[0066] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0067] Example 1

[0068] This embodiment introduces a method for resetting the question and answer text of an artificial intelligence large model, which resets and improves the question and answer text of an existing artificial intelligence large model based on Transformer, mainly including data processing, semantic encoding, cursor prediction and text output. The question and answer text of the artificial intelligence large model is first loaded through the data processing module, and each data in the question and answer text is separated, and the data text is spliced, and the head, connection and tail marks are set to obtain a complete spliced ​​text. Then, based on the rbt3 pre-trained model of Transformer, feature extraction and dynamic semantic encoding are performed on the spliced ​​text to obtain a complete text semantic vector, and then the text semantic vector is input into the cursor prediction module to obtain the cursor address of the key information of the data text, including the keyword position start cursor, the keyword position end cursor, the default position cursor, the pronoun position start cursor and the pronoun position end cursor. Finally, the question and answer text is reset and rewritten according to the five cursor addresses, and the reset text is obtained through the text output module.

[0069] like Figure 1 , Figure 2 As shown, the method of the present invention specifically comprises the following steps:

[0070] Step A: Process the question and answer text of the artificial intelligence large model to obtain a concatenated text containing tags.

[0071] In an embodiment of the present invention, the data processing module is responsible for loading and separating the question and answer texts in the multi-round question and answer process, based on the question and answer rounds of the user and the artificial intelligence large model, and separating the multi-round question and answer texts into multiple document samples according to the question and answer texts of two adjacent rounds.

[0072] Step A01: Generate multiple document samples by text separation based on the question and answer texts in the multi-round question and answer process of the artificial intelligence large model.

[0073] The format of each document sample is (a, b, current), where a represents the user's input text in the previous round (i.e., the question asked), b represents the output text of the artificial intelligence model in the previous round (i.e., the answer), and current represents the text to be reset in the current round (i.e., the user input text in the current round or the output text of the artificial intelligence model in the current round).

[0074] Step A02, splice the text data in the document sample in order, and set the head, connection and tail marks to obtain a complete spliced ​​text, recorded as S=([cls]a, [sep]b, [sep]current [sep]), wherein the [cls] mark is used to mark the starting position, that is, the head mark, and [sep] represents the text delimiter. In the present invention, when [sep] is used between two texts, it represents the connection mark, and when [sep] is used after the last text, it represents the tail mark.

[0075] Step B: Use the trained Transformer model to dynamically semantically encode the concatenated text containing tags to obtain a text semantic vector.

[0076] The Transformer model processes the input text sequence in parallel, rather than in a sequential order from front to back. In order to learn the positional relationship and contextual relationship between words in a sentence, in addition to embedding the words themselves during input, the present invention also adds position embedding, which is conducive to a better understanding of the contextual relationship.

[0077] Step B01: Use the Transformer-based rbt3 pre-trained model to convert each character in the marked concatenated text (including the mark symbol inserted in step A) into a vector form to obtain multiple word vectors. The formula is as follows:

[0078] (1)

[0079] in, Represents the i-th word vector in the concatenated text The input embedding of Represents the i-th word vector word embedding, Represents the i-th word vector The position of the embedding, , N is the length of the concatenated text.

[0080] In the embodiment of the present invention, the text is segmented into multiple tokens (which can be words, subwords or characters) by a tokenizer, and then each token is mapped to a unique index (ID). The corresponding word embedding matrix is ​​found through the index, and the index is converted into a vector form to obtain the word embedding. In order for the Transformer model to know the position of each word in the sentence, it is necessary to add a position embedding to each word. The position embedding is also a vector. The Transformer model generates position embeddings in two main ways: (1) fixed position encoding, where the position is calculated using a mathematical formula; (2) learnable position encoding, where the position vector is randomly initialized and is continuously adjusted and optimized during the training process.

[0081] Step B02: Calculate the similarity between different word vectors in the concatenated text, perform weighted summation on all word vectors, and obtain a text semantic vector.

[0082] (1) Each word vector Mapped into three new vectors of the same dimension, namely query vector, key vector, and value vector. The query vector is used to indicate what the current element wants to know, the key vector is used to indicate what other elements can answer, and the value vector is used to indicate what information other elements provide. These three vectors are used to calculate the correlation between the element and other elements.

[0083] (2) For different word vectors, by calculating the dot product between the query vector and all key vectors, a similarity score matrix can be obtained. This matrix reflects the degree of attention of each element to all other elements.

[0084] In the embodiment of the present invention, the calculation of similarity is shown as follows:

[0085] (2)

[0086] in, represents the similarity score between the i-th word vector and the j-th word vector in the concatenated text (i.e., the similarity score between a word and another word in the concatenated text), Q i represents the query vector of the i-th word vector, K j represents the key vector of the jth word vector, is the dimension of the key vector, The purpose is to keep the value stable.

[0087] (3) Use the softmax function to normalize the similarities between different word vectors and convert the similarity scores obtained in step B02 into probability distributions so that the sum of the values ​​of each row is equal to 1, thereby ensuring that the degree of attention of each element to other elements is a reasonable probability distribution. The formula is as follows:

[0088] (3)

[0089] in, represents the normalized similarity. is the query matrix, K is the key matrix, V is the value matrix, and the superscript T indicates transpose.

[0090] (4) The value vectors are weighted and summed according to the normalized similarity probability distribution to obtain the final output vector, namely the text semantic vector. The text semantic vector integrates the information of all characters in the concatenated text, but some characters have a greater impact depending on the degree of attention.

[0091] The present invention integrates context information by dynamically assigning different weights to each element through the similarity between each word and other words in the text, thereby obtaining a text semantic vector containing context information, which is conducive to better understanding of text semantic information in the subsequent text resetting process.

[0092] Suppose there is a sentence: "I like to eat apples because they are sweet." Among them, the keyword "apple" is processed. The attention mechanism may find that there is a high similarity score between "it" and "apple", so it gives a higher weight to the value vector of "apple", helping the model understand that "it" refers to "apple". When processing "because", the similarity can help the model understand that "sweet" is a description of "apple", so as to better grasp the logical relationship of the entire sentence.

[0093] Step C: Use the cursor prediction model to process the concatenated text and text semantic vector to obtain cursor addresses of multiple key information.

[0094] In an embodiment of the present invention, the cursor address of the key information mainly includes the keyword position start cursor, the keyword position end cursor, the default position cursor, the pronoun position start cursor and the pronoun position end cursor address. Through the cursor address, the keyword can be extracted to replace the pronoun of the text to be rewritten or inserted into the default position, thereby obtaining a complete semantic text after rewriting.

[0095] Step C01: Process the concatenated text using the cursor prediction model to obtain key information of the concatenated text, wherein the key information includes keywords, pronouns and default positions.

[0096] Step C02: According to the key information of the spliced ​​text, the cursor prediction model is used to obtain the position of the key information in the text semantic vector, and then the cursor addresses of multiple key information in the text semantic vector are obtained, so as to be reset according to the cursor address later.

[0097] The present invention first locks the position of the key information in the text form, and then finds the corresponding position in the text semantic vector through the cursor prediction model based on the position locking in the text form. The calculation process of the cursor address of the key information is as follows:

[0098] (1) The cursor prediction model is used to predict the probability that each element in the text semantic vector is the cursor address of key information, and the probability distribution of the cursor address of key information in the text semantic vector is obtained. The calculation formula is as follows:

[0099] (4)

[0100] in, Indicates the predicted probability of the cursor address where the i-th character in the text semantic vector is key information, Represents the output vector corresponding to the i-th character in the text semantic vector, is the learning parameter of the cursor prediction model, b is the bias value, and L is the total length of the text semantic vector.

[0101] (2) According to the probability distribution of the cursor address of the key information in the text semantic vector, the element position with the highest probability is taken as the cursor address of the key information.

[0102] The present invention can extract the start cursor and end cursor of the keyword position in the text semantic vector according to the keyword, extract the start cursor and end cursor of the pronoun position in the text semantic vector according to the pronoun, and extract the default position cursor in the text semantic vector according to the default position.

[0103] Step D: reset the text semantic vector according to the cursor address of the key information to obtain the reset text semantic vector.

[0104] The keywords of the previous round of question and answer text are extracted from the text semantic vector according to the keyword position start cursor address and the keyword position end cursor address; if there is a default position cursor address, the keywords are inserted into the default position cursor address; if there is a pronoun position start cursor address and a pronoun position end cursor address, the pronouns in the text semantic vector are replaced with keywords according to the pronoun position start cursor address and the pronoun position end cursor address.

[0105] Step E: Use the Transformer decoder to decode the reset text semantic vector to obtain the reset question and answer text.

[0106] The present invention uses a text question and answer reset model to reset the question and answer text of the artificial intelligence big model. The text question and answer reset model includes a Transformer model and a cursor prediction model. The present invention obtains a large amount of historical question and answer text by acquiring the conversation history between the user and the machine in the artificial intelligence big model, and uses the training samples to train the Transformer model and the cursor prediction model, and optimizes each model parameter, thereby improving the model's performance in feature extraction, semantic encoding and cursor prediction.

[0107] In this embodiment of the present invention, the model training process is as follows:

[0108] (1) Initialize model parameters and training parameters. Model parameters include learning rate and weight of Transformer model and cursor prediction model. Training parameters include iteration termination threshold and maximum number of iterations.

[0109] (2) Obtain a large amount of historical question-and-answer texts from the artificial intelligence big model, and separate the historical question-and-answer texts into multiple training samples. The format of each training sample is (a, b, current, right), where a represents the user's input text in the previous round, b represents the output text of the artificial intelligence big model in the previous round, current represents the text to be reset in the current round, and right represents the text after theoretically correct reset.

[0110] Because the purpose of the present invention is to enable the model to eventually automatically reset some unclear texts through model training and prediction, before that, the model needs to be given a part of training data for training. At this time, the training data needs to include the text information to be reset and the text information after theoretically correct reset.

[0111] Theoretically, the method to obtain the text after correct reset is:

[0112] 1) Text preprocessing: Perform preprocessing operations such as word segmentation and stop word removal on the input text to ensure the accuracy of subsequent analysis.

[0113] The code example is as follows:

[0114] import jieba

[0115] def preprocess(text):

[0116] tokens = list(jieba.cut(text))

[0117] return tokens

[0118] text = "I like apples, they are sweet."

[0119] tokens = preprocess(text)

[0120] 2) Co-reference resolution: Use the open source natural language processing spacy library to perform co-reference resolution on the pre-processed text to identify and parse the co-reference words in the text.

[0121] The code example is as follows:

[0122] import spacy

[0123] nlp = spacy.load("zh_core_web_sm")

[0124] doc = nlp(text)

[0125] for token in doc:

[0126] if token.dep_ == 'nsubj' and token.head.text == 'very sweet':

[0127] print(f"The pronoun '{token.text}' may refer to {token.head.text}")

[0128] # Or use a more complex coreference resolution model

[0129] from transformers import pipeline

[0130] coref_pipeline = pipeline('coreference-resolution', model='xlm-roberta-large')

[0131] resolved_text = coref_pipeline(text)

[0132] print("Resolved Text:", resolved_text)

[0133] 3) If there is default information in the text, add the default information. The code example is as follows:

[0134] def infer_missing_info(text, context):

[0135] if "because" not in text:

[0136] for keyword in context:

[0137] if keyword in text:

[0138] text = text.replace(keyword, f"because {keyword}")

[0139] break

[0140] return text

[0141] context_keywords = ["apple"]

[0142] inferred_text = infer_missing_info(resolved_text, context_keywords)

[0143] print("Inferred Text:", inferred_text)

[0144] The above three steps can obtain the theoretically correct reset text.

[0145] Concatenate the data in the training sample in the order of a, b, current and right, and set the head, connection and tail marks to obtain the complete concatenated text, which is recorded as S=([cls]a, [sep]b, [sep]current, [sep]right[sep]).

[0146] (3) The concatenated text S is used as the input of the semantic encoding module, and the Transformer model is used to perform dynamic semantic encoding to obtain the text semantic vector.

[0147] (4) By comparing the text to be reset and the theoretically correct reset text, the position of key information (keywords, pronouns, default positions) is obtained.

[0148] (5) According to the location of the key information, the cursor prediction model is used to predict the cursor address of the key information in the text semantic vector.

[0149] (6) Reset the text semantic vector according to the cursor address of the key information, and decode it to obtain the reset question and answer text.

[0150] (7) Calculate the model loss based on the reset question and answer text and the theoretically correctly reset text. If the model loss is greater than the iteration termination threshold, update the model parameters and return to step (3). If the model loss is less than the iteration termination threshold, end the iterative training and output the trained model. The cursor extraction method of the present invention can complete the sequence annotation of the training data. During the model training process, it is necessary to continuously learn the connection between the context of key information in the text and predict the keyword position start cursor, keyword position end cursor, default position cursor, pronoun position start cursor and pronoun position end cursor.

[0151] In order to verify the performance of the method of the present invention, the embodiments of the present invention respectively select the more common LSTM-Gen (a model for generating reset text based on a pointer generation network) and Trans-Gen (a model for generating reset text based on a pointer generation network) in the industry to compare with the method of the present invention and conduct a test experiment. The configuration of this test experiment is as follows: operating system: WIN10, CPU: Intel(R) Core(TM) i7-10750H@2.60GHz, memory: 32g, development language: Python3.7, development framework: Pytorch.

[0152] The experiment selected similarity and matching indicators to evaluate the actual effect of each method. The experimental comparison results are as follows:

[0153] Table 1

[0154]

[0155] According to the data in Table 1, it can be seen that the method of the present invention is significantly better than the LSTM-Gen method and the Trans-Gen method in terms of the similarity and precise matching between the reset text and the standard text. It can be seen that the ability to extract contextual semantic features of the text has a certain influence on the method model. The present invention adopts the Transformer model to fuse the context information, which improves the effect of resetting the text.

[0156] The experiment also compared the time consumption indicators of each method for resetting the question and answer text. The results are as follows:

[0157] Table 2

[0158]

[0159] According to the data in Table 2, it can be seen that the LSTM-Gen method consumes the most time and has the lowest performance. Although the time consumption of the Trans-Gen method has decreased, the overall time consumption is still relatively high. The improved method proposed in the present invention performs well in terms of time consumption, and the method using the pre-trained model is faster than the method using the native network to build the model.

[0160] Example 2

[0161] Based on the same inventive concept as in Example 1, this example introduces a question-answer text resetting device for a large artificial intelligence model, such as Figure 3 As shown, it includes a data processing module, a semantic encoding module, a cursor prediction module, a text reset module and a text output module.

[0162] The data processing module is used to obtain the marked spliced ​​text based on the question and answer text of the artificial intelligence large model.

[0163] The semantic encoding module is used to perform feature extraction and dynamic semantic encoding on the concatenated text using the Transformer-based rbt3 pre-trained model to obtain a text semantic vector.

[0164] The cursor prediction module is used to process the text semantic vector using the cursor prediction model to obtain the cursor address of multiple key information.

[0165] The text resetting module is used to reset the text semantic vector according to the cursor address of the key information to obtain the reset text semantic vector;

[0166] The text output module is used to use a decoder to decode the reset text semantic vector to obtain the reset question and answer text.

[0167] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.

[0168] Due to its built-in multi-head self-attention mechanism, Transformer itself has strong feature extraction capabilities, and it also has unique advantages in processing text sequences for long-term text dependency problems. Therefore, the present invention uses the Transformer model to extract the global context features of the text, which can accurately parse the pronoun reference relationship in the dialogue and effectively restore the incomplete expression text with missing semantics, thereby ensuring that the reset text semantics are complete and accurate. The method of the present invention adopts the idea of ​​cursor extraction, uses the cursor prediction model to extract the cursor address of the key information of the text, and extracts keywords according to the cursor address to replace the pronouns of the text to be rewritten or insert them into the default position to obtain the reset complete semantic text. Compared with the traditional idea of ​​pointer generation, this method has great advantages in the time of text reset. Compared with the prior art, the present invention can significantly improve the accuracy of text reset and greatly reduce the processing delay, and is particularly suitable for scenarios that require real-time response, such as the large-model robot question-answering system for power marketing.

[0169] The present invention is conducive to promoting the business growth of large artificial intelligence models. By supporting high-quality question-and-answer services, long-term and stable customer relationships are established, more new customers are attracted, and the market competitiveness of enterprises is enhanced. At the same time, the user experience is enhanced, the interaction is made more natural and smooth, and user stickiness is effectively improved.

[0170] The invention also promoted technological innovation, accumulated a wealth of technical experience in natural language processing and machine learning during the development and optimization process, and set an example for digital transformation in the industry.

[0171] This invention supports the organic integration of knowledge and quick questions and answers, greatly improves the business personnel's perception of professional knowledge, lowers the threshold for knowledge acquisition, improves the company's existing knowledge management system, accelerates the company's digital development process, and effectively enhances the social image of the power grid company.

[0172] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for resetting question and answer text of a large artificial intelligence model, characterized in that: The steps include: According to the question and answer text of the artificial intelligence large model, a concatenated text containing tags is obtained; Using a Transformer model to dynamically semantically encode the concatenated text to obtain a text semantic vector; Processing the concatenated text and the text semantic vector using a cursor prediction model to obtain cursor addresses of multiple key information in the text semantic vector; Resetting the text semantic vector according to the cursor address of the key information to obtain a reset text semantic vector; The reset text semantic vector is decoded by using a decoder to obtain a reset question and answer text.

2. The question and answer text resetting method according to claim 1, characterized in that: The question-answer text of the artificial intelligence large model is used to obtain a concatenated text containing a mark, including: Generate document samples according to question-and-answer texts of two adjacent rounds in the artificial intelligence big model, wherein the format of the document samples is (a, b, current), wherein a represents the input text of the previous round of the artificial intelligence big model, b represents the output text of the previous round of the artificial intelligence big model, and current represents the text to be reset in the current round, and the text to be reset in the current round includes the input text and output text of the current round of the artificial intelligence big model; The text data in the document sample are concatenated in order, and a head tag is set before a, a connection tag is set at the connection between a and b, and between b and current, and a tail tag is set after current to obtain a concatenated text containing tags.

3. The question and answer text resetting method according to claim 1, characterized in that: The method of using the Transformer model to dynamically semantically encode the concatenated text to obtain a text semantic vector includes: The Transformer-based rbt3 pre-trained model is used to process all characters in the concatenated text in parallel to obtain multiple word vectors. The formula is as follows: ； in, Represents the i-th word vector in the concatenated text The input embedding of Represents the i-th word vector word embedding, Represents the i-th word vector The position of the embedding, , N is the length of the concatenated text; Map each word vector into a query vector, a key vector, and a value vector, and calculate the similarity between different word vectors in the concatenated text. The calculation formula is as follows: ； in, represents the similarity score of the i-th word vector to the j-th word vector in the concatenated text, Q i represents the query vector of the i-th word vector, K j represents the key vector of the jth word vector, is the dimension of the key vector; The similarity between different word vectors is normalized using the softmax function to obtain the normalized similarity probability distribution. The formula is as follows: ； in, represents the normalized similarity. is the query matrix, K is the key matrix, and V is the value matrix; The value vectors of all word vectors are weighted and summed according to the normalized similarity probability distribution to obtain the text semantic vector.

4. The question and answer text resetting method according to claim 1, characterized in that: The step of processing the concatenated text and the text semantic vector using a cursor prediction model to obtain cursor addresses of multiple key information in the text semantic vector includes: The cursor prediction model is used to process the spliced ​​text to obtain the key information of the spliced ​​text; According to the key information of the spliced ​​text, the cursor prediction model is used to obtain the probability that each character position in the text semantic vector is the cursor address of the key information. The formula is as follows: ； in, The cursor address prediction probability representing the key information of the i-th character in the text semantic vector, represents the i-th character in the text semantic vector, is the learning parameter of the cursor prediction model, b is the bias value, and L is the total length of the text semantic vector; According to the probability that each character position in the text semantic vector is the cursor address of the key information, the character position with the largest probability is selected as the cursor address of the corresponding key information.

5. The question and answer text resetting method according to claim 4, characterized in that: The key information includes keywords, pronouns and / or default positions, and the cursor addresses of the key information include a keyword position start cursor address, a keyword position end cursor address, a default position cursor address, a pronoun position start cursor address and a pronoun position end cursor address.

6. The question and answer text resetting method according to claim 1 or 5, characterized in that: According to the cursor address of the key information, the text semantic vector is reset to obtain the reset text semantic vector, including: The keywords of the previous round of question and answer text are extracted from the text semantic vector according to the keyword position start cursor address and the keyword position end cursor address; if there is a default position cursor address, the keywords are inserted into the default position cursor address; if there is a pronoun position start cursor address and a pronoun position end cursor address, the pronouns in the text semantic vector are replaced with keywords according to the pronoun position start cursor address and the pronoun position end cursor address.

7. A question-answer text resetting device for a large artificial intelligence model, characterized in that: include: A data processing module is used to obtain a concatenated text containing a tag based on the question and answer text of the artificial intelligence large model; A semantic encoding module, used for dynamically semantically encoding the concatenated text using a Transformer model to obtain a text semantic vector; A cursor prediction module, used for processing the concatenated text and the text semantic vector using a cursor prediction model to obtain cursor addresses of multiple key information in the text semantic vector; A text resetting module, used for resetting the text semantic vector according to the cursor address of the key information to obtain a reset text semantic vector; The text output module is used to decode the reset text semantic vector using a decoder to obtain a reset question and answer text.

8. The question and answer text resetting device according to claim 7, characterized in that: The semantic encoding module is specifically used for: The Transformer-based rbt3 pre-trained model is used to process all characters in the concatenated text in parallel to obtain multiple word vectors. The formula is as follows: ； in, Represents the i-th word vector in the concatenated text The input embedding of Represents the i-th word vector word embedding, Represents the i-th word vector The position of the embedding, , N is the length of the concatenated text; Map each word vector into a query vector, a key vector, and a value vector, and calculate the similarity between different word vectors in the concatenated text. The calculation formula is as follows: ； in, represents the similarity score of the i-th word vector to the j-th word vector in the concatenated text, Q i represents the query vector of the i-th word vector, K j represents the key vector of the jth word vector, is the dimension of the key vector; The similarity between different word vectors is normalized using the softmax function to obtain the normalized similarity probability distribution. The formula is as follows: ； in, represents the normalized similarity. is the query matrix, K is the key matrix, and V is the value matrix; The value vectors of all word vectors are weighted and summed according to the normalized similarity probability distribution to obtain the text semantic vector.

9. The question and answer text resetting device according to claim 7, characterized in that: The cursor prediction module is specifically used for: The cursor prediction model is used to process the spliced ​​text to obtain the key information of the spliced ​​text; According to the key information of the spliced ​​text, the cursor prediction model is used to obtain the probability that each character position in the text semantic vector is the cursor address of the key information. The formula is as follows: ； in, The cursor address prediction probability representing the key information of the i-th character in the text semantic vector, represents the i-th character in the text semantic vector, is the learning parameter of the cursor prediction model, b is the bias value, and L is the total length of the text semantic vector; According to the probability that each character position in the text semantic vector is the cursor address of the key information, the character position with the largest probability is selected as the cursor address of the corresponding key information.

10. The question and answer text resetting device according to claim 7, characterized in that: The text resetting module is specifically used for: The keywords of the previous round of question and answer text are extracted from the text semantic vector according to the keyword position start cursor address and the keyword position end cursor address; if there is a default position cursor address, the keywords are inserted into the default position cursor address; if there is a pronoun position start cursor address and a pronoun position end cursor address, the pronouns in the text semantic vector are replaced with keywords according to the pronoun position start cursor address and the pronoun position end cursor address.

Images